You need to start with a Rev 3 or Rev 4 N64. Then you have to use a three channel video amplifier (THS7314) to amplify the signal to where its usable. After the amplifier you need a 75 ohm resistor on each if the three channels. From there you can either use a SCART television (good luck in the states) or a device that supports the uncommon sync rate (15.75kHz I believe). The PSone natively supports N64 RGB in this fashion. From there if you want to use it on anything other than the SCART TV or the PSone screen you need a GBS-8200 to convert it to VGA. To feed the signal into that you will need to get the Sync Line for an RGBS signal. You can either build a sync splitter (or some products exist called the Sync Strike) to strip the RGB Sync line from the composite video output. Or you can use the Y Luma line from the SVideo output.
Here's a link to a half decent
guide except instead of pulling the RGB signal from the resistors R8, R9, and R10 you need to get the signals from the capacitors next to them C124, C125, and C126.
If you want a more authentic look you can build a Scan Line Generator that will anti alias the image (CRT Screens already give this look). Mmmonkey has a guide on building one. And there also exists products out there that can do this too.
As long as you keep you wires short, shielded, and away from wires with power going through them you should get an acceptable image on screen. Personally I don't think its worth the effort. I've done these mods to three of my N64's and I have to say that a TV with a good scaler board and composite or svideo inputs will give better results with less effort. My $350 Samsung 40 inch TV from Wal-Mart does a really good job with its composite input. But for my monitor I bought a fairly pricey upscaler that does a really good job at converting the signal to VGA and cleaning up the image.
When I finish
my AIO portable I will have to do some comparison pictures of RGB vs Composite/Svideo through an upscaled. Here's a link to that work log in case you are interested in some pictures of the RGB amplifier and GBS-8200.
EDIT: Fixed the links and updated the N64 RGB output to 15.75kHz as thats what it outputs and VGA requires 31.5kHz.