It will not work, when you apply voltage to any one cart, the data pins will all interfere. While I have not actually tested this, someone tried it on NES, and on SNES, and I have tried it on a GBA. It will not work without finding a way to isolate all the data pins as well.
Of course, I could be wrong, but unless the N64 is significantly different, I think the same applies. Nothing to lose by trying!
Although I must say, if you only play those games, an iQue would not be a terrible choice, there is a guy who sells them for $90 with Dr. Mario and any other game for free, and $8 a game for as many other games as you want.