Not sure if you're running Mac or PC, but here's my take on the PC version:
Try messing around with the DMA buffer settings either in Nuendo or the dedicated software that runs your audio card (if applicable). Smaller values can result in pops and clicks when you record and larger values typically result in more latency.
Also using the right driver in Nuendo is important. If your card has an ASIO driver, go with that one instead of the Direct Sound or whatever crappy Windows driver it might be defaulting to.
Sorry if any of that is totally obvious, I'm not sure what you have and haven't tried yet.
But wait, there's more... If you're running 2000/XP, the tweaks on this page really do wonders for system performance in general -- but if your computer isn't a dedicated DAW, pay attention 'cause you might end up turning off your printing and networking services, etc.
http://www.modularmedia.co.uk/downloads/XPdawTweaks.htm
I just stumbled across this one today, but I haven't tried any of it out yet. If anyone finds this useful, be a sport and share with the group, eh?
http://mark-knutson.com/t3/
edit: Oh, and if you're running a lot of tracks in playback while you're recording, try disabling the tambourine and cowbell tracks and maybe kill the Castle Donnington reverb on the drum buss to get some more system resource "overhead" back. Heh.