Instrument Separation

use some kind of graphic analyzer. I'm not telling you to use your eyes but it's great to start with. You will see where the instruments sit and work from there. I highpass/lowpass almost everything to a certain degree. Don't look at the numbers, they're just pointers. use your ears and start with the highpass. Turn it up until the source starts to thin out, back up a bit. That's how I do it.

How I do it:

Guitars: Hipass from 80-100 hz, lowpass between 7-12khz, usually a small cut in the midsection to give space for the vocals, I cut a lot around 4khz (narrow!)

Bass: I cut A LOT of the lowmids. 500hz get's sucked out a lot. Wide cut... makes space for the guitars and sounds great imo (most of the times... use your ears), lowpass around 4-6khz, hipass (if needed) around 20-40hz. I always cut out some 60hz-80hz depending on were the kickdrum peaks. Boost at 80-120hz if needed.

Vocals: Hipass up to 200hz depending on the source. Vocals are a bitch, use your ears.

Drums: highpass EVERYTHING, I usually even highpass the kick (slate is boomy as fuck) up to 50hz. Toms depending on their tuning, use your ears...high tunes toms can get highpasses up to 180hz... depending on the slope of course :) Don't try to boost the toms all at the same frequency. Every tom sounds different and needs different treatment. So don't copy+paste the settings :). I usually highpass OH's up to 600hz, if possible lower.

Now start to listen for ugly peaks. Like that 4khz shit from the guitars (that I always find). It's covering the whole mix with this noise that's not doing anything good to the guitars. solo the guitars, narrow eq, search for that noise and pull out as much as needed without muffling the guitars. Next step... :)

ah yeah... something that will help you a lot too: Tight playing for all the instruments.

Some of my recent mixes were too separated so maybe this will help you :)
 
Toms are easy. There's always a reso peak that rings forever between 90-150hz, I zoom in on with a max q and suck it out. Cant stand it. Depending on your mics, tom tuning and if there was eq going in, you'll probably find alot can be cut around 2-300hz and 5-800hz. Medium q, sometimes pulling up to 10db out. Nice shelf from 6-8khz, you'll hear when you've found the right spot to give the drum some attack, clarity and click to pop over the mix. Floor toms love a bump at 80hz. Comp I'm usually using from the channel, 30ms/program dependent attack, 2-400ms release pulling anywhere between 3-10 db gain reduction til you find the sweet spot for fatness. Chuck a clipper on the tom bus.