Ulver - "Bergtatt: Et Eeventyr I Fem Capitler" is pure, romantic Norwegian black metal the way it should be done, with beautiful acoustic interludes. Their best. The follow up, "Kveldssanger" is a purely acoustic album, very folk-ish and Norwegian. Get it if you're into that sort of stuff. "Nattens Madrigal" is extremely raw and harsh. A lot of people dislike it due to the very minimalist production. Still, it's a good album. Their electronica albums and several EPs are also good, get them if you like experimental electronic music.
Satyricon - "Dark Medieval Times" is absolutely MANDATORY, and one of the three best black metal albums ever. Wonderful stuff, it has got so much more charm (and also better melodies, I think) than their second album, "The Shadowthrone", although that is a very good release too. "Nemesis Divina" is pretty good too.
Mayhem - "De Mysteriis Dom Sathanas" is also mandatory, and probably the second best black metal album ever. You HAVE to get it.
Borknagar - You can't really go wrong with Borknagar, I feel, although their latest effort, "Epic" was a bit of a dissapointment. "Quintessence", "Empiricism" and "The Archaic Course" are epic masterpieces (many will perhaps mislabel them as black metal, as they sure sound like a black metal band at times). "The Olden Domain" and "Borknagar" are also top notch.
Thorns - Many claim that without Thorns mastermind Snorre, the third wave of (Norwegian) black metal would never have existed. He influenced tons of Norwegian black metal bands through his demo tapes in the early- and mid 90s. Check out his split with Emperor if you want to find out what Thorns is all about.
Darkthrone - Every album up to and including "Total Death" is a must buy, especially "Under..", "A Blaze.." and "Transilvanian.." . You might also like their newer stuff, which is groovy black 'n roll-ish black metal.
Behemoth - Early stuff: Above average black metal. Later stuff: Killer death metal. Check out "Demigod" or "Satanica" to find out more about their blackened brand of death metal, or "Sventevith" if you want to check out their black metal stuff.