Actually in the case of Emperor, the music certainly did not become more mainstream. The production quality had increased, but I think that's only because the guys realized it was stupid to make their recordings inaudible. As time wore on Emperor grew technically as much as it did idealistically, and more ideas flourished. 'Prometheus' is a masterpiece and is certainly not mainstream in any way whatsoever.
With Opeth, it was more about styles. When they left the Orchid and Morningrise era behind them to start with MAYH it was almost like a new band. They incorporated new ideas into their music.. there was a major rehaul.. not every riff was harmonized, the song structures were more tight and the songs flowed better. It could be argued that MAYH still remains their heaviest album to date. You COULD certainly say that MAYH is alot more ear-friendly than the first two, but I like to believe that's largely because of the production. I mean In Flames did the dual-guitar thing too, and look where they are now.
Opeth then hit Still Life, which in my opinion is their peak.. the epitome of their style and music. It was like they nailed the perfect balance of progressive(ness) and metal, melancholy and soothing ambience. I love Orchid, and I always will, but I love Still Life just as well in a whole different sense... they are two different albums and Still Life is by no means compromised in any way.. if anything their technical abilities had increased substantially in that time, and the music had only increased in complexity. The only things mainstream-friendly on that album are the two ballads, Benighted and Face of Melinda, with the rest being amonst some of the most fucked up riff-induced pieces of musical literature I've ever heard in my entire life. In my opinion ever since Still Life, they have not reached this level of genius.
When Opeth hit BWP, this is when they started compromising. IMHO as soon as Steve Wilson hit the table, Opeth's music immediately had to accomodate that. Whether or not they actively changed anything, SW's influence moved Opeth away from that dark melancholic progressive realm they inhabited to try new things, new things that ultimately, and also unfortunately led them to Deliverance.
Deliverance is the product of rushed time in the studio, and whilst the band and most fans are happy with it, I certainly am not. Deliverance to me is the most boring of the Opeth albums, and gets quite tedious to listen to after you hear the ending of Deliverance 20, 000 times. It's nice that they tried new things with By The Pain I See in others, and certain production elements in other tracks, but that's about all... I couldn't see the genius of Opeth that I saw in times past.
In general practice, what you describe does not have to be the path of bands, yet it certainly is for many. For Emperor it certainly is not, whilst for Opeth it kind of is. I guess you could say in my mind Opeth started 'selling out' with BWP, because ironically when they started gaining more recognition for their music, I started appreciating it less. Not saying BWP is not an awesome fucking album by no means, but it pales in comparison to the giants of the past. Deliverance probably isn't even worth mentioning as the only high point to that seems to be the 15 yr old slipknot fans who love the end of Deliverance, despite having dissed the fuck out of Meshuggah. I don't think Opeth actively made a conscious decision that they would make their music more accessible, no I doubt they would ever do that... but that IS what has happened in my opinion. Coincidentally, ever since Steve Wilson took the helm I kept losing that feel I had for Opeth.
This is why I'm so fucking pissed off that he's taking the reigns again for the next album.