No, he's right in this case. An album that drifted from genre to genre would sound more like a compilation than a cohesive work. I can guarantee that you might listen to the individual tracks fairly regularly but you'd almost never listen to the whole thing in one go. There's a reason split albums tend to pick bands of the same ilk rather than slapping a drone band next to grindcore.
It all depends on how it's done, really. Although rare, compilations can be cohesive too. And I typically will listen to an album from start to finish regardless, so it wouldn't be an issue with me. I think if a band went into studio with the sole intent of making an album where every song sounds entirely different from each other, then I'd be inclined to agree with you. However, if a band absorbed all of their influences and ideas and were able to pool them all together effectively, then I don't see any problem. The Meads of Asphodel are an even better example than Akercocke for this and I have absolutely no qualms about them whatsoever, I love their music and style. They could probably mix together any combination of styles and you would still get something that still retains the band's character and integrity with enough strength to hold it together cohesively.
Whether you like those bands is up to you but basically, it can be done. It's just that most bands choose not to do so or are limited in their influences, creativity, ideas, vision etc. I personally like it when each song on an album stands out to me and is memorable in it's own right. I think a lot of metal bands tend to stay within the formula a bit too much.