My guess is these audiophile nutballs have something to their argument, but that it's pretty irrelevant in the grand scheme of things. I had a friend who used to rant and rave about this crap, and then I made him try out an ear testing program I have. Basically you input a wave file from a cd into it and an encoded mp3 of the same song, and it chops the song up into segments and you have to try and pick which is the mp3 and which is the wav. It then calculates your guesses, and if you got more than 66% of them right (or something like that) it considers that probably enough to not be dumb luck and you "pass."
And, this snobby audiophile who always railed about loud production distorting the original recording, always bought vinyl, cd, and original master recordings of everything, refused to rip anything to his computer in anything less than .flac files, couldn't differentiate between a wave and a 96 k/sec mp3! 192 is supposed to be the point at which it becomes hard for non-pros to distinguish, but he couldn't even manage 96. I'd imagine a lot of these alarmist snobs are exactly the same way. They like to make a gigantic fuss about something and buy multiple copies of an album to avoid something that's probably only detectable by computers and .5% of those who listen to it.