Well, I don't mean to be an ass, but by reencoding bitrates and reencoding BETWEEN FORMATS you are quite possibly doing this: introducing distortion and fuzziness. Volume levels has absolutely nothing to do with bitrates or formats whatsoever.
Reencoding from a lossy compressed format to another always, always, always decreases quality. You can go from 96k to 320 = quality decrease.