You might be scalding the 'normal' milk, that'll lead to a sour taste. If your coffee is as hot as mine tends to be it can chemically alter the milk in unpleasant ways.
Goddman! I hope all of you are talking about american standard cofee, because if you're talking about espresso, you really can't make a decent coffee....