Drummers are usually tone deaf so I'd recommend to tell him to STFU.
However I understand why we'd want to hear the drums from the drummer's perspective, but that would only make sense if you flip all the mics and not just the overheads, that shouldn't cause any issues afaik.
TBH, I tend to pan the drums from the drummer's perspective (even though I'm a guitarist) but everything needs to be placed correctly otherwise it's just retarded.
Cheers,
Necro