Oh yeah, I'm sure hearing the two languages will be a bit jarring, but there's a delay between the speaker and the translation—the earpiece needs the entirety of what's being said in order to properly parse it—so I think that'll help mitigate the issue. (Perhaps it'll even start to build one's vocabulary and understanding of the other language.)
It looks like they (smartly) started with a very small language set—English, French, Italian, Portuguese, and Spanish—so I'm hopeful that the panties-mishap you mentioned wouldn't be too likely. (Also, I'm curious if they're going to have something like Google Translate's interface on the phone that the earpiece is connected to. That way, when the translation happens, one can see the original statement and possible alternatives.)