Yeah, that bugs me too. But on the other hand people who are not willing to communicate with tourists in any other language that their own bug me even more. For instance when I go to Croatian sea or to Italy to shop I can understand some people don't speak English there but sometimes I think they just don't want to speak to me or anyone else anyway. They expect me to know Croatian/Italian just because I come from border country which is insignificant and way too small for them to even bother to understand the language. I mean, I can't possibly speak all the languages. I speak 5 languages already and I'm really fluent in 3 of them, so when I come to Italy I try to communicate in English if it doesn't go well then Romanian which is a little similar or if I come to Croatia then I speak Slovenian/Russian and I know that if they tried they could understand me but no way they would try.
