Racism? Frustration?...uh...whatever

Ever since I landed in the United States, Americans have taken the pains to understand the Indian English that I speak. Like for instance the day I landed in Washington, I was waiting for my connecting flight to Raleigh. I walked up to the gentleman in the American Airlines counter and asked him the gate number for the flight departing to Raleigh (which I pronounced Rallay). The guy strained his ears and asked me to repeat my question again. I obliged and he then realized it was pointless to make me repeat the query again. So, he started relaying each and every word that he could make sense of. Well it was quite a spectacle, two men who were at pains to understand each other even though they spoke the same language. Finally after a painstaking few minutes passed by we seemed to agree on everything except Raleigh. The AA guy threw his hands up in the air and told me that there is no place called Rallay in the United States. Yeah, he scared me out of my wits. You could imagine how it would have felt for a person who has come all the way from Madras to a non existent place to do Engineering. I then showed him my boarding card and pointed R A L E I G H on it. The man smiled at me which made his already creased face resemble a crumpled paper. The smile then gave way to hearty laughter which made the flab hanging out of his pants heave up and down like jello. I just stood there waiting for an explanation because I frankly thought the humor, if at all there was any, was too subtle for my taste. He then recovered himself and told me, "Sonny, its rolly not rallay, rolly as in dolly and you need to go to gate 24." I smiled at him sheepishly and walked towards 24. Over the past 2 years, consciously or unconsciously I have changed the way I pronounce certain words (for e.g. 'schedule') to make my life and the listeners life easier.
But, I had an altogether different experience last week. I called up XYZ to request some information. I introduced myself and asked the lady (she had an african american accent) my question. Her reply was, "Sir, You have a beautiful accent, I didn't understand a word of what you said, why don't you tell me your account number first to make things easier for me."
I was silent for a couple of seconds. I couldn't think of a smart repartee to say "**** *** ****" All I managed to say was, "thank you ma'am that wouldn't be necessary." and hung up. Maybe she had a rough day, or maybe she really had something against people who couldn't speak like Chris Tucker. I have problems understanding what the Chinese say but I never mock them on their face. I am not sure whether this qualifies as racism or probably an act of frustration.But, it certainly made me seethe with rage for the next hour or so.