Most people aren't zen monks who've mastered their emotions. Emotions are the polar opposite of logic, and you can reason that "oh, it's some random stranger, I shouldn't care what they think or say," all you want, but their words can still have an effect. "Words will never hurt me" isn't a statement of fact, it's a mantra one repeats to try to master one's feelings in the face of such abuse. So you can confidently state that it doesn't make sense to you that people are upset by strangers' words, but that doesn't change the fact that they are, and you shouldn't consider such dismissal the final word on the subject.