Came here to say a similar thing. The reaction can say a lot about the person's opinion/impression of you. It might be too much to try to correct for other people's insecurities; however, if you find that a large portion of people get upset by your phrasing, then it might be worth it to consider how you present yourself in general to others. It might just be you.