I don't particularly care for the term "lady", but I would never consider it patronizing or an insult.

I agree that people need to be more forgiving, this is just not worth the effort to worry about it.