Bees are pretty good about 'live and let live', at least in my experience. Unless you step on a bumblebee barefoot or something of the like, of course...
I grew up in the bad old 70s, full of bees and other stinging things, walking around barefoot pretty much all summer and I've never been stung on the foot. Likewise, we have a very bee-friendly yard, and our son (now 10) has never been stung by one.
Wasps and hornets? Him once, and myself 2-3 times, but that has nothing to do with plants, those suckers like making hives all over the place.
Try talking to people with bee phobias, instead of acting only on your own experiences. I get a high percentage of people who were either stung as a child or witnessed it happen to someone else.
It's very similar to your wasp experience, and some people's experience with certain dog breeds. What terrifies us is when a creature lashes out and we don't understand why. If we understand we feel we have control of the situation.
Most of us [lack] that feeling of control with wasps (I like parasitic wasps, I still hate the hornet family). Some people have this with chihuahuas or pits (hard to read body language). A lot of people who fear bees have the same thing.
I have to show them that you can put your face right up to bees foraging and nothing happens because they are busy and have no opinion on your proximity (it's the hive they care about).
People won't 'show up' for things they have no feelings about, or things they have negative feelings about. I don't want to deal with another generation of suburban bee-haters when I'm old and grey.
Wasps and hornets give bees a bad name.