That’s because it contains small amounts of the pollen you might be allergic to. So it’s really important that it’s raw, because you don’t want to cook the proteins that the pollen is made of.
FWIW, that's been thoroughly debunked. For the vast majority of people who suffer from seasonal allergies, they are not allergic to the same plants that bees pollinate.
It’s probably true that most seasonal allergies are cause by wind-pollinated plants, but bees sometimes help to pollinate those kinds of plants if they happen to be the closest food source. I have ash trees (Fraxinus) in my neighborhood, and even though they’re primarily wind-pollinated, I always see lots of bees gathering their pollen in the springtime. But bees would probably prefer flowers with more nectar if they’re available.