I think there was a discussion about this very subject several years ago.
The "conclusion" was that pollen MAY induce micro-marring or very fine scratches in the paint due to the very "pricky" natural structure of the pollen and how it is removed. Under a microscope they look like balls with sharp cones or pyramids (dager-like) on them and if wiped with ,say, just quick-detailer & microfiber or California duster, it MAY cause this micro-marring. I was suggested to WASH it off with low-pressure spray and do a two-bucket wash with a highly lubricated type soap.
I would also caution that tree sap is probably a MUCH bigger concern. Here in Wisconsin, locust tree sap will burn and etch your clear-coat if not removed in a short time.
However, that is of little immediate concern , as we are dealing with a MAJOR Winter Storm as we speak (or, type). Here in Green Bay, WI we have 8" on the ground and expect another 4-7" tonight thru Sunday. After shoveling for the 2-1/2 hours this morning in my own driveway and another 2 hours this afternoon at the neighbors, I am exhausted and my arms ache. Yes, I have a snow-blower and it was put to good use by my wife in helping me this afternoon. You know it is BAD when a grader with chains get stuck and a large front-end loader comes and pulls it out. There was a four-foot high bank of snow from the plowing at the neighbors across the corner sidewalk apron. THAT took a while to clear and the snow is like cement; very packy. So now I am resting, checking out Autopian forum.