Not sure this is entirely true. For instance, on the occasions I've had someone upset about "spying" on their family and property, I've been successful in changing their minds a few times by simply showing them what the wide-angle view is from...
Nobody is going to shut down the 0 to 400-foot-altitude airspace to drones. That would adversely affect the few people who use drones of domestic manufacture as well as those of us who fly DJI. YMMV, but I've never encountered any public...