Take off someplace else and fly over The Villages if that's what floats your boat.
A jurisdiction can enact an ordinance or pass a law which forbids taking off and landing drones within its boundaries, but it can't regulate the airspace above. That's the province of the FAA. If neighboring lands outside the jurisdiction boundaries don't have similar prohibitions, and if the FAA hasn't published any restrictions applicable to those areas, then it should be legal to take off and land outside jurisdiction boundaries and fly over it in FAA-regulated airspace. No doubt private and commercial pilots routinely fly over The Villages in their Cessnas and 737s in compliance with FAA regulations and nobody gives it a second thought.
What sets drones apart from other aircraft (other than helicopters) is their ability to fly lower and slower, and to hover. Plus, they're usually equipped with cameras. Concerns about intrusive behaviors raise some folks' hackles and may prompt jurisdictions to restrict drone use.