Pilots fly below 500ft because it is FUN!. The world looks wonderful from the air looking down at it. The world looks different at 300ft, different again at 500ft and different again at 1,000ft and again different at 2,000ft, 5,000ft and 10,000ft. When you are way up there, say in a slow flying aircraft, things move below you s l o w l y.. but when you drop down to 500ft, you get more of a sense of sped in your slow flying aircraft, plus the world looks so nice from that altitude.
That is why you fly at those altitudes, where it is allowed. I used to teach flying out in Colorado, many years ago. We had some pilot friends who flew for United, based in Denver. They would come up into the mountains from time to time to fly with us and we would drop down to 100ft or 50ft and sometimes 20ft above the ground and follow the terrain over open undulating prairie land, or skip over fresh hay bails in open fields early in the morning, just for the fun of it. These United pilots would whoop and yelp as we flew, saying they just loved the feeling of doing this type of flying in an open cockpit aircraft, more than any other type of flying.
So, the simple answer is because it is just plain fun and legal to do when you understand where you can do it. If you take the hierarchy of flying, everything shares the airspace and enjoys flying but as you move down the food chain, there are different rules to obey, and you must observe those rules to keep everyone safe and happy. Slower aircraft usually fly inside the pattern and often at lower altitudes, because that is where you need to be to be seen by faster aircraft in the pattern. Patterns in general, are always left hand at an airport unless something or structure or development necessitates a right hand pattern, but this is then noted on charts.
Faster aircraft know where to look to see if there are slower flying aircraft in that pattern, plus a pilot announces what they are doing and where they are in the pattern, at uncontrolled airports, so that everyone else in the pattern or on the ground or inbound, knows what is going on and can visualize where the other aircraft are in relation to themselves. This generally keeps everyone using the airspace, safe and happy.
Even and ultralight, a real ultralight, not just a small aircraft that most people just call an ultralight, has a right to be up in that airspace. However, they are the lowest on the food chain and therefore must yield to every other type of flying machine in the sky. Even in the pattern, every other flying aircraft has priority over them, other than, say, a powered paraglider. Even when in the pattern, if another heavier and faster aircraft is catching them up, the ultralight must yield and allow the other aircraft to land. The only time this does not apply is when they may be landing in an emergency situation. Then all other aircraft must give way. In fact any aircraft landing is an emergency has right of way over everything else.
So you can take the drone as being even lower on the food chain than an ultralight, and they must then give way to everything else out there flying, even if the drone is within their legal altitude of surface to 400ft. If you start to think that way, then you won't be getting in the way of anything else up there flying about. You won't be misguided about your right to be in that airspace and expect other aircraft not to be there, and everyone sharing the sky will be happy.
