Thanks for all the answers! The one thing I find disturbing is that the law applies even if you are on private land. While fully agreeing with regards to the safety of others, while I'm on my own land, provided I'm not located somewhere where it could be harmful, such as a flight path for an airport, I feel the FAA should have limited authority over my activities. So long as all activity remains within my boundaries and at 400 feet or lower, and I'm not engaged in otherwise illegal activities such as looking in neighbor's windows or scaring their animals, etcetera, of what concern is it to the government?
With amateur radio, there is an organization (ARRL) in the US that acts on behalf of ham radio operators to protect their hobby concerns from intrusion by the FCC. The same for firearm owners (NRA) vs multiple state, local and Federal laws and actions. Is there any such advocacy group for drone operators?