Just because it's legal doesn't make it right. We all are forced to put up with crap like intentionally loud exhaust systems, music too loud, litter bugs, and rude people. I try to avoid being one of these inconsiderate folks because they're not making the world a better place.
Approaching the neighbor to discuss why s/he is upset by your drone flying seems like a good idea. Fear of the unknown causes a lot of problems. I find that people can usually figure out that you're not up to any nefarious operations with an explanation. Definitely show some sunset photos.
Please do let us know how this worked out, and please DON'T tell the neighbor to **** off. Being a jerk reflects poorly on all of us.