No it's not. It's exactly the same. He's "bothering" me, not violating any laws, I asked him to stop, he politely declined. The height is relevant only in that his treehouse COULD be peering into my home. I can't prove it is, or if his kids bring devices that have cameras on them up inside to play with...
I'll tell you what: you can keep folding like a cheap suit everytime something you have EVERY RIGHT to do "bothers" someone (especially if they threaten you over it...), but I prefer to not back down in the face of someone that has an attitude so bad they follow me home, all in order to confront and threaten me over a drone passing by their house at 400ft (and the mild hum it emits).
Just because something bothers you does not mean everyone has to stop immediately or it makes them a bad person because they won't bend to your will. If they aren't breaking the law, you don't get to dictate what someone can or can't do just because your panties are in a wad over it.
If you are "neighborly" and approach me like an adult, explain why it bothers you so much, and ask me politely to stop, odds are I will at minimum consider it. Storm over and threaten to shoot it down if I do it again will NOT result in me "wanting you as a friend" or "bringing you beer" though... But if you are that type of person that appeases bullies than so be it. I am not. And every bully that you appease on THIS subject, grants more credibility to the idea that drones shouldn't be allowed anywhere. After all, even at 10,000 feet away they might "invade" privacy by being able to make out a fuzzy lump that sort of looks like someone's daughter.
We HAVE rules. Follow those rules, and there's no need to be weaker than circus lemonade anytime someone confronts you. The difference is: I respect your right to be weak. I'd prefer you weren't, as it is hurting my hobby, but it's your right to exercise, not mine.