Please remember that in the USA, 400 feet AGL, at the drone's location, is the maximum height limit. Even at that, altitude separation between drone and aircraft is not assured depending on where you are flying. As a general rule, where there are no people of structures, manned aircraft can fly as close to the ground as they like. Listening for aircraft engine noise, as a warning may help, though not always. Gliders often fly ridge if there's wind above 10 knots or so, and they are difficult to spot, make little noise, and "polish the rocks" as we say.