I'm with you there on that. Today I allowed it to do its own RTH fully since the ground was tarmac, but other wise I cancel when it's a metre or two off the ground and hand land. I often use a wide circular photo reflector as my launch pad but even then have to move it over to land the drone squarely on it.I also thought in the beginning that RTH was a recovery function for emergency cases. When I tried it the first time, to get familiair with my first Phantom (many years ago) I was surprised that it worked so well. Since than I always use it when the battery gets low or when I loose sight on the drone. In The Netherlands we always have to keep a drone in sight, which is not always easy as you know. When I don't see it anymore, I don't panic but after a while I think it is wise to bring it back in sight and press RTH. When I see it approach me I sometimes quit the RTH procedure and fly further. But often I do not interrupt the procedure and enjoy to see my toy land automatically at less than an inch from its original starting point.
My RTH-altitude is always 130 feet or more, to avoid tall trees.