I found that using the caching caused problems on my phone. I use a Nexus 5X and it's great, fast and all. The issue is namely that when the phone (or iPad Mini, actually) gets full (storage wise) then the app crashes, forcing me to fly the drone down to safety using only LOS and my flying skills. I've since turned caching off and it's more or less 100% stopped the issue. I then just limited the cache size to 2GB and made sure I would always have 4GB+ remaining after that (the phone is 32GB total). Seems to have helped a lot.
To answer your question though, assuming you have RTH set correctly and you have done a compass calibration recently or immediately before flying, and you've plenty of satellites , you can actually turn your controller off entirely and the drone will RTH. Sounds scary I know, but it will.