Flying purely FPV, like a lot of unexperienced and unqualified pilots tend to do, actually asks for a lot of training. It's called IFR, Instrument Flying Rules training and certification, not available for UAV pilots yet, at least in the countries I know.
Sure, flying FPV is fun, like you're in the cockpit or a playing a videogame and so on. Many UAV 'pilots' wouldn't even know how to fly otherwise. Learning to control your bird while watching it VLOS is quite different. Like someone here said already, even while having it close in VLOS, a sudden wind can become problematic, but at least you actually see the bird fighting it and you can take control, or even crashland it nearby.