Like anything else there are shades of gray. One could make a case for VLOS meaning that it is within visual parameters that you *could* see it, and not that you have to be staring at it 100% of the time. So if it is tracking you and your easily turn around and spot it though turned away from it for a few seconds or longer, does that break the VLOS rule? I think one could make a case that it does not, while other may make a case that it does.
Frankly I think the rules are a big mess and often comes down to authorities standing behind the "the law says what I say it is" position. A case in point is the recent interview with a FAA representative that sort of held that if you are flying around, turn your camera on on purpose and post the footage on social media where only friends follow you, you are in violation of Part 107. Personally I think that is a bogus interpretation and could easily be argued against in court if the hearing was on an even playing field.
So as I said... Shades of gray.