I had the same problem yesterday. Bright sun, very little wind, the Mini 3 was drifting back and forth, then just drifting away. I was hovering over water, so I'll need to test over land to see if that has anything to do with it. Much worse drift than my older Mavic Air.