Hi, there’s two possibilities that I can think of.
1. You were flying in ATTI mode
2. There were too many interference in your area, and it screwed up your compass and IMU.
I had a similar issue when I tried to take off besides a metal bridge, it just wonder around right after take off.
If...