First you should check if there is an enviromental issue. If there is metal objects nearby, or beneath you. And does the GPS get satellites or not, is there anything tall between the drone and the sky. Then you could try to calibrate the compass and the IMU. If these doesn't resolve the issue, you need to get it fixed.
What you can do, is use flight records to check what's going on. Download DJI assistant 2 and then you can download a DAT file from the drone.With CsvView, you can open the DAT. You can see from the DAT, if it lose satellites, does it lose compass, or like in my case, it has multiple issues.
Here's a short video that explains how to get and view the DAT:
Turn the captions on