Based on the evidence available and the current understanding of the DJI implementation, I don't think that's true or is, at least, very unlikely. During calibration the FC is simply trying to distinguish (and characterize) the apparently unchanging magnetic field (due to the aircraft components) from the external moving field (the earth's, plus any other local field). Unless the external field is too strong for that to work (in which case the calibration will fail), it really doesn't matter what the external field comprises, since it is subtracted out in the measurement of the aircraft's field.
The only real concern that I can see is if the local field also changes during the calibration process, which could happen if the local field gradient is large relative to the size of the aircraft, for example. However, the chances of that still producing an accepted calibration seems very small since the fluctuations would still be dominated by rotation, and a robust calibration algorithm should detect that.
I cannot recall any recent Phantom or Mavic logs that have been analyzed that have indicated a bad calibration. It's always magnetic distortion at IMU initialization or later in the flight, or magnetic components added to the aircraft without recalibration, which supports
@BudWalker's assessment.
That was not true back in the days of the
P2, but the calibration process seems to have become much more robust. That said, I have not calibrated either my
P4 or Mavic since purchase, because they fly just fine and have not requested recalibration.