If the area is dark then the downward looking sensor system (VPS) can not contribute to position holding and you are relying on GPS postion holding. GPS position holding it not as accurate as the VPS. see page 62 of the manual.
Could that be an answer to your circumstances.
Also, "between buildings" may reduce the number of satellites that are visible to the drone, if enough are masked, coupled with poor lighting, the drone could end up in ATTI mode with no automated position holding, so, be careful.