They generate heat while running as well. Last week I was in subfreezing temps (not extreme, but below 32F) and got the warning about battery temps when I fired it up. I found that if I launched the craft, then hovered (flew up and down the trail a bit) for 3-4 minutes, it warmed up to the point where the message went away.
You can monitor the temps on the battery screen in DJI Go 4 to see the temps rise as it sits there powering the props.
EDIT: in my "hover until warm enough" scenario, I did not go high or far and always kept the battery health screen up on DJI Go to monitor the cells. Definitely, do not FLY to warm the craft up.
That all uses juice, but so do dedicated electronic battery warmers. I have one for my old Phantom that you plug a battery into to warm up, but it used juice from the battery to do it.
Last year I was out in the winter desert and had that problem. I found that leaving the batteries on the dash of the car, because the sun was shining (even though it was freezing) did the trick.
Chris