They are a light modifier (reduction), nothing more. You reduce light by putting them on, so you need to choose the density of the filter based on the amount of light.
Sometimes there's not enough light to require a filter, so you use the original lens filter.
Then there's the fact that there is almost never a need for ND filters when doing still photography (with a drone--there are more cases with tripod mounted DSLRs, of course). They are primarily for video — leaving them on could lead to some soft / blurry photography.
So no, they should never be left on all the time unless a) you only do video and b) you always shoot in light bright enough that gives you too fast a shutter speed for what your desired frame rate.
Chris