They will remain turned off. However, quite a few intelligent flight modes prompt you to enable it if you want to use them, so I rarely have to manually turn them on. If you don't fly in close quarters or in areas with a lot of bottom obstacles, then they aren't essential. I fly mostly in wide open fields so I prefer to have them switched off most of the time.
It's not a biggie though if you need to toggle them last minute. While in flight, just go to the menu>sensors>advanced, and there you'll find the switch. More often than not I find myself having to turn them off right before grabbing it.
Isn't the downwards sensors required for landing on ground so that it can slow down when it gets closer to the ground?