You can alternate between two controllers, A and B, but you will have to link each 'new' controller each time you use it. So if you had been using A then went to use B you would have to link B to the drone and the next time you went to use A you would have to link A to the drone etc. etc..
Besides, I believe, most of the settings are stored on the drone. I am not what settings, if any, are stored elsewhere but those that are, if any, are, I suspect, stored in the App.
I swap phones quite frequently and I have even swapped a phone mid flight recently when the phone in use threw a tantrum and I needed/wanted a screen quickly.
When considering your reasons for it, see no real advantage in what you are contemplating. But there may be an advantage in the number of drone batteries worth of flights you could make using one charge of both controllers but, if starting with a fully charged battery in the controller I think each controller may last long enough to fly three batteries worth of drone flights, especially if you happen to be using Apple screens.
You may need multiple controller-to-screen-device cables, to accommodate the necessary cable lengths if nothing else.
If you are still thinking of two controllers it might be worth asking if a Mavic Mini controller will work with the Mini SE, a firmware update might be required but from memory the last time I looked they were the same model of controller. Obviously you would need to stick to the SE's FCC vs CE type version
