Next time, make sure you have the right props on the correct arms, securely installed.
What you've described here is exactly what will happen if you swap two props from their correct positions, putting upward thrust on one pair and downward thrust on the other pair.
The props are marked for...