When shooting video, the best shots are those where you fly quickly in a relatively straight line... left or right, forward or backwards.
Panning seems great when you're shooting it, but views poorly because it's not how our eyes work. When we "scan" a view, our eyes actually jump from one...