it seems that it will be easier to learn to fly this way.
I disagree. Flying in FPV mode is very different, and does take practice to get good. It feels different and in some ways is more difficult. The biggest difference is you have a very limited field-of-view: you can't see what's above or below or all-around the drone when in FPV (except what's directly in front of it).
You will need to be very comfortable at just flying the drone before you get good flying in FPV. I don't recommend trying to learn the basics of flying with
goggles on. It would be like learning to drive a car at night while wearing headphones: NOT safe! - just be good at driving the car FIRST.
Also, you won't be able (or want to) use the
goggles all the time. I almost always take-off and land with the
goggles flipped UP, especially if I'm in a new or unfamiliar location.
The
DJI goggles are cool, and I love them, but I still probably only use them maybe 20% of the time.
You should get to the point where you
instinctively know and feel that Left is Right and Right is Left when flying towards you, and you would respond appropriately in a sudden/panic situation.
Just my 2 cents.