You are mistaken. 107 fliers can go 400ft above a structure as long as they are within 400ft of the structure. This was put in the rules so drones could be used for inspections of towers and buildings that are close to or taller than 400ft.
Recreational fliers don't get that leeway.
This doesn't apply to terrain changes. You can't exceed 400ft AGL simply because you're within 400ft of higher terrain, like a steep slope or cliff.
You said: "Just be sure, in the U.S., to stay within 400’ laterally to the mountain and below 400’ above the summit/high point."
400ft lateral has nothing to do with allowed altitude for terrain. You can be 500ft lateral, just be sure you're below 400ft from ground directly beneath.
That statement sounded like the rule with allowing above 400ft AGL if within 400ft of structure, hence my comment.