No, that's not how the law is written.
If flying recreationally, you are limited to 400' AGL. That's four hundred feet above the ground that you're currently flying over. In other words, if you had an imaginary 400' long string hanging straight down, it's got to at least reach the ground at all times.
If you have a Part 107 license and are flying under Part 107 rules, you can fly up to 400' above the top point of a structure that is within 400 feet of your drone. So if you're inspecting a 900' tall tower, you can go up to 1300' above the ground.
The rules never measure your altitude from your takeoff point, even though the altitude above takeoff point is what DJI displays to you. It's always up to you to convert to the relevant altitude.