Thus "in theory". In fact, recreational pilots are limited to 400 AGL at all times. However, the reason I brought this up (in theory) is that is is ridiculously presumptive to think that if the cliff drops straight off, you have to drop your drone straight off within inches of the cliff face. The point was that you can descend off the cliff into the valley as long as you aren't being stupid about it.
A couple of points:
Firstly - note that even Part 107 doesn't allow 400 ft laterally from cliffs - that only applies to structures. 14 CFR §91.119 (b) and (c) give a clue why that would be:
§91.119 Minimum safe altitudes: General.
Except when necessary for takeoff or landing, no person may operate an aircraft below the following altitudes:
(a) Anywhere. An altitude allowing, if a power unit fails, an emergency landing without undue hazard to persons or property on the surface.
(b) Over congested areas. Over any congested area of a city, town, or settlement, or over any open air assembly of persons, an altitude of 1,000 feet above the highest obstacle within a horizontal radius of 2,000 feet of the aircraft.
(c) Over other than congested areas. An altitude of 500 feet above the surface, except over open water or sparsely populated areas. In those cases, the aircraft may not be operated closer than 500 feet to any person, vessel, vehicle, or structure.
Manned aircraft are not permitted within 500 ft of a structure, which is why it should be safe to fly a drone out to 400 ft, vertical or lateral. But that restriction does not apply to terrain - there is nothing preventing manned aircraft operating within 400 ft of a cliff in uncongested areas.
Secondly, if the cliff is vertical, then the question of how to treat that under Part 107 or the recreational exception is a good one. I've never seen it addressed by the FAA, but I've always worked on the assumption that as long as you are closer to the cliff than it would be reasonable to find a manned aircraft, then that should be fine.