I agree with your logic here and could easily support that viewpoint. However the only reference I have found in FAA documentation regarding airspace as it relates to recreational operators is that they are to stay out of Class B airspace. Other than that the only thing they are told to do is notify the airport and tower whenever they will be flying within 5 miles. I would be very interested to know what FAA documentation exists that says otherwise.
This is admittedly a gray area. The FAA clearly believes that recreational operators flying under the Part 101 model aircraft exemption ARE subject to airspace restrictions but the law passed in 2012 limits what they can regulate with regards to model aircraft. The best evidence I have found to date is the following from their document on Interpretation of the Model Aircraft Rule.
Congress directed that the FAA may not “promulgate any rule or regulation regarding a model aircraft, or an aircraft being developed as a model aircraft” if the aircraft is being operated, or being developed to be operated, pursuant to the five criteria enumerated in the statute as described above. P.L. 112-95, section 336(a). In other words, Congress has restricted the FAA from promulgating regulations, from the date when the statute was enacted, specifically regarding model aircraft that meet the terms of the statute.
However, the prohibition against future rulemaking is not a complete bar on rulemaking that may have an effect on model aircraft. As noted above, the rulemaking limitation applies only to rulemaking actions specifically “regarding a model aircraft or an aircraft being developed as a model aircraft.” P.L. 112-95, section 336(a). Thus, the rulemaking prohibition would not apply in the case of general rules that the FAA may issue or modify that apply to all aircraft, such as rules addressing the use of airspace (e.g., the 2008 rule governing VFR operations in the Washington, DC area) for safety or security reasons. See 73 FR 46803. The statute does not require FAA to exempt model aircraft from those rules because those rules are not specifically regarding model aircraft.
What FAA is saying here is that the already existing rules for aircraft such as airspace restrictions/classifications DO apply to model aircraft (including recreational drones) because those rules already existed before the law prevented the FAA from establishing and new laws on model aircraft.
As I said before, just applying logic here gives you the right answer. In my humble opinion. those who are trying to fly in controlled airspace under the model aircraft rule are just looking for a loophole to exploit.