The difference here is when a car goes 1mph over the speed limit, the risk of a catastrophic crash is not substantially increased. Cars don't run the risk of losing control just because they go over the PSL which is often posted unusually low and far below the car's capabilities under the conditions. When a car goes awry, there are all kinds of systems and safeguards and barriers in place to mitigate the possible damage and while they are not foolproof, the risks have been dramatically lowered lately.
With a drone, if you lift off the ground and something goes wrong, the drone could lose control and find it's way to do much harm to or interrupt aircraft traffic or at a minimum, increase the risk to manned aircraft. Unfortunately drones are not yet at the point where they don't seem to easily get away from their pilots and unfortunately we are not yet at the point where we can trust them fully once they get off the ground. The aircraft industry has done next to nothing (that I am aware of) to protect their aircraft from drones. Until the drone gets to the point where it is no longer on the short end of the stick and taken seriously, we should expect the lion's share of the safety obligation to rest with the drone and the drone operator to prevent catastrophe, to operate under strict rules, and bear the burden of almost any responsibility in the event of crash.