We have certain restrictions not shown on our 'ok to fly' app.
National Parks in certain states have no drone policies (while a few are totally ok), generally known / easy to find with a little research.
Places like local councils ('City of *'), parks and reserves, critical infrastructure, these are less known, and sometimes very hard to find out !!
Councils bury their policies in very hard to find by-laws, other places like our big water catchment dams can be no fly zones, but neither are on our app.
I personally think fly, if (after a little pre flight research into the place, and nothing is found re drone flight) all other air authority rules are able to be followed.
If someone stops and advises drones are not allowed / banned, whatever, then you should be given a warning and you do the right thing, advise you checked about drone policy and without (FAA, CAA, CASA etc) restrictions . . . you thought ok to fly, apologise, and move on.
If you get fined, then you take it to court, fight it, shouldn't be difficult to get off, especially if you can show research, emails sent (with no reply), etc.
If some land authority pressed prosecution when you could show the due diligence, then they are probably going to lose.
Common sense should be applied though, you wouldn't be flying over busy places, serious places like nuclear power plants, Government buildings like parliament, white house, etc.