On both sides of the border, taking off from someone else's property without permission is call Trespassing... And that is a whole different can of worms...
It depends a lot on where you are flying as far as I am concerned. My wife and I did a driving trip of the entire route from Calgary to Banff, and Jasper and the entire route was through national parks, which is, of course, forbidden. I did do...