I’m thinking that it’s your security settings in the ATV as when your out it won’t allow you to connect.
I bet it’s your security on the ATV
go into settings I think there’s a box to tick and allow anyone to connect.
If I’m right as well I think when I connected it sent a pass code to my phone.