Voice Control your drone with this app

[Yondu]

A few friends and I are developing a free app that allows you to control a DJI drone with your voice. This app will support all of the Mavic series drones. We are doing it by creating an AI agent using vision language models that understands voice commands and the environment. We hope that having a voice controlled drone will make it easy as pie to pilot a drone. We also hope it’ll enable multi-tasking, hands-free operation, and a whole world of applications we haven’t even thought about yet. As we finish up the app, we’d love to hear what features and functionalities you’d like us to include and what types of applications you’d use this app for. Please let us know what your thoughts are!

 

[lensimages]

A few friends and I are developing a free app that allows you to control a DJI drone with your voice. This app will support all of the Mavic series drones. We are doing it by creating an AI agent using vision language models that understands voice commands and the environment. We hope that having a voice controlled drone will make it easy as pie to pilot a drone. We also hope it’ll enable multi-tasking, hands-free operation, and a whole world of applications we haven’t even thought about yet. As we finish up the app, we’d love to hear what features and functionalities you’d like us to include and what types of applications you’d use this app for. Please let us know what your thoughts are!

Sounds very interesting, as I am 82 a bit shakey on my feet this could be very helpful as long as it is legally ok to use this sort of control. To be able to sit down in a field and control my drones with my voice seems ideal, please keep us info of progress, cheers Len

 

[Droning on and on...]

I applaud your creativity and effort

I would also caution you that this method of control may turn out to be impractical because necessary control inputs with sub-second reaction times, something easy with sticks and fingers, will be difficult to execute verbally.

 

[mavic3usa]

Smart drone:
"Hey drone, take off."
"Hey drone, return to home."
"Hey drone, increase speed to 25 mph."
"Hey drone, fly the waypoints mission called "backyard garden."
"Hey drone, hover at 150 feet for 15 minutes."

AI drone:
"Hey drone, fly out to the beach as quickly as possible and take 10 photos which includes at least 2 sailboats and 4 surfers."
"Hey drone, only when it is raining or just because it starts raining, launch the waterproof drone to the university parking lot (which is in a 50 ft LAANC grid) and slowly scan for unoccupied convertibles with the top down being careful not to fly over moving cars. Send a text message, not to the registered owner, but to the person who drove the car to the lot or currently has the keys when you detect."
"Hey drone, stage near the baseball field and then capture the half-time events at the high school football game (since drones are only allowed during non-play) including the players when the return to the field. Bypass the low-battery RTH and land on alternate landing pad #2 if necessary."
"Hey drone, during lunch time, fly to the city market and sample/count the foot traffic and include pets. Also break out the number of children and strollers. Do not include employees who work in a 2-mile radius."
"Hey drone, twice a day, check the vehicle license plates in the walmart parking lot next door for warrants only. If the vehicle is moving or the driver is checking out at the register and the warrant is high-level and in state, livestream and activate spotlight mode and trigger a "drone e-notify" alert."
"Hey drone, whenever I step out for lunch or log onto my all-hands 2 hour monthly zoom meeting, launch the drone to watch the back gate and send me a text message and live-stream to my iPhone or computer whenever an unauthorized vehicle approaches the gate. Follow any vehicle which appears suspicious."
"Hey drone, capture the early sunset from the east river bridge (when the moon is also visible, the skies have some fog, and remove the car traffic on the bridge but leave the foot traffic).
"Hey drone, watch the ski lift for traffic going up the mountain (for volume and equipment) and determine when it's best to capture the coolest run from snowboard-only customers on mogul slopes 2-4 only when the sun comes out. Focus on anyone who crashes for 1-2 minutes."

We already have smart things and have had them for a couple of decades; nothing special. Hopefully we get real AI in our drones soon, one day.

 

[Droning on and on...]

We already have smart things and have had them for a couple of decades; nothing special. Hopefully we get real AI in our drones soon, one day.

While your examples above are really cool, sincerely, I think the chances of the FAA allowing that degree of pilot-not-in-command are just about 0%.

Ever.

Especially after the idiocy of Google's Gemini AI that's been big news the last few days. Artificial Intelligence? More like Artificial Goebbels.

 

[mavic3usa]

While your examples above are really cool, sincerely, I think the chances of the FAA allowing that degree of pilot-not-in-command are just about 0%.

Ever.

FAA won't have a choice; the tide will be great. I believe we are soon to have a lower area of airspace that will permit these sort of activities otherwise, as you mentioned, we'll never have anything good. AI is supposed to make these drones safer and essentially crash free. Every inch of power line is in the AI database and it's real-time, growth rates of trees are predicable for limbs and branches, previous drone flights as well as photos and car videos and new reports of obstacles will be combined to paint the landscape, and finally a disabled drone has backup systems that allow recover and safe landing without ancient ideas like parachutes. A true AI powered drone just knows where it can fly and where it cannot, better than any pilot. Honestly I think this is 10 maybe 20 years off but if the Mavic 12 is just a Mavic 3 with better cameras then we all suck. The rate of evolution with technology is supposed to accelerate rapidly, it shouldn't take 50 to 100 years just to take 2 steps forward. But I think you nailed it, if the government continues to stand in the way of progress, we will have nothing.

 

[GFields]

Greetings from Birmingham Alabama USA, welcome to the forum! We look forward to hearing from you!

 

[Cafguy]

A.I. Drone:.." Buy me more Accessories or I'll cancel your Visa! Clean my sensors or NO Seinfeld FOR YOU! CMon Smart CAR- Lets go to the Charging station till morning- on the Chimps dime"
-No thanks I'll stick with my SpeedyBee

 

[RadioFlyerMan]

A few friends and I are developing a free app that allows you to control a DJI drone with your voice. This app will support all of the Mavic series drones. We are doing it by creating an AI agent using vision language models that understands voice commands and the environment. We hope that having a voice controlled drone will make it easy as pie to pilot a drone. We also hope it’ll enable multi-tasking, hands-free operation, and a whole world of applications we haven’t even thought about yet. As we finish up the app, we’d love to hear what features and functionalities you’d like us to include and what types of applications you’d use this app for. Please let us know what your thoughts are!

As an old programmer from way back, I can understand your motivation to develop such a thing. Asking if we can, rather than if we should.

But as an old radio control flyer from the 60’s turned drone pilot, I find the satisfaction of this hobby in finding the skills of flying with my hands. Where is the pride in a smoothly captured video clip obtained via a computer, if not from my hands?

I do l hope you’re successful. The young people can take the propellers off of airplanes and the steering wheels from cars to live fully autonomous lives. And if you can hit a home run with a computer, why bother going to the ball park. Not for me, thank you.

 

[Player-Pe]

As an old programmer from way back, I can understand your motivation to develop such a thing. Asking if we can, rather than if we should.

But as an old radio control flyer from the 60’s turned drone pilot, I find the satisfaction of this hobby in finding the skills of flying with my hands. Where is the pride in a smoothly captured video clip obtained via a computer, if not from my hands?

I do l hope you’re successful. The young people can take the propellers off of airplanes and the steering wheels from cars to live fully autonomous lives. And if you can hit a home run with a computer, why bother going to the ball park. Not for me, thank you.

As a person of the younger generation I welcome the era of autonomous channels that can do tasks faster and more efficiently (sometimes). However, I also have an "old soul" to know that TRUE gratification stems from knowing you've put sweat & tears into a project and it was a success instead of having someone/something do it for you. Big difference. Overall, I would welcome AI into the drone world, but personally I feel that at this point, for the average drone user, AI use would be the very last thing FAA would want to implement. Just my opinion.

 

[Torque]

When I was about 6 years old I had a vision, it is as clear today as it was that summer day 60 years ago. My Dad retired from the federal government when I was 5, so I grew up with an ever present father - a very good thing. Anyway, I was 6 and had just pushed my Dad's last button by asking him too many questions, and he had sent me off to play. I clearly remember opening the back gate and walking towards our corrals and chicken coop, wishing that I had someone I could ask my incessant questions to and always get and answer. Suddenly I could see "facts" and symbols floating before my eyes with this comforting knowledge that someday I would experience the ability to simply ask a question aloud and recieve and audio or visual answer about ANYTHING! Computers came along, then voice prompts using AI devices like Alexa and Google AI, and my premonition was coming to fruition! When Google glasses apeared, there was confirmation of the answers simply "floating before my eyes". Call me crazy, but since this is an anonymous forum, so I feel safe sharing this.
Anyway, I've been a fan and supporter of advancments in technology and AI and find them fascinating. I use Alexa to check traffic, start my car, start the vaccum, set the oven temp, set the thermostat, turn lights on and off, etc., etc.
I use Comma AI to drive my car - it does 75% of my driving for me. And I am certainly not a member of the younger generaton! Just an old guy who was fortunate enough to have a glimpse of the future when I was a child.

I welcome the voice commands and don't see much difference between limited voice commands and the hand gestures some drones have/had?
I'd love to focus just on piloting while giving voice commands to the camera! That would be fantastic.

 

[KaiCor]

A few friends and I are developing a free app that allows you to control a DJI drone with your voice. This app will support all of the Mavic series drones. We are doing it by creating an AI agent using vision language models that understands voice commands and the environment. We hope that having a voice controlled drone will make it easy as pie to pilot a drone. We also hope it’ll enable multi-tasking, hands-free operation, and a whole world of applications we haven’t even thought about yet. As we finish up the app, we’d love to hear what features and functionalities you’d like us to include and what types of applications you’d use this apps for. Please let us know what your thoughts are!

It's a very interesting idea. I think other people have thought about it too. Do you think it is possible to make voice control as fast as hand control? I think that for making a normal flight, it's a great idea. But if you need to do fast maneuvers, then hand control is better.

 

[Cafguy]

Well Before you actually speak your ideas are being bounced around your brain.- So not to much of a stretch to develop a controller that would read these signals in your Brian and interpret them into an action. Mavic 9 ish?

 

[Droning on and on...]

Well Before you actually speak your ideas are being bounced around your brain.- So not to much of a stretch to develop a controller that would read these signals in your Brian and interpret them into an action. Mavic 9 ish?

That will be the Mavic, Elon Edition.

 

[Jamesz]

It sounds like another step closer to the awaiting hell where we have relinquished the ability to control things directly, but instead rely on algorithms to translate (approximate?) our wishes into actions. In so many fields of life, part of the pleasure comes from combining knowledge with physical skills and judgement to achieve an end; even if its not always as predicted, but still very satisfying as a human . AI will make amazing differences to the world, but I think we will lose more than we gain overall.

 

[Droning on and on...]

It sounds like another step closer to the awaiting hell where we have relinquished the ability to control things directly, but instead rely on algorithms to translate (approximate?) our wishes into actions.

That's pretty much how it goes when I see my mechanic or doctor.

Thing is, I don't have any problem with relying on them "to translate (approximate?) [my] wishes into actions". I don't think twice about it, and they make mistakes.

 

[ScottTX]

As a hardware engineer with 40 years of experience, that started his career working with programmers that used machine language, AI is a tremendous paradigm shift. I am a named inventor on 37 patents, so "new stuff" is in my blood. Having said that AI still gives me pause.

 

[ScottTX]

Also, while on the subject of patents, have you disclosed this idea to a patent attorney?

 

[Yondu]

Guys I should probably provide some more context. I'm working on an early startup trying to make robots easier to use and control. We imagine a world where people can interact with their robots as easily as they would a pet. It’s a hard task but we’re starting off small by mastering drone navigation and command following. We’ve built an app that’ll let you voice control certain DJI drones. Here’s a demo of the initial functionality.
It’s limited to basic movement instructions, as well object tracking and following for now, but soon we hope to get it to a point where you can have access to expert level aerial maneuvers and command following. We're still awaiting Play Store approval and still developing the App Store version so fill the google form to get the download link emailed to you:
Voice Controlled DJI Drone
We hope that smart drones will enable new autonomous capabilities for aerial imagery in recreational and industrial use cases. We’re hoping for some feedback, whether it be specific functionality or use cases that you’re thinking of or anything else you’d want to comment on.

 

[Yondu]

Also, while on the subject of patents, have you disclosed this idea to a patent attorney?

i haven't haha