Add GPS Info from SRT File to individual Images extracted from Video File

[jonaskreiner]

Hi Guys,

So I want to do Photogrammetry with the Mavic 3. I have done so successfully using Images. For a new project I am looking to fly the drone quite slowely down a forestry road and do a 3D model of the road afterwards with dronedeploy. Instead of taking pictures, I am planning to film it (with multiple flights, angles) and then export individual frames with the VLC Scene Filter. While not having example footage on hand atm I assume the GPS data which is recorded in the SRT files will not be added to the images exported in the VLC player. I do however need each image to have this data which is recorded in the SRT file. Any quick way to do this automatically?

Other thought. Is there a Mode where I can capture regular Images at a constant rate of about 1 or 2 frames per second? Like some Timelapse feature or something?
Again I dont have the setup on hand right now. Will have a short time from getting the drone to showing results.

Any other thoughts on this?

Thanks
Jonas

 

[Not A Speck Of Cereal]

I don't have the M3 yet, but the manual states that it has TIMED shooting mode, with configurable intervals (see the specification section, which is the only details the M3 manual gives on this mode).

If it's the same as the previous models, then it probably works like the the following:

DJI Drones - How To Take Timed Shots - Drones Galaxy

DJI series drones like an Inspire, Phantom, or Mavic have the option to automatically take photographs at a timed interval. This option is called timed shots. Using timed shots can be used to take multiple photographs while in flight […]

dronesgalaxy.com

Chris

 

[mobilehomer]

Have you thought of using Hyperlapse?

 

[jonaskreiner]

I don't have the M3 yet, but the manual states that it has TIMED shooting mode, with configurable intervals (see the specification section, which is the only details the M3 manual gives on this mode).

If it's the same as the previous models, then it probably works like the the following:

DJI Drones - How To Take Timed Shots - Drones Galaxy

DJI series drones like an Inspire, Phantom, or Mavic have the option to automatically take photographs at a timed interval. This option is called timed shots. Using timed shots can be used to take multiple photographs while in flight […]

dronesgalaxy.com

Chris

Hi Chris,

This seems to only work with DJI Go 4 App, which to my understanding the Mavic 3 doesnt support. Only DJI Fly app

 

[jonaskreiner]

Have you thought of using Hyperlapse?

I am aware of the Hyperlapse existence. However I think it only goes down to 2s shutterintervals. Would much rather be closer to half a second. Also can I fly manually while using Hyperlapse?

 

[Not A Speck Of Cereal]

Hi Chris,

This seems to only work with DJI Go 4 App, which to my understanding the Mavic 3 doesnt support. Only DJI Fly app

That's weird, because it's stated in the M3 specifications in the M3 manual ('timed' is also mentioned one other place in the M3 manual). And of course, the M3 was not designed to use DJI Go 4.

Also, other models that use DJI Fly (not DJI Go 4) have timed shots. Here's a post I found on the DJI forums where an admin states it for the Mavic Air 2 (which uses Fly). Though they are only showing pages of the manual and not a DJI Fly screen.

You might ask on the DJI official forum "Why does the Mavic 3 manual mention Timed Shooting, but I can't find it in the DJI Fly app?"

However, the specs state that there timed shooting only goes down to 2 seconds, not 1 second or less. This is probably because the camera is not built for speed, at least not at full resolution (timed shooting is at 20MP).,

You can look at the LITCHI app to see if it offers timed shooting and if it shoots faster than 2 second intervals.

Chris

 

[jonaskreiner]

I ended up writing a Matlab file (using exiftool.exe) that allows me to extract video frames and attach the geotags in order to be used with drone deploy photogrammetry. See Code and attached Zip below. You do need Matlab >2019 in order to use this.


% Welcome to the Matlab based Frame Extractor developed by MCI Medtech Department
%
% This tool allows you to extract frames from a Video at a regular rate and geotag them with Location Data.
% Specifically for DJI Drone mp4 Videos with corresponding SRT files. You do need Matlab >2019 installed.
%
% The program structure is one main folder containing the matlab file, the exiftool.exe and an empty "Source" folder.
%
% 1. Copy your MP4 and SRT file into the Source folder. Ensure they have the identical name and make sure they are backed up!
% 2. Open the Matlab file and adjust the Framextractionrate and the Filenamebasis according to the source files.
% 3. Hit Run
% 4. Remove files from Source folder. Cut the folder "ProcessedFrames/xxx" to your desired location.
% 5. It is recommended to remove blurry pictures afterwards!
%
% Start all over

%% Cleanup
clc, clearvars, tic

%% Define Parameters
FrameExtractionRate = 10; %% Adjust
aFilenameBasis = 'DJI_0995'; %% Adjust

aFilenameSRT = append('Source\', aFilenameBasis, '.SRT');
aFilenameVideo = append('Source\', aFilenameBasis, '.MP4');
cPathExif = "exiftool.exe";
cMak = " -Make=Hasselblad";
cMod = " -Model=L2D-20c";
cFL = " -FocalLength=12";
cFL35 = " -FocalLengthIn35mmFormat=24";
cLonRef = " -GPSLongitudeRef=E";
cLatRef = " -GPSLatitudeRef=N";
cFilename = "Frame";

%% Import and Extract relevant Meta Data
RawDataArray = fileread(aFilenameSRT);
RawDataStrings = convertCharsToStrings(RawDataArray);

bLatitude = extractBetween(RawDataStrings, "latitude: ", "] [long");
bLongitude = extractBetween(RawDataStrings, "longitude: ", "] [rel");
bAltitude = extractBetween(RawDataStrings, "abs_alt: ", "] </font");

%% Calculate Times
frameNumber = length(bLatitude);
ExportedFrames = ceil(frameNumber/FrameExtractionRate);
timeTotal = num2str(ceil(ExportedFrames*5/60));

disp(['1/4 Starting the Magic! Total estimated Time: ', timeTotal ' minutes'])

%% Import Video
disp('2/4 Importing Video.')
VIDEO = VideoReader(aFilenameVideo);

%% Export Video Frames
mkdir(['ProcessedFrames',aFilenameBasis])
disp(['3/4 Writing Exif and exporting ', num2str(ExportedFrames), ' Frames. Highest max Frame Number ', num2str(frameNumber)]);
for i = 1:FrameExtractionRate:frameNumber
% Extract Frame
frames = read(VIDEO,i);
imwrite(frames,['ProcessedFrames' ,aFilenameBasis, '/', aFilenameBasis, '_Frame' int2str(i), '.jpg']); % write this frame to file

% Write Exif
cLon = append(" -GPSLongitude=-", bLongitude(i));
cLat = append(" -GPSLatitude=", bLatitude(i));
cAlt = append(" -GPSAltitude=", bAltitude(i));
cFile = append(" ProcessedFrames",aFilenameBasis, "/", aFilenameBasis, "_Frame" , string(i), ".jpg");
fCommand = append(cPathExif, cMak, cMod, cFL, cFL35, cLonRef, cLon, cLatRef, cLat, cAlt, cFile);
system(fCommand);
disp([num2str(ceil(i/FrameExtractionRate)),' out of ', num2str(ceil(frameNumber/FrameExtractionRate)), ' finished.']);
delete(append("ProcessedFrames" ,aFilenameBasis, "/" ,aFilenameBasis, "_Frame" , string(i), ".jpg_original"));
end

%% Finish Up

clear ans i frames VIDEO vid time RawDataArray RawDataStrings timeTotal timeExifwrite timeExport timeImport frameNumber
timeActual = num2str(toc/60);
disp(['4/4 Finished Exporting and Geotagging every ', num2str(FrameExtractionRate), 'th Frame to a total of ', num2str(ExportedFrames), ' Frames fo Folder ProcessedFrames from File ', aFilenameBasis])
disp(['Process took ', timeActual, ' minutes. Find Frames in Folder >>ProcessedFrames<<'])

 

[Jagerbomb52]

A fellow on the parrot forum was doing the same thing I think. Check out his post on this. Parrot Anafi and embedded metadata in video files

 

[Jagerbomb52]

There is also a program called 3DF Zephyr (not free) that will do all this for you. You will need a pretty fast computer but works very well. It will take a video and extract the photos needed to render your final project.

 

[Mickanside]

I ended up writing a Matlab file (using exiftool.exe) that allows me to extract video frames and attach the geotags in order to be used with drone deploy photogrammetry. See Code and attached Zip below. You do need Matlab >2019 in order to use this.


[...]

Hi mates,

I came to this forum through this thread. jonaskreiner, I tested your script, and it works wonderfully, so thank you!!
I found it annoying to have to run the script manually for each video, and to change its parameters, so I share with you a modified version that automatically detects and processes the entire Source folder.

Thank you and have a good afternoon!

% Welcome to the Matlab based Frame Extractor developed by MCI Medtech Department
%
% This tool allows you to extract frames from a Video at a regular rate and geotag them with Location Data.
% Specifically for DJI Drone mp4 Videos with corresponding SRT files. You do need Matlab installed.
%
% The program structure is one main folder containing the matlab file, the exiftool.exe and an empty "Source" folder.
%
% 1. Copy your MP4 and SRT file into the Source folder. Ensure they have the identical name and make sure they are backed up!
% 2. Open the Matlab file and adjust the Framextractionrate and the Filenamebasis according to the source files.
% 3. Hit Run
% 4. Remove files from Source folder. Cut the folder "ProcessedFrames/xxx" to your desired location.
% 5. It is recommended to remove blurry pictures afterwards!
%
% Start all over

%% Cleanup
clc, clearvars, tic

%% Define Parameters
FrameExtractionRate = 24;             %% Adjust

%% Processing every files of Source folder
filesList = strrep({dir('Source/*.MP4').name}, '.MP4', '')
for filesListIteration = 1:numel(filesList)
    aFilenameBasis = char(filesList(filesListIteration))
    aFilenameSRT = append('Source\', aFilenameBasis, '.SRT');
    aFilenameVideo = append('Source\', aFilenameBasis, '.MP4');
    cPathExif = "exiftool.exe";
    cMak = " -Make=Hasselblad";
    cMod = " -Model=L2D-20c";
    cFL = " -FocalLength=12";
    cFL35 = " -FocalLengthIn35mmFormat=24";
    cLonRef = " -GPSLongitudeRef=E";
    cLatRef = " -GPSLatitudeRef=N";
    cFilename = "Frame";

    %% Import and Extract relevant Meta Data
    RawDataArray = fileread(aFilenameSRT);
    RawDataStrings = convertCharsToStrings(RawDataArray);

    bLatitude = extractBetween(RawDataStrings, "latitude: ", "] [long");
    bLongitude = extractBetween(RawDataStrings, "longitude: ", "] [rel");
    bAltitude = extractBetween(RawDataStrings, "abs_alt: ", "] </font");

    %% Calculate Times
    frameNumber = length(bLatitude);
    ExportedFrames = ceil(frameNumber/FrameExtractionRate);
    timeTotal = num2str(ceil(ExportedFrames*5/60));

    disp(['1/4 Starting the Magic! :) Total estimated Time: ', timeTotal ' minutes'])

    %% Import Video
    disp('2/4 Importing Video.')
    VIDEO = VideoReader(aFilenameVideo);

    %% Export Video Frames
    mkdir(['ProcessedFrames',aFilenameBasis])
    disp(['3/4 Writing Exif and exporting ', num2str(ExportedFrames), ' Frames. Highest max Frame Number ', num2str(frameNumber)]);
    for i = 1:FrameExtractionRate:frameNumber
        % Extract Frame
        frames = read(VIDEO,i);
        imwrite(frames,['ProcessedFrames' ,aFilenameBasis, '/', aFilenameBasis, '_Frame' int2str(i), '.jpg']); % write this frame to file

        % Write Exif
        cLon = append(" -GPSLongitude=-", bLongitude(i));
        cLat = append(" -GPSLatitude=", bLatitude(i));
        cAlt = append(" -GPSAltitude=", bAltitude(i));
        cFile = append(" ProcessedFrames",aFilenameBasis, "/", aFilenameBasis, "_Frame" , string(i), ".jpg");
        fCommand = append(cPathExif, cMak, cMod, cFL, cFL35, cLonRef, cLon, cLatRef, cLat, cAlt, cFile);
        system(fCommand);
        disp([num2str(ceil(i/FrameExtractionRate)),' out of ', num2str(ceil(frameNumber/FrameExtractionRate)), ' finished.']);
        delete(append("ProcessedFrames" ,aFilenameBasis, "/" ,aFilenameBasis, "_Frame" , string(i), ".jpg_original"));
    end

    %% Finish Up

    clear ans i frames VIDEO vid time RawDataArray RawDataStrings timeTotal timeExifwrite timeExport timeImport frameNumber
    timeActual = num2str(toc/60);
    disp(['4/4 Finished Exporting and Geotagging every ', num2str(FrameExtractionRate), 'th Frame to a total of ', num2str(ExportedFrames), ' Frames fo Folder ProcessedFrames from File ', aFilenameBasis])
    disp(['Process took ', timeActual, ' minutes. Find Frames in Folder >>ProcessedFrames<<'])
end

 

[Miro_Rava]

Hi guys, I just encountered this problem and I couldn't find any ready to use tool for this, so I created my own with a gui in python, i also packaged it in an exe file, you can do this by yourself using auto-py-to-exe.
It basically takes as input the folder containing the videos and maybe also photos, the starting altitude (since as geotag dji uses relative altitude and not absolute), and the interval in seconds at which each frame has to be saved.
Videos without .SRT files will be discarded automatically.
Using this, I basically dump all my videos and photos of a site inside a folder, and then run the tool.
After I can use the folder with reality capture or other photogrammetry softwares.

I think this is a cool tool since it is self contained without the need for exif tools or others. Obviously to run it you need python and some of the libraries installed, but you can also use the .exe directly which is truly self contained without the need for python

 

[GadgetGuy]

Hi guys, I just encountered this problem and I couldn't find any ready to use tool for this, so I created my own with a gui in python, i also packaged it in an exe file, you can do this by yourself using auto-py-to-exe.
It basically takes as input the folder containing the videos and maybe also photos, the starting altitude (since as geotag dji uses relative altitude and not absolute), and the interval in seconds at which each frame has to be saved.
Videos without .SRT files will be discarded automatically.
Using this, I basically dump all my videos and photos of a site inside a folder, and then run the tool.
After I can use the folder with reality capture or other photogrammetry softwares.

I think this is a cool tool since it is self contained without the need for exif tools or others. Obviously to run it you need python and some of the libraries installed, but you can also use the .exe directly which is truly self contained without the need for python

Sounds great! Where can we find your tool?

 

[Miro_Rava]

Sounds great! Where can we find your tool?

I'm sorry, I forgot to add the link to my github... , I will add it in a few hours when I arrive home

 

[Miro_Rava]

Sounds great! Where can we find your tool?

for the ones that want a simple solution without installing python, i have the exe hosted on my website at this link:
https://www.miro-rava.com/documents/DJI_GPS_Video_Editor.exe
I didn't put it on github because it is not a finished project yet, this exe opens a gui that does what i explained above

----this is the gui with customtkinter: (save it in a file and name the extension as .pyw

import customtkinter
import subprocess
import sys
import os

customtkinter.set_appearance_mode("light")  # Modes: "System" (standard), "Dark", "Light"
customtkinter.set_default_color_theme("green")  # Themes: "blue" (standard), "green", "dark-blue"

app = customtkinter.CTk()
app.geometry("600x280")
app.title("Altitude Image Processor")


def resource_path(relative_path):
    """ Get absolute path to resource, works for dev and for PyInstaller """
    try:
        # PyInstaller creates a temp folder and stores path in _MEIPASS
        base_path = sys._MEIPASS
    except Exception:
        base_path = os.path.abspath(".")

    return os.path.join(base_path, relative_path)


def start_batch():

    subprocess.Popen(["python", resource_path("alt_changer.py"), folder_path.get(), entry_1.get(),entry_2.get()])

def browse_folder():
    # Allow user to select a directory and store it in global var
    # called folder_path
    global folder_path
    filename = customtkinter.filedialog.askdirectory()
    folder_path.set(filename)


frame_1 = customtkinter.CTkFrame(master=app)
frame_1.pack(pady=20, padx=20, fill="both", expand=True)

folder_path = customtkinter.StringVar()
folder_path.set("No Folder Selected")

button_1 = customtkinter.CTkButton(text="Chose Image Folder", master=frame_1, command=browse_folder)
button_1.pack(pady=10, padx=10)

label_2 = customtkinter.CTkLabel(master=frame_1, textvariable=folder_path, justify=customtkinter.LEFT)
label_2.pack(pady=10, padx=10)

entry_1 = customtkinter.CTkEntry(master=frame_1, placeholder_text="Enter Altitude in m")
entry_1.pack(pady=10, padx=10)

entry_2 = customtkinter.CTkEntry(master=frame_1, placeholder_text="Frames Interval in s")
entry_2.pack(pady=10, padx=10)

button_2 = customtkinter.CTkButton(text="Start Batch Processing", master=frame_1, command=start_batch)
button_2.pack(pady=10, padx=10)

if __name__ == '__main__':
    app.mainloop()

-----and this is the main file that processes the videos: (save it in the same folder as the gui in a .py file called "alt_changer.py"

import os
import sys
import cv2
import piexif
import re
import exif
from PIL import Image
from fractions import Fraction

files = os.listdir(sys.argv[1])
percentage = 0

def progress_bar(current, total, bar_length=20):
    fraction = current / total

    arrow = int(fraction * bar_length - 1) * '-' + '>'
    padding = int(bar_length - len(arrow)) * ' '

    ending = '\n' if current == total else '\r'

    print(f'Progress: [{arrow}{padding}] {int(fraction*100)}%', end=ending)

def parse_srt_file(srt_path):
    latitudes = []
    longitudes = []
    altitudes = []

    with open(srt_path, 'r') as file:
        srt_content = file.read()

    pattern = r'\[latitude: ([\d.-]+)\] \[longitude: ([\d.-]+)\] \[altitude: ([\d.-]+)\]'
    matches = re.findall(pattern, srt_content)
    for match in matches:
        latitude = float(match[0])
        longitude = float(match[1])
        altitude = float(match[2])

        altitudes.append(altitude)
        latitudes.append(latitude)
        longitudes.append(longitude)

    return latitudes, longitudes, altitudes

# Convert the latitude and longitude to the required format (Rational)
def degrees_to_rational(number):
    degrees = int(abs(number))
    minutes = int((abs(number) - degrees) * 60)
    seconds = int(((abs(number) - degrees - minutes / 60) * 3600) * 100)

    return [(degrees, 1), (minutes, 1), (seconds, 100)]

print("Elaborating Videos with Timestamps if present:")

for videoPath in files:
    if videoPath[-4:] in [".MOV", ".mov", ".MP4", ".mp4"]:
        video_path = os.path.join(sys.argv[1], videoPath)
        video_name = os.path.splitext(os.path.basename(video_path))[0]
        srt_path = os.path.join(sys.argv[1], video_name + ".srt")
        capture = cv2.VideoCapture(video_path)
        frame_count = int(capture.get(cv2.CAP_PROP_FRAME_COUNT))
        frames_to_extract = range(0, frame_count-1, int(float(sys.argv[3]) * capture.get(cv2.CAP_PROP_FPS)))
        try:
            latitudes, longitudes, altitudes = parse_srt_file(srt_path)
        except FileNotFoundError:
            print(f"WARNING ---> Skipping Video: {video_name} ---> NO .SRT File found")
            continue
        for frame_index in frames_to_extract:
            capture.set(cv2.CAP_PROP_POS_FRAMES, frame_index)
            success, frame = capture.read()

            if success:
                frame_path = os.path.normpath(os.path.join(sys.argv[1], f'{video_name}_frame_{frame_index}.jpg'))
                cv2.imwrite(frame_path, frame)
                try:
                    frame_latitude = latitudes[frame_index]
                    frame_longitude = longitudes[frame_index]
                    frame_altitude = altitudes[frame_index]+int(sys.argv[2])
                except Exception as e:
                    print(f"Image not processed because ---> {e}")
                    continue

                if frame_latitude is not None and frame_longitude is not None and frame_altitude is not None:
                    try:
                        # Open the image file
                        image = Image.open(frame_path)

                        # Get the existing EXIF data
                        exif_dict = piexif.load(frame_path)

                        # Preserve existing GPS data
                        gps_info = exif_dict.get("GPS", {})
                        altitude_ref = gps_info.get(piexif.GPSIFD.GPSAltitudeRef, 0)
                        gps_version = exif_dict.get(piexif.GPSIFD.GPSVersionID, (2, 3, 0, 0))

                        new_lat_rational = degrees_to_rational(frame_latitude)
                        new_lon_rational = degrees_to_rational(frame_longitude)

                        # Update the GPS data in the EXIF metadata
                        exif_dict["GPS"] = {
                            piexif.GPSIFD.GPSLatitudeRef: 'N' if frame_latitude >= 0 else 'S',
                            piexif.GPSIFD.GPSLatitude: new_lat_rational,
                            piexif.GPSIFD.GPSLongitudeRef: 'E' if frame_longitude >= 0 else 'W',
                            piexif.GPSIFD.GPSLongitude: new_lon_rational,
                            piexif.GPSIFD.GPSVersionID: (2, 3, 0, 0),
                            piexif.GPSIFD.GPSAltitude: Fraction.from_float(frame_altitude).limit_denominator().as_integer_ratio(),
                            piexif.GPSIFD.GPSAltitudeRef: 0,
                        }

                        # Encode the EXIF data and save it back to the image
                        exif_bytes = piexif.dump(exif_dict)
                        image.save(frame_path, exif=exif_bytes)

                        print(f'{video_name}_frame_{frame_index}.jpg')
                    except Exception as e:
                        print(f"Image not processed because ---> {e}")


                else:
                    print(f"No GPS data found for frame: {frame_path}")
            else:
                print(f"Error extracting frame at index {frame_index}")

        capture.release()

print("Elaborating Images:")

for imagePath in files:
    if imagePath[-4:] in [".JPG", ".PNG", ".jpg", ".png"]:
        percentage += 100/len(files)
        full_imagePath = os.path.join(sys.argv[1], imagePath)
        with open(full_imagePath, 'rb') as image_file:
            img = exif.Image(image_file)
            img.gps_altitude = img.gps_altitude + int(sys.argv[2])
        ext = full_imagePath[-4:]
        tempPath = full_imagePath[:-4]
        with open(f"{tempPath}_mod{ext}", 'wb') as test_image_file:
            test_image_file.write(img.get_file())
        os.remove(full_imagePath)
        print(f"{percentage:.2f}%  --> Changed GPS Altitude of: {full_imagePath[-12:-4]}")
print("100.00% --> Done!!")