Hacking a Ghost XL

sugarlata — Mon, 24 Feb 2025 11:54:52 +0000

1. My Problem

I just got a Ghost XL, and while there are many great features, the app and downloading workflow for me was going to be a little too laborious. I want something that will automatically search out for my device, download any media to my NAS, delete it from the device, and then disconnect.

The included mobile app connects to the Ghost over WiFi, so there should be some way to perform this myself and bring in some automation. Let's see what I can do...

2. Research

The first step is figuring out how it works. If it's WiFi, then an HTTP API will likely be associated. I might have a look around and see what documentation there is...

Drift Link. An open-source app for connecting to the Ghost XL, excellent.

This blog looks pretty good, what else do we have here...

How to integrate a Drift camera into your own application. This is what I'm talking about. This is a great starting place to begin experimenting and see what's available.

Side note: I love it when companies leave some traces of documentation or hints for custom integration. Shout out to Drift Innovation on this one. It doesn't have to be perfect, but enough to get started is usually pretty good!

3. Lab Work

Next, I'll see what the calls are like. Looking through the second blog article it seems the listed calls are for live streaming - not what I'm looking for, I want a way to interact with existing media. I might have a look through the Android App Source.

I know that the IP address is 192.168.42.X and with a quick search I get...

public static String SERVER_IP = "http://192.168.42.1";

public final static String LOCAL_START_STREAM_WITH_URL = "http://%s/cgi-bin/foream_remote_control?start_rtmp_with_param=%s&stream_res=%s&stream_bitrate=%s";
public final static String LOCAL_START_STREAM = "http://%s/cgi-bin/foream_remote_control?start_rtmp";
public final static String LOCAL_STOP_STREAM = "http://%s/cgi-bin/foream_remote_control?stop_rtmp";

public final static String LOCAL_SETTING_ZOOM = "http://%s/cgi-bin/foream_remote_control?dzoom=%s";
public final static String LOCAL_SETTING_EXPOSURE = "http://%s/cgi-bin/foream_remote_control?exposure=%s";
public final static String LOCAL_SETTING_BITRATE = "http://%s/cgi-bin/foream_remote_control?stream_bitrate=%s";
public final static String LOCAL_SETTING_FILTERS = "http://%s/cgi-bin/foream_remote_control?filter=%s";
public final static String LOCAL_SETTING_RESOLUTION = "http://%s/cgi-bin/foream_remote_control?stream_res=%s";
public final static String LOCAL_SETTING_STREAM_FRAMERATE = "http://%s/cgi-bin/foream_remote_control?stream_framerate=%s";
public final static String LOCAL_SETTING_VIDEO_RESOLUTION = "http://%s/cgi-bin/foream_remote_control?video_resolution=%s";
public final static String LOCAL_SETTING_VIDEO_FRAMERATE = "http://%s/cgi-bin/foream_remote_control?video_framerate=%s";
public final static String LOCAL_SETTING_VIDEO_BITRATE = "http://%s/cgi-bin/foream_remote_control?video_bitrate=%s";
public final static String LOCAL_REBOOT = "http://%s/cgi-bin/foream_remote_control?reboot";
public final static String LOCAL_SHUTDOWN = "http://%s/cgi-bin/foream_remote_control?power_off";
public final static String LOCAL_SETTIME = "http://%s/cgi-bin/foream_remote_control?set_time=%s";
public final static String LOCAL_GETCAMSETTING = "http://%s/cgi-bin/foream_remote_control?get_camera_setting";
public final static String LOCAL_GETCAMSTATUS = "http://%s/cgi-bin/foream_remote_control?get_camera_status";
public final static String LOCAL_START_RECORD = "http://%s/cgi-bin/foream_remote_control?start_record";
public final static String LOCAL_STOP_RECORD = "http://%s/cgi-bin/foream_remote_control?stop_record";
public final static String LOCAL_SET_LED = "http://%s/cgi-bin/foream_remote_control?led=%s";
public final static String LOCAL_SETTING_MIC_SENSITIVITY = "http://%s/cgi-bin/foream_remote_control?mic_sensitivity=%s";
public final static String LOCAL_LIST_FOLDERS = "http://%s/cgi-bin/foream_remote_control?list_folders=/tmp/SD0/DCIM";
public final static String LOCAL_LIST_FILES = "http://%s/cgi-bin/foream_remote_control?list_files=/tmp/SD0/DCIM/%s";
public final static String LOCAL_DEL_FILES = "http://%s/cgi-bin/foream_remote_control?delete_media_file=%s";

(excerpt from BossDefine.java)

Ok, this is looking promising. Now for testing some of these endpoints.

First I'll try the list files. I'm not sure what the last string substitution is, so will leave it blank:

GET http://192.168.42.1/cgi-bin/foream_remote_control?list_files=/tmp/SD0/DCIM

I use Thunder Client in VS Code (for personal use), however, with only GET requests I can also do this in the browser if I need. The response is:

<?xml version="1.0" encoding="utf-8"?>
<Response>
    <Files>{"Path":"100MEDIA/VID00008.MP4","CreateTime":"Feb 24 17:49:54 2025","Size":6852780,"Thumb":0},</Files>
    <Amount>1</Amount>
</Response>

This is proving to be surprisingly easy. I only had a test video recorded, which was showing up. It seems like the path uses 100MEDIA, so I'll use that instead of blank. What about downloading files...

Looking back to BossDefine.java I can't see anything from what I've extracted above, but I do see this definition:

public final static String FILE_PATH = SERVER_IP + "/DCIM/";

Again, let's give it a go

GET http://192.168.42.1/DCIM/100MEDIA/VID00008.MP4

Download prompt, and it's the right video, so I think I've got enough to start.

4. The Solution

Using a clean architecture with some adjustments I'm going to segregate components out. I'll skip the domain -> repo definition as this is a small project. This is the structure I'm thinking of:

└───app
    │   constants.py
    │   main.py
    │   __init__.py
    │
    ├───data
    │   │   __init__.py
    │   │
    │   ├───data_sources
    │   │   │   __init__.py
    │   │   │
    │   │   ├───ap
    │   │   │       __init__.py
    │   │   │
    │   │   ├───drift
    │   │   │       __init__.py
    │   │   │
    │   │   └───file_io
    │   │           __init__.py
    │   │
    │   └───models
    │           __init__.py
    │
    └───domain
        │   __init__.py
        │
        └───use_cases
                connect_ghost.py
                connect_normal.py
                download_files.py
                get_file_list.py
                __init__.py

I've been using uv for a couple of projects and can see why it gets so much love. I'll use it here too.

Data Sources

There will be three data sources, one each for managing WiFi, API calls to the Ghost (or Drift), and File IO.

AP

I'm going to run this from my laptop to start, have never managed WiFi using Python and a quick Google search shows WinWiFi is a good solution, after some testing, this will do what I need.

I also need to both connect to the Ghost and then connect back to my normal network. Loguru I've found to be much better than the built-in Python logger, so I'll use that for feedback.

import winwifi

from loguru import logger

from app.constants import GHOST_WIFI_AP, NORMAL_WIFI_AP


class APManager:

    def _connect(ssid: str):

        connected_ssid = winwifi.WinWiFi.get_connected_interfaces()[0].ssid

        if ssid == connected_ssid:
            return

        logger.debug(f"Connecting to {ssid}")
        try:
            winwifi.WinWiFi.connect(ssid)
        except RuntimeError as e:
            logger.error(f"Could not connect to {ssid} ({e})")
            raise e

        logger.debug(f"Completed Connection to {ssid}")

    @classmethod
    def connect_ghost(cls):
        cls._connect(GHOST_WIFI_AP)

    @classmethod
    def connect_normal(cls):
        cls._connect(NORMAL_WIFI_AP)

Drift

I've tested the API calls, so I can create a class to call the endpoints. A problem I know I'm going to run into is the big videos on the Ghost are about 2Gb, which will need to be downloaded in chunks and written to file, instead of pulling into RAM, and then writing at the end. I'd usually want to provide a bridge between the File IO data source and the Drift data source, but for a quick and dirty, I'm just going to break this rule and allow the Drift data source to also perform File IO operations.

import json
import pytz
import datetime
import requests
import xmltodict

from loguru import logger

from app.data.models import GhostFile


class DriftConnection:

    BASE_URL = "http://192.168.42.1"
    GET_FILE_LIST = "/cgi-bin/foream_remote_control?list_files=/tmp/SD0/DCIM/100MEDIA"
    DOWNLOAD_FILE = "/DCIM/100MEDIA/"
    DELETE_FILE = (
        "/cgi-bin/foream_remote_control?delete_media_file=/tmp/SD0/DCIM/100MEDIA/"
    )
    MELBOURNE_TIMEZONE = pytz.timezone("Australia/Melbourne")
    CHUNK_SIZE = 8192 * 4

    @classmethod
    def get_file_list(cls) -> list[GhostFile]:

        logger.debug("Getting file list")
        resp_xml = requests.get(f"{cls.BASE_URL}{cls.GET_FILE_LIST}")

        resp_xml.raise_for_status()

        resp = xmltodict.parse(resp_xml.text)

        files_str = resp["Response"]["Files"]

        if not files_str:
            return []

        if files_str[-1] == ",":
            files_str = files_str[:-1]

        files = json.loads(f"[{files_str}]")

        return [
            GhostFile(
                created_time=cls.MELBOURNE_TIMEZONE.localize(
                    datetime.datetime.strptime(file["CreateTime"], "%b %d %H:%M:%S %Y")
                ),
                filename=file["Path"],
                size=file["Size"],
            )
            for file in files
        ]

    @classmethod
    def get_file(cls, file: GhostFile, local_path: str):

        download_url = f"{cls.BASE_URL}{cls.DOWNLOAD_FILE}{file.filename}"

        with requests.get(download_url, stream=True) as r:
            r.raise_for_status()

            completion = 0
            with open(local_path, "wb") as f:
                for chunk in r.iter_content(chunk_size=cls.CHUNK_SIZE):
                    f.write(chunk)
                    completion += cls.CHUNK_SIZE
                    logger.debug(
                        f"Getting {file.filename}: {round(100 * completion/float(file.size),3):.3f}%"
                    )

    @classmethod
    def delete_file(cls, file: GhostFile):

        delete_url = f"{cls.BASE_URL}{cls.DELETE_FILE}{file.filename}"
        requests.get(delete_url)

File IO

Part of the feature set is to delete the files from the Ghost once they're copied. When copying files and deleting them, I always want to make sure that they are copied. I'd usually perform some sort of hash comparison, but without access to the filesystem on the Ghost, I'll simply compare sizes, needing a File IO data source.

import os


class FileIO:

    def get_local_file_size(path: str) -> int:

        return os.path.getsize(path)

Use Cases

I'll segregate each action into use cases. This will be:

connect_ghost.py
connect_normal.py
download_files.py
- I'll include deleting files here, just for simplicity instead of creating another use case.
get_file_list.py

main.py

This will be the script that ties everything together. Need to change the WiFi network, get the files, download each one, and then set the WiFi back to normal.

I'll use env variables for determining the location to download to, and the Ghost WiFi, and Normal WiFi.

from app.domain.use_cases.download_files import download_files
from app.domain.use_cases.connect_normal import connect_normal
from app.domain.use_cases.connect_ghost import connect_ghost
from app.domain.use_cases.get_file_list import get_file_list


def main():

    try:
        connect_ghost()
    except RuntimeError:
        return

    file_list = get_file_list()
    download_files(file_list)

    try:
        connect_normal()
    except RuntimeError:
        return


if __name__ == "__main__":
    main()

5. Conclusion

There are a few more sections that are required, to bring everything together. The full working code can be found here

Here's the output:

2025-02-24 22:46:13.057 | DEBUG    | app.data.data_sources.ap:_connect:17 - Connecting to GHOST XL-C0627
2025-02-24 22:46:19.877 | DEBUG    | app.data.data_sources.ap:_connect:24 - Completed Connection to GHOST XL-C0627
2025-02-24 22:46:19.877 | DEBUG    | app.data.data_sources.drift:get_file_list:26 - Getting file list
2025-02-24 22:46:19.986 | DEBUG    | app.data.data_sources.drift:get_file:67 - Getting VID00008.MP4: 0.478%
2025-02-24 22:46:20.012 | DEBUG    | app.data.data_sources.drift:get_file:67 - Getting VID00008.MP4: 0.956%
2025-02-24 22:46:20.025 | DEBUG    | app.data.data_sources.drift:get_file:67 - Getting VID00008.MP4: 1.435%
2025-02-24 22:46:20.041 | DEBUG    | app.data.data_sources.drift:get_file:67 - Getting VID00008.MP4: 1.913%
2025-02-24 22:46:20.042 | DEBUG    | app.data.data_sources.drift:get_file:67 - Getting VID00008.MP4: 2.391%
...
2025-02-24 22:46:27.074 | DEBUG    | app.data.data_sources.drift:get_file:67 - Getting VID00008.MP4: 98.503%
2025-02-24 22:46:27.184 | DEBUG    | app.data.data_sources.drift:get_file:67 - Getting VID00008.MP4: 98.981%
2025-02-24 22:46:27.185 | DEBUG    | app.data.data_sources.drift:get_file:67 - Getting VID00008.MP4: 99.460%
2025-02-24 22:46:27.186 | DEBUG    | app.data.data_sources.drift:get_file:67 - Getting VID00008.MP4: 99.938%
2025-02-24 22:46:27.186 | DEBUG    | app.data.data_sources.drift:get_file:67 - Getting VID00008.MP4: 100.416%
2025-02-24 22:46:27.540 | DEBUG    | app.data.data_sources.ap:_connect:17 - Connecting to <NORMAL WIFI SSID>
2025-02-24 22:46:34.341 | DEBUG    | app.data.data_sources.ap:_connect:24 - Completed Connection to <NORMAL WIFI SSID>

Finally, I manually verified the video is now in my Downloads folder, and no longer on my Ghost XL.

Mission Accomplished.

6. Bonus: Futures Changes

Performing some experiments to identify the best chunk size would be ideal. I'd use a trial and error approach, and graph the results in excel to see whether there was any pattern to choose a local minimum.

Beyond this, dockerizing and setting up on a raspberry pi would be ideal, or potentially one of my NUCs at home which is hardwired. I wouldn't mind something that can access my NAS while still able to connect to the Ghost via WiFi.

DEV Community: sugarlata