How I bundled llama.cpp, FLUX.1, XTTS-v2, and Kokoro into one Electron app that ships on Windows, macOS and Linux

PocketCoreAI — Mon, 17 Aug 2026 09:23:30 +0000

I just shipped Pocket Core AI — a desktop app that runs four AI inference pipelines completely offline. This post is a technical walkthrough of how it's built, the problems I hit, and the solutions that actually worked.

This is the post I wish existed when I started building.

THE ARCHITECTURE OVERVIEW

┌─────────────────────────────────────┐
│ Electron Shell │
│ (Node.js + Chromium) │
│ │
│ ┌─────────────────────────────┐ │
│ │ UI Layer │ │
│ │ (HTML + CSS + JavaScript) │ │
│ └─────────────┬───────────────┘ │
│ │ HTTP localhost │
│ ┌─────────────▼───────────────┐ │
│ │ Python FastAPI Backend │ │
│ │ (PyInstaller executable) │ │
│ │ │ │
│ │ ┌──────────────────────┐ │ │
│ │ │ llama.cpp (chat) │ │ │
│ │ │ diffusers (images) │ │ │
│ │ │ ONNX (TTS) │ │ │
│ │ │ XTTS-v2 (cloning) │ │ │
│ │ └──────────────────────┘ │ │
│ └─────────────────────────────┘ │
└─────────────────────────────────────┘

The key insight: Electron handles the UI and OS integration. Python handles all AI inference. They communicate via HTTP to localhost. The Python backend is bundled as a platform-specific executable via PyInstaller so the user never needs Python installed.

THE PYTHON BACKEND

FastAPI serves all AI operations through REST endpoints:

python
# server.py — simplified structure
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware

app = FastAPI()

# Only allow localhost — never exposed to network
app.add_middleware(
    CORSMiddleware,
    allow_origins=["http://localhost:*"],
    allow_methods=["*"],
    allow_headers=["*"],
)

@app.post("/api/chat")
async def chat(request: ChatRequest):
    return StreamingResponse(
        generate_response(request),
        media_type="text/event-stream"
    )

@app.post("/api/generate-image")
async def generate_image(request: ImageRequest):
    result = image_generator.generate(
        prompt=request.prompt,
        negative_prompt=request.negative_prompt,
        width=request.width,
        height=request.height,
        steps=request.steps,
        seed=request.seed
    )
    return {"image_base64": result.base64, 
            "generation_time": result.time}

@app.post("/api/tts")
async def text_to_speech(request: TTSRequest):
    audio = tts_engine.speak(
        text=request.text,
        voice=request.voice,
        speed=request.speed
    )
    return {"audio_base64": audio.base64,
            "duration": audio.duration}

@app.post("/api/clone-voice")
async def clone_voice(request: VoiceCloneRequest):
    audio = voice_cloner.clone(
        text=request.text,
        reference_audio=request.reference_audio,
        language=request.language
    )
    return {"audio_base64": audio.base64}

BUNDLING WITH PYINSTALLER

The tricky part is bundling four different AI frameworks together. The .spec file needs careful attention:

python
# pocketai-backend.spec
a = Analysis(
    ['main.py'],
    pathex=['.'],
    binaries=[],
    datas=[
        # Include Kokoro model files
        ('models/kokoro', 'models/kokoro'),
        # Include any required config files
        ('config', 'config'),
    ],
    hiddenimports=[
        # llama-cpp-python
        'llama_cpp',
        'llama_cpp.llama',
        # diffusers / FLUX
        'diffusers',
        'diffusers.pipelines',
        'diffusers.pipelines.flux',
        'transformers',
        'accelerate',
        # ONNX / Kokoro
        'onnxruntime',
        'kokoro_onnx',
        # XTTS
        'TTS',
        'TTS.api',
        'TTS.tts.configs',
        # Web search
        'newspaper',
        'newspaper3k',
        'bs4',
        'requests_cache',
        # FastAPI
        'fastapi',
        'uvicorn',
        'uvicorn.logging',
        'uvicorn.loops',
        'uvicorn.loops.auto',
        'uvicorn.protocols',
        'uvicorn.protocols.http',
        'uvicorn.protocols.http.auto',
        # Other common hidden imports
        'multipart',
        'aiofiles',
    ],
    hookspath=[],
    runtime_hooks=[],
    excludes=[
        'tkinter',  # Not needed, saves space
        'matplotlib',  # Only if not used
        'PIL._tkinter_finder',
    ],
)

KEY LESSON: Missing hidden imports fail silently in development and loudly in production. Test your PyInstaller build on a completely clean VM before shipping.

THE CROSS-PLATFORM BUILD WITH GITHUB ACTIONS

yaml
# .github/workflows/build.yml
name: Build PocketCore AI

on:
  workflow_dispatch:
  push:
    tags:
      - 'v*'

jobs:
  build:
    strategy:
      matrix:
        include:
          - os: windows-latest
            name: windows-installer
          - os: macos-latest  
            name: macos-dmg
          - os: ubuntu-latest
            name: linux-appimage

    runs-on: ${{ matrix.os }}

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python 3.11
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      # Linux only — free disk space first
      - name: Free disk space (Linux)
        if: matrix.os == 'ubuntu-latest'
        run: |
          sudo rm -rf /usr/share/dotnet
          sudo rm -rf /usr/local/lib/android
          sudo rm -rf /opt/ghc
          sudo apt-get clean
          docker system prune -af 2>/dev/null || true

      - name: Download Kokoro TTS model
        run: |
          pip install huggingface_hub
          python -c "
          from huggingface_hub import snapshot_download
          snapshot_download(
              repo_id='hexgrad/Kokoro-82M',
              local_dir='./models/kokoro',
              allow_patterns=['config.json', 
                             '*.pth', 
                             'voices/*.pt']
          )"

      - name: Build Python backend
        run: |
          pip install --no-cache-dir -r backend/requirements.txt
          pip install pyinstaller
          cd backend
          pyinstaller pocketai-backend.spec --noconfirm

      - name: Set up Node.js 18
        uses: actions/setup-node@v4
        with:
          node-version: '18'

      - name: Build Electron app
        run: |
          cd app
          npm install
          npm run dist

      - name: Upload installer
        uses: actions/upload-artifact@v4
        with:
          name: pocketai-${{ matrix.name }}
          path: app/dist/*.{exe,dmg,AppImage}
          if-no-files-found: error

THE DEPENDENCY CONFLICT THAT COST ME A DAY

ERROR: Cannot install misaki==0.7.4 
and kokoro 0.7.16

kokoro 0.7.16 depends on misaki>=0.7.16

This fails on CI across all three platforms simultaneously. The fix:

# requirements.txt
# Before (broken):
misaki==0.7.4
kokoro==0.7.12

# After (working):
misaki>=0.7.16
kokoro>=0.7.16

Simple once you know it. Took too long to find because the error message was buried in pip's dependency resolution output.

THE USB GHOST MODE IMPLEMENTATION

When the app detects removable media, it switches all data paths to the USB drive:

python
# utils/path_manager.py

import platform
import os
import subprocess
from pathlib import Path

def detect_removable_media() -> bool:
    """Detect if app is running from removable media"""
    current_path = Path(os.path.abspath(__file__))

    if platform.system() == "Windows":
        return _windows_detect(current_path)
    elif platform.system() == "Darwin":
        return _macos_detect(current_path)
    else:
        return _linux_detect(current_path)

def _windows_detect(path: Path) -> bool:
    try:
        import wmi
        c = wmi.WMI()
        drive_letter = str(path.drive)
        for disk in c.Win32_LogicalDisk():
            if disk.DeviceID == drive_letter:
                return disk.DriveType == 2  # 2 = Removable
    except Exception:
        return False
    return False

def _macos_detect(path: Path) -> bool:
    try:
        result = subprocess.run(
            ['diskutil', 'info', str(path.anchor)],
            capture_output=True, text=True
        )
        return 'Removable Media: Yes' in result.stdout
    except Exception:
        return False

def _linux_detect(path: Path) -> bool:
    try:
        mount_point = str(path.anchor)
        with open('/proc/mounts', 'r') as f:
            for line in f:
                parts = line.split()
                if len(parts) >= 2 and parts[1] == mount_point:
                    device = parts[0].replace('/dev/', '')
                    removable_path = f'/sys/block/{device}/removable'
                    if os.path.exists(removable_path):
                        with open(removable_path) as rf:
                            return rf.read().strip() == '1'
    except Exception:
        return False
    return False

def get_data_root() -> Path:
    """Return the appropriate data directory"""
    if detect_removable_media():
        # Run from USB — store everything on the drive
        app_dir = Path(os.path.abspath(__file__)).parent
        return app_dir / "PocketCoreData"
    else:
        # Run from install — use OS app data directory
        if platform.system() == "Windows":
            return Path(os.environ['APPDATA']) / "PocketCoreAI"
        elif platform.system() == "Darwin":
            return Path.home() / "Library" / "Application Support" / "PocketCoreAI"
        else:
            return Path.home() / ".config" / "PocketCoreAI"

XTTS-V2 MEMORY LEAK WORKAROUND

XTTS-v2 has a memory leak during long sessions. Our fix — reinitialise periodically:

python
# core/voice_cloner.py

class VoiceCloner:
    def __init__(self):
        self.model = None
        self.generation_count = 0
        self.MAX_GENERATIONS = 50

    def _ensure_model_loaded(self):
        if self.model is None or \
           self.generation_count >= self.MAX_GENERATIONS:
            # Reinitialise to clear leaked memory
            if self.model is not None:
                del self.model
                import gc
                gc.collect()
                if torch.cuda.is_available():
                    torch.cuda.empty_cache()

            self.model = TTS("tts_models/multilingual/multi-dataset/xtts_v2")
            self.generation_count = 0

    def clone(self, text: str, reference_audio: str, 
              language: str) -> bytes:
        self._ensure_model_loaded()

        # Generate audio
        output = self.model.tts(
            text=text,
            speaker_wav=reference_audio,
            language=language
        )

        self.generation_count += 1
        return output

WHAT I'D DO DIFFERENTLY

Test on a clean VM before every release PyInstaller hidden imports that work in dev silently fail in bundled executables. I discovered this the hard way on the first user install reports.
Start cross-platform CI earlier I built on Windows and assumed it would work on macOS and Linux. It didn't — different hidden imports, different CUDA paths, different filesystem conventions.
The model bundling strategy from day one I changed my approach to model delivery three times. Final approach (bundle small models, download large models on first use) should have been the first approach.

WHAT I SHIPPED AND WHERE TO GET IT

Pocket Core AI is live at getpocketcore.com. $89 one-time for the Pro tier — all four inference pipelines, all platforms, USB Ghost Mode, lifetime updates.

The source isn't open (it's a commercial product) but I'm happy to discuss any implementation detail in the comments.

Questions, criticism, better approaches to any of this — I'm reading all the comments.

DEV Community: PocketCoreAI