I just shipped Pocket Core AI — a desktop app that runs four AI inference pipelines completely offline. This post is a technical walkthrough of how it's built, the problems I hit, and the solutions that actually worked.
This is the post I wish existed when I started building.
THE ARCHITECTURE OVERVIEW
┌─────────────────────────────────────┐
│ Electron Shell │
│ (Node.js + Chromium) │
│ │
│ ┌─────────────────────────────┐ │
│ │ UI Layer │ │
│ │ (HTML + CSS + JavaScript) │ │
│ └─────────────┬───────────────┘ │
│ │ HTTP localhost │
│ ┌─────────────▼───────────────┐ │
│ │ Python FastAPI Backend │ │
│ │ (PyInstaller executable) │ │
│ │ │ │
│ │ ┌──────────────────────┐ │ │
│ │ │ llama.cpp (chat) │ │ │
│ │ │ diffusers (images) │ │ │
│ │ │ ONNX (TTS) │ │ │
│ │ │ XTTS-v2 (cloning) │ │ │
│ │ └──────────────────────┘ │ │
│ └─────────────────────────────┘ │
└─────────────────────────────────────┘
The key insight: Electron handles the UI and OS integration. Python handles all AI inference. They communicate via HTTP to localhost. The Python backend is bundled as a platform-specific executable via PyInstaller so the user never needs Python installed.
THE PYTHON BACKEND
FastAPI serves all AI operations through REST endpoints:
python
# server.py — simplified structure
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
app = FastAPI()
# Only allow localhost — never exposed to network
app.add_middleware(
CORSMiddleware,
allow_origins=["http://localhost:*"],
allow_methods=["*"],
allow_headers=["*"],
)
@app.post("/api/chat")
async def chat(request: ChatRequest):
return StreamingResponse(
generate_response(request),
media_type="text/event-stream"
)
@app.post("/api/generate-image")
async def generate_image(request: ImageRequest):
result = image_generator.generate(
prompt=request.prompt,
negative_prompt=request.negative_prompt,
width=request.width,
height=request.height,
steps=request.steps,
seed=request.seed
)
return {"image_base64": result.base64,
"generation_time": result.time}
@app.post("/api/tts")
async def text_to_speech(request: TTSRequest):
audio = tts_engine.speak(
text=request.text,
voice=request.voice,
speed=request.speed
)
return {"audio_base64": audio.base64,
"duration": audio.duration}
@app.post("/api/clone-voice")
async def clone_voice(request: VoiceCloneRequest):
audio = voice_cloner.clone(
text=request.text,
reference_audio=request.reference_audio,
language=request.language
)
return {"audio_base64": audio.base64}
BUNDLING WITH PYINSTALLER
The tricky part is bundling four different AI frameworks together. The .spec file needs careful attention:
python
# pocketai-backend.spec
a = Analysis(
['main.py'],
pathex=['.'],
binaries=[],
datas=[
# Include Kokoro model files
('models/kokoro', 'models/kokoro'),
# Include any required config files
('config', 'config'),
],
hiddenimports=[
# llama-cpp-python
'llama_cpp',
'llama_cpp.llama',
# diffusers / FLUX
'diffusers',
'diffusers.pipelines',
'diffusers.pipelines.flux',
'transformers',
'accelerate',
# ONNX / Kokoro
'onnxruntime',
'kokoro_onnx',
# XTTS
'TTS',
'TTS.api',
'TTS.tts.configs',
# Web search
'newspaper',
'newspaper3k',
'bs4',
'requests_cache',
# FastAPI
'fastapi',
'uvicorn',
'uvicorn.logging',
'uvicorn.loops',
'uvicorn.loops.auto',
'uvicorn.protocols',
'uvicorn.protocols.http',
'uvicorn.protocols.http.auto',
# Other common hidden imports
'multipart',
'aiofiles',
],
hookspath=[],
runtime_hooks=[],
excludes=[
'tkinter', # Not needed, saves space
'matplotlib', # Only if not used
'PIL._tkinter_finder',
],
)
KEY LESSON: Missing hidden imports fail silently in development and loudly in production. Test your PyInstaller build on a completely clean VM before shipping.
THE CROSS-PLATFORM BUILD WITH GITHUB ACTIONS
yaml
# .github/workflows/build.yml
name: Build PocketCore AI
on:
workflow_dispatch:
push:
tags:
- 'v*'
jobs:
build:
strategy:
matrix:
include:
- os: windows-latest
name: windows-installer
- os: macos-latest
name: macos-dmg
- os: ubuntu-latest
name: linux-appimage
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: '3.11'
# Linux only — free disk space first
- name: Free disk space (Linux)
if: matrix.os == 'ubuntu-latest'
run: |
sudo rm -rf /usr/share/dotnet
sudo rm -rf /usr/local/lib/android
sudo rm -rf /opt/ghc
sudo apt-get clean
docker system prune -af 2>/dev/null || true
- name: Download Kokoro TTS model
run: |
pip install huggingface_hub
python -c "
from huggingface_hub import snapshot_download
snapshot_download(
repo_id='hexgrad/Kokoro-82M',
local_dir='./models/kokoro',
allow_patterns=['config.json',
'*.pth',
'voices/*.pt']
)"
- name: Build Python backend
run: |
pip install --no-cache-dir -r backend/requirements.txt
pip install pyinstaller
cd backend
pyinstaller pocketai-backend.spec --noconfirm
- name: Set up Node.js 18
uses: actions/setup-node@v4
with:
node-version: '18'
- name: Build Electron app
run: |
cd app
npm install
npm run dist
- name: Upload installer
uses: actions/upload-artifact@v4
with:
name: pocketai-${{ matrix.name }}
path: app/dist/*.{exe,dmg,AppImage}
if-no-files-found: error
THE DEPENDENCY CONFLICT THAT COST ME A DAY
ERROR: Cannot install misaki==0.7.4
and kokoro 0.7.16
kokoro 0.7.16 depends on misaki>=0.7.16
This fails on CI across all three platforms simultaneously. The fix:
# requirements.txt
# Before (broken):
misaki==0.7.4
kokoro==0.7.12
# After (working):
misaki>=0.7.16
kokoro>=0.7.16
Simple once you know it. Took too long to find because the error message was buried in pip's dependency resolution output.
THE USB GHOST MODE IMPLEMENTATION
When the app detects removable media, it switches all data paths to the USB drive:
python
# utils/path_manager.py
import platform
import os
import subprocess
from pathlib import Path
def detect_removable_media() -> bool:
"""Detect if app is running from removable media"""
current_path = Path(os.path.abspath(__file__))
if platform.system() == "Windows":
return _windows_detect(current_path)
elif platform.system() == "Darwin":
return _macos_detect(current_path)
else:
return _linux_detect(current_path)
def _windows_detect(path: Path) -> bool:
try:
import wmi
c = wmi.WMI()
drive_letter = str(path.drive)
for disk in c.Win32_LogicalDisk():
if disk.DeviceID == drive_letter:
return disk.DriveType == 2 # 2 = Removable
except Exception:
return False
return False
def _macos_detect(path: Path) -> bool:
try:
result = subprocess.run(
['diskutil', 'info', str(path.anchor)],
capture_output=True, text=True
)
return 'Removable Media: Yes' in result.stdout
except Exception:
return False
def _linux_detect(path: Path) -> bool:
try:
mount_point = str(path.anchor)
with open('/proc/mounts', 'r') as f:
for line in f:
parts = line.split()
if len(parts) >= 2 and parts[1] == mount_point:
device = parts[0].replace('/dev/', '')
removable_path = f'/sys/block/{device}/removable'
if os.path.exists(removable_path):
with open(removable_path) as rf:
return rf.read().strip() == '1'
except Exception:
return False
return False
def get_data_root() -> Path:
"""Return the appropriate data directory"""
if detect_removable_media():
# Run from USB — store everything on the drive
app_dir = Path(os.path.abspath(__file__)).parent
return app_dir / "PocketCoreData"
else:
# Run from install — use OS app data directory
if platform.system() == "Windows":
return Path(os.environ['APPDATA']) / "PocketCoreAI"
elif platform.system() == "Darwin":
return Path.home() / "Library" / "Application Support" / "PocketCoreAI"
else:
return Path.home() / ".config" / "PocketCoreAI"
XTTS-V2 MEMORY LEAK WORKAROUND
XTTS-v2 has a memory leak during long sessions. Our fix — reinitialise periodically:
python
# core/voice_cloner.py
class VoiceCloner:
def __init__(self):
self.model = None
self.generation_count = 0
self.MAX_GENERATIONS = 50
def _ensure_model_loaded(self):
if self.model is None or \
self.generation_count >= self.MAX_GENERATIONS:
# Reinitialise to clear leaked memory
if self.model is not None:
del self.model
import gc
gc.collect()
if torch.cuda.is_available():
torch.cuda.empty_cache()
self.model = TTS("tts_models/multilingual/multi-dataset/xtts_v2")
self.generation_count = 0
def clone(self, text: str, reference_audio: str,
language: str) -> bytes:
self._ensure_model_loaded()
# Generate audio
output = self.model.tts(
text=text,
speaker_wav=reference_audio,
language=language
)
self.generation_count += 1
return output
WHAT I'D DO DIFFERENTLY
Test on a clean VM before every release PyInstaller hidden imports that work in dev silently fail in bundled executables. I discovered this the hard way on the first user install reports.
Start cross-platform CI earlier I built on Windows and assumed it would work on macOS and Linux. It didn't — different hidden imports, different CUDA paths, different filesystem conventions.
The model bundling strategy from day one I changed my approach to model delivery three times. Final approach (bundle small models, download large models on first use) should have been the first approach.
WHAT I SHIPPED AND WHERE TO GET IT
Pocket Core AI is live at getpocketcore.com. $89 one-time for the Pro tier — all four inference pipelines, all platforms, USB Ghost Mode, lifetime updates.
The source isn't open (it's a commercial product) but I'm happy to discuss any implementation detail in the comments.
Questions, criticism, better approaches to any of this — I'm reading all the comments.
Top comments (0)