DEV Community: yamayu-dev

Building local hybrid search with Qdrant and BGE-M3

yamayu-dev — Mon, 06 Jul 2026 13:38:47 +0000

I wanted a local semantic search setup for internal documents: no hosted vector database, no embedding API, and enough flexibility to compare dense search, sparse search, hybrid search, and reranking.

The interesting part was not "how to start Qdrant." It was how to store both dense and sparse vectors for the same document chunk, then switch the retrieval strategy at query time.

This is the shape I ended up testing:

Qdrant running locally in Docker
BGE-M3 as a local embedding model
Qdrant named vectors: dense and sparse on the same point
dense search, sparse search, hybrid search with RRF
optional cross-encoder reranking with BAAI/bge-reranker-v2-m3

This is a small PoC, not a benchmark. The goal is to make the moving parts explicit.

Why BGE-M3 with Qdrant named vectors?

BGE-M3 can produce dense embeddings and sparse lexical weights from the same model family. Qdrant can store multiple vectors on a single point using named vectors.

That combination is convenient for document search because each chunk can be stored once:

PointStruct(
    id=chunk_id,
    vector={
        "dense": dense_vector,
        "sparse": sparse_vector,
    },
    payload={
        "text": chunk_text,
        "path": path,
    },
)

At query time, I can choose:

using="dense" for semantic search
using="sparse" for lexical-style search
both, then fuse rankings with Reciprocal Rank Fusion
a second reranking stage for the top candidates

The practical benefit is that I do not need to rebuild the collection just to compare retrieval modes.

Local setup

The environment I used:

macOS on Apple Silicon
Python 3.13
Docker 28.x
Qdrant in a local Docker container

Install the Python packages:

python -m pip install --upgrade pip setuptools wheel
python -m pip install -U \
  "qdrant-client>=1.15.0" "pydantic>=2.7,<2.12" \
  sentence-transformers FlagEmbedding scipy \
  torch torchvision torchaudio \
  pandas pymupdf

Download BGE-M3 locally:

python - <<'PY'
from huggingface_hub import snapshot_download

snapshot_download(
    "BAAI/bge-m3",
    local_dir="./models/bge-m3",
    local_dir_use_symlinks=False,
)
print("Downloaded to ./models/bge-m3")
PY

Start Qdrant with persistent local storage:

docker run --name qdrant \
  -p 6333:6333 \
  -v $(pwd)/qdrant_storage:/qdrant/storage \
  qdrant/qdrant

After startup, the dashboard is available at:

http://localhost:6333/dashboard

Test corpus

For the first pass, I used a tiny mixed-language corpus. It is too small to prove quality, but good enough to check whether each retrieval path behaves as expected.

I kept the sample texts in Japanese because the real target is Japanese internal documentation. For readers who do not read Japanese, this is the meaning of each row:

id	Text	Meaning
1	今日は天気がいい	The weather is nice today
2	年末年始の休暇規定を確認してください	Check the year-end and New Year vacation policy
3	申請の締切日は12月15日です	The application deadline is December 15
4	API 標準設計ガイドの参照はこちら	See here for the standard API design guide
5	会議は月曜日に延期になりました	The meeting was postponed to Monday
6	正しいPythonの環境構築方法	How to set up a proper Python environment
7	Please check the holiday policy for New Year vacation.	English sentence with similar holiday-policy meaning

TEXTS = [
    "今日は天気がいい",
    "年末年始の休暇規定を確認してください",
    "申請の締切日は12月15日です",
    "API 標準設計ガイドの参照はこちら",
    "会議は月曜日に延期になりました",
    "正しいPythonの環境構築方法",
    "Please check the holiday policy for New Year vacation.",
]

QUERY = "正月の休暇と申請締切"
TOP_K = 5

The query means:

Query	Meaning
正月の休暇と申請締切	New Year vacation and the application deadline

In this corpus, the relevant answers are split across multiple sentences:

holiday policy: 年末年始の休暇規定を確認してください
deadline: 申請の締切日は12月15日です
related English sentence: Please check the holiday policy for New Year vacation.

Normalizing BGE-M3 sparse output

One detail that is easy to miss: BGE-M3 sparse output may come back in different shapes depending on the library path. I normalized both forms into Qdrant's SparseVector.

from typing import Any
from qdrant_client.http.models import SparseVector


def to_qdrant_sparse_list(m3_output: dict[str, Any]) -> list[SparseVector]:
    sv_list: list[SparseVector] = []

    if "sparse_vecs" in m3_output:
        for sp in m3_output["sparse_vecs"]:
            sv_list.append(
                SparseVector(
                    indices=sp.indices.tolist(),
                    values=sp.data.tolist(),
                )
            )
    elif "lexical_weights" in m3_output:
        for lw in m3_output["lexical_weights"]:
            items = sorted((int(k), float(v)) for k, v in lw.items())
            sv_list.append(
                SparseVector(
                    indices=[i for i, _ in items],
                    values=[v for _, v in items],
                )
            )
    else:
        raise ValueError("No sparse output found in BGEM3 result")

    return sv_list

Creating dense, sparse, or hybrid collections

For a dense-only collection:

q.create_collection(
    collection_name=COLL,
    vectors_config={
        "dense": qm.VectorParams(
            size=dim,
            distance=qm.Distance.COSINE,
        )
    },
)

For a sparse-only collection:

q.create_collection(
    collection_name=COLL,
    vectors_config=None,
    sparse_vectors_config={
        "sparse": qm.SparseVectorParams(),
    },
)

For the hybrid version, store both named vectors:

q.create_collection(
    collection_name=COLL,
    vectors_config={
        "dense": qm.VectorParams(
            size=dim,
            distance=qm.Distance.COSINE,
        )
    },
    sparse_vectors_config={
        "sparse": qm.SparseVectorParams(),
    },
)

That last shape is the one I would normally use for document search. It keeps the operational model simple: one point per chunk, both retrieval signals attached.

Upserting both vectors on the same point

Dense embeddings came from SentenceTransformer using the local BGE-M3 directory. Sparse vectors came from BGEM3FlagModel.

from FlagEmbedding import BGEM3FlagModel
from qdrant_client.http.models import PointStruct
from sentence_transformers import SentenceTransformer


dense_model = SentenceTransformer("./models/bge-m3")
m3 = BGEM3FlagModel("BAAI/bge-m3", use_fp16=True)

dense_vecs = [
    dense_model.encode(t, normalize_embeddings=True).tolist()
    for t in TEXTS
]

sparse_out = m3.encode(TEXTS, return_sparse=True)
sparse_vecs = to_qdrant_sparse_list(sparse_out)

points = []
for id_, text, dense_vec, sparse_vec in zip(
    range(1, len(TEXTS) + 1),
    TEXTS,
    dense_vecs,
    sparse_vecs,
):
    points.append(
        PointStruct(
            id=id_,
            vector={
                "dense": dense_vec,
                "sparse": sparse_vec,
            },
            payload={"text": text},
        )
    )

q.upsert(collection_name=COLL, points=points)

Dense and sparse search

Dense search uses the dense vector name:

def search_dense(q, dense_model, qtext: str, top_k: int = 5):
    qvec = dense_model.encode(qtext, normalize_embeddings=True).tolist()
    res = q.query_points(
        collection_name=COLL,
        query=qvec,
        using="dense",
        limit=top_k,
        search_params=qm.SearchParams(hnsw_ef=128),
    )
    return res.points

Sparse search uses the sparse vector name:

def search_sparse(q, m3, qtext: str, top_k: int = 5):
    out = m3.encode([qtext], return_sparse=True)
    q_sp = to_qdrant_sparse_list(out)[0]
    res = q.query_points(
        collection_name=COLL,
        query=q_sp,
        using="sparse",
        limit=top_k,
    )
    return res.points

Hybrid search with RRF

For the hybrid mode, I searched dense and sparse separately, then fused the ranked IDs with Reciprocal Rank Fusion.

def rrf_fuse(id_lists: list[list[int]], k: int = 60) -> list[int]:
    score: dict[int, float] = {}
    for id_list in id_lists:
        for rank, pid in enumerate(id_list, start=1):
            score[pid] = score.get(pid, 0.0) + 1.0 / (k + rank)

    return [
        pid
        for pid, _ in sorted(
            score.items(),
            key=lambda x: x[1],
            reverse=True,
        )
    ]

The search function is intentionally boring:

def search_hybrid_rrf(q, dense_model, m3, qtext: str, top_k: int = 5):
    dense_hits = search_dense(q, dense_model, qtext, top_k=top_k)
    sparse_hits = search_sparse(q, m3, qtext, top_k=top_k)

    dense_ids = [h.id for h in dense_hits]
    sparse_ids = [h.id for h in sparse_hits]
    fused_ids = rrf_fuse([dense_ids, sparse_ids])[:top_k]

    by_id = {h.id: h for h in dense_hits}
    for h in sparse_hits:
        by_id.setdefault(h.id, h)

    return [by_id[i] for i in fused_ids if i in by_id]

I used RRF here because it only needs ranks, not comparable raw scores from different retrieval systems.

Optional cross reranking

After first-stage retrieval, I optionally reranked the candidates with BAAI/bge-reranker-v2-m3.

from FlagEmbedding import FlagReranker


def cross_rerank(qtext: str, hits, max_chars: int = 1400, batch_size: int = 16):
    rr = FlagReranker("BAAI/bge-reranker-v2-m3", use_fp16=True)

    pairs = []
    keep = []
    for h in hits:
        text = (h.payload.get("text") or "")[:max_chars].replace("\n", " ")
        if text.strip():
            pairs.append([qtext, text])
            keep.append(h)

    scores = rr.compute_score(pairs, normalize=True, batch_size=batch_size)

    for h, s in zip(keep, scores):
        h.payload["_cross"] = float(s)

    keep.sort(key=lambda x: x.payload.get("_cross", 0.0), reverse=True)
    return keep

This is a second stage, not a replacement for the index. In a larger system I would keep the rerank candidate set bounded, for example by reranking only the top 20 to 50 candidates.

What the small test showed

I ran six combinations:

python test_index_and_rerank.py --index dense  --rerank none
python test_index_and_rerank.py --index dense  --rerank cross
python test_index_and_rerank.py --index sparse --rerank none
python test_index_and_rerank.py --index sparse --rerank cross
python test_index_and_rerank.py --index both   --rerank none
python test_index_and_rerank.py --index both   --rerank cross

Dense search without reranking put the holiday policy first, the English holiday sentence second, and the deadline third:

INDEX = DENSE, SEARCH = DENSE   RERANK=NONE
Query: "正月の休暇と申請締切"

score=0.727  id=2  text=年末年始の休暇規定を確認してください
score=0.664  id=7  text=Please check the holiday policy for New Year vacation.
score=0.617  id=3  text=申請の締切日は12月15日です
score=0.474  id=5  text=会議は月曜日に延期になりました
score=0.378  id=1  text=今日は天気がいい

Sparse search put the deadline first and the holiday policy second:

INDEX = SPARSE, SEARCH = SPARSE   RERANK=NONE
Query: "正月の休暇と申請締切"

score=0.097  id=3  text=申請の締切日は12月15日です
score=0.071  id=2  text=年末年始の休暇規定を確認してください
score=0.001  id=6  text=正しいPythonの環境構築方法
score=0.000  id=5  text=会議は月曜日に延期になりました

Hybrid search plus cross reranking gave this order in the same small test:

INDEX = BOTH, SEARCH = HYBRID(RRF)   RERANK=CROSS
Query: "正月の休暇と申請締切"

score=0.617  cross=0.030  id=3  text=申請の締切日は12月15日です
score=0.727  cross=0.025  id=2  text=年末年始の休暇規定を確認してください
score=0.664  cross=0.020  id=7  text=Please check the holiday policy for New Year vacation.
score=0.474  cross=0.000  id=5  text=会議は月曜日に延期になりました
score=0.001  cross=0.000  id=6  text=正しいPythonの環境構築方法

I would not read too much into the exact scores from such a small corpus. The useful observation was simpler:

dense search captured semantic and cross-language similarity better in this test
sparse search surfaced lexical matches such as the deadline sentence
hybrid search let both paths contribute candidates
reranking changed the order of the top candidates, so it should be evaluated separately from retrieval

Issues I hit

The main problems were ordinary integration issues:

Vector dimension mismatch: if the model dimension does not match the collection's configured vector size, recreate the collection.
Sparse vector support: use a recent qdrant-client; I used qdrant-client>=1.15.0.
Pydantic compatibility: I pinned pydantic>=2.7,<2.12 in this environment.
Model download size: the first BGE-M3 download needs several GB of disk space.
Docker persistence: without -v $(pwd)/qdrant_storage:/qdrant/storage, deleting the container also removes the data.

How I would use this for real documents

For actual internal documents, I would keep the same retrieval structure and add a document ingestion pipeline:

extract text from .txt, .md, and .pdf
split into chunks with overlap
store dense and sparse vectors on the same Qdrant point
include payload fields such as path, page, chunk_index, and text
query with dense, sparse, or hybrid mode depending on the use case
optionally rerank a bounded candidate set

My default would be to ingest both dense and sparse vectors from the beginning. Even if the first UI only exposes semantic search, having both vectors available makes later experiments much cheaper.

Qdrant docs:

Named vectors: https://qdrant.tech/documentation/concepts/vectors/#named-vectors
Sparse vectors: https://qdrant.tech/documentation/concepts/vectors/#sparse-vectors

BGE-M3:

Model: https://huggingface.co/BAAI/bge-m3
FlagEmbedding: https://github.com/FlagOpen/FlagEmbedding

Happy to answer questions.

Building on-device Video Notes in a macOS app

yamayu-dev — Fri, 03 Jul 2026 11:16:01 +0000

Building on-device Video Notes: SpeechAnalyzer, Foundation Models, and libmpv in a shipping macOS app

My macOS video player, Reel, has a feature called Video Notes: pick a local video, click Generate, and get a timestamped transcript, an optional translation, and a structured summary — entirely on-device. No API keys, no upload, works on an mkv file of a two-hour lecture.

The pipeline is four stages, each on a different technology:

Apple's 2025 APIs demo beautifully. Shipping them is a different story — the interesting bugs are all in the last stage. Here's what each stage looks like in production, and every workaround I needed.

Stage 1: audio extraction with headless libmpv (`ao=pcm`)

Why not AVFoundation? Because it isn't enough for a local-video library: mkv and webm — both common in the wild — won't open at all. Reel already ships libmpv for playback, and libmpv has a trick that makes it a universal audio extractor without any encoder: the PCM audio output.

guard let h = mpv_create() else { return nil }
func opt(_ k: String, _ v: String) { _ = mpv_set_option_string(h, k, v) }
opt("terminal", "no"); opt("config", "no"); opt("msg-level", "all=no")
opt("vid", "no"); opt("vo", "null")          // headless: no video, no window
opt("ao", "pcm")                             // "play" audio into a WAV file
opt("ao-pcm-file", out.path)
opt("ao-pcm-waveheader", "yes")
opt("audio-samplerate", "16000")             // what speech models want
opt("audio-channels", "mono")
guard mpv_initialize(h) >= 0 else { mpv_terminate_destroy(h); return nil }
// mpv_command(h, ["loadfile", url.path]) …then pump events until END_FILE

You "play" the file with video disabled and the audio device replaced by a WAV writer; decoding runs much faster than realtime. Anything mpv can demux — mkv, webm, avi, and the other containers AVFoundation rejects — becomes a 16 kHz mono WAV. Two production notes:

Pump mpv_wait_event until MPV_EVENT_END_FILE, with a deadline. A pathological input that never emits END_FILE would otherwise hang your pipeline forever.
Sanity-check the output (exists, > 1 KB) before declaring success; a video with no audio track "succeeds" into an empty file.

Stage 2: transcription with SpeechAnalyzer (macOS 26)

SpeechAnalyzer / SpeechTranscriber is the new speech stack — on-device, fast (it chewed through long files far quicker than realtime in my testing), and notably it does not require Apple Intelligence, just a model download. Three things the sample code doesn't emphasize:

1. Ask for timestamps via attributeOptions. I want a transcript that seeks the player when you click a line, which means per-segment timing:

let transcriber = SpeechTranscriber(
    locale: locale,
    transcriptionOptions: [],
    reportingOptions: [],
    attributeOptions: [.audioTimeRange]   // ← per-run timing, for click-to-seek
)

The timing comes back as attributed-string runs; read the first run carrying an audioTimeRange to get a segment's start time.

2. Model download is your job to surface. First use of a locale requires the asset:

if let request = try await AssetInventory.assetInstallationRequest(supporting: [transcriber]) {
    try await request.downloadAndInstall()
}

This can take a while on first run — show progress, don't just spin.

3. Consume results concurrently with feeding the file. transcriber.results is an async sequence; collect it in a Task while analyzer.analyzeSequence(from:) consumes the audio file, then finalizeAndFinish(through:) the last sample. Collecting after the fact deadlocks; the collector must already be draining.

Stage 3: translation — the framework that only lives inside SwiftUI

The Translation framework does on-device EN↔JA (and more) with no Apple Intelligence requirement. Its one architectural surprise: you can't just instantiate a translation session in your service layer. A TranslationSession is only vended through SwiftUI's .translationTask modifier, so the translation stage is driven from the view/model layer, not from the same stateless service as the rest of the pipeline. Plan your architecture around that asymmetry; my pipeline service does extraction/transcription/summary, and the model that owns the SwiftUI surface drives translation.

Stage 4: summarization with Foundation Models — where the production issues showed up

This is the stage that "works in the demo" and then fails four different ways on real content. Every workaround below is in shipping code.

4a. The default guardrails refuse ordinary content

My first end-to-end test summarized a tourism clip fine, then refused an ordinary interview video. The default safety guardrails false-positive on perfectly normal human conversation — and summarizing the user's own file is exactly the use case Apple provides relaxed guardrails for:

private static let summaryModel =
    SystemLanguageModel(guardrails: .permissiveContentTransformations)

Switching to .permissiveContentTransformations (the mode intended for content-transformation tasks over user-provided material) cut the false blocks dramatically.

4b. Tell guardrail refusals apart from real failures

When the model does refuse, users deserve a different message than "generation failed":

} catch let error as LanguageModelSession.GenerationError {
    switch error {
    case .guardrailViolation, .refusal: return .blocked   // won't change on retry
    default:                            return .none
    }
}

.blocked gets its own UI string; retrying a safety block is pointless, so don't.

4c. The context window is small — map-reduce long transcripts

A single respond(to:) with a long transcript just throws: context window exceeded, no summary for exactly the videos that need one most. The fix is the classic map-reduce: chunk the transcript (~6,000 characters per request), summarize each chunk with a neutral prompt, then summarize the concatenated partial summaries with the user's preferences applied to the final pass. The chunker splits on whitespace, which means word boundaries in space-separated languages. Japanese has no internal spaces, but the transcript is assembled by joining timestamped segments with spaces, so it still chunks correctly — on segment boundaries. If even the combined partials exceed the budget, ship the concatenation rather than fail.

4d. The on-device model gets stuck in repetition loops

Occasionally the model emits the same sentence over and over — a classic greedy-decoding failure. Three defenses, all needed:

A little temperature. GenerationOptions(temperature: 0.6) avoids most loops outright.
Collapse what still repeats. Post-process: dedupe identical adjacent lines, then identical adjacent sentences within a line.
Detect degenerate output and retry once. If, after collapsing, more than half the sentences are duplicates, the model looped — regenerate once, then give up rather than show garbage:

private static func isDegenerate(_ text: String) -> Bool {
    let units = text.split(whereSeparator: { $0 == "\n" || $0 == "。" || $0 == "." })
        .map { $0.trimmingCharacters(in: .whitespaces) }
        .filter { !$0.isEmpty }
    guard units.count >= 3 else { return false }
    return Set(units).count * 2 < units.count   // over half are duplicates
}

4e. Smaller lessons from the same stage

Plain text beat Guided Generation for this task. Structured (guided) output occasionally came back malformed; a strictly-specified plain-text format ("1–2 sentence overview, blank line, 3–6 bullets") parses trivially and never broke.
Don't summarize the un-summarizable. Below ~140 characters of transcript, a "summary" just restates the input; skip the stage and point users at the transcript.
Cross-language output needs shouting. Producing a Japanese summary of an English transcript works, but only if the instructions say, in effect, CRITICAL: write the ENTIRE summary, including headings, in Japanese — a polite request gets you mixed-language output.
The transcript is noisy input — say so. One instruction line ("the transcript is auto-generated; silently correct obvious proper-noun misrecognitions") noticeably improves summaries of technical content.

Degrade gracefully

SystemLanguageModel.default.availability tells you whether Apple Intelligence is on. When it isn't, only the summary stage dies — transcription, translation, and frame captures all still work, so the feature stays useful on machines without Apple Intelligence. Check availability up front and disable exactly one stage, not the feature.

Assembly notes

Generate once, persist, reload instantly. The pipeline takes seconds to minutes; results are written as JSON (plus captured frames) in Application Support, keyed by a path-derived stable video ID. Regeneration is explicit.
Every stage is independent. Extraction/transcription need nothing from Apple Intelligence; translation is optional and on-demand; summary is the only gated stage. Model the pipeline that way and each capability degrades separately.
Frame captures reuse the player's engine. Timestamped captures come from the same headless-libmpv screenshot path the player uses for thumbnails, so "capture this moment" costs nothing extra. The end goal is a portable Markdown export — summary, highlights with images, transcript — that opens in Obsidian or GitHub.

The takeaway

On-device AI on the Mac in 2026 is genuinely shippable, but the difficulty is inverted from what you'd expect: the AI part (transcription quality, summary quality) mostly just works, while the production engineering — guardrail false positives, context budgeting, repetition loops, framework lifecycle quirks, graceful degradation — is where the real work lives. None of it is hard once you know it's coming. Now you do.

I built this for Reel, a local video player for macOS, available on the Mac App Store. The companion piece — getting libmpv itself through App Store review — is here.

Shipping a libmpv-based video player on the Mac App Store

yamayu-dev — Thu, 02 Jul 2026 22:53:25 +0000

Shipping a libmpv-based video player on the Mac App Store

Most well-known mpv-based Mac players, including IINA, distribute outside the Mac App Store — a store build has been an open request for IINA for years. When I built Reel, a local-video player/library app for macOS, I wanted to know what makes that true, and whether it could be done anyway. It can. It took a month from first commit to approval, and almost none of the hard parts were about playing video.

This is a field report of every wall I hit: a JIT crash that only happens under the App Store's rules, LGPL compliance with static linking, two sandbox traps that pass locally and fail later, and one design rejection. If you're putting any FFmpeg/libmpv-family library into a sandboxed Mac app, this should save you a week or two.

Setup: Swift / SwiftUI app. Playback is AVFoundation plus libmpv (via MPVKit, statically linked, LGPL build) for containers where AVFoundation isn't enough, such as mkv, webm, and avi. Distribution: Mac App Store, sandboxed, with StoreKit 2 for a tip jar.

Wall 1: LuaJIT inside libmpv gets your process killed — only on the App Store build

During development (sandbox off), libmpv worked perfectly. The moment I built with the Mac App Store configuration — App Sandbox plus Hardened-Runtime-style entitlement restrictions — the process died inside mpv_initialize(), before playing anything.

The culprit is the pair of JIT entitlements:

com.apple.security.cs.allow-jit — allowed on the Mac App Store
com.apple.security.cs.allow-unsigned-executable-memory — not allowed on the Mac App Store

libmpv statically links LuaJIT, and LuaJIT's machine-code allocator requests plain RWX executable memory rather than MAP_JIT memory. With only allow-jit, AMFI kills the process.

I confirmed it with an A/B test on the actual signed builds:

Entitlements on the binary	`mpv_initialize()`	Result
`allow-jit` only (the most MAS permits)	SIGKILL (exit 137)	AMFI kills the process
`allow-jit` + `allow-unsigned-executable-memory` (dev config)	succeeds, playback works	fine — but unshippable

"Just disable Lua scripts" does not work

My first hope was configuration: load-scripts=no, osc=no, ytdl=no, config=no. No effect. LuaJIT's allocator grabs executable memory during init regardless of whether any script will ever run. If LuaJIT is linked in, you crash.

The fix: rebuild libmpv with Lua disabled

Reel doesn't use mpv's Lua scripting at all — no OSC, no user scripts. So the fix was a one-line change to the mpv build:

- -Dlua=luajit
+ -Dlua=disabled

A script applies this and rebuilds Libmpv.xcframework. Verification:

libmpv build	Entitlements	`mpv_initialize()`
standard (LuaJIT)	`allow-jit` only	SIGKILL 137
Lua disabled	`allow-jit` only	succeeds, playback works

Lua symbols in the rebuilt xcframework: 0 (previously ~50 undefined LuaJIT references)
All the mpv API I use (mpv_create, mpv_initialize, mpv_render_context_create, …) intact — zero functional loss for this app

A pleasant side effect: dropping LuaJIT removed the need for allow-unsigned-executable-memory and disable-library-validation entirely, so the final signature is tighter than the dev build ever was. The shipping entitlements file is now just: app-sandbox, network.client (StoreKit), files.user-selected.read-only, files.bookmarks.app-scope, and cs.allow-jit.

Lesson: if you statically link any library that JITs (LuaJIT, some JS engines, …), test against App Store entitlement rules first. And ask whether you actually use the feature that needs the JIT — the cheapest fix may be compiling it out.

Wall 2: LGPL compliance with static linking

libmpv and its FFmpeg dependencies are LGPL (I use MPVKit's LGPL variant — no GPL components, no x264/x265, so there's no App Store incompatibility at the license level). But static linking triggers the LGPL's re-linking obligation: users must be able to relink the application against a modified version of the library.

There are three classic ways to satisfy it. Dynamic linking doesn't get you anything on the Mac App Store (users can't swap a dylib inside a signed, store-installed bundle anyway) and would have meant restructuring the build, so I went with a written offer, valid for three years:

An in-app Licenses window (App menu → Licenses…) listing every library, its license, and upstream source URL, with the full LGPL texts bundled in Reel.app/Contents/Resources
The written offer text included in the App Store description's notes: on request, I provide the machine-readable object code of the app plus linked libraries, sufficient to relink against a user-modified library
A repo script that reproduces my only modification to the libraries (-Dlua=disabled), and another script that collects the .o files and link inputs from the exact release build into an archive for anyone who requests it

Not legal advice — this is the reading I implemented, and it passed review. The parts most people forget: keep the build inputs of the shipped release around for the duration of the offer, and remember the offer must ship with the binary (store listing + in-app), not just sit in your repo.

Wall 3: two sandbox traps that pass locally and fail later

Playing files the user picked, across relaunches, needs security-scoped bookmarks. The happy path is well documented:

NSOpenPanel → url.bookmarkData(options: .withSecurityScope)
next launch → resolve → startAccessingSecurityScopedResource()

The traps are not.

Trap A: the bookmark entitlement key has a decoy spelling

The correct key is:

com.apple.security.files.bookmarks.app-scope

The variant without files. (com.apple.security.bookmarks.app-scope) signs and runs locally just fine — and is then rejected by App Store submission validation as an unsupported entitlement. Everything works on your machine, so nothing warns you until you upload.

Trap B: a dropped folder's security scope dies with the closure

Shipped in 1.0, added folder drag-and-drop in 1.1, and hit this: opening a folder from Finder/Dock (.onOpenURL) worked, but dropping the same folder onto the window silently did nothing — drop highlight appears, nothing imports.

Debug logging showed the URL was arriving. The failure chain:

NSItemProvider.loadObject(ofClass: URL.self) → returns nil for Finder file URLs
loadItem + URL(dataRepresentation:) → yields a URL without a security scope; startAccessingSecurityScopedResource() returns false and a later fileExists check quietly fails
Root cause: the scoped URL from loadInPlaceFileRepresentation is only valid until that closure returns. I was hopping to the main actor with Task { @MainActor in … } — by the time the task ran, the scope was gone.

The fix: mint the security-scoped bookmark inside the closure, while the scope is alive, and pass the bookmark bytes onward:

provider.loadInPlaceFileRepresentation(forTypeIdentifier: UTType.folder.identifier) { url, _, _ in
    guard let url else { return }
    let ok = url.startAccessingSecurityScopedResource()
    defer { if ok { url.stopAccessingSecurityScopedResource() } }
    // The scope is alive *now* — capture it as a bookmark before leaving the closure.
    let bookmark = try? url.bookmarkData(options: .withSecurityScope,
                                         includingResourceValuesForKeys: nil, relativeTo: nil)
    Task { @MainActor in store.importDroppedBookmark(bookmark, fallback: url) }
}

The asymmetry that makes this hard to spot: URLs from .onOpenURL come via LaunchServices and their scope persists, so the "same" feature works through one door and not the other.

Wall 4: App Review — Guideline 4, the un-reopenable main window

First 1.0 submission was rejected under Guideline 4 (Design): close the main window with the red button and there's no menu item to get it back. Fair.

The fix:

Made the main window a dedicated single Window scene (not WindowGroup), so the system lists it in the Window menu and reopening works
Added an explicit Window → Main Window (⌘0) item that also brings it forward when a player window covers it
The app keeps its menu bar alive with the main window closed

In the resubmission notes I wrote exactly what changed and how I verified it ("clean build; close main window with the red button; ⌘0 reopens it"). Plain reproduction steps, no argument. Approved.

One postscript: an older branch still had the WindowGroup implementation, and I later nearly merged it back over the reviewed fix. Whatever implementation passed review is the canonical one — guard it.

Smaller things that also cost time

You can't attach a new build to an approved version. New changes require a new version number. I learned this trying to add drag-and-drop to the already-approved 1.0 → it shipped as 1.1.
App Store screenshots must have no alpha channel (macOS: 2880×1800). Screen captures have alpha; converting to JPEG kills it reliably. Also, a plain UI dump looks like nothing at search-result thumbnail size — I compose the first screenshot (headline, rounded corners, shadow, dark background) with a small Python/PIL script over real captures.
String Catalogs: only strings that go through LocalizedStringKey are auto-extracted, and xcodebuild won't write keys back into the catalog for you. Budget manual passes.

The checklist I wish I'd had

Statically linking anything that JITs? Test mpv_initialize() (or equivalent) under App Store entitlements, not your dev entitlements. allow-unsigned-executable-memory is not available to you.
Do you actually use the JIT-dependent feature? Compiling it out may be the whole fix.
LGPL + static linking → written offer + relink materials + bundled license texts, and keep the release's build inputs.
Bookmark entitlement key is com.apple.security.files.bookmarks.app-scope — the files.-less spelling only fails at submission.
Security-scoped URLs from drag-and-drop die with the provider closure — bookmark them in place.
Close your main window with the red button. Can a reviewer get it back from the menu bar?
Approved version = new build needs a new version number.
Screenshots: no alpha, and make the first one legible at thumbnail size.

Building the player was the easy part. Getting it — safely, license-compliantly, review-provenly — into people's hands is where the month went. But the walls are all climbable, documented above, and mostly one-time costs.

Reel, a local video player for macOS, is available on the Mac App Store. Happy to answer questions about any of the above.

The companion piece — building the on-device Video Notes pipeline (SpeechAnalyzer + Foundation Models) on top of this player — is here.

DEV Community: yamayu-dev

Building local hybrid search with Qdrant and BGE-M3

Why BGE-M3 with Qdrant named vectors?

Local setup

Test corpus

Normalizing BGE-M3 sparse output

Creating dense, sparse, or hybrid collections

Upserting both vectors on the same point

Dense and sparse search

Hybrid search with RRF

Optional cross reranking

What the small test showed

Issues I hit

How I would use this for real documents

Building on-device Video Notes in a macOS app

Building on-device Video Notes: SpeechAnalyzer, Foundation Models, and libmpv in a shipping macOS app

Stage 1: audio extraction with headless libmpv (ao=pcm)

Stage 2: transcription with SpeechAnalyzer (macOS 26)

Stage 3: translation — the framework that only lives inside SwiftUI

Stage 4: summarization with Foundation Models — where the production issues showed up

4a. The default guardrails refuse ordinary content

4b. Tell guardrail refusals apart from real failures

4c. The context window is small — map-reduce long transcripts

4d. The on-device model gets stuck in repetition loops

4e. Smaller lessons from the same stage

Degrade gracefully

Assembly notes

The takeaway

Shipping a libmpv-based video player on the Mac App Store

Shipping a libmpv-based video player on the Mac App Store

Wall 1: LuaJIT inside libmpv gets your process killed — only on the App Store build

"Just disable Lua scripts" does not work

The fix: rebuild libmpv with Lua disabled

Wall 2: LGPL compliance with static linking

Wall 3: two sandbox traps that pass locally and fail later

Trap A: the bookmark entitlement key has a decoy spelling

Trap B: a dropped folder's security scope dies with the closure

Wall 4: App Review — Guideline 4, the un-reopenable main window

Smaller things that also cost time

The checklist I wish I'd had

Stage 1: audio extraction with headless libmpv (`ao=pcm`)