DEV Community: DMJ Jones

Your certificate inventory is exact. Your tickets are not.

DMJ Jones — Sun, 20 Sep 2026 10:22:11 +0000

Every certificate team has read this ticket:

We renewed the certificate yesterday and I can see the new one in the portal, but about half of our customers still get a warning.

Two things are true at once here. Your inventory knows exactly what is deployed, down to the serial and the second of expiry. And the person who filed the ticket has told you almost nothing you can query on.

The gap between those two sentences is where certificate triage actually happens, and it is the part nobody automates. I spent a week building a triage tool to see how much of that gap a model can close, and — more usefully — where it should not be allowed anywhere near.

The split

The rule I started with, and never had cause to break: if the answer is computable, compute it.

Expiry maths, chain building for a given client's trust store, SAN coverage including wildcards, whether a CAA record permits the CA your ACME client is using, which endpoints serve which serial — none of that belongs in a model. It is arithmetic and string matching, and a model can only make it less reliable.

What code cannot do is read "about half of our customers" and know that means some nodes behind a load balancer still serve the old certificate. Or read "trust anchor for certification path not found, but the website works in Chrome on the same phone" and understand that the phrase on the same phone is the whole diagnosis, because it rules out the trust store and points at a missing intermediate.

So: code owns the facts, the model reads the prose, and a fixed policy in ordinary code decides what happens. The model never acts.

Don't hand it the inventory

The tempting design is to give the model the estate and ask which certificate the ticket is about. It does not survive contact with a real inventory.

A synthetic estate of 500 certificates costs about 20,000 tokens to describe, which is most of a request before you have asked anything. At 10,000 certificates it does not fit at all. Accuracy also drops as you pad the input with irrelevant detail, which is the documented failure mode for this class of model and the intuitive one for any other.

What works is the boring thing. Code extracts the CN or hostname — from the intake form if the reporter gave one, otherwise from the ticket text — and matches it against the inventory exactly: common name, SANs, wildcards, and every endpoint serving that host. Only the matches go to the model, capped at twelve. The other 488 certificates never leave the server.

That cap is what makes the cost independent of the estate. Triaging a ticket against 500 certificates and against ten million costs the same, because the model sees the same handful either way.

When it cannot tell, it asks

Plenty of tickets name no host at all. "Old iPhones stopped loading product images this morning." For those, one question to the model — which service in the catalogue is this about — is enough for code to do the lookup.

But sometimes the honest answer is that nobody knows yet. A ticket about mTLS failures between two internal services mapped to a service with 81 certificates, one per pod. Twelve is the cap, 81 is well past it, and picking one would be theatre.

So the tool asked the reporter for the exact CN, along with the error, where it fails and since when.

The answer came back as orders-17. Code then found, in one exact lookup, a pod still serving a certificate that had expired three hours earlier — while the renewed one sat in the inventory, issued and never deployed. cert-manager had done its job. The pod never reloaded.

Asking "which CN?" costs one message. Guessing across 81 certificates costs an incident review.

What the model is actually for

The model I used is Jev, TypeSafe's System One model, which does not generate text at all. You send it state and typed questions, and it returns typed answers with probabilities: one option from a set you defined, a position on a scale you defined, or the probability that a yes/no statement is true. There is nothing to parse and no prose to second-guess.

Triage became two requests. The first gets the ticket and nothing else, and answers what the reporter is describing, how urgent it sounds, and which service it concerns. The second gets the matched certificates and the findings the automated checks produced, and asks one yes/no question per finding: would this problem cause the failure this person is describing?

Keeping the inventory out of the first request is deliberate. It means the reading of the symptoms and the reading of the evidence are independent, so when they agree, the agreement is worth something. That cross-check is the part I would keep in any design, whatever model sits behind it.

Then I measured it against an ordinary LLM

The obvious question: why not just call a normal model? So I built the same pipeline again on gemini-3.5-flash-lite — same lookup code, same questions, same policy, structured JSON output — and ran both over the same tickets.

Same 9 tickets, same code, same policy

Jev (jev-1.13.0)          8 of 9 causes   596 ms    $0.09 / 1,000 tickets
gemini-3.5-flash-lite     8 of 9 causes 1,420 ms    $0.94 / 1,000 tickets

Accuracy tied, and they failed on different tickets, which is about what nine samples can tell you. Ten times the cost and a bit over twice the latency is a real difference at volume, but it is not the interesting part.

The interesting part was the probabilities. Asked "would this finding cause what the reporter describes?", the LLM answered 1.00. Every finding, every ticket, including a stale-certificate finding paired with a ticket about payment timeouts that had nothing to do with TLS. Jev gave that pair 0.29.

The LLM did not act wrongly on it, but only because its self-reported cause confidence happened to land on 0.50 that time — and its confidences across the run were 0.50, 0.80, 0.85, 0.90, 0.95. Round numbers it wrote about itself.

A threshold is only as good as the number you put it on.

If you are gating an automatic action on a probability, that is the whole ballgame. A model that says 1.00 to everything has not given you a probability. It has given you a shape that looks like one.

What I would require before letting this act

No probability moves a key. Issuance, revocation and export stay behind RBAC and human approval. A model can rank, link and explain. It does not get to sign.
Narrow in code first. A handful of candidates, never the estate. It is cheaper, more accurate, and the cost stops scaling with your inventory.
Ask a person a specific question when the narrowing is ambiguous. "Which CN?" beats a confident guess.
Write the probability, the thresholds and the model version into the audit log next to the decision. A decision you cannot replay six months later is not auditable, and "the model said so" is not a root cause.
Pin the model version. Thresholds are tuned against specific behaviour, and an alias like jev-latest moves underneath you.
Measure wrong automatic actions, not accuracy. Accuracy is a number for a slide. The operational question is how often the system did something it should not have.

Where this goes

Ticket triage was the demo, because a ticket is the clearest case of prose meeting exact data. The same shape fits three more jobs I care about rather more, and they are next for CertPilot:

Discovery. CT logs surface a certificate for one of your domains that the inventory cannot match by name. Is it a team you do not know about, a SaaS vendor operating legitimately, or a lookalike worth an alert?
Renewal failures. A durable queue with retries is correct design and wrong behaviour for a CAA misconfiguration — it will retry twenty times and burn the CA's rate limit. Code reads the structured ACME error; something else has to read the prose the CA sends back and decide retry, back off, or hand over.
Rollout safety. During a deployment wave, the signal that you broke an old client usually arrives as a support ticket, not a failed probe. Link fresh tickets to the hosts just deployed, and hold the wave.

The demo runs on a synthetic estate with invented tickets, so treat the numbers as a demonstration rather than a benchmark: fourteen labelled tickets, nine of them in the LLM comparison because of a free-tier quota. What is real is the model traffic, the latency and the cost, and every request and response is on screen.

CertRadar is here if you want to click through it. Every step shows the exact JSON that went to the model, the exact JSON that came back, and what the decision cost.

I co-engineered a macOS Soundcore headphone controller with Claude over a weekend

DMJ Jones — Mon, 14 Sep 2026 19:54:11 +0000

My Soundcore Space 2 pairs with my MacBook and sounds exactly as it should. But
the moment I want to turn noise cancelling off, or pull the bass back, I have to
pick up my phone and open the Soundcore app — because Anker only ships that app
for Android and iOS.

The headphones are right there. The Mac is right there. The controls are on a
third device.

So over a single weekend, I decided to pair-program and co-engineer a solution with
Claude to see if we could reverse-engineer the protocol and build
SoundcoreBridge: a native macOS menu bar app
and CLI that speaks the same protocol. And it actually worked.

This is the story of how that weekend went, what co-engineering low-level hardware
code with an LLM looks like in practice, and the two occasions I confidently
concluded macOS was broken when the bug was mine.

Finding the door

Bluetooth devices advertise their services over SDP. Dumping the Space 2's
records shows the usual audio profiles — A2DP, AVRCP, handsfree — and then some
more interesting entries:

ch 30   PaulWang-Spp   0cf12d31-fac3-4553-bd80-d6832e7d1402   <- vendor control
ch 17   RFCOMM COM     df21fe2c-2515-4fdb-8886-f12c4d67927c
ch 12   TOTA           0x1101                                  <- firmware OTA
ch 13   BESOTA         66666666-...                            <- chipset OTA
ch 16   IOSSPP         00000000-deca-fade-deca-deafdecacaff     <- Apple iAP2

That first UUID shares its prefix (0cf12d31-fac3-4553-bd80-) with the
documented Soundcore control UUID; the suffix is per-model. Channel 30 is the
door.

Channels 12 and 13 flash firmware. Those are hard-blocked in my code by service
identity, and I'd suggest anyone doing this does the same before writing a
single byte.

The frame format

Prior art from other Soundcore projects gave me the shape:

08 EE 00 00 | 00 | 06 81 | 10 00 | 00 5F 02 00 00 01 | EF
└─ magic ─┘  seq  └ cmd ┘ └ len ┘ └───  payload  ──┘  checksum

Two-byte command, little-endian total length, and a checksum that's just the sum
of every preceding byte truncated to 8 bits. Device replies swap the magic to
09 FF 00 00.

Before touching hardware I implemented the codec and checked it against six
packets documented for other models. All six reproduced byte-for-byte. That
five-minute investment meant that later, when nothing worked, I could rule out
the encoder completely.

Two days of "macOS blocks this"

Then I tried to open the channel, and nothing happened.

openRFCOMMChannelAsync returned success and the delegate callback never fired.
The synchronous variant returned kIOReturnError immediately. I tried channel
30, then 17, then channel 1 (handsfree — definitely exists), then channel 2
(nothing advertised). Identical failure every time.

So I ran what I thought was a decisive control experiment: I tried opening
RFCOMM to a completely different device — an Android phone, on standard
services like OBEX and phonebook access. Same failure.

Two devices, seven channels, standard services, from both a bare binary and a
signed .app. bluetoothd logs showed no connect attempt ever reaching the air.
I checked TCC (granted), App Sandbox (not sandboxed), the baseband link
(connected). I wrote it up as a platform limitation and started designing a
Raspberry Pi bridge.

I was wrong.

Claude — who I was pair-programming with throughout the weekend — pushed back on my
theory and surfaced SonyBridge, an
open-source project doing exactly this on macOS. It exists. It works. That single
pointer broke my confirmation bias. I read its implementation, reproduced its
approach in a standalone spike with Claude, and the channel opened.

Diffing my spike against my app turned up three things:

1. The first openRFCOMMChannelAsync in a process always fails. It appears
to lazily initialise IOBluetooth's run-loop source, and the delegate never fires
for that attempt. A retry succeeds. My spike had accidentally made two throwaway
attempts against the phone before the one that worked — which is also why my
"control experiment" failed identically on both devices. It was measuring my own
bug twice and I read it as corroboration.

2. IOBluetooth delivers delegate callbacks on the run loop of the thread that
opened the channel — not on a dispatch queue. I was awaiting a Swift
continuation, which never sees them. You have to pump the run loop:

func pump(_ seconds: TimeInterval, until: () -> Bool = { false }) {
    let deadline = Date().addingTimeInterval(seconds)
    while Date() < deadline {
        RunLoop.current.run(until: Date().addingTimeInterval(0.05))
        if until() { return }
    }
}

This is also why CoreBluetooth worked fine for me all along while IOBluetooth
never did — CoreBluetooth uses a dispatch queue. I had that evidence and read it
backwards.

3. Never close a failed attempt. The close lands asynchronously and tears
down whichever channel succeeds next.

Reads work. Writes vanish.

With the channel open, 01:01 returns 103 bytes, and there's readable ASCII in
there straight away — 01.59, 1402, and the headset's own MAC. Firmware,
model, serial.

Writes were another matter. I sent what I was sure was the noise-cancelling
command and got nothing. No error. No NAK. No state change. The device
acknowledged reads perfectly and acted as though writes had never arrived.

I burned a lot of time guessing command bytes. What actually solved it was
capturing the Android app's own traffic: enable Bluetooth HCI snoop log in
developer options, drive the app, pull a bug report, and scan the capture for
the frame magic.

The capture showed a handshake the app performs on connect:

01:01                    device info
05:01  payload 01        capability table
05:81  (no payload) x2
05:81  payload 01
05:81  (no payload)
18:85  payload 01

Until that runs, the device discards every write in silence. That's not
documented anywhere I could find, and I don't think I'd ever have guessed it.

The same capture caught something subtler. The sound-mode write is 06:81 with
payload [mode, level, 02, 00, 00, 01]. I'd been building that payload by
reading the current block back and modifying it — but the read returns FF in
byte 2 where the write requires 02. Echoing back what you read gets you
silently ignored. That single byte was most of a day.

Mapping the rest by diffing

Once writes worked, the remaining fields fell out of a simple loop: hold the
channel open from the Mac, poll the state blob, change one setting on the phone,
and diff the bytes.

CHANGED 01:01
  byte[71] (0x47): 02 → 01

That's the ANC mode byte. Same method found the EQ preset at byte 23, the eight
band values at 25–32, and — after an embarrassing detour — the ANC strength in
the high nibble of byte 72, not byte 70 as I first assumed. Byte 70 is a
constant 05: the number of levels supported, not the level selected.

Battery is byte 0, and it reads 0–9 rather than a percentage. That's Apple's
accessory convention (AT+IPHONEACCEV): percent is (level + 1) × 10. I only
worked that out because the raw byte said 5 while both macOS and the phone app
said 60%.

The official app has a bug

While verifying the strength levels I found something I didn't expect.

I set level 4 from the Mac, reconnected the phone, and watched the exchange. The
app asked the headset for its sound mode. The headset replied truthfully:

11:46:10  app  → 06:01
11:46:10  dev  → 00 4F FF 00 00 01     (level 4)
          app displays: 2

It queries, receives the correct answer, and renders its own cached value
instead. So if you change anything from another host, Anker's app lies to you
about the current state. Mine reads it from the device.

Making it work for more than one model

The frame format is identical across Soundcore devices. What differs is where
fields sit in the state blob, which value means which sound mode, and how many
EQ bands there are. So a device is a data structure, not new protocol code:

static let space2 = DeviceProfile(
    modelCode: "1402", displayName: "Soundcore Space 2",
    batteryOffset: 0, ancModeOffset: 71, ancLevelOffset: 72,
    eqPresetOffset: 23, eqBandsStart: 25, eqBandCount: 8,
    features: [.battery, .soundMode, .ancLevel, .equaliser, .customEQ]
)

Unknown models resolve to a read-only profile: battery and firmware, no writes.
I own exactly one Soundcore device, so that's the only one whose offsets I've
confirmed. Shipping guessed offsets to someone else's headphones isn't a risk
worth taking for the sake of a longer compatibility table.

What I'd tell myself at the start

Verify by reading back, not by absence of an error. Every real bug in this
project was silent. The device never once told me I was wrong — it just ignored
me. confirmed: byte[71] = 0x00 is worth more than any number of successful
writes.

Check that your control experiment isn't measuring the same bug. Mine tested
a second device and gave me false confidence, because the defect was in the code
common to both paths.

Prior art is the cheapest debugging tool there is. I'd written off the whole
platform. One link to a project that already did it turned two days of wrong
conclusions around in an hour.

Ground your AI pairing in real packet captures. Claude excelled at spotting
byte patterns, generating Swift codecs, and writing offline tests, but only
because every prompt was grounded in real hex dumps and HCI logs. Without ground-truth
captures, pair-programming on hardware quickly turns into mutual guesswork.

SoundcoreBridge is MIT, Swift, macOS 13+. The full protocol map — state blob
layout, handshake, all 22 EQ presets with band values, and the IOBluetooth traps
— is in the repo, so if you'd rather write your own client than use mine, the
hard part is already done.

github.com/mervin008/soundcorebridge

Not affiliated with Anker or Soundcore. Firmware-update channels are blocked in
code; the protocol work is interoperability reverse engineering on hardware I own.

Ai apps at i3

DMJ Jones — Fri, 06 Dec 2024 15:16:30 +0000