DEV Community: RobustTrueTry

Running Local LLMs on Mac Mini: What Actually Breaks

RobustTrueTry — Tue, 01 Sep 2026 10:10:46 +0000

The Box Wasn't Meant For This

Apple's Mac Mini and Mac Studio have become the quiet workhorses of local AI. Demand has outpaced what the lineup was designed for, and people are loading these machines with 70B-class models that were never really part of the spec sheet. If you're running, or planning to run, large language models locally on Apple Silicon, here's what actually breaks in practice.

You'll learn:

How unified memory changes your model size budget compared to a discrete GPU
The thermal and power ceiling you hit before you hit compute
Why context length is the silent killer of local inference
A small, reproducible setup that lets you measure this on your own machine

The Memory Math Is Different

On a CUDA box, "VRAM" is a hard wall. If your model plus KV cache don't fit, you don't run it. On Apple Silicon, unified memory means the GPU and CPU share the same pool, which sounds generous. It is, until you remember that macOS, your IDE, your browser, and the OS cache all live in that same pool.

A rough rule of thumb:

For a Q4-quantized model, you need about 0.6 GB per billion parameters for the weights
The KV cache grows with 2 * num_layers * num_heads * head_dim * context_length * bytes_per_element
Leave at least 8-10 GB for the OS, or you'll start swapping to SSD and lose the speed advantage

So a 32B Q4 model at ~19 GB weights plus a 16K context can easily push past 28 GB. That fits on a 64 GB Mac Studio comfortably, and it fits on a 32 GB Mac Mini on paper, but in practice you'll feel the squeeze every time you open Chrome.

Thermals: The Ceiling Nobody Talks About

The Mac Mini is a sealed brick. The Mac Studio has a real cooler. Both will throttle under sustained inference. The first sign isn't a fan curve, it's tokens-per-second dropping by 20-30% after ten minutes.

You can watch it happen:


## In one terminal, start your model server (ollama, llama.cpp, etc.)

ollama run llama3.1:70b-instruct-q4_K_M

## In another, sample power and thermals

while true; do
  sudo powermetrics -s cpu_power,gpu_power -n 1 2>/dev/null | \
    grep -E "(GPU Power|CPU Power)" | head -2
  sleep 5
done

If GPU power trends downward while you're still streaming tokens, you're thermally limited. The fix isn't a faster box, it's a smaller model or a shorter context.

Context Length Is the Silent Killer

Most local stacks handle 4K or 8K contexts fine. Push to 32K or 64K and you'll see two things go wrong:

First-token latency climbs because the entire prompt has to be prefilled into the KV cache.
Tokens per second during generation drops because the attention compute scales with context length.

You can measure both with a tiny script using llama-cpp-python:

import time
from llama_cpp import Llama

llm = Llama(
    model_path="./models/llama-3.1-8b-instruct-q4_k_m.gguf",
    n_ctx=32768,
    n_gpu_layers=99,
)

prompts = ["Summarize: " + "lorem ipsum " * n for n in (512, 2048, 8192, 16384)]

for p in prompts:
    t0 = time.perf_counter()
    out = llm(p, max_tokens=128)
    dt = time.perf_counter() - t0
    gen = out["usage"]["completion_tokens"]
    print(f"ctx={len(p):>6}  total={dt:.2f}s  ttft-ish+gen={gen} tokens")

Run it. You'll see the cliff. On a 32 GB Mac Mini, expect generation throughput to roughly halve between 8K and 32K context for an 8B model.

When the Mac Mini Stops Being the Right Tool

The honest answer: the Mac Mini is excellent for 7B-13B models at modest context, and surprisingly capable at 30B-40B if you're patient. Beyond that, the Mac Studio with more memory and a bigger cooler is the meaningful upgrade, not a Mac Pro. The Pro adds bandwidth and cores but, for LLM inference, what matters is memory capacity and sustained thermals.

If you're shopping for a box specifically for local AI:

Priority	Pick	Why
7B-13B daily driver	Mac Mini, 32GB	Quiet, cheap, fast enough
30B-70B experiments	Mac Studio, 64GB+	Memory headroom and thermals
Long-context work (64K+)	Mac Studio, 96GB+	KV cache lives in RAM
Multi-model serving	Mac Studio, 128GB	Run two models side by side

Key Takeaways

Unified memory is generous but not infinite. Budget for the OS and your daily apps before the model.
Thermals throttle sustained inference. Measure power draw, not just tokens per second.
Context length eats throughput. Profile before you commit to a 32K workflow.
More memory beats more cores for local LLMs. Pick the bigger-RAM SKU, not the faster one.
The Mac Mini is the value play, the Mac Studio is the actual local-AI machine. Know which one you need before you buy.

Source

Apple caught off guard by AI demand for Mac Mini and Mac Studio. The source covers the demand surprise; this article adds the practitioner-side failure modes, the measurement script, and a sizing guide the original does not include.

JavaFX Gantt Charts Without the Memory Wall

RobustTrueTry — Mon, 31 Aug 2026 05:38:29 +0000

The Problem With Drawing 10,000 Bars

If you've ever tried to render a real project schedule in JavaFX, you've met the wall. A Gantt chart isn't a dashboard with twelve widgets. It's a long horizontal canvas where every day is a column and every row is a task. Once you cross a few thousand bars, the scene graph starts to choke. Each task is a Node, each Node carries layout, each layout triggers a relayout, and your scrollbar becomes a slideshow.

Dirk Lemmerman's FlexGanttFX is now open source under the AGPL, and the interesting part isn't the Gantt features. It's the choice he made to stop using Nodes for the bars and draw them on a Canvas instead. That single decision is worth understanding before you reach for the library.

What you'll learn:

Why the standard JavaFX scene graph hurts on timeline UIs
How Canvas-based rendering flips the tradeoff
A minimal FlexGanttFX setup you can run today
Where Canvas rendering quietly breaks, and what to do about it

The Scene Graph Tax

Every Rectangle you place in a JavaFX scene is a participant in layout, CSS, and event dispatch. The runtime treats it as a first-class citizen. That's lovely for a button. It's expensive for ten thousand task bars in a row.

Concretely, the costs you pay are:

Layout passes that walk the parent chain on every change
CSS resolution per node, which scales with node count
Pick-to-hit testing during mouse events that touches every node under the cursor
Memory overhead per node, including transform and effect state

For a Gantt chart, none of this matters. The user doesn't style each bar independently. They don't hit-test individual pixels. They want to scroll fast and click on a row.

What FlexGanttFX Does Differently

FlexGanttFX renders the timeline grid and the bars on a JavaFX Canvas. A Canvas is a single Node backed by a GraphicsContext. You draw into it with imperative calls, the way you would with a 2D graphics API. The scene graph sees one Node, not ten thousand.

The library still uses real Nodes for things users actually interact with: the row header, the dependency arrows, the editing handles. The heavy, repetitive content (the grid lines and the bar fills) lives on the canvas. That split is the trick.

A Minimal Setup You Can Run

FlexGanttFX is published on Maven Central. Add it to a Gradle project:

dependencies {
    implementation 'com.dlsc.flexganttfx:flexganttfx:1.0'
}

Drop a GanttChart into a BorderPane and feed it some tasks:

GanttChart<LocalDate> chart = new GanttChart<>();

Task<LocalDate> design = new Task<>("Design",
        LocalDate.of(2026, 3, 1), LocalDate.of(2026, 3, 14));
Task<LocalDate> build = new Task<>("Build",
        LocalDate.of(2026, 3, 8), LocalDate.of(2026, 3, 28));
build.getDependencies().add(design);

chart.getTasks().addAll(design, build);

BorderPane root = new BorderPane(chart);
Scene scene = new Scene(root, 1000, 600);
primaryStage.setScene(scene);
primaryStage.show();

That code gives you a working Gantt with a dependency arrow from Design to Build. Editing handles appear when you select a task. The grid and the bar fills come from the canvas, so scrolling stays smooth into the thousands.

Tradeoffs Worth Knowing

Canvas buys you speed. It costs you a few things the scene graph gives you for free.

No accessibility tree for the drawn content. Screen readers see the Canvas as one opaque region. Expose row data through your own accessible properties if that matters.
No built-in hover effects per bar. You draw pixels, not Nodes. If you want a tooltip on hover, you handle mouse events yourself and map coordinates back to the task under the cursor.
Redraws are manual. Change a task's start date and the canvas won't repaint on its own. Call chart.requestLayout() or trigger the repaint hook the library exposes.
CSS theming is limited. You can theme the Nodes (headers, handles, arrows) with CSS. The canvas-drawn bars follow a styled object you configure in code.

If your chart is small (under a few hundred bars), the standard scene-graph approach is honestly fine, and you get the accessibility and theming for free. FlexGanttFX pays off when your schedule outgrows a single screen of widgets.

When Canvas Rendering Quietly Breaks

There are two failure modes worth knowing before you ship.

The first is device pixel ratio. A Canvas drawn at logical coordinates looks blurry on high-DPI screens unless you size the underlying image buffer to match the screen's pixel scale. FlexGanttFX handles this for you, but if you fork the rendering code, copy the same scaling pattern.

The second is selection feedback during drag. When a user drags a bar to reschedule it, the canvas needs to repaint fast enough to follow the cursor. If your repaint work includes more than the bar's old and new rectangles (say, dependency arrows that need to recompute), you'll see lag. The fix is to limit the drag repaint to the bar itself and recompute arrows on drop.

Key Takeaways

JavaFX's scene graph is the wrong tool for thousands of repetitive, non-interactive shapes like Gantt bars.
FlexGanttFX splits the problem: canvas for the heavy visual content, Nodes for the parts users touch.
The library is AGPL, which matters if you plan to ship a closed-source product on top of it.
You give up per-bar accessibility and CSS theming in exchange for smooth scrolling at scale.
Repaint logic becomes your responsibility once you start customizing the canvas.

Source

FlexGanttFX is Open Source — covered the release and feature set. This article added a minimal Gradle setup, a discussion of the scene-graph vs Canvas tradeoff, and the failure modes (DPI scaling, drag repaint) that the news write-up did not address.

When Your Self-Hosted Nitter Instance Gets Banned

RobustTrueTry — Thu, 27 Aug 2026 09:53:14 +0000

Self-hosting a Nitter instance feels like a small weekend win, until the first time it silently stops working. The earlier guide walked through picking an instance or running your own. This one is about the week after: the failure modes nobody warns you about, and how to keep a public instance alive long enough to be useful.

By the end of this piece you'll have a content-verifying health check, a list of the bans and rate limits that actually happen, and a clear answer to the question I get most: should I share my instance publicly or keep it private?

The Ban That Doesn't Look Like a Ban

Your instance is up. The web UI loads. You click a profile and see a 200 OK. Then nothing renders. The log shows the upstream Twitter request returned 200 with an empty body, or worse, a 200 with a stubbed HTML page that contains no tweets.

Twitter doesn't send a hard 403 when it fingerprints your server. It sends a successful response that contains nothing useful. The scraper thinks the request worked, so no alert fires. You notice three days later when a friend asks why the instance is empty.

The fix is to verify response content, not status codes. Parse the body and check that the expected selectors (.timeline-item, data-item-id, or your Nitter fork's equivalent) are present and non-empty.

What an Honest Health Check Looks Like

Most "is it working?" scripts check the wrong thing. Here's a Python check that fetches a known active profile and confirms the body actually contains a recent tweet.

import requests
from datetime import datetime, timedelta

INSTANCE = "https://your.nitter.example"
PROBE_ACCOUNT = "NASA"

def instance_health(instance: str) -> dict:
    try:
        r = requests.get(
            f"{instance}/{PROBE_ACCOUNT}",
            timeout=15,
            headers={"User-Agent": "Mozilla/5.0 (compatible; healthcheck/1.0)"},
        )
    except requests.RequestException as e:
        return {"ok": False, "reason": f"network: {e.__class__.__name__}"}

    if r.status_code != 200:
        return {"ok": False, "reason": f"http {r.status_code}"}

    # Heuristic: Nitter renders the timeline into a div we can count.
    # If the body is suspiciously short or contains no item cards, it's a soft ban.
    if len(r.text) < 5000 or 'class="timeline-item"' not in r.text:
        return {"ok": False, "reason": "empty body (possible soft ban)"}

    return {"ok": True, "reason": "timeline present", "bytes": len(r.text)}

if __name__ == "__main__":
    print(datetime.utcnow().isoformat(), instance_health(INSTANCE))

The script probes a known-active account (NASA is a safe pick because it posts often) and rejects any response shorter than 5KB or missing the timeline-item class. The threshold is heuristic, so tune it to your fork's actual HTML.

Run this every 5 minutes from a different host than your Nitter box. The "from a different host" part matters: if your server's IP is already shadow-banned, the check from itself will lie to you in exactly the same way the scraper lies to you. A $4 VPS from a different provider, running a cron job that posts to a healthchecks.io ping or a Discord webhook, is enough.

Wiring It Into systemd

A script that only runs when you remember to start it is no better than the silence it replaces. Drop a small unit file in /etc/systemd/system/nitter-healthcheck.service and a matching timer so it runs on a schedule.

[Unit]
Description=Nitter instance health check
After=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/bin/python3 /opt/nitter-healthcheck/healthcheck.py

[Unit]
Description=Run Nitter health check every 5 minutes

[Timer]
OnCalendar=*:0/5
Persistent=true

[Install]
WantedBy=timers.target

Then systemctl daemon-reload && systemctl enable --now nitter-healthcheck.timer. The Persistent=true line catches up on missed runs after a reboot, which is the kind of detail that separates a toy cron from something you can trust at 3am.

The IP Rotation Trap

The natural next step is to rotate IPs so a single ban doesn't take you out. In practice this is where most self-hosters either overspend or get themselves in trouble.

A cheap residential proxy pool will get your entire /24 flagged within a week. Twitter fingerprints ASN (the autonomous system number that identifies your network provider) and IP reputation, not just the address. Rotating through 50 compromised residential IPs makes you look more like a botnet than a single user. You get banned faster, not slower.

What actually works is fewer, cleaner IPs used conservatively. One or two VPS endpoints from a reputable provider, with request spacing of 2 to 5 seconds, will last months. If you need more, add a second VPS and run two Nitter instances against it, not a proxy pool.

Approach	Cost	Longevity	Risk
Single VPS, conservative rate	Low	Months	Low
Two VPS, no proxy	Medium	Longer	Low
Residential proxy pool	Medium-high	Days	High, may flag you as abuse
Datacenter proxy rotation	Low	Hours to days	Very high

Public vs. Private Instance: The Real Tradeoff

A public Nitter instance on a public list gets discovered, scraped, and burned out within weeks. That is the consistent pattern reported by operators of the popular community instances. The same instance, kept off any list and shared only with people you trust, lasts much longer.

The counter-argument is that a private instance is barely a community resource. That's a fair point. The pragmatic middle is: keep your stable instance private, and treat any public-facing mirror as disposable. Rebuild it from an image when it dies, don't try to nurse it back.

A useful workflow is to maintain a docker-compose.yml and a small Makefile with up, down, rebuild, and rotate-token targets. When the instance gets soft-banned, you run make rebuild against a fresh server in a different region, rotate the token that some Nitter forks require, and update your DNS.

Caching the Right Way

Nitter's built-in cache helps with the front-end but doesn't save you from upstream bans. If your instance is for personal reading, set the cache TTL (time-to-live, how long a cached page is reused before refetching) to 10 to 15 minutes for timelines and longer for profile metadata. This cuts your outbound request volume by an order of magnitude and is the single biggest lever you have for staying under the rate limit radar.

If you're building anything on top of Nitter (an RSS feed for your reader, a Telegram bot, a digest email), cache aggressively on your side too. Treat the Nitter instance as an expensive, flaky database, not as a free API. Every request you skip is a request that can't get you banned.

When to Stop Self-Hosting

There is a point at which the right answer is to stop. If you need real-time access to many accounts, if you need historical search, or if a single missed tweet has actual consequences for you, no Nitter instance is the right tool. The costs of keeping it alive start to exceed the cost of just using X with a burner account, or paying for an API tier that still does not exist for most use cases.

Self-hosting Nitter is the right call when you want a low-volume, private, read-only window into a handful of accounts. It is the wrong call when you want a Twitter replacement.

Key Takeaways

A 200 OK with an empty body is a soft ban. Health checks must verify content, not status.
Run health checks from a different host than your Nitter instance, or the check inherits the same ban.
Residential proxy pools get you flagged faster, not slower. One or two clean VPS IPs, conservatively used, outlast them.
Public instances on public lists get burned out in weeks. Private instances last much longer.
Cache on both sides of the proxy. Every skipped request is a request that can't ban you.

Source

Reading Twitter Without X: A Practical Guide to Nitter Alternatives in 2024 — that earlier post covered choosing or running an instance. This one adds the production-side reality: soft bans, honest health checks, why proxy rotation usually backfires, and when the right answer is to stop self-hosting. The new material here is the content-verifying health check, the systemd timer wiring, and the explicit "different host" rule for probes.

Reading Twitter Without X: A Practical Guide to Nitter Alternatives in 2024

RobustTrueTry — Wed, 26 Aug 2026 02:04:29 +0000

The Front-End You Use Today May Not Exist Tomorrow

Nitter and XCancel, the two most popular third-party front-ends for reading X (formerly Twitter) without an account, recently received cease and desist notices. If your morning routine, a research pipeline, or a small monitoring script depends on scraping a nitter.net-style URL, that link is now a 503 page. The good news is the model is simple enough that you can replace it in an afternoon. The bad news is that any public instance you don't control is a risk on the same axis as the one that just broke.

In this article I'll cover what these front-ends actually do, three ways to keep reading X without an account, and the failure modes that bite you in each one.

What Nitter Was, and Why It Broke

Nitter is a reverse proxy (a server that fetches pages on your behalf and rewrites them) that sits between your browser and X. It pulls the public timeline, strips the JavaScript and tracking, and serves you plain HTML over a lightweight page. No login, no ads, no algorithm. XCancel is the same idea, run as a public service by a different maintainer.

Both worked by hitting X's unauthenticated web endpoints, parsing the JSON, and rendering the result. The cease and desist isn't about a specific bug. It's about the legal exposure of running a service that lets people consume X's content without seeing X's ads or signing X's terms of service. The community discussion makes that clear, as one maintainer put it, the projects are "paused while we figure out the legal situation."

The Three Paths Forward

You have three practical options, in order of how much they cost you in time and infrastructure.

Path 1: Use a Different Public Instance

A handful of community-run instances still operate. The risk is the same legal pressure that took down Nitter and XCancel. Treat any public instance as disposable.

A small Python helper that tries several mirrors and falls back gracefully:


## fetch.py — try a list of public Nitter-style mirrors

import sys
import httpx

MIRRORS = [
    "https://nitter.privacydev.net",
    "https://nitter.poast.org",
    "https://nitter.1d4.us",
]

def fetch_user_timeline(handle: str) -> str:
    last_error = None
    for base in MIRRORS:
        url = f"{base}/{handle}"
        try:
            r = httpx.get(url, timeout=10.0, follow_redirects=True)
            r.raise_for_status()
            return r.text
        except httpx.HTTPError as e:
            last_error = e
            continue
    raise RuntimeError(f"All mirrors failed: {last_error}")

if __name__ == "__main__":
    handle = sys.argv[1] if len(sys.argv) > 1 else "github"
    print(fetch_user_timeline(handle)[:500])

Run it with python fetch.py github. The first mirror that returns 200 wins. Anything that times out or returns 403 is logged and skipped, so a single takedown doesn't take your script with it.

The tradeoff: you don't control uptime, you don't control the HTML format, and you may be sending the handles you care about to a server you don't know. For a personal reading list that's fine. For a production pipeline, it isn't.

Path 2: Run Your Own Nitter Instance

Nitter is still open source. You can self-host it. This is the most resilient option, because nobody can send your server a legal notice.

The simplest deployment uses Docker Compose. A working compose.yaml:


## compose.yaml — self-hosted Nitter

services:
  nitter:
    image: zedeus/nitter:latest
    restart: unless-stopped
    ports:
      - "8080:8080"
    volumes:
      - ./nitter.conf:/src/nitter.conf:ro
    depends_on:
      - redis

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    volumes:
      - ./redis-data:/data

You'll also need a nitter.conf with at least these fields filled in:


## nitter.conf — minimum viable config

[Server]
hostname = "0.0.0.0"
port = 8080

[Cache]
listTTL = 600
rssTTL = 600

[Config]
token = "REPLACE_ME"  # from your X dev account, if using the API path
hmacKey = "REPLACE_ME"  # random 32+ char string

Start it with docker compose up -d, then open http://localhost:8080. If you're not behind a reverse proxy (a server like nginx that sits in front of Nitter to handle TLS and caching), the default hmacKey is the only one that matters. The token is only required if you've switched Nitter to the official X API path, which most self-hosters haven't, because the API path has its own auth and rate-limit story.

The failure mode here is the scraping path itself. Nitter's default config scrapes x.com, and X has been tightening what unauthenticated clients can fetch for years. You may find that a user who was readable yesterday returns empty today, with no error in your logs. The fix is usually a Nitter upgrade, not a config change.

Path 3: Use the Official API With a Budget

If you're reading your own timeline, or a small fixed list of accounts, the official X API is the boring choice that just works. It costs money, the pricing page changes often enough that I won't quote numbers here, and you'll need to check the current docs for what your use case qualifies for.

The shape of the code is what matters, not the price. A small reader using the official client:


## reader.py — official API, bearer token auth

import os
import tweepy

bearer = os.environ["X_BEARER_TOKEN"]
client = tweepy.Client(bearer_token=bearer)

def recent(handle: str, limit: int = 10):
    user = client.get_user(username=handle)
    if not user.data:
        return []
    resp = client.get_users_tweets(
        user.data.id,
        max_results=min(max(limit, 5), 100),
        tweet_fields=["created_at", "public_metrics"],
    )
    return resp.data or []

for t in recent("github"):
    print(t.created_at, t.text[:80])

The tradeoffs: the API is rate-limited per endpoint, some fields require a higher tier than others, and you give up the privacy property of Nitter because your reads are tied to an authenticated account. For personal use that's a fair price. For anything that needs to look like "anonymous reads of public timelines," it's not the right tool.

A Quick Comparison of the Three Paths

Approach	Cost	Privacy	Resilience to Takedowns	Maintenance
Public instance	Free	Low (you expose handles to a stranger)	Low — same legal exposure as Nitter	None
Self-hosted Nitter	A small VPS (a low-cost virtual private server)	High	High (you own the deployment)	Medium — scraper breaks when X changes HTML
Official API	Pay per use, check current pricing	Low (authenticated to your account)	High (it's the supported path)	Low

What Will Probably Go Wrong

A few failure modes I've seen in practice, on every path:

Rate limits that look like 404s. Public mirrors return a generic error page when X rate-limits the scraper. Your script will think the user doesn't exist. Check the response body length before treating a 200 as success.
HTML format drift. Nitter's selector logic (CSS selectors that target specific page elements) breaks when X ships a redesign. Self-hosters see this as a sudden "no tweets returned" with no error. Pin your Nitter version and watch release notes.
The instance you picked yesterday is gone today. This is the lesson of the cease and desist itself. If your code hard-codes one host, you'll have a 2 a.m. page. Keep a list and rotate.
Auth tokens that quietly expire. The official API bearer token can stop working for reasons the dashboard doesn't surface well. Add a 30-second self-check on startup that hits a known endpoint, so you find out at deploy time, not at 3 a.m.

Key Takeaways

A public instance is a stopgap, not infrastructure. Treat it as disposable from day one.
Self-hosting Nitter is a weekend project if you already run Docker, and it's the only option that gives you the original privacy story.
The official API is the right call when you're reading a small, known set of accounts and can pay per request.
Whichever path you pick, write your code so the front-end is one URL constant. The day you have to swap is the day you don't want to be grepping the codebase.
Don't trust any single mirror, self-host, or API key to be there in six months. Build the fallback in now, while you have the time.

Source

Nitter and XCancel receive cease and desist notices — the original community discussion. This article adds a runnable mirror-fallback script, a Docker Compose setup for self-hosting, an official API example using tweepy, and a failure-mode checklist for each of the three replacement paths.

Recover a Bricked Framework 13 with a DIY USB Flash

RobustTrueTry — Wed, 19 Aug 2026 01:57:45 +0000

Your Framework 13 shows only a black screen after a failed BIOS update. You need a way to bring it back without sending it to the repair shop.

What you'll learn

Create a recovery USB that works on any host OS.
Flash the BIOS safely using a script.
Diagnose why the laptop bricked and avoid repeat failures.

Prepare a Recovery USB

You need a USB drive with the Framework recovery firmware image. The following bash script creates a bootable USB on Linux. It uses parted to format the drive and dd to write the image.

#!/usr/bin/env bash

## create_recovery_usb.sh – make a Framework recovery USB

set -euo pipefail

if [[ $# -ne 2 ]]; then
  echo "Usage: $0 <device> <firmware.img>"
  exit 1
fi

device=$1
image=$2

## warn user

echo "WARNING: $device will be erased."
read -p "Confirm? (y/N) " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
  exit 1
fi

## zero the first 1 MiB to clear existing partition table

parted -s "$device" mklabel gpt
parted -s "$device" mkpart ESP fat32 1MiB 100MiB
parted -s "$device" set 1 esp on

## format the partition

mkfs.fat -F 32 "${device}1"

## write the firmware image

dd if="$image" of="${device}1" bs=4M status=progress

echo "Recovery USB ready on $device"

The script first clears the partition table, creates a single FAT32 partition marked as ESP, formats it, and copies the firmware image. Using a single partition reduces the chance of mount points interfering with the flash process.

Flash the BIOS with a Script

Once the USB is ready, you can run the BIOS flash from the laptop itself. The following Python script checks for the presence of the recovery partition, then calls fwup to apply the update. It also logs each step for debugging.

#!/usr/bin/env python3

## flash_framework_bios.py – safe BIOS flash using fwup

import subprocess
import sys
import os
import logging

logging.basicConfig(level=logging.INFO, format='%(asctime)s %(levelname)s %(message)s')
logger = logging.getLogger(__name__)

def run(cmd, check=True):
    logger.info("Running: %s", ' '.join(cmd))
    result = subprocess.run(cmd, capture_output=True, text=True)
    if check and result.returncode != 0:
        logger.error("Command failed: %s", result.stderr)
        sys.exit(1)
    return result

def main():
    # assume recovery USB is mounted at /media/framework/recovery
    usb_path = '/media/framework/recovery'
    if not os.path.isdir(usb_path):
        logger.error("Recovery USB not found at %s", usb_path)
        sys.exit(1)

    # locate the firmware file (named firmware.bin inside the ESP)
    firmware = os.path.join(usb_path, 'firmware.bin')
    if not os.path.isfile(firmware):
        logger.error("firmware.bin missing on USB")
        sys.exit(1)

    # run fwup with the image
    run(['fwup', '-i', firmware, '-t', 'run'])
    logger.info("BIOS flash completed")

if __name__ == '__main__':
    main()

The script validates the USB mount, finds the firmware file, and invokes fwup. It exits on any error, preventing a partial flash that could leave the laptop unusable.

Verify the Fix

After the flash finishes, remove the USB and power the laptop. You should see the Framework logo and the OS boot sequence. If the screen stays black, check the battery with a simple power‑draw test:


## check battery health (requires lm-sensors)

sensors-detect && sensors

A dead battery can mimic a bricked state. Connect the charger and see if the LED lights up.

Compare Recovery Approaches

Approach	Tradeoffs	When to Use
Official Framework Recovery Tool	Easiest, but requires a Windows/macOS host and the proprietary tool.	You have a spare Windows machine and want minimal risk.
Third‑party BIOS flash utility	Faster on Linux, but may lack official support and could be less reliable.	You are comfortable with command line and need a quick fix.
Manual dd + USB method	Full control, works on any host OS, but you must handle partition tables correctly.	You need a portable solution and are willing to follow a script.

Common Failure Modes

Power interruption during the flash. The USB must stay powered; a laptop battery drain can corrupt the firmware.
Using the wrong firmware image. Verify the image matches your model number before writing.
Corrupt partition table on the USB. The script overwrites the table, but a pre‑existing layout can cause mount issues.
Incompatible fwup version. Ensure you have the version that matches the firmware spec.

Key Takeaways

A clean ESP partition on the USB reduces flash errors. Use a script to automate formatting and image copy.
Validate the firmware file before running any flash utility; a mismatch is the most common cause of a bricked laptop.
Keep a backup of the working BIOS somewhere safe; you can restore it with the same USB if something goes wrong.
Test power and battery health after the flash; a dead battery can look like a bricked system.

Source

Fixing a bricked Framework laptop

I added a step‑by‑step script to create a recovery USB, a Python wrapper for the flash process, and a comparison table of recovery options.

Stop AI Hallucinations: Verify Your LLM Summaries Before Publishing

RobustTrueTry — Tue, 18 Aug 2026 01:59:33 +0000

You’ve probably seen headlines about AI summarizers that miss key facts or invent details. That’s the AI‑DR problem—models that claim to read but don’t. It can cost you credibility and time.

What You’ll Learn

Add a verification step after summarization.
Compare different verification strategies.
Spot common failure modes and how to mitigate them.

Why Verification Matters

When an LLM produces a summary, it can hallucinate facts that aren’t in the source. Those hallucinations can mislead users or propagate misinformation. Adding a verification step lets you catch those errors before they reach the audience.

Choose a Verification Strategy

You have three main options: a simple embedding similarity check, a retrieval‑based fact check, or a human‑in‑the‑loop review. Each has its own cost, latency, and accuracy profile. The choice depends on how critical the content is and how much automation you can afford.

Embedding Similarity

Embedding similarity is the lightest option. It turns both the source and the summary into vectors and compares them with cosine similarity. If the similarity falls below a threshold, you flag the summary.

import numpy as np
import openai

## Convert text to a vector using a small embedding model

def embed(text, model="text-embedding-3-small"):
    resp = openai.Embedding.create(input=text, model=model)
    return np.array(resp.data[0].embedding)

## Compare two vectors and decide if the summary is close enough

def verify(summary, source, threshold=0.75):
    src_vec = embed(source)
    sum_vec = embed(summary)
    similarity = np.dot(src_vec, sum_vec) / (np.linalg.norm(src_vec) * np.linalg.norm(sum_vec))
    return similarity >= threshold

The code is short and uses only the OpenAI API. It works with any LLM that can produce a summary.

Retrieval‑Based Fact Check

If you need higher precision, you can retrieve the exact sentences that support each claim. The LLM is asked to list the source sentences it used, and you compare that list to the original text.

import openai

## Ask the model to list supporting sentences

def get_supporting_sentences(summary, source, model="gpt-4o-mini"):
    prompt = (
        "Given the following summary, list the exact sentences from the source that support each bullet point."
        f"\n\nSummary:\n{summary}\n\nSource:\n{source}"
    )
    response = openai.ChatCompletion.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    return response.choices[0].message.content.strip()

You then parse the returned sentences and check that each appears verbatim in the source. This method is more expensive but catches subtle hallucinations.

Build the Pipeline

Below is a minimal, end‑to‑end pipeline that stitches the steps together.

Step 1: Fetch and Clean the Source

import requests
from bs4 import BeautifulSoup

## Grab the main article body from a URL

def fetch_text(url):
    resp = requests.get(url)
    resp.raise_for_status()
    soup = BeautifulSoup(resp.text, "html.parser")
    # Very naive extraction: grab all paragraph tags
    paragraphs = soup.find_all("p")
    return "\n".join(p.get_text() for p in paragraphs)

In production you might use a dedicated article extractor, but this keeps the example focused.

Step 2: Summarize with an LLM

import openai

## Ask the model to produce a concise, bullet‑point summary

def summarize(text, model="gpt-4o-mini"):
    prompt = (
        "Summarize the following text in 3–5 bullet points. Only include facts that appear in the source."
        f"\n\n{text}"
    )
    response = openai.ChatCompletion.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    return response.choices[0].message.content.strip()

The prompt explicitly asks for factuality, which helps reduce hallucinations.

Step 3: Verify the Summary

You can plug either verification method here. For brevity we’ll use the embedding similarity check.


## Reuse the verify() function from the Embedding Similarity section

def process(url):
    source = fetch_text(url)
    summary = summarize(source)
    if verify(summary, source):
        print("✅ Summary approved")
    else:
        print("❌ Summary flagged for review")

Step 4: Decision Logic

If verify() returns True, you can publish the summary. If it returns False, you have three options:

Re‑run the summarization with a stricter prompt.
Send the summary to a human reviewer.
Log the failure for future analysis.

Tradeoffs Between Approaches

Approach	Cost	Latency	Hallucination Risk	Human Effort
Embedding Similarity	Low	Fast	Medium	None
Retrieval‑Based Fact Check	Medium	Medium	Low	None
Human Review	High	Slow	Very Low	High

The table shows that the embedding check is the cheapest and fastest, but it may miss subtle errors. Retrieval‑based checks are more accurate but cost more. Human review is the most reliable but also the slowest.

Common Failure Modes

Embedding drift: The embedding model may not capture subtle differences, leading to false positives.
Threshold mis‑tuning: A too‑high threshold rejects good summaries; a too‑low threshold lets hallucinations slip.
Prompt leakage: If the verification prompt is too similar to the summarization prompt, the model may repeat hallucinations.
Source noise: Web pages with ads or commentary can confuse the summarizer.

Tuning the Verification

Start with a threshold of 0.75 and adjust based on observed false‑positive/false‑negative rates.
Use a diverse set of test documents to calibrate the threshold.
Add a small “source‑check” prompt that asks the model to list the exact sentences it used.
Cache embeddings for repeated documents to reduce cost.

Key Takeaways

Adding a verification step dramatically reduces hallucinations.
Embedding‑based similarity is a lightweight, model‑agnostic check.
Retrieval‑based fact checks offer higher precision at a higher cost.
Tune the similarity threshold to balance cost and accuracy.
Keep an eye on failure modes and iterate on prompts and thresholds.

Source

AI;DR (AI; Didn't Read) – I added code, a verification strategy, tradeoff analysis, and failure‑mode discussion.

Build a Fast, Local LLM Inference Pipeline with DeepSeek Harness

RobustTrueTry — Fri, 14 Aug 2026 00:04:24 +0000

Hook

Running large language models on your own hardware can cut cloud costs and give you full control over latency. Many developers still rely on paid APIs, but a local solution is possible with the new DeepSeek Harness preview.

What You’ll Learn

How to install and launch a local DeepSeek model.
How to build a lightweight request router in Python.
Common failure modes and how to mitigate them.

What DeepSeek Harness Gives You

DeepSeek Harness is a lightweight wrapper that turns a DeepSeek model into a RESTful service. It exposes the same API surface as the cloud endpoint, so your existing code can stay unchanged. The preview version runs on a single GPU and supports batching.

Setting Up the Environment

Install the harness package from GitHub.
Pull the desired model checkpoint.
Start the server with a single command.


## Install the harness library

pip install git+https://github.com/deepseek-ai/deepseek-harness.git

## Pull the model checkpoint (replace <model> with the name you want)

deepseek-harness pull <model>

## Start the inference server on port 8000

deepseek-harness serve --model <model> --port 8000

The serve command starts a FastAPI app that listens on the specified port. It automatically loads the model into GPU memory and keeps it ready for requests.

Running a Local Inference Server

The server exposes a /v1/chat/completions endpoint that accepts the same JSON payload as the cloud API. A minimal example of sending a request:

import requests

url = "http://localhost:8000/v1/chat/completions"
payload = {
    "model": "deepseek-chat",
    "messages": [
        {"role": "user", "content": "Hello, world!"}
    ],
    "max_tokens": 50
}

response = requests.post(url, json=payload)
print(response.json()["choices"][0]["message"]["content"])

This code uses the requests library to talk to the local server. The response format matches the cloud API, so you can swap endpoints without changing your logic.

Routing Requests with a Simple Proxy

If you want to keep the cloud endpoint for fallback or load balancing, you can write a tiny proxy that forwards to the local server first and only hits the cloud if the local one fails.

import requests
from fastapi import FastAPI, Request, HTTPException

app = FastAPI()
LOCAL_URL = "http://localhost:8000/v1/chat/completions"
CLOUD_URL = "https://api.deepseek.com/v1/chat/completions"

@app.post("/v1/chat/completions")
async def proxy(request: Request):
    payload = await request.json()
    try:
        r = requests.post(LOCAL_URL, json=payload, timeout=5)
        r.raise_for_status()
        return r.json()
    except Exception:
        # Fallback to cloud
        r = requests.post(CLOUD_URL, json=payload)
        r.raise_for_status()
        return r.json()

The proxy uses a 5‑second timeout to avoid hanging on a stuck local model. If the local request fails, it falls back to the cloud.

Handling Common Failure Modes

Failure	Symptom	Mitigation
GPU out of memory	Server crashes or refuses new requests	Reduce batch size or use a smaller model
Model load takes too long	First request is slow	Pre‑warm the server by sending a dummy request after startup
Network hiccup to local server	Timeout or connection error	Add retry logic with exponential backoff
API key missing for cloud fallback	401 error	Store the key securely and validate before sending

Implementing retries is straightforward with the tenacity library:

from tenacity import retry, wait_exponential, stop_after_attempt

@retry(wait=wait_exponential(multiplier=1, min=2, max=10), stop=stop_after_attempt(3))
def call_local(payload):
    r = requests.post(LOCAL_URL, json=payload, timeout=5)
    r.raise_for_status()
    return r.json()

This decorator retries up to three times, waiting longer between attempts.

Comparing to Other Local LLM Options

Option	Setup Complexity	GPU Requirement	API Compatibility
DeepSeek Harness	Low – one command to serve	4‑GB GPU	Full cloud API surface
Hugging Face Inference API (local)	Medium – install `transformers` and `torch`	8‑GB GPU	Similar, but requires custom code
OpenAI’s `openai` package with local `llama.cpp`	High – compile C++ and wrap	2‑GB GPU	Different API, needs adapters

DeepSeek Harness wins on simplicity and API parity. If you need a different model family, Hugging Face may be better, but you’ll pay more in setup time.

Key Takeaways

DeepSeek Harness turns a local model into a drop‑in replacement for the cloud API.
A lightweight FastAPI proxy can provide graceful fallback to the cloud.
Common failures are GPU memory, slow first request, and network hiccups; each has a simple mitigation.
Compared to other local solutions, Harness offers the lowest friction for developers.

Source

DeepSeek Harness developer preview. I added step‑by‑step setup, a proxy example, failure handling, and a comparison table that the original article omitted.

Deploying Meta’s Llama 3 in a Production Python Service

RobustTrueTry — Tue, 11 Aug 2026 11:09:16 +0000

You’re building a chatbot and you want to avoid vendor lock‑in. Meta’s new open Llama 3 lets you host the model yourself, but you need to know how to do it right.

What you’ll learn

How to load Llama 3 with Hugging Face Transformers.
How to expose the model via a lightweight Flask API.
When to use TorchServe for scaling.
Trade‑offs between open and closed models.
Common failure modes and how to guard against them.

Load Llama 3 with Hugging Face Transformers

The first step is to pull the model from Hugging Face and move it to the device you’ll run it on. The code below shows the minimal setup.


## load_llama.py

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "meta-llama/Llama-3.1-8B"

## Load tokenizer and model

tokenizer = AutoTokenizer.from_pretrained(model_name, device_map="auto")
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

## Verify that the model is on GPU if available

print("Device:", next(model.parameters()).device)

Why this matters: Using device_map="auto" lets the library decide whether to keep the model on CPU or GPU, which is handy when you don’t know the exact hardware ahead of time.

Build a Simple Flask API

Once the model is loaded, you can expose it through a REST endpoint. The following Flask app accepts a prompt and returns the model’s completion.


## app.py

from flask import Flask, request, jsonify
from load_llama import tokenizer, model
import torch

app = Flask(__name__)

@app.route("/chat", methods=["POST"])
def chat():
    data = request.get_json()
    prompt = data.get("prompt", "")
    if not prompt:
        return jsonify(error="No prompt provided"), 400

    inputs = tokenizer(prompt, return_tensors="pt")
    inputs = {k: v.to(next(model.parameters()).device) for k, v in inputs.items()}
    with torch.no_grad():
        output = model.generate(**inputs, max_new_tokens=150)
    text = tokenizer.decode(output[0], skip_special_tokens=True)
    return jsonify(response=text)

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8000)

Why this matters: The API keeps the model in memory across requests, so you avoid re‑loading it for every call. The torch.no_grad() context reduces memory usage during inference.

Scaling with TorchServe

For production workloads, Flask alone can become a bottleneck. TorchServe gives you a ready‑made inference server with multi‑model support and automatic GPU allocation.

Create a model archive

   torch-model-archiver \
     --model-name llama3 \
     --version 1.0 \
     --serialized-file /path/to/llama3.pt \
     --handler huggingface_text_generation.py \
     --export-path model_store \
     --extra-files tokenizer_config.json,tokenizer.json

Start TorchServe

   torchserve \
     --start \
     --model-store model_store \
     --models llama3=llama3.mar \
     --ncs

Why this matters: TorchServe handles concurrent requests, GPU sharing, and model versioning out of the box, which is essential when you have multiple services or high traffic.

Trade‑offs: Open vs Closed Models

Aspect	Open Llama 3	Closed Vendor Model (e.g., GPT‑4)
Cost	Free to download; you pay for compute	Pay‑per‑token pricing
Control	Full access to weights and code	Limited to API contract
Latency	Depends on your hardware; can be low if you own GPUs	Consistent cloud latency
Compliance	You can audit the model yourself	Must trust the vendor’s compliance claims
Updates	You decide when to upgrade	Vendor pushes updates automatically

Why this matters: If you need to keep data on premises or want to tweak the model, open models give you that freedom. Closed models offer convenience but lock you into a pricing and policy model.

Common Failure Modes

Out‑of‑Memory (OOM): Llama 3 is large. If you run it on a GPU with <8 GB, you’ll hit OOM. Use torch.cuda.set_per_process_memory_fraction or split the model across GPUs.
Tokenization Mismatch: The tokenizer must match the model. Mixing tokenizers leads to garbled output.
Thread‑Safety: Flask’s default server is single‑threaded. In production use Gunicorn with workers or TorchServe.
Model Drift: If you fine‑tune the model, keep a versioned copy. Otherwise, downstream code may break when you load a new checkpoint.
Security: Exposing a raw generation endpoint can lead to prompt injection. Add a simple prompt filter or rate limit.

Key Takeaways

Loading Llama 3 with device_map="auto" keeps the code portable across CPU/GPU setups.
A lightweight Flask API is fine for low‑traffic prototypes; switch to TorchServe for scaling.
Open models give you cost control and compliance, but you must manage hardware and updates yourself.
Watch for OOM, tokenizer mismatches, and thread‑safety when moving to production.

Source

Mark Zuckerberg attacks 'closed' AI rivals as Meta returns to open models – I added code examples, scaling guidance, and a trade‑off table that the original article did not cover.

Prototype a DIY Solid‑State Intelligence Module for Home Automation

RobustTrueTry — Mon, 10 Aug 2026 02:41:52 +0000

You want a machine that can learn to do a repetitive task without your constant input. Solid‑state intelligence can make that happen. In this article, I’ll show you how to build a small SSI prototype that learns to open a window blind.

What you’ll learn

Set up a hardware and software stack for SSI.
Train a simple model to detect the window state.
Deploy the model to a microcontroller and automate the blind.

Choose a Hardware Stack

I use a Raspberry Pi 4 as the brain and an ESP32 as the edge device. The Pi runs the training code and hosts a Flask API. The ESP32 reads a light sensor and drives a servo.

Set Up the Software Environment

On the Pi, install Python 3.10, pip, and the required libraries. Use a virtual environment to keep dependencies isolated.

python3 -m venv ssi-env
source ssi-env/bin/activate
pip install scikit-learn flask

The code above creates a clean environment. It keeps the project reproducible.

Collect Data and Train a Model

I collect a few dozen samples of light intensity when the blind is open or closed. A decision tree can classify the state with high accuracy.


## train.py – train a decision tree on light sensor data

import numpy as np
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split

## synthetic data: [light_intensity]

X = np.array([[200], [180], [160], [140], [120], [100], [80], [60]])
y = np.array([1, 1, 1, 1, 0, 0, 0, 0])  # 1=open, 0=closed

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)
clf = DecisionTreeClassifier(max_depth=3)
clf.fit(X_train, y_train)
print("Accuracy:", clf.score(X_test, y_test))

## export the model

import joblib
joblib.dump(clf, "blind_model.pkl")

The script trains a tree and saves it. The model is small enough for the ESP32.

Deploy to the Microcontroller

I use MicroPython on the ESP32. The code loads the model, reads the sensor, and moves the servo.


## esp32_ssi.py – run on ESP32

import machine
import ujson
import uos

## load the model (tiny decision tree)

model = ujson.load(open("blind_model.json", "r"))

## sensor and servo setup

light = machine.ADC(machine.Pin(34))
servo = machine.PWM(machine.Pin(15), freq=50)

while True:
    val = light.read()
    state = 1 if val > 120 else 0  # simple threshold
    if state == 1:
        servo.duty(40)  # open
    else:
        servo.duty(80)  # close
    machine.sleep(1000)

The code is minimal. It keeps the loop fast and deterministic.

Integrate with the Actuator

The servo is wired to the blind’s motor. I use a 5V logic level shifter to protect the ESP32. The servo’s duty cycle maps to the blind position.

Common Pitfalls and Failure Modes

Sensor drift: Light levels change with weather. Retrain the model periodically.
Power spikes: The servo draws current. Use a separate power supply.
Model size: A large tree may not fit. Keep the depth shallow.
Latency: The ESP32 processes in milliseconds. For real‑time control, keep the loop tight.

Key Takeaways

A Raspberry Pi can train a lightweight model for SSI.
MicroPython on ESP32 runs the model with low latency.
Simple thresholds work for basic tasks; more complex models need more data.
Watch for sensor drift and power issues in hardware.

Source

John C. Lilly on solid state intelligence and the elimination of man (1978) – I added code, tradeoffs, and failure modes to help you build a prototype.