DEV Community: ArshTechPro

How Dopamine Works: The Architecture of a Modern iOS Jailbreak

ArshTechPro — Tue, 11 Aug 2026 14:06:56 +0000

Most developers will never jailbreak a phone. That is fine. This article is not a how-to, and there is no install guide here.

What makes Dopamine worth reading as an engineer is the constraint it operates under: it has to run a full package ecosystem on an operating system that was explicitly designed to make that impossible, without modifying a single byte of the system partition, and it has to survive every process launch on the device.

Dopamine is a rootless, semi-untethered jailbreak by opa334 and évelyne, written mostly in C and Objective-C. Version support depends heavily on chip and iOS version, so check the README rather than trusting any number in a blog post.

Let's walk the architecture.

The rules of the game

Every design decision downstream falls out of what iOS enforces. Five things matter:

The system volume is sealed. Since iOS 15, the root filesystem is a cryptographically sealed snapshot. You cannot write to /usr/lib and have the device boot. So the entire jailbreak has to live somewhere else.

Code must be signed. The kernel refuses to execute pages that do not carry a valid signature chain. There is also a trust cache, a kernel-side list of hashes that are allowed to run. A tweak you compiled ten seconds ago is in neither.

Processes are sandboxed. Even as root, a process only sees what its sandbox profile allows.

Libraries are validated. Library validation means a process will typically only load libraries signed by the same team as the main binary. That is the single biggest obstacle to loading third-party code into Apple's own processes.

Memory is protected below the kernel. On modern arm64e chips there are protection layers beneath the kernel itself (PPL, and later SPTM) that guard page tables, plus pointer authentication on function pointers. Kernel read/write alone is no longer enough.

So: no writable system directory, no way to sign code, no way to load unsigned libraries, and a kernel you cannot fully trust yourself inside. Everything below is a response to one of those five facts.

Two layers

The repo splits cleanly in two:

Application/   Objective-C, UIKit. The app you tap. Orchestration, UI, logs.
BaseBin/       C, Mach, assembly. The runtime that lives on the device afterwards.

Application is the installer and control panel. BaseBin is the actual jailbreak. Almost everything interesting is in BaseBin, and it keeps running long after the app is closed.

Phase 1: exploits as plugins

The app has a DOExploitManager that selects an exploit based on the device's chip family and OS build, and a DOJailbreaker that drives the whole sequence.

The part worth stealing here is not the exploits, it is the plugin boundary. Each exploit ships as a bundle with an Info.plist declaring what it supports. The repo has several of them side by side, and the wiki has a page on adding new ones. The orchestration code does not care which one runs. It asks the manager for something compatible with this device, runs it, and gets a set of capabilities back.

This is a hardware abstraction layer, applied to bugs. When a new technique appears, you add a bundle instead of rewriting the jailbreak. Given how quickly individual entry points get patched, that boundary is the reason the project survived across four-plus years of OS releases.

Phase 2: from a bug to a stable primitive

Raw exploitation gives you something awkward and fragile. What the rest of the system wants is a clean interface: read kernel memory, write kernel memory, call a kernel function, mark this page executable.

That translation lives in libjailbreak. It also holds a table of kernel structure offsets that vary per Darwin version, because struct layouts change between iOS releases and there are no headers for the ones that matter.

That table is why version support is enumerated so precisely, and why "it should probably work on the next point release" is never true. A wrong offset is not a bug report, it is a kernel panic.

Note the layering discipline: exploitation is one module, primitives are another, and every consumer above talks to the primitive API only. A large percentage of the codebase never has to know how privileges were obtained.

Phase 3: the rootless bootstrap

The system volume is sealed, so Dopamine installs into a randomized path under /private/preboot, and exposes it at /var/jb.

If you have ever built software that must be relocatable, this will look familiar. It is /usr/local versus /usr, or a container volume mount versus the base image. Every package is compiled to reference /var/jb/... instead of /, the actual location is randomized per install, and the symlink hides that indirection from everything above.

The environment itself is a Procursus bootstrap: a proper Debian-style userland with dpkg, so packages install through Sileo or Zebra using ordinary .deb semantics.

Two things fall out of this design. Restoring the device is mostly a matter of deleting a directory, not repairing a system partition. And system updates do not fight with a modified root. "Rootless" sounds like a limitation; in practice it made jailbreaks dramatically less destructive.

Phase 4: a capability server inside PID 1

Here is the design decision I find most interesting.

You have kernel read/write. The naive approach is to hand that to every process that needs it. That is a disaster: any of them can panic the kernel, and every one of them is now a privilege escalation target.

Dopamine does the opposite. It injects a hook into launchd (PID 1), and inside it runs jbserver, a Mach service that owns the privileged primitives. Everything else is a client that sends requests over Mach or XPC via libjailbreak. Requests are organized into domains, and callers are checked for what they are allowed to ask for.

That is a broker pattern, straight out of browser sandbox design. One privileged component, a narrow typed API, everyone else unprivileged. The clients cannot corrupt the kernel because they never touch it.

Putting it inside launchd also solves persistence: PID 1 never dies while userspace is alive, so the jailbreak state outlives the app entirely.

Phase 5: getting into every process

For tweaks to work, code has to load into arbitrary system processes. Two components handle this.

dyldhook patches the dynamic linker itself. It runs before the process's main, checks the process in with jbserver to receive its sandbox extensions and environment info, and handles library validation by making sure a library's signature is registered in the trust cache before the kernel evaluates it.

systemhook.dylib is inserted via DYLD_INSERT_LIBRARIES and does the ongoing work: loading tweaks, and hooking posix_spawn and execve so that every child process inherits the injection.

That last detail is the whole trick. Think LD_PRELOAD, except it re-preloads itself into everything it spawns. Inject once into PID 1, and the property propagates down the entire process tree by induction. You never have to enumerate processes or race a launch.

Actual tweak hooking is delegated to ElleKit, an open-source hooking library that replaced the old proprietary Substrate.

Phase 6: making the system not notice

This is where most of the engineering hours actually went, and it is the least glamorous part.

Once you modify a running process, the OS starts noticing. csops reports the process as invalid. On iOS 16 the networking policy layer began checking code signing validity, which meant modified processes silently lost network access. So systemhook hooks those paths and re-validates.

On arm64e, fork() breaks, because the child needs to inherit memory protections and signing state that the kernel will not copy for it. The fix, forkfix, is a small masterpiece of pragmatism: hook __fork, use a pipe pair to freeze the child immediately after it appears, have the parent ask jbserver to apply the necessary fixups to the child PID, then let it continue.

There is also a jetsam multiplier, because processes carrying a stack of injected tweaks blow through memory limits and get killed.

None of this is the exciting part of a jailbreak. All of it is why it is usable.

Semi-untethered, and the userspace reboot

Everything above lives in memory. A real reboot wipes it, which is what "semi-untethered" means: the device boots stock, and you reopen the app to re-apply.

There is a middle option, though. A userspace reboot tears down and restarts userland without a kernel boot, which means the jailbreak state can be re-established without re-running an exploit. Practically, it turns "something broke, reboot and start over" into a thirty-second operation. launchdhook is what makes it possible, because PID 1 is where the state lives.

The map

  Dopamine.app
        |
        | selects + runs
        v
  Exploit bundle  --->  libjailbreak  (kernel primitives, version offsets)
                              |
                              v
                        launchdhook  in PID 1
                              |
                        +-----+-----+
                        |  jbserver |  <---- Mach / XPC ---- every process
                        +-----+-----+
                              |
   /var/jb  ------------------+   trust cache, sandbox extensions, fixups
   (Procursus bootstrap)
                              |
     dyldhook + systemhook injected into each process on spawn
                              |
                        ElleKit -> tweaks

What transfers to normal software

Strip away the iOS specifics and there are four patterns here worth borrowing:

Isolate the volatile part behind an interface. Exploits are plugins with declared compatibility. When your dependency on the outside world is guaranteed to break, make replacing it a config change.
Centralize dangerous capability, distribute access. One broker owns the primitive, everyone else gets a narrow API. This is browser sandboxing, syscall filtering, and jbserver, all the same shape.
Make your install root relocatable and additive. Never modify what you do not own. Add a prefix and indirect through it.
Propagate through inheritance, not enumeration. Hooking spawn beats scanning for processes, in the same way that fixing a base image beats patching running containers.

Notes

Jailbreaking is legal in many jurisdictions but not all, it voids your warranty, and the same mechanisms described here are why a jailbroken device is a weaker security boundary than a stock one. Read the code for the engineering. That is where the value is.

Xcode 27: Everything Developers Need to Know

ArshTechPro — Mon, 10 Aug 2026 10:27:47 +0000

Xcode 27 is in beta right now (beta 4 at the time of writing), and it is a big release. Apple's official release notes are lots of lines of radar numbers and one-line bug fixes, which is great as a reference and terrible as a read.

Can you even run it?

Before anything else, check these three lines:

Xcode 27 requires a Mac running macOS Tahoe 26.4 or later.
Xcode 27 only installs and runs on Apple silicon Macs. If you are still on an Intel Mac, this release is the end of the line for you.
On-device debugging supports iOS 17+, tvOS 17+, watchOS 10+, and visionOS. Older devices are no longer debuggable.

It ships with Swift 6.4 and the SDKs for iOS 27, iPadOS 27, tvOS 27, watchOS 27, macOS 27, and visionOS 27.

Part 1: The breaking changes (read this section)

I am putting these first because they are the parts that turn into a red build log on Monday morning. Everything else is upside.

Intel is being phased out of your build settings

If a target's minimum deployment target is macOS 27.0 or DriverKit 27.0, it will no longer build Universal by default. The ARCHS_STANDARD setting drops x86_64 once MACOSX_DEPLOYMENT_TARGET or DRIVERKIT_DEPLOYMENT_TARGET is 27.0 or higher.

If you still ship to Intel Macs, you have two options:

Keep your minimum deployment target below macOS 27.0.
Explicitly add x86_64 back to the ARCHS build setting.

The macOS 27 SDK can still back-deploy Universal apps down to macOS 12, so this is a default change, not a hard removal.

The old linker is gone

ld64 has been removed and the -ld_classic flag is no longer supported. If you have that flag lingering in OTHER_LDFLAGS from some 2023-era workaround, delete it now. This one is a hard failure, not a warning.

On Demand Resources is deprecated

NSBundleResourceRequest and the whole On Demand Resources system are deprecated. The replacement is Background Assets, which got a solid round of improvements this release (more on that below).

PreviewProvider is deprecated

PreviewProvider and its family of preview modifiers are now deprecated. If you still have the old struct-based previews hanging around, this is your nudge to move to the #Preview macro.

Before:

struct ContentView_Previews: PreviewProvider {
    static var previews: some View {
        ContentView()
    }
}

After:

#Preview {
    ContentView()
}

A real Swift source break

This one is subtle. A computed property that has both an init accessor and an array or dictionary literal as its initial value will no longer compile if the getter is declared before the init accessor. This is a known source break from SE-0508.

Broken:

struct S {
    var _strings: [String]
    var strings: [String] = ["hello"] {
        get { _strings }
        @storageRestrictions(initializes: _strings)
        init { _strings = newValue }
    }
}

The fix is to reorder: declare the init accessor first, then the getter. Annoying, but a one-line move.

Duplicate Clang module names now fail

The Swift dependency scanner was optimized to skip redundant header searches, and the tradeoff is that every Clang module reachable from a single dependency scan must have a unique module name. Previously the scanner tolerated duplicates; now it may error out.

The two situations that trigger this in the real world:

An SDK or project that vends the same Clang module name from more than one location on the header search path.
Vendored third-party sources shipping a module.modulemap that redeclares a module already in the SDK.

If you get a mysterious scanning error after upgrading, this is the first thing to check.

C++ changes worth knowing

If you have C++ in your app, a few things moved:

The minimum macOS deployment target for the C++ standard library is now 11.0.
multimap::find and multiset::find no longer guarantee returning an iterator to the first equal element. libc++ used to do this by accident, and the Standard never promised it. Use lower_bound or equal_range if you were relying on it.
lower_bound and upper_bound on std::map and std::set behave differently for comparators that are not a strict weak order. Defining _LIBCPP_ENABLE_LEGACY_TREE_LOWER_UPPER_BOUND gets the old behavior back, but that escape hatch is going away, likely next release.
bitset::operator[] now returns bool, which is actually what the Standard says it should do.
std::allocator is now trivially default-constructible.

The upside is significant: associative and unordered containers got up to 11x faster in some functions, several algorithms got up to 3x faster, and distance on non-random-access segmented iterators improved dramatically. A number of C++ papers landed too, including std::optional<T&>, zip, and std::views::indices(n).

Part 2: Agentic coding is now a first-class citizen

This is the headline feature, and it is a lot more than a chat box.

The assistant moved out of the sidebar

The coding assistant now lives in the editor area, not the navigator, with a redesigned conversation transcript. Artifacts the agent produces (code diffs, plans, SwiftUI preview snapshots) show up next to the transcript, and you can annotate code snippets and plan documents to give targeted inline feedback without leaving the conversation.

The sidebar is now dedicated purely to organizing conversations: real-time status, unread indicators, drag-and-drop grouping, archiving, renaming, multi-select for bulk actions, and a context menu to open conversations in new tabs, windows, or editor panes.

There is also a New Conversation button in the toolbar that works from anywhere in Xcode, with a status indicator you can click to jump to whichever conversation needs your attention.

Plan mode

Planning is now a proper feature rather than a prompting trick. Plans appear as editable Markdown artifacts next to the conversation. You review, annotate, discuss changes, and approve before the agent writes any code.

This is the workflow I would recommend for anything non-trivial: get the plan right first, then let it execute.

Agents can actually run your app

This is the part that changes what agents are useful for. In Xcode 27, agents can:

Boot simulators, install and launch apps, synthesize touch events, and capture screenshots to verify UI behavior.
Manipulate the active run state, read and interact with the debugger console.
List and switch between schemes and run destinations.
Inspect and modify build settings, compiler flags, entitlements, and Info.plist keys.
Access project insights such as crashes, disk writes, energy, hangs, and launch issues affecting your shipped app.

In other words, an agent can now write a fix, build it, run it, look at the screen, and check whether it worked.

Gemini, ACP, and plugins

Google Gemini is now available in the coding assistant alongside the existing options.
Xcode supports the Agent Client Protocol (ACP).
Agents can be extended with plugins containing skills, MCP servers, and ACP agent configurations. Skills are invokable as slash commands with completion support.
Apple ships its own specialists for targeted tasks like localization, UIKit resizing, and accessibility.
Two security-focused skills landed in beta 3: adopt-c-bounds-safety for a file-by-file -fbounds-safety adoption workflow, and audit-xcode-security-settings to suggest security-oriented build settings and entitlements.

Plugin authors can customize how their MCP servers appear in the UI using _meta fields:

{
  "name": "MyGreatPlugin",
  "description": "An awesome MCP server configuration.",
  "version": "1.0.0",
  "mcpServers": {
    "MyGreatMCP": {
      "type": "http",
      "url": "...",
      "tools": ["*"],
      "_meta": {
        "ideToolIconPath": "./icon.svg",
        "ideToolIconRendersAsTemplate": true,
        "ideToolTitles": {
          "whoami": "Who Am I",
          "get-current-email": "Get Current Email Message"
        }
      }
    }
  }
}

A sandbox for agents

Coding Intelligence includes a new security layer that monitors and controls filesystem access by coding agents and any processes they spawn. It is opt-in via Coding Intelligence settings. If you are nervous about letting an agent loose in your repo, turn this on before you do anything else.

One known issue to watch

If the "Implement the plan?" confirmation bar appears while the agent is still streaming, clicking Yes or No can start a new agent turn on top of the in-flight one and leave the conversation in a broken state. Wait for the agent to finish responding before confirming.

Part 3: Device Hub replaces the Simulator workflow

Device Hub is the new unified interface for both simulators and physical devices. Two features stand out.

Wireless pairing

You can now pair iPhone, iPad, and Apple Watch running OS 27 or later over a network. Click the + button in the Device Hub sidebar and choose "Pair Nearby Device". No cable needed for iPhone and iPad, and watch pairing is noticeably more reliable.

Mouse and trackpad gestures on iOS

Standard Mac gestures (scrolling, pinching, rotating) now work with UIKit components on iOS devices, physical or simulated.

There is a nuance worth understanding here. When you scroll with a pointing device, UIEvent.EventType.scroll is emitted. Pinch or rotate produces UIEvent.EventType.transform. But clicking with a mouse produces a simulated finger touch of type UITouch.TouchType.direct, not UITouch.TouchType.indirectPointer.

That is a convenience hybrid, not a faithful simulation. To validate real pointer behavior, use "Simulate Trackpad or Mouse" from the Device menu, test on a physical iPad with a paired pointing device, or use iPhone Mirroring.

Simulator boot is faster

Simulator runtimes now ship with a pre-built dyld cache, which makes the first launch of a simulator much faster. Small change, noticeable every single day.

Device Hub rough edges

Beta software, so expect some friction:

Game controllers only work with the visionOS simulator.
Video and input for a physical Apple Vision Pro are not supported (use AirPlay to view remotely). Everything else, like settings and DeviceFS, works.
Two-finger touches cannot be sent.
Scrolling over an Apple Watch face does not emulate the digital crown. Move the pointer over the crown in the bezel instead.
Devices may not appear during parallel testing even though tests are running. Disable parallel runs if you want to watch UI tests execute.
Downloaded app data containers land in a folder named after the bundle identifier rather than a proper .xcappdata bundle. The workaround is to rename the folder with an .xcappdata extension and nest the contents inside an AppData subfolder.

Part 4: Previews and Playgrounds

Previews got real quality-of-life work this cycle.

Argument grids

#Preview(arguments:) renders a grid of previews, one per argument. Click any cell to open it in Interactive mode. This is great for state matrices.

#Preview(arguments: [
    Order.empty,
    Order.singleItem,
    Order.manyItems
]) { order in
    OrderSummaryView(order: order)
}

Resizable canvas

iOS previews get a Resizable Canvas mode so you can view your view in arbitrarily sized containers rather than fixed device frames. Helpful for adaptive layout work, and no longer constrained to specific size ratios.

Other improvements

Each #Preview and #Playground tab can be pinned independently in the canvas.
You can preview your UI in a different localization.
Code inside #Preview now explicitly runs on the main actor, so calling main-actor-isolated APIs no longer produces concurrency warnings or runtime check failures. That was a real annoyance under strict concurrency.
Holding Command routes zoom and scroll events to the canvas. Toggleable via Editor > Canvas.
Error messages across previews are meaningfully better: clearer timeouts, full diagnostics on MCP tool failures, and a placeholder view instead of a silent macOS fallback when a runtime is not installed.

Known issue: standalone Swift files opened by double-clicking in Finder may fail to run #Playground or #Preview blocks. Use File > Open, or drag onto the Dock icon.

Part 5: Localization becomes an agent workflow

This is probably the most immediately practical feature for small teams.

Agents can now translate strings in String Catalogs, from a single feature to an entire project, into one or more languages. Xcode handles the plumbing: it adds languages to your project settings, creates missing String Catalogs, and feeds context to the agent as it translates.

The String Catalog editor has a Generate Translations button, and you can right-click specific strings to translate just those.

Supporting changes that make this workable in a real pipeline:

A localization comment of "do not translate" automatically marks the string as Don't Translate in String Catalogs and translate="no" in exported XLIFFs.
Exported XLIFFs use state-qualifier="leveraged-mt" to flag machine-translated strings, so your human translators know what to review.
You can annotate translations in String Catalog artifacts when agents are translating.
Exporting localizations now extracts NSLocalizedString and similar macros from header files, not just implementation files. That is a fifteen-year-old bug report finally closed.
The "Prepare Project for Localization" tool surfaces newly added strings as artifacts, and shows keys removed because they no longer appear in source.

The honest take: this handles the bulk of the work, but you still want a professional translator reviewing the output before you ship to a new market.

Part 6: Instruments got a serious upgrade

If you do performance work, this might be the most valuable section of the release.

Swift Concurrency tooling

A new Swift Executors instrument shows tracks for the Cooperative Thread Pool, the Main Actor, and any type conforming to TaskExecutor or SerialExecutor. Full capture on OS 27; older systems show "Unknown executor".
Task tracks now group into Swift Task Collections, sorted by name or creation site. You can switch a collection track between showing task lifetimes and task states.
Tasks, Collections, Actors, and Executors have a Profile detail showing a call tree built from data captured while the task was running. Requires recording alongside Time Profiler or CPU Profiler.
Selecting a bar chart interval in an Actor or Executor queue plot lists the waiting tasks in the inspector.
Tasks and Actors whose lifetime started before the trace now show up, on a best-effort basis.
language swift task tree in LLDB prints a tree of every Swift Task the debugger knows about.

Foundation Models instrument

New instrument for tracing and debugging Foundation Models usage: instructions, prompts, responses, token usage, and inference performance. If you are shipping on-device AI features, you now have visibility into them.

System Trace and QoS

System Trace unifies system calls, VM faults, and thread states into a single plot, with a blending algorithm that keeps dense regions readable when zoomed out. You can walk the chain of scheduling events for a thread with left/right arrow keys, and the inspector offers quick actions like pinning the thread that made another thread runnable.

The System Trace template also graphs thread priority over time, which makes priority inversion and resource starvation much easier to spot. Thread Activity now displays effective QoS by default, with requested QoS available from the track dropdown.

Everything else in Instruments

Graphs no longer rescale to the local maximum when you pan the timeline, so comparisons across tracks are consistent. Manual rescale is under View > Rescale.
Pinned tracks are restored from the previous run, and can be saved and restored explicitly via View > Track States.
A new inspector shows details for the selected event with quick actions for pinning, filtering tracks, and filtering the detail view.
os_log data can be overlaid on process and thread tracks via the Track Graph Display popover.
os_signpost gets a track per signpost name, nested under the category.
The SwiftUI instrument records more detail about layout passes and why a layout computation was not cached, plus a "Summary of Updates" focus action on the view hierarchy.
Allocations marks tagged allocations with a (tagged) suffix when the process runs with Memory Integrity Enforcement.
Drag and drop .atrc, .logarchive, or .sample files onto the sidebar to create a run for each.
Memory usage when importing .atrc files dropped by roughly 1.5 GB on average.
Animation Hitches supports visionOS 27.0 and later.

On the command line, xctrace record accepts recording options as JSON (--show-recording-options prints the available ones), xctrace export can restrict a time range, and export now takes .atrc and .logarchive directly instead of requiring an import step first.

Deprecation: Instruments now requires iOS 17, watchOS 10, or tvOS 17 as a minimum on target devices.

Part 7: Organizer and production insights

The Organizer quietly became a planning tool rather than a crash dump viewer.

Insights Overview summarizes high-impact performance regressions across metrics and diagnostic reports, so you can prioritize instead of guessing.
The Hitches metric replaces Scrolling and covers all animations in your app, not just scroll.
Storage metrics track Documents & Data and app size across releases, which catches cache bloat and bundle growth.
AI-driven analysis generates recommendations for Crash, Energy, Disk Write, Hang, and Launch diagnostics, with links into your source and the coding assistant.
Metric goals now cover Battery Usage, Disk Writes, Hang Rate, Hitches, Memory, and Storage. Similar-app goals are supported for Hang Rate, on-screen Battery Usage, Disk Writes, and Storage, and Launch Time baselines were recalibrated.

Combined with agents having access to these insights, the loop is: Organizer flags a hang regression, you ask the agent about it, the agent reads the diagnostic and proposes a fix.

Part 8: Testing

XCUIVoiceOverService is a new UI testing API for verifying VoiceOver behavior. You can drive VoiceOver from UI tests and assert on focus, spoken output, and navigation. Accessibility testing that actually tests accessibility.
A launch test file template that opts into runsForEachTargetApplicationUIConfiguration, so the test runs across every combination of orientation, localization, and appearance your app supports.
Test plans let you choose how the system responds when the target app crashes during UI testing: off, warning, failure (the default), or fatal failure.
Mixing frameworks now warns you: calling an XCTest assertion inside a Swift Testing test (or the reverse) produces a warning-severity runtime issue. Configurable via the new interoperability setting in your test plan.
Filters were added to the test plan configurations tab, plus recent tests and open tests filters in the Test Navigator.
Large Swift Testing suites with many parameterized cases perform significantly better.
Test Repetition Mode now repeats individual Swift Testing cases instead of the whole test plan.

On the SwiftPM side, swift test summarizes failures at the end of a run, and supports --maximum-repetitions with --repeat-until [pass|fail] for hunting flaky tests. Only cases matching the condition get repeated.

swift test --maximum-repetitions 20 --repeat-until fail

Part 9: Everything else worth a mention

Interface Builder builds without a simulator

A new IB compilation mode, toolchain, is enabled by default for UIKit documents. It compiles IB documents without downloading a simulator, which is a genuine win for CI and build servers.

Background Assets

Since On Demand Resources is deprecated, this matters:

Asset-pack manifests support path wildcards, file exclusion, hard-coded source roots, and custom destination subpaths.
Localized asset packs: the system delivers only the packs matching the user's preferred languages, cutting storage usage.
A Steam Asset Converter turns Steam depots into asset packs.
Xcode can serve asset packs to your app while debugging on device. Set a Background Asset Packs folder in the Run scheme action's Options tab.

StoreKit testing

New configuration UI for In-App Purchase offer codes, plus off-device purchase options to test them through the Transaction Manager.
Subscription Bundles and Subscription Suites can be configured for local testing.
Volume purchase transactions can be created for 1-month and 1-year auto-renewing subscriptions.

Swift and C++ interoperability

C++ constructors with default parameter expressions no longer require passing every argument explicitly from Swift.
Swift closures convert to std::function instances.
__counted_by and __noescape-annotated std::span parameters map to Swift Span without the experimental feature flag. Return values via __lifetimebound still need it.
Safe wrappers can be generated for functions taking std::span directly, without hiding the instantiation behind a typedef, as long as the parameter is __noescape.
The new SWIFT_REFCOUNTED_PTR macro bridges smart pointers to intrusively reference-counted types into Swift classes.

Debugging

LLDB can inspect data types with ~Copyable fields in the standard library and system frameworks.
LLDB ships with an MCP server (lldb-mcp).
In projects using bridging headers, LLDB imports explicitly built Swift modules and PCH from DerivedData directly. This can dramatically speed up the first po in a debug session, which has historically been painfully slow.

Documentation search

You can search developer documentation using natural language, with results matched semantically rather than by keyword.

Icon Composer 2.0

Supports a sharper rendering mode for the 2027 operating systems, with refractivity, outside specular, and deeper shadows. Edit the new properties in the group inspector and preview either design generation from the toolbar. One icon covers all OS versions.

Migration checklist

If you want the short version, here it is:

Confirm you are on an Apple silicon Mac with macOS Tahoe 26.4+.
Remove -ld_classic from your linker flags.
Decide your Intel story: lower your deployment target, or add x86_64 to ARCHS explicitly.
Migrate PreviewProvider to #Preview.
Plan the move from On Demand Resources to Background Assets.
Check for duplicate Clang module names if dependency scanning starts failing.
Fix any init accessor ordering issues flagged by the Swift compiler.
If you use C++, audit multimap/multiset find usage and any custom comparators.
Turn on the agent filesystem security layer before letting agents run.
Drop your minimum device targets for testing to iOS 17 or later.

Reference
Source: Xcode 27 Release Notes

Reasonix - Deepseek: A Terminal Coding Agent Built Around the Thing Everyone Else Ignores

ArshTechPro — Wed, 05 Aug 2026 18:27:59 +0000

Most terminal coding agents are architecturally similar: a loop, a tool registry, some context management, a TUI. Reasonix picks a different thing to optimize for, and it is a thing that shows up on your bill rather than in a demo video.

The tagline is "engineered around prefix-cache stability — leave it running." That phrase is doing a lot of work, so let's unpack it.

Why prefix caching is the whole pitch

DeepSeek's API, like several others, caches the prefix of your prompt. If the next request starts with the exact same token sequence as the previous one, the provider serves those tokens from cache and bills them at a small fraction of the normal input rate. Cache hits are dramatically cheaper than cache misses.

Here is the catch: it is a prefix cache. The match has to start at token zero and run forward. Change one character near the top of your context and every token after it is a miss.

Now think about what a typical agent harness does over a long session. It re-summarizes the conversation. It injects a fresh timestamp or a re-scanned directory tree at the top. It reorders tool definitions. It rewrites the system prompt when you switch modes. Every one of those is a mutation near the front of the context, and every one of them silently invalidates the entire cache.

The result is an agent that feels fine and costs several times what it should. You do not notice, because nothing errors. You just watch the number go up.

Reasonix's central design constraint is: don't do that. Keep the front of the context stable, append rather than mutate, and put churn where it costs least.

What that looks like in practice

A small, stable environment summary is injected at startup rather than regenerated each turn.
Stale tool output gets snipped and pruned before summary compaction kicks in, so a giant cat result from twenty turns ago is not still sitting in your prefix.
The built-in tool schema contract is documented and regression-reviewed, because a silent tool-definition reshuffle is a cache invalidation with no visible symptom.
Two-model mode (executor plus planner) runs each model in its own separate, cache-stable session instead of interleaving them into one context.

That last one is the neatest idea in the project. The naive way to add a planner is to inject planning turns into the same conversation, which trashes cache stability for both roles. Keeping them in separate sessions means each one's prefix stays intact.

The rest of the architecture

A single static Go binary. CGO_ENABLED=0, cross-compiles to six targets, and the only dependency is a TOML parser. No Node runtime, no Python venv, no dependency tree to audit.

Config-driven, not model-hardcoded. Providers, agent settings, enabled tools, and plugins all live in a reasonix.toml. DeepSeek ships as a preset, but any OpenAI-compatible endpoint is a config entry rather than a code change. Secrets come from the environment and are never written into the config file. Despite the repo name, this is not DeepSeek-only, and it is not an official DeepSeek project — it is a community project that treats DeepSeek as the first-class default.

MCP client. External tools run as subprocesses over stdio JSON-RPC, or over Streamable HTTP for remote servers. If you already have an .mcp.json, drop it in the project root and it is read as-is. Server prompts show up as slash commands, and resources are pulled into a message with @server:uri.

Permissions and sandbox are separate mechanisms. Permissions are policy: each tool call is evaluated deny, then ask, then allow, then fallback, and approvals are stored as reusable rules like Bash(go test:*) rather than one-off clicks. The sandbox is enforcement: file writers refuse any path outside the workspace root, resolving symlinks and .. so a link cannot tunnel out.

One caveat worth knowing before you use the autonomous mode: bash itself is jailed via Seatbelt on macOS, but on other platforms it currently runs unconfined. On Linux or Windows, your deny list is the real boundary, not the sandbox. Write it accordingly.

Getting started

npm i -g reasonix    # or: brew install esengine/reasonix/reasonix

reasonix setup       # pick a provider, set your key
reasonix             # start an interactive session

That is genuinely the whole first run. There is also a desktop app and a VS Code extension that drive the same local engine, plus prebuilt archives on every release if you would rather not go through npm. Everything else — the TOML schema, permission rules, MCP plugins, custom slash commands, two-model setup — is in the Guide, and none of it is required to try the thing.

Two commands worth knowing on day one: /init generates project instructions, and /branch <turn> forks the conversation from an earlier checkpointed turn. That second one is a much better recovery move than clearing context and re-explaining everything when the agent goes down a bad path.

Is it worth a look?

Yes, if: you already pay for DeepSeek or another OpenAI-compatible endpoint and run long agent sessions, you are cost-sensitive enough to care about cache hit rates, or you want a single static binary instead of a Node or Python install. The MCP support, config-first design, and permission rules are all solid, and the project has real traction — tens of thousands of stars, an active contributor list, CI, code-signed Windows builds, and a bilingual Discord.

Probably not, if: you are locked into a provider without prefix caching, in which case the core optimization is irrelevant and you should pick a harness on other merits. Also skip it if you need a mature autonomous-mode sandbox on Linux or Windows today — that is still on the roadmap.

Things to go in knowing. The active branch is main-v2, not main, which is mildly confusing when you clone. If you are on a 0.x config, there is a migration guide, and your old MCP servers are still read as a lowest-priority source. The issue and PR counts are large, which is what you would expect from a fast-moving project at this size — read it as activity, not instability, but check the changelog before upgrading anything you depend on. And the repo name genuinely does mislead people: this is a community project, not something DeepSeek ships.

The reason I would look at it even if you never install it is the design argument. Most agent harnesses treat context management as a correctness problem: keep the model informed, drop the irrelevant. Reasonix treats it as an economics problem too, where the ordering and stability of your context is a cost variable you control. That reframing is worth twenty minutes of reading whatever tool you end up using.

Repo: https://github.com/esengine/DeepSeek-Reasonix

LoopX: A Control Plane for AI Agents That Have to Keep Working for Days

ArshTechPro — Wed, 05 Aug 2026 18:23:46 +0000

If you have ever pointed a coding agent at a multi-day goal, you know the failure mode. It is not that the model writes a bad function. It is that on turn 40, the agent no longer remembers what the objective was, which decision you already made, what is out of scope, or what the last run actually proved. The context window rolled over, and the plot went with it.

LoopX is an attempt to fix that specific problem. It calls itself "loop engineering for long-running AI agents," and it is a local control plane that sits above your agent runtime rather than replacing it.

The one-sentence version

Your agent (Codex, Claude Code, Cursor, whatever) executes bounded loops. Something (a heartbeat, a cron job, you hitting enter) triggers the next loop. LoopX holds the state that has to survive between those loops.

The project draws the separation like this:

Layer	Role
Codex / Claude Code / Cursor	Execute a bounded agent loop: read, write, run commands, respond
Goal mode / automation / CLI / TUI	Trigger or schedule the next loop
LoopX	Preserve goals, gates, todos, run history, quota, evidence, handoff state

That third row is the whole product. LoopX is not an executor and not an autonomous production controller. It is a state kernel with a CLI.

Why "just use a todo file" isn't enough

A TODO.md plus a long system prompt gets you surprisingly far. It falls over once any of these become true:

The goal changed halfway through, and nothing recorded why.
A decision genuinely needs a human, and that request evaporated into a chat message nobody read.
Two agents are touching the same repo and neither knows who owns what.
The last run claimed success, and there is no artifact proving it.
Some work is safe and read-only, some crosses into writes, production, or private data, and the distinction lives only in your head.

LoopX makes those things explicit and machine-readable, which is what lets a loop run longer without becoming less accountable.

The concepts, in plain English

Lifetime goals. A durable project intention that outlives one chat thread. Importantly, a lifetime goal does not hand the agent open-ended autonomy: only the next bounded transition is executable.

User gates. A concrete decision that belongs to you, recorded as a first-class object instead of a sentence in a transcript. The loop can see that it is blocked on a human.

Safe fallback. When one lane is gated, audited side paths can keep moving without bypassing the gate. This is the part I find most interesting: the alternative designs are usually "block everything" or "let the agent decide," and both are bad.

Todo ownership. Todos are tagged user or agent, with a claimed_by field so multiple agents can coordinate instead of colliding.

Quota. A guard that answers whether an automatic turn should run right now, wait, ask the user, self-repair, or stay quiet. Practically, this is your defense against a heartbeat loop burning tokens on turns that cannot produce a verified transition.

Run history and evidence. Compact append-only events for progress, validation, blockers, rewards, and quota spend.

Public/private boundary checks. A local scan that tries to keep credentials, raw logs, local paths, and private state out of anything you publish.

Setup

Requirements are refreshingly light: Python 3.11+, curl, tar, and a macOS or Linux shell. The Python package has no runtime dependencies outside the standard library. Git is only needed if you want to contribute.

1. Install (no clone)

curl -fsSL https://raw.githubusercontent.com/huangruiteng/loopx/main/scripts/install-from-github.sh | bash
export PATH="$HOME/.local/bin:$PATH"
loopx doctor

The installer drops a release snapshot under ~/.local/share/loopx/releases/, a CLI wrapper in ~/.local/bin, a man page, and reusable agent skills under ~/.codex/skills. As always, read a piped install script before running it if that matters to you.

Updates go through an explicit interface rather than re-running the installer blind:

loopx update --check     # read-only
loopx update --dry-run   # read-only
loopx update --execute

2. Kick the tires without touching a real repo

loopx demo
cd /tmp/loopx-demo
loopx status
loopx quota should-run --goal-id demo-goal
loopx history --goal-id demo-goal

This creates a disposable goal with one user todo and one agent todo. You should see ok: True and a should_run=True / state=eligible quota response. Do this first. It takes thirty seconds and tells you whether the mental model clicks for you.

3. Connect a real project

cd /path/to/your-project
loopx bootstrap \
  --goal-id your-project-goal \
  --objective "Improve this project through bounded, verified goal segments." \
  --goal-doc GOAL.md

loopx connect is an alias for bootstrap. This creates:

your-project/
  .loopx/registry.json
  .codex/goals/your-project-goal/ACTIVE_GOAL_STATE.md

~/.codex/loopx/
  goals/<goal-id>/runs/

Add these to .gitignore before you commit anything:

.loopx/
.codex/goals/
.opencode/goals/
goals/**/ACTIVE_GOAL_STATE.md

That state is live local runtime data. Committing a controller's active goal state is how private paths and internal notes end up in a public repo.

A healthy connection means loopx doctor passes, both files above exist, loopx status shows who acts next, and none of it is staged for commit.

4. The agent-first path

The docs actually push you toward not running these commands yourself. Paste something like this into Codex, Claude Code, or Cursor from your project root:

Connect the current project to LoopX. Do not clone the LoopX repository.
If `loopx` is not on PATH, install it with the official no-clone installer.

Then run `loopx doctor`. Working only from the current project root:
1. If LoopX state already exists, reuse it. Do not overwrite the goal or objective.
2. If the project is not connected, prefer `loopx connect`; use `loopx bootstrap`
   only when state clearly needs initialization.
3. Ensure `.loopx/`, `.codex/goals/`, and `.local/` are ignored.
4. Set up the thin LoopX heartbeat for this surface.
5. Stop after setup and report the active state id, current user gate, top agent
   todo, and next safe action.

Do not start longer delivery work in this setup turn.

One caveat worth knowing before you try this with a non-Codex agent: LoopX can only drive an agent that exposes at least one control hook, such as shell execution, a goal/task command, an automation hook, or its own scheduler. Without one, LoopX still tracks state, but you run the commands by hand.

5. The daily loop

loopx status
loopx history --goal-id your-project-goal
loopx quota should-run --goal-id your-project-goal

Adding work:

loopx todo add --goal-id your-project-goal --role agent \
  --text "Run the next bounded validation slice."

Diagnosing is also meant to be delegated. loopx diagnose deliberately emits an agent-facing evidence packet rather than a verdict, so you ask your agent to run it and reason from it: can this project self-drive, what blocks it, what exact question needs your answer, what happens next.

6. Heartbeats and quota (the token-spend part)

An automatic turn is supposed to check quota before working and record spend exactly once after validated writeback:

loopx quota should-run --goal-id your-project-goal
loopx heartbeat-prompt --thin --goal-id your-project-goal
loopx quota spend-slot --goal-id your-project-goal --slots 1 --source heartbeat --execute

Spend is not appended for quiet skips, preflight failures, or dry runs. should-run returns a fairly rich contract: whether delivery may run, what it is waiting on (user, controller, external evidence, health, quota), the work lane, todo summaries, and the spend policy.

7. Before you publish anything

loopx check --scan-path README.md --scan-path docs/ --scan-path examples/

8. Optional local dashboard

There is a read-first React dashboard for inspecting projects, todos, gates, and evidence across a global registry. It is explicitly experimental: the CLI stays the source of truth and browser writes require local opt-in. Check the repo docs for the current path, since it moved between the README and the getting-started guide.

What it deliberately does not do

The maintainer is unusually direct about this, which is a good sign. LoopX is not a full agent platform, not an autonomous production controller, and not a replacement for your runtime. Project ownership and dangerous permissions stay with the human. It is a local coordination substrate.

Is it worth a look?

Yes, if: you are running agents on goals that span days, you have heartbeat or monitor-style turns firing on a schedule, you coordinate a controller agent with scoped side agents, or you have already been bitten by an agent confidently redoing work it finished last Tuesday. The demo costs you a minute, and the ideas are portable even if you never adopt the tool.

Probably not yet, if: you use agents for single-session tasks, you are on Windows without WSL, you need something battle-tested with a release history and a large user base, or you do not use Codex-family or shell-capable agents. The value shows up proportionally to how long your loops run. Short loops do not drift, and the ceremony will feel like overhead.

Repo: https://github.com/huangruiteng/loopx

Qwen3.8-Max Just Went GA: A Developer's Guide to Alibaba's 2.4T Model

ArshTechPro — Mon, 03 Aug 2026 22:40:19 +0000

Alibaba made Qwen3.8-Max generally available on August 3, 2026.
This is a practical rundown for developers: what the model is, what it costs, how to call it, and where the claims still need a pinch of salt.

The one-line version

Qwen3.8-Max is a 2.4-trillion-parameter Mixture-of-Experts model with a 1M-token context window, native text/image/video input, OpenAI-compatible API, and pricing of $2 in / $6 out per million tokens. Open weights are promised for next week.

First, the naming confusion

The version number trips people up, so let's clear it up:

Qwen3 is the open-weight model family you've probably used (Qwen3-32B, Qwen3-235B-A22B, etc.)
Qwen3.5 / 3.6 / 3.7 / 3.8 are successive flagship generations, not point releases of Qwen3
"Max" is the top tier of each generation

So Qwen3.8-Max is not "Qwen 3, version 8." It is the newest flagship, and it succeeds Qwen3.7-Max from May 2026. Qwen describes it as their most capable model to date, and the first open-weight model at Max scale.

Also note there is a Qwen3.8-Max-Preview (July 19) and now Qwen3.8-Max (August 3, GA). If you're reading a blog post from July, it's about the preview, and half its "not disclosed yet" list has since been answered.

The specs that matter

Spec	Value
Total parameters	2.4 trillion (sparse MoE)
Active parameters per token	~95 billion (reported)
Context window	1,000,000 tokens
Max input	991K tokens (983K with thinking on)
Max output	131K tokens
Max reasoning budget	262K tokens
Input modalities	Text, image, video
Output	Text
Rate limits	2M tokens/min, 15K requests/min
Model ID	`qwen3.8-max`

Why "95B active" is the number to care about

This is the part worth understanding properly, because "2.4 trillion parameters" is close to meaningless on its own.

A sparse Mixture-of-Experts model is not one giant network. Each layer holds many specialist sub-networks ("experts"), and a router picks a small handful for each token. The rest sit idle for that token.

Qwen's own smaller models make the pattern obvious from the naming: Qwen3-235B-A22B carries 235B total parameters but activates 22B per token, and Qwen3-30B-A3B activates roughly 3B.

So:

Total parameters ≈ how much the model knows (and how much memory you'd need to host it)
Active parameters ≈ how much compute each token actually costs

At 2.4T total / ~95B active, roughly 4% of the network fires per token. That's why Alibaba can sell it at $2/M input rather than something ruinous.

One honesty note: this figure needs a small asterisk. MarkTechPost's launch coverage stated Alibaba had not disclosed the activated-parameter count, while benchmark trackers report that Qwen's own August 3 release post lists 2.4T total with 95B active. Other coverage advises treating the number as reported rather than confirmed until Alibaba publishes a model card. Use it for rough cost intuition, not for capacity planning.

And to be clear: 95B active does not mean you can serve this on a 95B-sized box. A serving system still needs fast access to the full expert pool, plus attention state, routing machinery, multimodal components and runtime buffers. The whole 2.4T checkpoint has to be resident somewhere.

Pricing, and the one trick that dominates your bill

Item	Price per 1M tokens
Input	$2.00
Output	$6.00
Input (implicit cache read)	$0.25
Explicit cache creation	$2.50
Explicit cache read	$0.17

Here's the thing to internalize: cached input is eight times cheaper than fresh input, which means prefix stability drives your cost more than prompt length does.

That single sentence should change how you architect against this model. Concretely, imagine an agent loop with a 200K-token stable prefix (system prompt, tool schemas, codebase context) running 50 turns:

No caching:      50 × 0.2M × $2.00                = $20.00
Explicit cache:  (0.2M × $2.50) + (50 × 0.2M × $0.17) = $2.20

Roughly a 9x difference, from nothing but keeping your prefix byte-stable. Practical implications:

Put everything static at the front of your prompt, and everything variable at the end
Don't inject timestamps, request IDs or shuffled context into your system prompt
If you're rebuilding the prefix per request, you're paying 8x for no reason

Compare against the field: Kimi K3 runs $3.00 input / $15.00 output per million tokens, so Qwen3.8-Max undercuts it meaningfully, particularly on output.

Calling the API

Integration is deliberately boring, which is the point. The hosted API is OpenAI- and DashScope-compatible, so integration is a base-URL and model-ID change. It also supports the Anthropic protocol, so tools such as Cursor, Cline, Codex and Claude Code can point at it through existing integrations.

Here's the multimodal example straight from the model page, using the DashScope SDK:

import os
import dashscope

dashscope.base_http_api_url = "https://dashscope-intl.aliyuncs.com/api/v1"

messages = [
    {
        "role": "user",
        "content": [
            {"image": "https://example.com/your-image.jpeg"},
            {"text": "What is depicted in the image?"}
        ]
    }
]

response = dashscope.MultiModalConversation.call(
    api_key=os.getenv('DASHSCOPE_API_KEY'),
    model='qwen3.8-max',
    messages=messages
)

print(response.output.choices[0].message.content[0]["text"])

If you're already on the OpenAI SDK, you point base_url at Alibaba's compatible-mode endpoint and change the model string to qwen3.8-max. Grab the exact base URL from docs.qwencloud.com rather than guessing, since it differs between the international and China-domestic deployments.

Watch out for the two regional endpoints. Availability is split between an international deployment (qwencloud.com) and a China-domestic one (platform.qianwenai.com), each requiring separate registration and billing. Keys are not interchangeable.

It's also on third-party gateways already. Vercel's AI Gateway added it as alibaba/qwen3.8-max on August 2 at provider pricing with no markup.

Supported features

Function calling, structured outputs, batches, prefix completion and fine-tuning are all supported. Five built-in tools ship on the Responses API: code_interpreter, web_search, web_extractor, t2i_search and i2i_search.

The built-in tools are worth a look before you hand-roll your own. If you're currently maintaining a custom web-search tool wrapper, that's now a server-side flag.

Benchmarks: the honest read

Unlike the July preview, GA came with actual numbers. Alibaba published a benchmark table with the formal launch showing 86.6 on Terminal-Bench 2.1, 67.7 on SWE-bench Pro, and 92.6 on GPQA Diamond, with the strongest gains in multimodal and agentic categories rather than general reasoning.

Filling in the competitive picture:

Benchmark	Qwen3.8-Max	Claude Fable 5	Notes
Terminal-Bench 2.1	86.6	84.6	GPT-5.6 Sol (max) leads at 88.8
SWE-bench Pro	67.7	80.0	Fable 5 ahead
FrontierSWE	73.5	88.8	Fable 5 ahead
GPQA Diamond	92.6	—	Up marginally from 3.7-Max's 92.4
PaperBench	93.0	—	Qwen leads
IFBench	82.8	—	Qwen leads

It also tops most vision rows, including OSWorld-Verified at 86.1, Parametric CAD Bench at 91.5, and OmniDocBench 1.5 at 92.1. Against its own predecessor the jump is large: DeepSWE 1.1 moves from 21.6 to 56.6, FrontierSWE from 40.7 to 73.5, and JobBench from 31.3 to 53.4.

Two caveats that belong in any fair reading, both flagged by MarkTechPost: the multimodal table benchmarks against Qwen3.7-Plus rather than Qwen3.7-Max, which flatters the generational delta; and Alibaba's own RL scaling curve peaks at 0.725 near 4,000 training environments, then declines to 0.719 and 0.689.

Open weights: read the fine print

Alibaba confirmed open weights ship next week, along with a second checkpoint, Qwen3.8-27B. Releases are expected the week of August 10 via Alibaba Cloud Model Studio. This marks Alibaba's return to open-sourcing its top-tier models after keeping several recent flagships proprietary earlier this year.

What's still missing

Being straight about the gaps:

No model card. The GA launch still did not provide an official training and safety model card. No training data disclosure, no safety evaluation methodology.
No published license. Until the weights land, there's nothing to review on commercial-use terms.
Active parameter count is reported, not officially confirmed in a spec sheet.
No independent Artificial Analysis score yet.

Should you use it?

Try it now if: you're doing multimodal work (documents, video indexing, screenshots, UI automation), you want long-context agent runs at a fraction of Western frontier pricing, or you have an existing OpenAI/Anthropic-protocol setup where testing costs you a base-URL change and an afternoon.

References:

Qwen3.8-Max model page (QwenCloud) — canonical specs, pricing, rate limits

Running this in production yet? I'd be interested in real latency and tokens-per-second numbers, since Alibaba hasn't published throughput figures.

AirLLM Runs a 70B Model on a 4GB GPU. It's True, and That's Not the Interesting Part

ArshTechPro — Mon, 03 Aug 2026 22:23:56 +0000

AirLLM's README opens with a line that sounds like it can't be true:

AirLLM dramatically reduces inference memory usage, letting 70B large language models run on a single 4GB GPU card — without quantization, distillation, or pruning.

So let's actually check it. Is the claim real? How do you set it up? And — the question nobody asks loudly enough — should you?

TL;DR: The claim is technically true and the engineering is legitimate.

The trick, in one paragraph

Here's the thing about a transformer: it's a stack of layers, and it runs them in order.

Input → Layer 1 → Layer 2 → Layer 3 → ... → Layer 80 → Output

When Layer 1 is computing, layers 2 through 80 are just... sitting in your VRAM. Doing nothing. Taking up space.

Normal inference loads all 80 layers into GPU memory because keeping them there is fast. AirLLM asks the obvious follow-up question: what if we didn't? Load Layer 1, run it, throw it away, load Layer 2, run it, throw it away.

Now your VRAM requirement isn't "the size of the model." It's "the size of the single largest layer."

For a 70B model at full FP16 precision, that's roughly 1.75GB per layer. Which fits in 4GB with room to spare. The model is still 140GB — it just lives on your disk instead of your GPU, streaming through one slice at a time.

That's it. That's the whole idea. And it genuinely works.

Fact-check: is the claim real?

Yes — with an asterisk the size of the model itself.

Let's take the claims one at a time.

"70B on a single 4GB GPU"

Real. The math checks out (~1.75GB per layer at FP16), and it's been independently reproduced by enough people that this isn't in dispute.

"Without quantization, distillation, or pruning"

Real, and this is the genuinely interesting part. Most "run big models on small hardware" tricks work by making the model worse — squashing weights from 16 bits to 4, which costs you some accuracy. AirLLM doesn't have to. You get the actual, unmodified, full-precision model.

(Quantization is optional here — you can pass compression='4bit' to make it faster. But you don't have to, and that's the distinction they're drawing.)

The bigger numbers, too

The README's scaling table looks absurd but follows from the same logic:

Model	Size	Claimed VRAM
Llama 3.x 70B	70B	~4 GB
Llama 3.1 405B	405B	~8 GB
DeepSeek-V3	671B	~12 GB
Qwen3-235B (MoE)	235B	~3 GB
Kimi K3 (MoE)	2.8T	~3.7 GB

Notice something weird? The 2.8-trillion-parameter model needs less VRAM than the 671B one.

That's not an error. Those are Mixture-of-Experts models. An MoE layer contains hundreds of "expert" sub-networks, but each token only routes to a handful of them. Per the v3.1.0 release notes, Kimi K3 holds 896 experts per layer and routes each token to just 16 — so while a full layer's experts expand to ~55GB, a single token only actually needs ~1GB of them. AirLLM streams just those experts instead of the whole layer.

Sparser model → smaller working set → less VRAM. Counterintuitive, correct.

What the headline leaves out

Here's the part that isn't in the big bold text. From AirLLM's own v3.1.0 release notes, measured on an RTX 6000 Ada:

Metric	Value
Peak VRAM during generation	3.72 GB
One-time init	900 seconds
Generation speed	292 s/token, disk-bound

To be clear about what that means: a 100-token response would take just over 8 hours.

Credit where it's due — the maintainer publishes this honestly in the release notes. It's just not the number on the marketing line.

For more typical setups, community reports land in the range of:

70B on a decent NVMe: roughly 5–35 seconds per token
70B on a MacBook: reports as low as ~0.07 tokens/sec (~14 s/token)
For comparison, llama.cpp with a quantized 70B on an RTX 4090: 8–15 tokens per second

So the honest version of the claim is:

AirLLM doesn't make 70B fast on a 4GB GPU. It makes 70B possible on a 4GB GPU.

Why it's slow (do this math once and you'll never be confused again)

This is worth internalizing, because it explains everything and it's not complicated.

To generate one token, the model must run every single layer. Which means AirLLM must read the entire model off disk — for every token.

So:

seconds per token ≈ model size on disk ÷ disk read speed

Let's plug in a 70B model at FP16 (~140GB):

Storage	Speed	Time per token
Gen4 NVMe SSD	~7 GB/s	~20 s
Gen3 NVMe SSD	~3.5 GB/s	~40 s
SATA SSD	~0.5 GB/s	~280 s
Spinning HDD	~0.15 GB/s

Your disk is your inference engine. The GPU is barely working — it's sitting idle waiting for data. This is why AirLLM users report their fans screaming and their laptop becoming unusable: the bottleneck is I/O and CPU, not compute.

Two consequences that fall right out of this formula:

Use compression='4bit'. It shrinks the bytes you have to read by ~4x. The README advertises up to 3x speedup, and now you know exactly why — this isn't about faster math, it's about moving less data.
RAM is secretly your best upgrade. If your system RAM can hold a big chunk of the model, the OS page cache serves layers from memory instead of disk. This is why people with 128GB machines report dramatically better numbers than the raw disk math predicts.

Setup

Requirements — read this part before you start

Disk space is the #1 thing that will bite you. AirLLM downloads the model, then decomposes it into per-layer shards. For a while, you have both copies on disk.

For a 70B FP16 model, budget:

~140GB (original download)
+ ~140GB (layer shards)
= ~280GB free space

The single most common error in the repo's FAQ — safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer — is, per the maintainers, almost always just you ran out of disk.

You'll also want:

An NVMe SSD (not SATA, definitely not HDD)
As much system RAM as you can get
A Hugging Face token for gated models like Llama
Patience. Real, genuine patience.

1. Install

pip install airllm

For the 4-bit compression speedup (recommended — see the math above):

pip install -U bitsandbytes

2. Run it

from airllm import AutoModel

model = AutoModel.from_pretrained(
    "Qwen/Qwen3-32B",
    compression='4bit',        # ~3x faster; skip for full precision
    delete_original=True,      # deletes the original after splitting — saves ~50% disk
    profiling_mode=True,       # logs per-layer timing so you can see the bottleneck
)

input_text = ['What is the capital of United States?']

input_tokens = model.tokenizer(
    input_text,
    return_tensors="pt",
    return_attention_mask=False,
    truncation=True,
    max_length=128,
    padding=False,             # avoids a common tokenizer error
)

generation_output = model.generate(
    input_tokens['input_ids'].cuda(),
    max_new_tokens=20,
    use_cache=True,
    return_dict_in_generate=True,
)

print(model.tokenizer.decode(generation_output.sequences[0]))

That's the whole API. Swapping to a 671B model is a one-line change:

model = AutoModel.from_pretrained("deepseek-ai/DeepSeek-V3")   # 671B, ~12GB VRAM

Start small. Please run an 8B model first to validate your setup before committing 280GB and several hours to a 70B download.

3. Useful config flags

Flag	What it does
`compression`	`'4bit'` or `'8bit'` block-wise quantization — biggest speed lever
`delete_original`	Deletes the original download after splitting; halves disk usage
`layer_shards_saving_path`	Put shards on a different (faster/bigger) drive
`profiling_mode`	Logs time consumption per layer
`hf_token`	For gated models (Llama, etc.)
`prefetching`	Overlaps loading and compute (~10% gain); on by default

Common errors, decoded

Error	Actual cause
`MetadataIncompleteBuffer`	Out of disk space. It's basically always this.
`401 Client Error... Repo is gated`	Pass `hf_token='...'`
`Asking to pad but the tokenizer does not have a padding token`	Set `padding=False`
`ValueError: max() arg is an empty sequence`	Use `AutoModel`, not a specific model class

macOS

Works on Apple Silicon only. Install mlx plus torch, and make sure you're on native (not Rosetta) Python. Same code otherwise.

Is it worth checking out?

Depends entirely on which of these you are.

Skip it if you want to chat with a big model

This is the fantasy that draws people in, and it just doesn't work. Interactive chat needs ~20+ tokens/sec. AirLLM gives you seconds-to-minutes per token. You will not be having a conversation.

Skip it if you're doing serious volume

Every token reads tens of GB off your SSD. Consumer NVMe drives are rated for a finite number of terabytes written; hammering one with continuous full-model reads and shard rewrites is not what it was designed for. Also, your machine will be effectively unusable while it runs.

It's genuinely great for offline batch work

This is the real use case, and it's underrated.

The killer insight: the expensive part is loading a layer, not using it. So if you load Layer 1 and run 50 prompts through it before moving on, you amortize that cost 50 ways.

One reported benchmark: 35 s/token for a single prompt vs 5.3 s/token when batching 50 — a 6.6x improvement for free.

So if you have 10,000 documents to classify overnight and no GPU budget, AirLLM is a legitimately reasonable tool. Latency doesn't matter when nobody's waiting.

It's great if you need full precision, specifically

Research on quantization effects, numerical reproducibility, evaluating a model as-published — cases where a 4-bit approximation defeats the point. AirLLM is close to the only way to do this on hardware you already own.

The verdict

The claim is real. 70B on 4GB VRAM, full precision, no tricks in the "lying" sense. The engineering is clever and the MoE expert-streaming work is legitimately impressive.

The framing is the problem. "Run 70B on a 4GB GPU" implies you get 70B-quality answers on cheap hardware. What you actually get is 70B-quality answers eventually — at a pace measured in minutes per token, with your SSD as the engine and your GPU mostly idle.

AirLLM didn't remove the cost of running a huge model. It moved it — out of VRAM, into time and disk I/O. Whether that's a good trade depends entirely on whether you have more time than money.

Links:

AirLLM on GitHub

Have you actually run this on real hardware? I'd love to hear your tokens/sec and your disk setup in the comments — the community numbers vary wildly and more data points would help.

OpenWorker: Andrew Ng's Local-First AI Coworker, Explained for Developers

ArshTechPro — Wed, 29 Jul 2026 12:10:20 +0000

OpenWorker shipped in late July 2026. It is MIT-licensed, runs on your own machine, and takes your API key instead of selling you inference.

The pitch is narrow and worth repeating exactly: it is an agent that hands you finished work, not a chat transcript. A drafted document on disk. A Slack reply with the real numbers in it. A calendar that has actually been rearranged.

There are a lot of desktop agents right now. This post is about what makes this one structurally different, what state it is actually in, and how to get it running.

What it is in one paragraph

OpenWorker is a desktop app: a Tauri shell around a React UI, sitting on top of a local Python agent server. You give it an outcome ("prepare a customer brief from these three files and the Jira tickets"). It decomposes that into steps, reaches into your files, terminal, and connected SaaS apps, and produces an artifact. Before anything consequential happens - sending a message, running a shell command, writing to your calendar - it stops and asks.

The engine is built on aisuite, Ng's provider-agnostic LLM library. That matters more than it sounds like: OpenWorker is explicitly positioned as a reference implementation of what you can build on aisuite, so the codebase doubles as a worked example if you are building your own harness.

The four things that actually distinguish it

1. There is no OpenWorker inference service

You paste a key, or you point it at Ollama and use none at all. The curated list covers OpenAI, Anthropic, Google, plus OpenAI-compatible vendors like DeepSeek, GLM, Kimi, Qwen, MiniMax, Mistral, and Grok, plus open-weight models through Together and Fireworks. Roughly thirty models are marked as verified for tool-calling work; you can point it at any other model string and accept the risk yourself.

The practical consequence: your cost is your provider bill, and swapping models is a dropdown, not a migration.

2. The permission model is typed, not a confirmation dialog

This is the part I would actually read the source for. Most agent projects treat approvals as UI polish. OpenWorker classifies every tool call into one of four risk classes:

Risk class	Meaning
`read`	No side effects
`write_local`	Mutates the workspace, path-scoped
`exec`	Runs commands
`external`	Side effects that leave your machine

Five permission modes then decide what happens to each class. discuss and plan are read-only. interactive is the default and asks before writes, commands, and external actions. auto allows everything but stays path-scoped. custom auto-approves a list of tools you name.

Two design choices stand out. First, unattended runs do not raise the autonomy ceiling - they change where the human is reached. Prompts that would have appeared inline get routed to an inbox and the session suspends until you answer. The agent does not quietly gain permissions because you walked away. Second, standing "always allow" rules are restricted to the external class only. Shell commands ask every single time, on purpose.

3. It has an explicit prompt-injection posture

The shipped operator persona instructs the model to treat content coming from tools, logs, web pages, files, and incoming messages as untrusted data, not as instructions. That is written into the persona the app ships with, rather than being left as an exercise for the user. For an agent that reads your Slack and your inbox, this is not optional, and it is good to see it stated rather than assumed.

4. Local-first is close to literal

Conversations, connector tokens, and model keys live in a local secret store. Model calls go straight from your machine to whichever provider you configured. The only cloud component is a small optional broker that handles OAuth handshakes for one-click connectors, and tokens are handed to your machine rather than stored server-side. You can skip sign-in entirely and wire up connectors with manually created credentials.

What it can do out of the box

Real deliverables - documents, spreadsheets, reports, web pages, written to disk as files you can open.
25+ connectors - GitHub, Slack, Jira, Notion, Linear, HubSpot, Outlook, monday.com, Gmail, Google Calendar, and more, plus your terminal and local filesystem. Anything reachable over MCP plugs in too, with per-tool control.
Slack as an entry point - mention @OpenWorker in a channel, a session opens on your desktop, the work runs with your local tools, and the answer comes back as a thread reply.
Scheduled automations - morning briefs, weekly reports, a standing watch on a channel. Runs land in the app with full transcripts.

Setup

Option A: install the app (five minutes)

Download from the repo's release links: macOS (Apple Silicon), macOS 12+, signed and notarized, auto-updating. Or Windows 10/11 x64 - note that Windows builds are not code-signed yet, so SmartScreen will warn you.
Open the app and add a model key, or point it at a running Ollama instance.
Give it a real task. Not "hello" - something with a file attached and a defined output.

Leave it in interactive mode for the first few sessions. Watching what it asks permission for is the fastest way to build an accurate mental model of what it is doing.

Option B: run from source

Prerequisites: Python 3.10+, Node 20+, and the Rust toolchain via rustup if you want the desktop shell rather than the browser UI.

git clone https://github.com/andrewyng/openworker
cd openworker

# 1. One-time bootstrap - creates the Python venv at .venv
#    On Windows, run this from Git Bash or WSL
bash packaging/setup_dev_env.sh

# 2. Start the local agent server
.venv/bin/openworker-server --cwd ~/some/project --port 8765
# Windows: .venv\Scripts\openworker-server.exe

Then in a second terminal:

cd surfaces/gui
npm install
npm run dev        # browser UI on the Vite dev port

For the full desktop app instead of the browser UI, replace that last step with npm run tauri dev from surfaces/gui/. The Tauri shell launches the window and supervises the Python server itself, so you can skip step 2 in that case.

The server is FastAPI on uvicorn, bound to 127.0.0.1:8765 by default.

Running the tests:

.venv/bin/pytest              # backend
cd surfaces/gui
npm test                      # GUI unit tests
npm run e2e                   # hermetic end-to-end

Note the --cwd flag on the server. That is the workspace root the agent's path-scoped file access is anchored to. Point it at a scratch directory for your first runs, not at your home folder.

Where to look in the code

Directory	Contents
`coworker/`	Python backend - agent engine, model providers, connectors, MCP client, memory, automations
`surfaces/gui/`	React UI plus the Tauri shell
`stt/`	Rust speech-to-text sidecar for voice input
`packaging/`	macOS DMG and Windows installer builds, auto-update manifest
`docs/`	Design specs and decision logs
`tests/`	Backend test suite

If you are here to learn rather than to use, docs/ and the permission engine in coworker/ are the highest-value reads. The design logs explain choices rather than just documenting APIs.

Is it worth looking into?

Yes, if any of these describe you:

You want a desktop agent that touches real work systems but you are unwilling to hand a vendor your Slack tokens and your filesystem.
You are building an agent harness and want a substantial, production-shaped reference for permission gating, connector management, and MCP integration. Roughly 32k lines of Python with 78 backend test modules is a real codebase to learn from, not a demo.
You have compliance constraints that rule out hosted agents. Local-first plus BYO-key plus MIT is a rare combination.
You want to run agents against fully local models. The Ollama path needs no key at all.

Hold off if:

You need stability. This is v0.1.x, self-labeled open beta, with a handful of contributors and a repo that is days old. It auto-updates, which is convenient and also means the thing under you moves.
You expect a coding agent. It works with your terminal and files, but the framing is knowledge work - briefs, reports, inbox triage, calendar - not writing your service.

The honest summary

The interesting claim here is not "agent that does tasks." Everyone is shipping that. The interesting claim is that the boring infrastructure - who is allowed to do what, where secrets live, what counts as an instruction versus data - has been designed as a typed system rather than bolted on. That is the part worth reading even if you never use the app.

It is early. Expect rough edges, expect the Windows signing situation to resolve, expect the connector list to shift. But if you have been waiting for a credible open, local, model-agnostic option in this category, this is the first one that looks like it was built by people who thought about the failure modes first.

Repo: github.com/andrewyng/openworker
Site: openworker.com

Openship: Your Deployment Tool Shouldn't Compete With Your App for RAM

ArshTechPro — Sun, 26 Jul 2026 16:59:30 +0000

The self-hosted PaaS drawer is not empty. Coolify, Dokploy, CapRover, Dokku, Kamal. If you have wanted a Heroku you actually own, someone has already built you three of them.

So when Openship showed up in July 2026 and hit several thousand GitHub stars in about a week, the reasonable reaction was skepticism. Another dashboard for docker compose up.

Having read through it, there is one architectural decision here that is actually different, and it is the kind of difference you feel on a 4 GB VPS. This post covers what that decision is, how to get the thing running three different ways, and an honest read on whether you should put anything you care about on it yet.

The one idea that matters

Every self-hosted PaaS in that drawer runs its entire control plane on your server. The dashboard, the database backing it, the build system, the CI runner, the webhook listener, and your actual apps all share one box and one set of resources.

You notice this the first time a Next.js build on a 4 GB VPS starves the running production container and takes your site down. Your deployment tool became a reliability risk for the thing it deploys.

Openship inverts it. The control plane runs on your machine, or on a separate box, and drives your servers over SSH. Builds happen locally. What gets shipped to production is a finished container image.

You                              Your server
┌──────────────────────┐         ┌──────────────────────┐
│  Desktop / Web UI    │   SSH   │                      │
│  Build pipeline      │────────→│  Your apps. Only.     │
│  CLI / API           │         │                      │
└──────────────────────┘         └──────────────────────┘

The practical consequences are worth spelling out:

Your production server does not need build-sized resources. It runs containers. That is it. A small VPS goes further.

Your deployment tool has no public attack surface. In the desktop configuration, Openship exposes no port and has no public URL. The only thing reachable from the internet is an app you deliberately gave a domain to.

Builds use your laptop's CPU, which is almost certainly faster than your VPS's shared vCPUs.

The tradeoff is honest and the maintainers state it: the dashboard is only up while your machine is. No teammate access, no phone access, and push-to-deploy webhooks need a stable public endpoint that a laptop is not. If you need any of those, you run the control plane on a server instead, at which point you have re-created the normal arrangement, just deliberately rather than by default.

Building on the server is also available as a config flag. The README's phrasing is that it is a config flag, not a religion.

What is actually in the box

Openship bundles more than deployment:

Area	What you get
CI/CD	Push-to-deploy, preview environments, staging and prod flows, rollbacks
Stacks	Node, Python, Go, Rust, PHP, Ruby, Java, .NET, Docker, monorepos
Data	Postgres, MySQL, MongoDB, Redis, workers, WebSockets, storage
Domains and SSL	Let's Encrypt automation via OpenResty and certbot, wildcards, auto-renewal
CDN	Edge caching, HTTP/3, Brotli, instant purge
Mail	Built-in SMTP with DKIM, SPF, DMARC
Backups	Scheduled database and volume backups, one-click restore, exportable
Automation	REST API, and MCP so AI agents can drive it

The built-in mail server is the unusual one. Running your own SMTP with correct DKIM, SPF, and DMARC is genuinely annoying, and most self-hosted PaaS tools tell you to go use SES or Mailgun. Whether it survives contact with real deliverability is a separate question, but it is a real differentiator on the feature list.

Under the hood it is a TypeScript monorepo: an Electron desktop app, a Next.js dashboard (React 19, Tailwind 4, xterm.js for terminals), and a CLI built on Commander and Clack. A single OPENSHIP_TARGET environment flag decides whether an instance behaves as desktop, selfhosted, or cloud, which is why the same codebase powers the local app and the managed SaaS.

Pick your setup first

The install path depends entirely on who is using it and whether it needs to be up when your machine is off.

Setup	Use when	Where the control plane runs
Desktop app	Just you, private	Your machine, driving servers over SSH
Self-hosted on a server	Team, always-on, CI, push-to-deploy	A Linux box at a public URL, invite-only
Openship Cloud	You want zero ops	Managed

Decide before you install. Switching later is possible but it is not a no-op.

Path 1: Desktop app (solo, and the recommended one)

Download the build for your OS from openship.io, or let the CLI fetch it:

openship install

On Linux the download is an AppImage:

chmod +x Openship.AppImage && ./Openship.AppImage

On first launch it asks what you want to manage:

This Machine — manage the local box.
Another Server — add a remote server over SSH with host, user, and key. There is a Test Connection button. Use it before you go further, because SSH problems surface much more clearly here than mid-deploy.

This is the configuration where Openship's architecture pays off most. Nothing about the platform is exposed. It is worth starting here even if you eventually want the server setup, because you learn the model without also debugging a public deployment.

Path 2: CLI on a server (teams and CI)

Point your domain's DNS A record at the server first. Doing this after the fact means waiting on propagation while the certificate step fails repeatedly.

Install the CLI, which bundles the API and dashboard:

npm i -g openship
# or
curl -fsSL https://get.openship.io | sh

Then bring it up:

openship up --public-url https://ops.example.com --managed-edge

What each part does:

openship up installs Openship as a background service that starts on boot and auto-restarts, then runs a setup wizard that creates the first admin and attaches your domain.
--public-url makes the dashboard reachable at your domain. Login is required and public signup is disabled; everyone else joins by invite.
--managed-edge installs OpenResty and provisions a free Let's Encrypt certificate, routing your domain to the dashboard. Omit this if you already run your own reverse proxy, or you will have two things fighting over ports 80 and 443.

For an attached, non-service run:

openship up --foreground

A nice touch: once it is up, Openship registers itself as an app in its own dashboard. You manage its domain, tail its logs, and see it listed as Live alongside everything else. Self-hosting the control plane as a first-class app rather than a special case is the sort of detail that suggests someone actually ran this in anger.

If you lose the admin password:

openship reset-admin-password

That works locally on the box without signing in, using the internal token. Which also means: anyone with shell access to that server owns your Openship instance. Treat SSH access accordingly.

Path 3: Docker Compose (and why the maintainers steer you away)

git clone https://github.com/oblien/openship.git && cd openship
cp .env.example .env
docker compose up -d

This works, but the docs explicitly label it as not the recommended path, and the reason is worth understanding rather than ignoring.

The compose stack gives the control-plane container access to the host Docker daemon. That is effectively host-level privilege. A container that can talk to the host's Docker socket can start a privileged container, mount the host filesystem, and escape. It is a well-understood escalation path, not a theoretical one.

There are legitimate reasons to want a containerized control plane. Just know that you are trading the security posture that was the point of the architecture in the first place.

Deploying something

Once the control plane is running, the app workflow is short:

cd my-project
openship init          # links this directory to a project (.openship/project.json)
openship deploy        # triggers a deployment
openship logs <deploymentId> -f    # stream build and runtime logs

Openship detects the stack, builds it, wires up config, and ships the container. No YAML pipeline, no Dockerfile required unless you want one. Existing Docker Compose files can be deployed as-is.

The auto-detection claim is the one I would test against your actual repo before believing it. Framework detection is easy for a vanilla Next.js app and gets progressively less magical as your monorepo gets weirder.

CLI cheat sheet

Instance management:

Command	Does
`openship up [--foreground]`	Start as a service, or attached
`openship stop`	Stop the service
`openship status [--json]`	Running state, ports, API health
`openship open`	Open the dashboard in a browser
`openship update`	Update CLI and bundled server
`openship doctor`	Diagnose config, context, runtime

Deploy and inspect:

Command	Does
`openship init`	Link current directory to a project
`openship deploy`	Trigger a deployment
`openship logs <id> [-f] [--tail N]`	View or stream deployment logs
`openship deployment`	List and manage deployments
`openship project`	List and manage projects
`openship service`	Services within a stack
`openship domain`	A project's domains

Infrastructure and auth:

Command	Does
`openship server`	Manage self-hosted SSH servers
`openship system`	Read and update instance settings
`openship mail`	Mail server setup
`openship backup`	Backup policies and schedules
`openship login` / `logout`	Authenticate with a Personal Access Token
`openship context`	Switch which instance the CLI talks to
`openship api <method> <path>`	Authenticated request to any API route

--json works on most read commands, which makes this scriptable in CI. And openship api following the gh api pattern is a good call, since it means you are never blocked waiting for a CLI command to be written for an endpoint that already exists.

The MCP angle, with a caveat

Openship ships MCP support so AI agents can drive deployments. The pitch is roughly: add the MCP server, and your agent can do the work.

Worth pausing on. You are handing an agent deploy permissions, secrets access, and production rollback control. Standard MCP hygiene applies and applies harder here: scope the tools you expose, point agents at staging rather than production, use a Personal Access Token with the narrowest permissions that work, and keep an audit trail of what the agent did.

An agent that can deploy can also deploy the wrong thing at 3am while nobody is watching.

How ready is this, actually

Being straight with you, since dev.to posts that skip this part are advertising.

Known rough edges. As of July 2026 there are open reports of breakage in the CLI installer and in the self-hosted SSH deploy path, including deploys failing immediately after a successful Docker build. The issue queue is active, which is a good sign for responsiveness and a clear sign it is still early.

License. This one matters and is easy to get wrong. Openship launched under AGPL-3.0 plus the Commons Clause, which is source-available rather than open source and prohibited offering it as a hosted service.

Roadmap gaps. Multi-node clusters, load balancing UI, private networking, and visual CI/CD pipelines are all listed as coming, not shipped. If clustering is a requirement, this is not your tool yet.

Should you use it

Try it now if you are a solo developer or small team, you have side projects or non-critical services, and the local-build architecture appeals to you. The desktop app is genuinely low-risk to experiment with, since it does not expose anything and does not need a server to evaluate.

The architecture is the real contribution here. Keeping build infrastructure off the production box is correct, and it is a bit odd that it took this long for a self-hosted PaaS to treat it as the default rather than an option. Even if Openship itself does not win the category, that idea should propagate.

Links:

Repo: github.com/oblien/openship

If you run it on a real VPS, especially the SSH deploy path, post what broke in the comments. Early projects get better fastest when the failure reports are specific.

Unlimited-OCR: Parsing a 40-Page PDF in One Pass Without Your GPU Melting

ArshTechPro — Fri, 24 Jul 2026 17:19:34 +0000

If you have ever built a document-parsing pipeline, you know the ritual. Split the PDF into pages. Run each page through the OCR model. Stitch the outputs back together. Then write a pile of glue code to fix the tables that got cut in half at a page boundary, and the heading that lost its section, and the footnote that ended up orphaned three pages away from its reference.

Baidu's Unlimited-OCR is a bet that you should not have to do any of that. It parses dozens of pages in a single forward pass, and it is MIT licensed.

This post covers what the model actually does differently, and then three concrete ways to get it running: a quick Transformers script, an SGLang server for real throughput, and a Docker image if you would rather not touch a Python environment at all.

The problem it solves

Standard transformer decoding has a cost that most OCR benchmarks quietly hide: the KV cache grows with every token you generate.

The KV cache is the model's short-term memory. Every token the model produces gets appended to it so future tokens can attend backwards. For a chatbot writing three paragraphs, this is fine. For an OCR model transcribing a 40-page technical manual into 30,000 tokens of Markdown, it is not. Memory climbs, attention cost climbs with it, and generation gets slower the longer it runs. Somewhere around page eight your throughput graph stops looking like a line and starts looking like a cliff.

The industry workaround has been chunking. Process one page, dump the cache, process the next page, and accept that the model has no idea what came before.

Unlimited-OCR attacks the cache growth directly. The team replaced every attention layer in the decoder with something they call Reference Sliding Window Attention (R-SWA).

The intuition is close to how a human copy-typist works. You do not hold every word you have already typed in your head. You hold the last sentence or two, and you keep glancing back at the source document. R-SWA does the same thing: the decoder keeps a fixed-size window of recently generated tokens, but it retains permanent access to the original image tokens. The KV cache is implemented as a queue with a fixed capacity, so when a new token arrives, the oldest one in the window gets evicted.

The result is a cache size that is constant rather than growing. Memory and per-token latency stay flat whether you are on token 500 or token 30,000.

What you actually get

3B total parameters, 500M activated. It is a Mixture-of-Experts model, so the compute cost at inference is closer to a 500M model than a 3B one.
32K max output length, which is what makes multi-page single-pass parsing viable.
93.23 on OmniDocBench v1.5, about 6.2 points above the DeepSeek-OCR baseline it was continue-trained from. Worth noting because efficiency work usually costs you accuracy; here it did not.
MIT license. Commercial use, no asterisks.

The architecture is DeepSeek-OCR's DeepEncoder (SAM-ViT-B plus CLIP-L) feeding an MoE decoder. The encoder's aggressive visual token compression is what keeps the image side of the cache small enough for the whole thing to work.

One thing the paper flags that is easy to miss: R-SWA is not OCR-specific. It is a general-purpose attention mechanism for any long-horizon transcription-shaped task, ASR included.

Before you start

You need an NVIDIA GPU. There is no CPU or Apple Silicon path in the official repo.

The weights are bfloat16, so roughly 6 GB just to load the model, before activations and image tokens. A 12 GB card is a sane floor for single-image work; for multi-page runs at 32K context you will want more headroom. The maintainers tested on Python 3.12.3 with CUDA 12.9.

If you just want to see output before committing to any of this, there is a Hugging Face Space you can drop a file into.

Path 1: Transformers (start here)

This is the fastest way to a working result. Set up the environment:

python -m venv .venv
source .venv/bin/activate

pip install torch==2.10.0 torchvision==0.25.0 transformers==4.57.1
pip install Pillow==12.1.1 matplotlib==3.10.8 einops==0.8.2
pip install addict==2.4.0 easydict==1.13 pymupdf==1.27.2.2 psutil==7.2.2

Pin these versions. The model ships custom modeling code via trust_remote_code, and that code is written against these specific releases. Version drift here produces confusing import errors rather than clean failures.

Now a single image:

import torch
from transformers import AutoModel, AutoTokenizer

model_name = 'baidu/Unlimited-OCR'

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_name,
    trust_remote_code=True,
    use_safetensors=True,
    torch_dtype=torch.bfloat16,
)
model = model.eval().cuda()

model.infer(
    tokenizer,
    prompt='<image>document parsing.',
    image_file='your_image.jpg',
    output_path='your/output/dir',
    base_size=1024, image_size=640, crop_mode=True,   # gundam mode
    max_length=32768,
    no_repeat_ngram_size=35, ngram_window=128,
    save_results=True,
)

Two details in there matter more than they look:

The prompt must literally start with <image>. It is not decoration, it is the placeholder token the visual features get spliced into. Drop it and you will get output that looks like the model hallucinating a document it never saw.

no_repeat_ngram_size and ngram_window are load-bearing. Long-horizon generation on repetitive layouts, think a table of contents or a price list, can send the model into a loop where it happily emits the same row forever. These parameters block any 35-gram from repeating inside a sliding window. Do not remove them because they look like tuning noise.

gundam vs base

Single-image inference gives you two configurations, and the naming is not self-explanatory:

Mode	Settings	Use for
gundam	`base_size=1024, image_size=640, crop_mode=True`	Single images, especially dense ones. Crops the image into tiles and processes each, so small text survives.
base	`base_size=1024, image_size=1024, crop_mode=False`	Whole image at once. Required for multi-page.

Multi-page and PDF paths only support base mode. That is a hard constraint, not a default.

Multiple pages

model.infer_multi(
    tokenizer,
    prompt='<image>Multi page parsing.',
    image_files=['page1.png', 'page2.png', 'page3.png'],
    output_path='your/output/dir',
    image_size=1024,
    max_length=32768,
    no_repeat_ngram_size=35, ngram_window=1024,
    save_results=True,
)

Note that ngram_window jumps from 128 to 1024 here. Across many pages there is legitimately more repeated structure, so the repeat-detection window has to widen to keep up.

PDFs

There is no direct PDF entry point. You rasterize first, then feed the images to infer_multi:

import os, tempfile
import fitz  # PyMuPDF

def pdf_to_images(pdf_path, dpi=300):
    doc = fitz.open(pdf_path)
    tmp_dir = tempfile.mkdtemp(prefix='pdf_ocr_')
    mat = fitz.Matrix(dpi / 72, dpi / 72)
    paths = []
    for i, page in enumerate(doc):
        out = os.path.join(tmp_dir, f'page_{i+1:04d}.png')
        page.get_pixmap(matrix=mat).save(out)
        paths.append(out)
    doc.close()
    return paths

model.infer_multi(
    tokenizer,
    prompt='<image>Multi page parsing.',
    image_files=pdf_to_images('your_doc.pdf', dpi=300),
    output_path='your/output/dir',
    image_size=1024,
    max_length=32768,
    no_repeat_ngram_size=35, ngram_window=1024,
    save_results=True,
)

300 DPI is the recommended default. Going lower to save memory costs you small text and table rules, which is usually the exact content you cared about.

Path 2: SGLang server (for anything real)

Transformers is fine for evaluating the model. For a service handling concurrent requests, run SGLang and talk to it over an OpenAI-compatible API.

Set up the environment. Note that SGLang ships as a local wheel in the repo rather than from PyPI:

uv venv --python 3.12
source .venv/bin/activate

uv pip install wheel/sglang-0.0.0.dev11416+g92e8bb79e-py3-none-any.whl
uv pip install kernels==0.11.7
uv pip install pymupdf==1.27.2.2

Small heads-up: the README prose mentions pinning kernels==0.9.0 while the command block right beneath it installs 0.11.7. Follow the command block. If you hit kernel-related errors, that mismatch is the first thing to check against the current repo state.

Launch the server:

python -m sglang.launch_server \
    --model baidu/Unlimited-OCR \
    --served-model-name Unlimited-OCR \
    --attention-backend fa3 \
    --page-size 1 \
    --mem-fraction-static 0.8 \
    --context-length 32768 \
    --enable-custom-logit-processor \
    --disable-overlap-schedule \
    --skip-server-warmup \
    --host 0.0.0.0 \
    --port 10000

The flags that are not optional:

--enable-custom-logit-processor — the no-repeat-ngram processor runs as a custom logit processor. Without this flag, requests referencing it fail.
--attention-backend fa3 — FlashAttention 3, which requires Hopper-class hardware. On older GPUs you will need a different backend.
--page-size 1 and --disable-overlap-schedule — R-SWA's cache eviction does not play nicely with the usual paged-attention and overlapped-scheduling optimizations.

Then a client. The image goes in as a base64 data URL, standard OpenAI vision format, with a couple of model-specific extras:

import base64, json, os
import requests
from sglang.srt.sampling.custom_logit_processor import (
    DeepseekOCRNoRepeatNGramLogitProcessor,
)

server_url = "http://127.0.0.1:10000"
session = requests.Session()
session.trust_env = False

def encode_image(image_path):
    ext = os.path.splitext(image_path)[1].lower()
    mime = "image/jpeg" if ext in (".jpg", ".jpeg") else f"image/{ext.lstrip('.')}"
    with open(image_path, "rb") as f:
        data = base64.b64encode(f.read()).decode("utf-8")
    return {"type": "image_url", "image_url": {"url": f"data:{mime};base64,{data}"}}

def generate(prompt, image_paths, image_mode, ngram_window):
    content = [{"type": "text", "text": prompt}] + [
        encode_image(p) for p in image_paths
    ]
    payload = {
        "model": "Unlimited-OCR",
        "messages": [{"role": "user", "content": content}],
        "temperature": 0,
        "skip_special_tokens": False,
        "images_config": {"image_mode": image_mode},
        "custom_logit_processor": DeepseekOCRNoRepeatNGramLogitProcessor.to_str(),
        "custom_params": {"ngram_size": 35, "window_size": ngram_window},
        "stream": True,
    }
    response = session.post(
        f"{server_url}/v1/chat/completions",
        headers={"Content-Type": "application/json"},
        data=json.dumps(payload),
        timeout=1200,
        stream=True,
    )
    response.raise_for_status()

    chunks = []
    for line in response.iter_lines(chunk_size=1, decode_unicode=True):
        if not line or not line.startswith("data: "):
            continue
        data = line[len("data: "):]
        if data == "[DONE]":
            break
        delta = json.loads(data)["choices"][0].get("delta", {}).get("content", "")
        if delta:
            print(delta, end="", flush=True)
            chunks.append(delta)
    return "".join(chunks)

generate("document parsing.", ["your_image.jpg"],
         image_mode="gundam", ngram_window=128)

temperature: 0 and skip_special_tokens: False are both deliberate. This is transcription, not generation, so any sampling randomness is pure downside. And the special tokens carry layout structure you want in the output.

The 1200 second timeout is also not paranoia. Long documents take a while, which is precisely why you want streaming.

Batch processing

The repo ships infer.py, which starts the SGLang server for you and fires concurrent requests at it:

# A directory of images
python infer.py \
    --image_dir ./examples/images \
    --output_dir ./outputs \
    --concurrency 8 \
    --image_mode gundam

# A PDF
python infer.py \
    --pdf ./examples/document.pdf \
    --output_dir ./outputs \
    --concurrency 8 \
    --image_mode gundam

Useful extras: --model_dir accepts a local path or a Hugging Face ID, --gpu sets CUDA_VISIBLE_DEVICES, and --server_log puts server output somewhere you can read it.

Start concurrency low. Eight is the documented example, but the right number depends on your VRAM and how long your documents are.

Path 3: vLLM via Docker

If you want to skip environment management entirely:

# Default, CUDA 13.0
docker pull vllm/vllm-openai:unlimited-ocr

# Hopper GPUs, CUDA 12.9
docker pull vllm/vllm-openai:unlimited-ocr-cu129

docker run --rm --gpus all --network host --ipc host \
    vllm/vllm-openai:unlimited-ocr \
    baidu/Unlimited-OCR \
    --trust-remote-code \
    --logits_processors vllm.model_executor.models.unlimited_ocr:NGramPerReqLogitsProcessor \
    --no-enable-prefix-caching \
    --mm-processor-cache-gb 0

Same story as SGLang regarding the ngram processor, it has to be registered at server start. Prefix caching and the multimodal processor cache are both disabled because R-SWA's fixed-window cache invalidates the assumptions those optimizations make.

Full details are in the official vLLM recipe.

Parameter cheat sheet

Parameter	Single image	Multi-page / PDF
mode	gundam or base	base only
`image_size`	640 (gundam) / 1024 (base)	1024
`crop_mode`	True (gundam) / False (base)	False
`ngram_window`	128	1024
`no_repeat_ngram_size`	35	35
`max_length`	32768	32768
prompt	`<image>document parsing.`	`<image>Multi page parsing.`

When this is the wrong tool

Reach for something else if you need bounding boxes and per-word confidence scores, since this is an end-to-end model producing Markdown, not a detection-plus-recognition pipeline. Same if you are OCRing short receipts or single lines, where a 3B model on a GPU is enormous overkill compared to PaddleOCR or Tesseract. And if you have no NVIDIA GPU, there is no supported path today.

Wrapping up

The interesting claim here is not the benchmark number. It is that a fixed-size KV cache made the model both faster and more accurate on long documents, when efficiency work almost always costs you quality somewhere.

If R-SWA generalizes the way the authors suggest, the same trick applies to any task where a model transcribes a long input into a long output while keeping the source in view. Long-form ASR is the obvious next one.

Repo: github.com/baidu/Unlimited-OCR

Kimi Code CLI: A Beginner-Friendly Guide to Moonshot AI's Terminal Coding Agent

ArshTechPro — Sun, 19 Jul 2026 04:49:26 +0000

If you have used tools like Claude Code or similar terminal-based AI agents, Kimi Code CLI will feel familiar. It is Moonshot AI's open-source coding agent that lives in your terminal, reads and edits your code, runs shell commands, and works through tasks step by step based on the results it gets back.

This post explains what Kimi Code CLI is, why it might be useful to you, and walks through installing and using it for the first time.

What is Kimi Code CLI

Kimi Code CLI is an AI coding agent that runs directly in your terminal. Instead of copying code back and forth between a chat window and your editor, you describe a task in plain language, and the agent:

Reads and edits files in your project
Runs shell commands
Searches through your codebase
Fetches web pages when it needs external information
Decides its next step based on the output it gets, rather than following a fixed script

It works out of the box with Moonshot AI's Kimi models, and it can also be pointed at other compatible providers like Anthropic, OpenAI, or Google by editing a config file.

The project is written in TypeScript, MIT licensed, and distributed as a single binary, so you do not need a Node.js setup just to run it.

Why it stands out

A few design choices make it worth trying if you already use a terminal-based AI agent, or are looking for your first one:

Single-binary install. No Node.js, no PATH conflicts, no global package clutter. One script and you are done.
Fast startup. The terminal UI (TUI) loads in milliseconds.
Video input. You can drop a screen recording into the chat, and the agent can turn it into working code or a short summary. This is unusual among coding agents.
Conversational MCP setup. Model Context Protocol (MCP) servers can be added and authenticated by talking to the CLI with /mcp-config, instead of hand-editing JSON files.
Plugin ecosystem. You can install skills, MCP servers, and data sources from a marketplace or directly from a GitHub repo, with the trust level of each shown upfront.
Subagents. Built-in coder, explore, and plan subagents can run in isolated contexts in parallel, keeping your main conversation clean.
Lifecycle hooks. You can run local commands at key points, for example to block a risky tool call, log decisions, or trigger a desktop notification.
Editor integration via ACP. Using the Agent Client Protocol, you can drive a Kimi Code CLI session from Zed, JetBrains IDEs, or any other ACP-compatible client.

Installing Kimi Code CLI

You have two main installation paths: the official install script, or npm.

Option 1: Install script (recommended)

macOS or Linux

curl -fsSL https://code.kimi.com/kimi-code/install.sh | bash

Windows (PowerShell)

irm https://code.kimi.com/kimi-code/install.ps1 | iex

On Windows, install Git for Windows first, since Kimi Code CLI uses the bundled Git Bash as its shell environment. If Git Bash is in a non-standard location, set the KIMI_SHELL_PATH environment variable to the full path of bash.exe.

Homebrew (macOS/Linux)

brew install kimi-code

The script downloads the latest release, verifies its checksum, and adds the kimi executable to your PATH.

Option 2: Install via npm

If you already have Node.js 22.19.0 or later:

node --version
npm install -g @moonshot-ai/kimi-code

Or with pnpm:

pnpm add -g @moonshot-ai/kimi-code

Verify the install

Open a new shell session and run:

kimi --version

If that prints a version number, you are ready to go.

First launch and login

Move into a project directory and start the interactive UI:

cd your-project
kimi

On first launch, log in from inside the CLI:

/login

This opens a platform selector with two options:

Kimi Code (OAuth) — a device-code flow. You open a link on any device, sign in, and enter a code to authorize.
Kimi Platform API key — paste in a key from platform.kimi.com or platform.kimi.ai.

To log out later, run /logout.

If you would rather use Anthropic, OpenAI, Google, or another compatible provider, edit ~/.kimi-code/config.toml directly and add your API key there.

Trying your first task

Once logged in, just describe what you want in plain English. A good first prompt is to have it explore your project:

Take a look at this project's directory structure and briefly describe what each directory is for.

Kimi Code CLI will call its file-reading and search tools automatically to gather context before answering. Read-only operations run without asking for confirmation. Anything that modifies files or runs shell commands will prompt you to approve it first.

You can also hand it a concrete coding task:

Add a function in src/utils that converts any string to kebab-case, and add a unit test for it.

The agent plans the steps, edits the code, runs the tests, and reports back what it did.

Running a single instruction without the interactive UI

kimi -p "Take a look at this project's directory structure"

Resuming your previous session

kimi -c

Useful commands to know

Command	What it does
`/help`	Opens the command and shortcut panel
`/new`	Starts a new session, clearing current context
`/sessions`	Browse and resume past sessions
`/model`	Switch the active model
`/compact`	Manually compress context to free up tokens
`/fork`	Fork the current session, keeping history but continuing separately
`/mcp-config`	Add and authenticate MCP servers conversationally
`/exit`	Exit the CLI

Handy keyboard shortcuts:

Shortcut	What it does
`Esc`	Interrupt streaming output or close a popup
`Ctrl-C`	Interrupt output; press twice while idle to exit
`Shift-Tab`	Toggle Plan mode
`Ctrl-S`	Inject a message mid-stream
`Ctrl-O`	Collapse or expand tool output

Using it inside your editor (ACP)

Kimi Code CLI supports the Agent Client Protocol, which lets editors like Zed and JetBrains drive a session directly.

For Zed, add this to ~/.config/zed/settings.json:

{
  "agent_servers": {
    "Kimi Code CLI": {
      "type": "custom",
      "command": "kimi",
      "args": ["acp"],
      "env": {}
    }
  }
}

Then open a new conversation in Zed's Agent panel. No extra login is needed since it reuses your existing CLI credentials.

Where your data lives

Kimi Code CLI stores config files, session records, logs, and its update cache under ~/.kimi-code/ by default. If you want to move that elsewhere, set the KIMI_CODE_HOME environment variable to a different path.

Upgrading and uninstalling

Upgrade

kimi upgrade

Or, if you installed via npm:

npm install -g @moonshot-ai/kimi-code@latest

Uninstall

If you installed via the script, just delete the kimi executable. If you installed via npm:

npm uninstall -g @moonshot-ai/kimi-code

Building it from source

If you want to contribute or run the latest code from main:

git clone https://github.com/MoonshotAI/kimi-code.git
cd kimi-code
pnpm install

pnpm dev:cli    # run the CLI in dev mode
pnpm test       # run tests
pnpm typecheck  # TypeScript check
pnpm lint       # oxlint
pnpm build      # build all packages

This requires Node.js 24.15.0 or later and pnpm 10.33.0. Check CONTRIBUTING.md in the repo for the full contribution guide.

Wrapping up

Kimi Code CLI brings together a lot of what people already like about terminal-based AI coding agents: a fast single-binary install, a clean TUI, subagents, MCP support, and editor integration through ACP. The video input feature and the conversational MCP setup are the two pieces that stand out as genuinely different from similar tools.

If you want to try it, the fastest path is the install script followed by /login. From there, just start describing what you want done.

Links

Repository: https://github.com/MoonshotAI/kimi-code

Strix: Give Your App a Free AI Pentester Before It Ships

ArshTechPro — Thu, 02 Jul 2026 17:13:58 +0000

If you have ever pushed code to production and quietly hoped nobody probes it too hard, this project is for you.

Strix is an open-source tool that runs autonomous AI agents against your application the way a real attacker would: it spins up your app, pokes at it, tries to break in, and proves whether a vulnerability is real before it tells you about it. No SaaS lock-in required, no waiting weeks for a pentest firm to get back to you.

In this article I will walk through what Strix actually does, how it is different from the static analysis tools you already have, and how to run your first scan.

The Problem With Most "Security Scanning"

Most security tooling developers use day to day falls into two buckets:

Static analysis (SAST) — scans your source code for risky patterns. Fast, but full of false positives, and it has no idea how your app actually behaves at runtime.
Dependency scanners — flag known CVEs in your packages. Useful, but they say nothing about the business logic bugs you wrote yourself.

Neither of these actually runs your application and tries to exploit it. That's traditionally been the job of a human penetration tester, and hiring one is slow and expensive.

Strix tries to close that gap by giving AI agents an actual hacker's toolkit and letting them attack a running instance of your app, the same way a person would.

What Strix Actually Does

Strix agents come with:

A full HTTP proxy for inspecting and manipulating requests and responses
Browser automation for testing things like XSS, CSRF, and auth flows across multiple tabs
Terminal access for running commands and testing interactively
A Python runtime for writing custom exploits on the fly
Reconnaissance tooling for mapping out the attack surface
Static and dynamic code analysis

Instead of one agent doing everything, Strix uses a "graph of agents" model, so multiple specialized agents can work in parallel on different parts of your app and share what they find with each other.

Critically, when Strix reports a vulnerability, it comes with an actual proof-of-concept demonstrating that the exploit works, not just a pattern match that says "this line looks suspicious." That is the main thing that separates it from a linter with a security label on it.

It can detect things like:

Access control issues — IDOR, privilege escalation, auth bypass
Injection attacks — SQL, NoSQL, command injection
Server-side flaws — SSRF, XXE, insecure deserialization
Client-side flaws — XSS, prototype pollution, DOM issues
Business logic bugs — race conditions, workflow abuse
Auth issues — JWT vulnerabilities, broken session management
Infrastructure misconfigurations and exposed services

Installing and Running Your First Scan

You need two things before you start:

Docker, running locally (Strix's agents operate inside a sandbox container)
An API key from a supported LLM provider — OpenAI, Anthropic, Google, or a local model via Ollama/LMStudio

Install it:

curl -sSL https://strix.ai/install | bash

Configure which model powers the agents:

export STRIX_LLM="openai/gpt-5.4"
export LLM_API_KEY="your-api-key"

Then point it at a target:

strix --target ./app-directory

That's it. On first run it will pull the sandbox Docker image, then start testing. Results land in strix_runs/<run-name>.

Strix will also remember your config after the first run, saving it to ~/.strix/cli-config.json so you don't have to set environment variables every time.

Beyond a Local Folder

Strix isn't limited to scanning a directory on your machine. A few other ways to point it at something:

# Review a GitHub repo directly
strix --target https://github.com/org/repo

# Black-box test a live deployed app
strix --target https://your-app.com

# Test both the source code and the deployed app together
strix -t https://github.com/org/app -t https://your-app.com

If your app needs a login, you can hand Strix credentials and let it test authenticated flows:

strix --target https://your-app.com \
  --instruction "Perform authenticated testing using credentials: user:pass"

You can also steer it toward specific concerns instead of a generic sweep:

strix --target api.your-app.com \
  --instruction "Focus on business logic flaws and IDOR vulnerabilities"

For more detailed rules of engagement, scope, or exclusions, hand it a file instead of a one-liner:

strix --target api.your-app.com --instruction-file ./instruction.md

Running It Headless

If you want to run Strix as part of an automated job rather than interactively, use non-interactive mode:

strix -n --target https://your-app.com

It prints findings in real time and exits with a non-zero status code if it finds vulnerabilities, which makes it straightforward to fail a build on real findings.

Wiring It Into CI/CD

This is where Strix gets genuinely useful for a team: instead of running a security scan occasionally, you run it on every pull request and block insecure code before it merges.

A minimal GitHub Actions setup:

name: strix-penetration-test

on:
  pull_request:

jobs:
  security-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
        with:
          fetch-depth: 0

      - name: Install Strix
        run: curl -sSL https://strix.ai/install | bash

      - name: Run Strix
        env:
          STRIX_LLM: ${{ secrets.STRIX_LLM }}
          LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
        run: strix -n -t ./ --scan-mode quick

A couple of practical notes if you're setting this up:

Use fetch-depth: 0 in the checkout step. On PR runs, Strix automatically scopes a quick review to just the changed files, and it needs full git history to resolve that diff correctly. If it can't, pass --diff-base explicitly.
Store your LLM credentials as GitHub secrets, not plain environment variables.

Configuration Options Worth Knowing

export STRIX_LLM="openai/gpt-5.4"
export LLM_API_KEY="your-api-key"

# Optional: point at a local model instead
export LLM_API_BASE="your-api-base-url"

# Optional: enables search capability for the agents
export PERPLEXITY_API_KEY="your-api-key"

# Optional: control how much the model "thinks"
export STRIX_REASONING_EFFORT="high"  # default is high; quick scans use medium

As for which model to run it with, the project currently recommends:

OpenAI GPT-5.4 (openai/gpt-5.4)
Anthropic Claude Sonnet 4.6 (anthropic/claude-sonnet-4-6)
Google Gemini 3 Pro Preview (vertex_ai/gemini-3-pro-preview)

It also supports Vertex AI, Bedrock, Azure, and local models — worth checking the LLM providers docs if you want to run something self-hosted.

Where This Fits in Your Workflow

A reasonable way to think about where Strix fits alongside what you probably already run:

Tool type	What it catches	What it misses
SAST / linters	Risky code patterns	Runtime behavior, business logic
Dependency scanners	Known CVEs in packages	Bugs in your own code
Strix	Exploitable, validated vulnerabilities with a working PoC	Anything outside the scope you give it

Strix is not a replacement for code review or a full professional pentest on a critical system, but as a fast, repeatable layer that actually tries to exploit your app before an attacker does, it fills a gap that static tooling structurally can't.

Trying It Out

If you want to run this against something disposable first rather than your production app, that's a reasonable way to get a feel for it — spin up a small local project, point Strix at it, and see what it turns up.

One important note directly from the maintainers: only test applications you own or have explicit permission to test. You are responsible for using it ethically and legally.

Repo: github.com/usestrix/strix

WWDC 2026 - Build Intelligent Siri Experiences with App Schemas

ArshTechPro — Thu, 02 Jul 2026 09:28:07 +0000

In the iOS 27 release, Siri takes a real step forward. It can now reach into the actual content inside your app, take actions on your behalf, and understand what is on screen right now. The way you plug into all of that as a developer is the App Intents framework, and the piece that makes it click is App Schemas.

This guide walks through the core ideas from Apple's session Build intelligent Siri experiences and turns them into something you can act on.

If App Intents is completely new to you, it is worth watching "Get to know App Intents" first, since everything below builds on that foundation.

The big picture: three new powers for Siri

There are three ways Siri gets more capable this year, and it helps to hold all three in your head before diving into code.

First, Siri can access your app's entities, meaning the real content inside your app. Someone can ask "When and where is my next meeting?" and Siri answers directly, because it understands what a meeting is, which one is relevant, and which properties to return.

Second, Siri can take actions using your app's intents. A request like "Send my latest report to Mary" works because your intent describes the action, its parameters, and when it is safe to run. Siri handles the language understanding; your app just does the work.

Third, Siri can understand on-screen context. When you annotate your views with entities, someone can say "Explain this text" or "Forward the last one" and Siri knows exactly what content they mean.

Everything below is built on one central concept, so let us start there.

App Entities: describing the content you already have

An App Entity is a structured description of the content inside your app. You are not building a new data model. You are describing content you already have in a way the system can reason about.

Think about the nouns in your app. A calendar app has events. A mail app has messages. A photos app has photos and albums. UnicornChat has Contacts, Conversations, and Messages. Each of these is an App Entity.

An entity describes three things: what the thing is, how it is identified, and which properties matter (a title, a date, some text). That is it.

Modeling an entity is step one, but on its own it is not enough for Siri to find it or talk about it. For that, your entity needs to conform to an App Schema.

App Schemas: giving Siri a shared vocabulary

An App Schema gives Siri a predefined understanding of a common concept, such as a message, a contact, or a document. When your entity conforms to a schema, Siri already knows how to reason about it. Instead of treating your app as a black box, it understands that a UnicornChat message is a message.

This is the key mental shift. You do not teach Siri your app's vocabulary or define training phrases. You declare your data against a shape Siri already knows, and the language understanding comes for free. Because schemas are system-defined, your integration automatically benefits as Siri improves and expands to new languages and dialects.

Here is how UnicornChat contributes a message's text content to Apple Intelligence by conforming to the messages message schema:

// Contributing message content to Apple Intelligence

@AppEntity(schema: .messages.message)
struct MessageEntity: IndexedEntity {

    // The text content of the message
    @Property(indexingKey: \.textContent)
    var body: AttributedString?
}

That single schema: .messages.message is what lets Siri understand a request like "Show my last message from Flare."

Entity resolution: turning words into real objects

Once your content is modeled, the next question is how Siri finds the right one. That process is called entity resolution. When someone says "Open UnicornChat with Glow," Siri resolves that "Glow" refers to a specific contact, finds the match, and fills in the entity with its properties.

But people do not always name things exactly. They describe them. Someone might say "the best windsurfing in Carmel," which is a meaning, not a text match. To support that, Siri needs semantic search, and there are two ways to power resolution.

Option 1: IndexedEntity (the best experience)

The primary path is adopting IndexedEntity. When you do, your entities are added to the system's semantic index in Spotlight. That lets Siri match on meaning rather than exact text, understand relationships between entities, and answer questions over your content.

For example, "Show the messages with Flare about movies" is not a string match. Siri can find messages that reference movie titles because it is running a semantic query over UnicornChat's indexed messages.

Notice the indexingKey in the earlier code sample. That is how you tell Spotlight which properties, like the message body, should be searchable. IndexedEntity gives you semantic matching, fewer follow-up questions, and the most natural language understanding.

Option 2: EntityStringQuery (when indexing is not feasible)

Not everything can be indexed ahead of time. Your dataset might be huge, live on a server, or change too frequently. In those cases you use EntityStringQuery. Siri hands you the person's raw input string, and your app is responsible for finding and returning the matches.

// An interface that locates entities using arbitrary string input

struct ContactQuery: EntityStringQuery {
    func entities(matching string: String) async throws -> [ContactEntity] {
        let predicate = #Predicate<Person> { person in
            person.name.localizedStandardContains(string)
        }
        let descriptor = FetchDescriptor<Person>(predicate: predicate)
        let matches = try modelContext.fetch(descriptor)
        return matches.map(\.entity)
    }
}

You lose semantic understanding here, but you gain full control over how you search. The rule of thumb: index when you can, use a string query when you cannot.

App Intents and action schemas: letting Siri do things

Entities on their own are just information. Things get interesting when you combine them with actions, and actions come from App Intents.

When you define an App Intent, that action can appear across the system: in Shortcuts, Spotlight, Widgets, and more. People can discover and trigger it even without Siri. You describe what the action does, define its parameters, and implement the behavior. The system handles surfacing and suggesting it.

To bring an action to Siri specifically, you adopt an action schema. Just as entities use schemas to be understood, actions use schemas to become executable by Siri. Think of a schema as a specialized App Intent, shaped so Siri knows how to process it. The schema defines the kind of action, the structure Siri expects, and how it maps to natural language. That is what lets Siri confidently handle "Send a message to Mary" or "Play my focus playlist."

Domains: grouping schemas into complete experiences

A single schema defines a single action, but apps usually need a set of them. That is why schemas are grouped into domains such as mail, photos, and messages. A domain is a category of contract between your app and Siri. When you adopt a domain, you implement its predefined schemas, map them to your app's functionality, and Siri immediately knows how to talk about your app in that category.

In UnicornChat, sending a message means adopting the sendMessage schema from the messages domain. In Xcode you start typing the schema name, autocomplete shows the available schemas grouped by domain, and you pick the one you want. Your job is then to map the schema's parameters (the recipient, the message content) onto your existing messaging flow: process the parameters, pass them into your send logic, and return the sent message back to the system as an entity.

The payoff looks like this in practice. You say "Send a message to Glow in UnicornChat, saying 'What movies do you recommend?'" Siri resolves Glow through your entity query, invokes your intent, and sends the message, all without opening the app.

Working across apps: on-screen awareness and content transfer

Many real requests span multiple apps. "Email my wife this reply from Bubbles" starts in one app and finishes in another. That combines two capabilities: understanding what the person is looking at, and moving that content elsewhere.

On-screen awareness

To let Siri understand references like "this message" or "that conversation," you connect what is visible to your entities. There are two APIs, for two situations.

Use UserActivity when there is one primary thing on screen, like a single document or a compose view. Use view annotations when several meaningful items are visible at once, like rows in a conversation. Here is the view annotation approach in UnicornChat, attaching each message row to its entity:

// Working across apps - View annotations

List {
    ForEach(messages) { message in
        MessageRow(message: message)
            .appEntityIdentifier(
                EntityIdentifier(
                    for: MessageEntity.self,
                    identifier: message.id
                )
            )
    }
}

With that in place, someone can say "Edit this message" or "Forward the last one" and Siri resolves the entity straight from the view.

Exporting content to another app

Content transfer is what lets other apps act on your entities. You enable it by conforming your entity to Transferable and providing an IntentValueRepresentation. UnicornChat exports a ContactEntity as a system IntentPerson, which powers requests like "Call this contact." Your app does not need to know what happens next; it just describes its content accurately.

// Working across apps - Exporting content to another app

extension ContactEntity: Transferable {

    static var transferRepresentation: some TransferRepresentation {
        IntentValueRepresentation(
            exporting: \.person
        )
    }
}

Receiving content: resolve or import

When content comes into your app, it either refers to something that already exists or represents something new. You decide which path to take.

If the incoming content should match an existing entity, use IntentValueQuery. This is conceptually like an entity query, but scoped to an intent parameter. Here UnicornChat receives an IntentPerson from another app and matches it to an existing contact:

// Working across apps - IntentValueQuery

struct ContactEntityQuery: IntentValueQuery {

    func values(for input: [IntentPerson]) async throws -> [ContactEntity] {
        let names = input.map(\.displayName)
        let descriptor = FetchDescriptor<Contact>()
        let contacts = try model.mainContext.fetch(descriptor)
        let matches = contacts.filter { contact in
            names.contains(where: { name in
                contact.name.localizedStandardContains(name)
            })
        }
        return matches.map(\.entity)
    }
}

If the incoming content should create something new, add an importing closure to your IntentValueRepresentation. This converts the incoming value into a brand new entity, such as creating a new unicorn from an IntentPerson. Your app stays in control of how that content is stored.

// Working across apps - IntentValueRepresentation

extension ContactEntity: Transferable {

    static var transferRepresentation: some TransferRepresentation {
        IntentValueRepresentation(exporting: \.person, importing: { intentPerson in
            let contact = Contact(importing: intentPerson)
            ContactManager.shared.contacts.append(contact)
            return contact.entity
        })
    }
}

Many apps use both: resolve when the content already exists, import when it does not.

Best practices and tooling

A few things separate a working integration from a great one.

Some Siri scenarios need more than one schema. If you adopt sendMessage, Xcode may raise a build error telling you that you also need draftMessage, because a complete send flow requires a way to draft and confirm. This is a design hint delivered at build time rather than a silent runtime failure. Xcode even offers a fix-it that generates a stub adoption for you to fill in. If your app mutates UI state in an intent, remember to run that work on the main actor.

When it comes to testing, work outward in layers. Start with AppIntentsTesting, a framework that lets you exercise your intents in isolation with no Siri involved, so you can validate business logic fast. Then use the Shortcuts app to inspect how your intent's parameters are shaped and exposed. Next check Spotlight to confirm your entities are indexed, discoverable, and linkable. Finally test end to end with Siri, where natural language, entity resolution, on-screen context, and cross-app workflows all come together.