DEV Community

Cover image for Letting an agent drive the app in osFoundry Operator, with every action on the record
tauchiii
tauchiii

Posted on

Letting an agent drive the app in osFoundry Operator, with every action on the record

"Which of my GPU endpoints are still billing?" is a simple question, but answering it usually means clicking through three screens, reading the numbers, and coming back to the chat to type them out. Handing that to an AI agent sounds good until you ask what it actually clicked. osFoundry Operator is built around that question: the agent works in a live pane of the app right next to the conversation, and every action it takes is written down.

A conversation beside a live pane of the app

osFoundry is a desktop app that puts chat, agents, files and tools in one window. Operator splits the screen in three. On the left is the list of Operator sessions. In the middle is the conversation. On the right is a live dashboard pane: a real copy of the osFoundry app that Operator opens screens in. Above the pane you see which screen it is on (here Dashboard /models/endpoints), with Home and Project shortcuts, Reload dashboard and Open in a new window. You can drag the divider to give the pane more room.

osFoundry Operator with the session list on the left, the conversation in the middle and the live dashboard pane on the right showing the Endpoints screen (example data)

In this example Sammy asks which GPU endpoints are billing. Operator opens Endpoints in the pane, reads the cards back, and answers in the chat: one endpoint is Active at $1.45/hr and has cost $7.54 so far, one is still Creating and has started billing, and one is Stopped. The answer comes from the screen it just opened, and the screen is still there to check. It also offers to stop the active endpoint, and says it will ask before it presses Stop.

The activity strip: what Operator actually did

Under the pane is a thin strip that shows whether Operator is idle or working and when it last acted. Open it, and the Log lists every action Operator took in this pane, newest first: here two navigations, to Models and then Endpoints, each marked "surface changed, read back". The log belongs to the session, so each Operator session keeps its own trail. Copy as report copies the whole log as text, for a ticket or a teammate, and Clear empties it.

The activity strip under the osFoundry Operator pane expanded to the Log, listing two navigate actions with the time and the screen each one opened (example data)

This is the part that makes Operator easy to trust. You don't have to believe a summary of what the agent says it did. The log and the pane show it.

Setting up osFoundry with Operator

The second session in the example is "Set up osFoundry". Here the strip has a second tab, Setup, with a checklist that reads the app's live state rather than what Operator reported: a provider key, a fallback model, the on-device runtime, the local embedding model and the router role. In the example three of five steps are done, and for each open step the checklist explains why (for example "Downloaded, but not selected for local memory yet").

The Setup tab of the osFoundry Operator activity strip showing 3 of 5 setup steps done, with Ask Operator and Not now on the open steps (example data)

Each open step has two links. Ask Operator puts a request for that one step in the message box, and you press Send. Not now hides the step in this session, without marking it done. One rule is fixed: a step that needs an API key is never handled in chat. Operator opens the settings dialog, and you type the key there yourself.

If you try to leave while Operator is still working, it tells you that the run keeps going in the background and keeps using credits, but loses the pane it was operating, so its next step on that screen will fail. You can stay, or leave anyway. Stop ends what Operator is doing.

The built-in browser, with a companion panel

Operator works inside osFoundry. For pages on the web there is the built-in browser, which keeps the page next to the tools that work on it. The left panel has Search, Opened, Bookmark, Read Later and History. The centre is the page itself, with tabs, an address bar and zoom. The right side is the companion panel, with Maestro, Page, Links and Actions tabs.

The osFoundry built-in browser with a saved search on the left, a page in the centre and the companion Page panel on the right showing the extracted article (example data)

The Page tab reads the live page and shows it as clean text, which you can view Rendered or as Markdown, and Read again refreshes it after the page changes. With a model assigned to the Summarizer role, it can also summarize the page. The Links tab lists the links extracted from the page with their URLs, so you can follow the ones that matter without hunting through the layout. Maestro, osFoundry's main assistant, can work beside the page in the same panel.

The Links tab of the osFoundry browser companion panel listing four links extracted from the page with their URLs (example data)

Use case: checking spend at the end of the day

Before logging off, Sammy opens Operator and asks which endpoints are still billing. Operator opens the Endpoints screen in the pane and lists them, and the activity log shows the two screens it visited. One endpoint is idle, so Sammy asks Operator to stop it. Operator asks first, then stops it, and the stop shows up in the log and on the card in the pane. The next morning Sammy copies the log as a report into the team's cost thread.

Questions people ask first

Does Operator click things without asking?
Operator works in its own pane and records every action it takes there. In the example it asks before stopping an endpoint, and steps that need a secret are never done in chat: you type the key in the settings dialog.

Where can I see what it did?
In the activity strip under the pane. The Log lists each action with its time and the screen it opened, and Copy as report copies it as text.

Is Operator the same as the built-in browser?
No. Operator drives the osFoundry app itself in a live pane. The built-in browser is for web pages, with a companion panel that reads the page, extracts links and lets Maestro work beside it.

Is it free?
osFoundry is free to use on your own device. Work that runs on osFoundry's servers, and model calls through osFoundry, use credits.

The desktop app is free to download at osfoundry.io/download.

Top comments (0)