Agent skills give coding agents reusable instructions for tasks such as preparing a release. They can also include executable code: a script that checks a file or produces a report another step will use.
When that work reaches a delivery pipeline, we need a way to decide whether the result is good enough to publish. Where should the skill's code run, and how should its output affect that decision?
OpenSandbox provides APIs for creating temporary environments and running commands. I wanted to test how it could run a skill's helper between building a Python package and publishing it, with GitHub Actions enforcing the result.
I built a small workflow around a skill that checks a release manifest—the file listing a package and its version. The experiment tests whether valid input can reach publication, while a missing version or an incorrect report stops the release.
Where does the check fit?
The pipeline has three jobs: build, check, and publish. The build creates a wheel, Python's installable package format, and its release manifest. The check runs the skill's helper. Publication depends on both jobs succeeding.
The OpenSandbox check sits between building the candidate and publishing it. Code review and application tests belong to the broader delivery process.
The publish job declares these dependencies:
publish:
needs: [build, check]
runs-on: ubuntu-24.04
timeout-minutes: 5
permissions:
contents: write
GitHub Actions skips dependent jobs when a prerequisite fails. A failed check prevents the publish job from starting.
What code comes from the skill?
This example's release-checking skill supplies a Python CSV validator and SKILL.md instructions describing its input and output requirements.
The valid manifest has one package:
package,version
opensandbox-release-demo,0.1.0
The helper requires both fields and unique package names. It returns JSON containing the row count and package identities. A missing version produces a diagnostic naming the row and exits with status 7.
The delivery job invokes a reviewed helper explicitly. An agent can follow the skill while preparing release files, while CI runs the same executable check before publication. The pipeline's acceptance rules determine whether that release step can continue.
How does OpenSandbox run the check?
The check job starts an OpenSandbox server on its GitHub runner. Its Python client creates a temporary environment, uploads the skill and manifest, verifies file readback, and runs the helper through the OpenSandbox SDK.
The complete client includes this command invocation:
execution = await sandbox.commands.run(
command, opts=RunCommandOpts(timeout=timedelta(seconds=30))
)
The server and SDK are pinned to version 1.2.0. The sandbox uses a Python image pinned by digest, a three-minute lifetime, and explicit resource limits. It receives the skill files and synthetic manifest; the GitHub publication token belongs to the publish job.
The caller reads the expected package name and version from the built wheel's metadata. It accepts the report only when execution succeeds and the returned fields match that candidate. Before publication, it also rechecks the wheel and manifest checksums against the accepted evidence.
A finally block requests sandbox destruction even after rejection. The caller then checks for a 404 from a new lifecycle client. A cleanup failure leaves the release check blocked.
What passed, and what blocked publication?
I ran three separate GitHub Actions executions at the same source revision. Each execution built its own candidate before reaching the OpenSandbox check.
| Scenario | Helper exit | Check job | Publish job |
|---|---|---|---|
| Valid manifest | 0 | Passed | Created prerelease |
| Missing version | 7 | Failed | Skipped |
| Incorrect report | 0 | Failed | Skipped |
The missing-version case changes the CSV input. The wrong-report case uses a separate control script that returns 99 rows while exiting successfully. The caller expects one package, so it rejects that report.
The valid run created a demo prerelease. I downloaded its wheel without signing in, matched its checksum to the accepted candidate, and installed it in a fresh environment. Both rejected runs had skipped publish jobs and returned 404 for their expected release tags. All three sandbox cleanup lookups also returned 404.
The retained evidence connects each decision to its source revision, helper, candidate checksums, process output, and cleanup lookup. A failed run's JSON record remains available even though its publish job was skipped.
When should a team use this pattern?
I'd consider OpenSandbox when an agent application needs to create temporary environments for individual script jobs, with a defined runtime, selected input files, and collected results. This release check shows how that execution service can participate in an automated delivery decision.
For a simple validator, your existing CI container may already provide a suitable environment. Compare that support with the lifecycle work an OpenSandbox service adds. The recommendation depends on your agent application's execution needs.
The companion repository includes the workflow, skill, tests, and reproduction commands. This check covers manifest fields and agreement with the built package; teams still define their other testing, approval, and deployment requirements.
The useful result is a release check your team can inspect, rerun, and enforce before publication. The skill supplies the procedure, OpenSandbox runs its helper, and the pipeline acts on the observed result.
If your agent workflow needs that execution step, check out OpenSandbox.
Top comments (0)