Piyush Gupta

Posted on Aug 22

From Sandbox to Review Queue: My GSoC 2026 Project with OWASP OWTF

#opensource #gsoc #python #security

When I started GSoC in May, my plan was to build a runtime sandbox for community plugins. By week two my mentor had talked me out of it, and I ended up spending the rest of the summer building a review queue instead. This post is about how that happened and what I actually shipped.

Quick summary

Project: Community Driven Plugin Ecosystem for OWTF
Org: OWASP Foundation
Mentors: Abraham Aranguren, Viyat Bhalodia
What got shipped: Six pull requests against owtf/owtf, around 6,000 lines of Python and TypeScript, 153 backend unit tests, and a trust model doc.
Working mirror of this post: gist

If you only want the code, here are all my PRs on OWTF.

The problem I was trying to solve

OWTF is a security testing framework, and until this summer its plugin catalogue was static. If you wrote a detection for some new attack pattern, your options were: open a PR against the framework itself (high bar, slow), or keep the plugin to yourself. Most useful plugins never made it upstream because of that.

The Community Plugin Marketplace fixes this. Any authenticated user can upload a Python plugin through the web UI. The plugin is validated at upload time, lands in a pending queue, and waits for an admin to look at the source. Once approved, the plugin gets mirrored into OWTF's standard plugin table. From that point on, the runner, the worklist, and the report generator all treat it exactly like a built-in plugin.

The pivot

My accepted proposal called for a sandbox. Community plugins would run inside something like a subprocess with dropped privileges, so that a malicious plugin could not do too much damage.

Then Viyat said this in Slack:

A sandbox in Python that talks to the same postgres, the same file system, the same target scope as OWTF itself is not really a security boundary.

I sat with that for a couple of days and realised he was right. A plugin that runs inside OWTF has to see the target, has to read config, has to write results. Any "sandbox" I put around that is going to have holes the size of the API surface itself. Doing it properly meant running each plugin in its own container. That is a lot of orchestration, and it was not going to fit into a summer.

So the trust model changed. Instead of pretending we had runtime isolation, we made admin source code review the actual security boundary. The sandbox got replaced with two smaller things that are honest about what they do:

A static AST validator that rejects obviously dangerous code at upload time.
Serializer discipline so server-only stuff like file_path and raw source never leaves the API.

The scope shrank. The value didn't. And it forced me to write down what the marketplace protects against and what it does not, which became docs/community_plugin_trust_model.md. Honestly, that doc is the thing I am proudest of. It will stop the next person from making the same mistake I did.

What actually shipped

Six pull requests, one job each:

PR	What it does
#1456	Data model plus an idempotent DB upgrader that runs on server start (no Alembic needed)
#1457	AST validator for uploads, with alias tracking
#1458	Admin role, JWT and `@admin_required` decorators, and an `owtf-admin` CLI
#1459	Manager plus REST endpoints (upload, list, mine, review, approve, reject)
#1460	Runner integration so approved plugins run through OWTF's standard `PluginRunner`
#1461	React and TypeScript marketplace UI plus the trust model doc

The validator, since it is the interesting bit

The static validator walks the AST of an uploaded plugin before the file even hits disk. It rejects:

Imports: os, sys, subprocess, socket, ctypes, signal, resource, pickle
Functions: eval, exec, compile, __import__, input
Dangerous calls: os.system, os.popen, subprocess(...) with shell=True
File writes: open(...) in write or append modes

The catch is that a naive "just block subprocess" rule falls apart the second somebody aliases the import. So the validator tracks aliases too:

from subprocess import run as process
process("id", shell=True)   # still rejected

Any name that resolves back to a blocked module or function is caught, even through renames and re-imports. Without that, the whole check is theatre.

Two things I got wrong the first time

One: SQLite lies about foreign keys.

My first cut of the runner integration synthesised community plugin keys at query time. All my local tests passed. All of them. And the moment the same code hit prod postgres, it crashed on a foreign key check. Turns out SQLite (my dev database) does not enforce FKs the same way PostgreSQL does, so the entire class of bug was invisible to my test suite.

The fix was to stop faking it. On approval, I now insert a real "mirror row" into the standard plugins table with source = "community". That way the FK to test_groups.code is respected and every existing query, worklist call, and report generator just works.

The lesson I would pass to anyone: if your CI runs on SQLite and prod runs on PostgreSQL, set up a TEST_POSTGRES_URL for FK and transaction sensitive tests. Otherwise you are going to ship bugs that your test suite is blind to.

Two: 6,500 line PRs are not PRs.

I opened my first pull request as one giant thing covering everything: model, validator, admin role, API, runner, UI. Reviews were slow and shallow because nobody could hold that much context in their head, myself included. Viyat suggested I split it. I did, and the six-PR version got reviewed faster, with sharper feedback, and each PR merged on its own timeline.

Splitting felt like starting over. It was not. It was just the actual work of shipping code that other people can review.

Gotchas for anyone who touches this next

Do not synthesise community plugin keys at query time. Persist the mirror row.
Plugin.code has a foreign key to test_groups.code. Use outer joins to keep community plugins visible in existing queries.
file_path must never leave the server. There is a pytest assertion (TestSerializersNeverLeakFilePath) that fails loudly if it does.
SIGALRM timeout enforcement only works on the main thread.
Import ordering matters: owtf/models/plugin.py has to import test_group before metadata.create_all(), or the FK breaks on fresh installs.

Done and not done

Done this summer:

Full upload, review, approve, and run flow, end to end
AST validator with alias tracking and its own test suite
Admin role, decorators, env-based seeding, and a CLI
REST API with role scoped serializers
Marketplace UI with Browse, Upload, Pending Review, and My Plugins tabs
Startup DB upgrader so existing installs migrate without Alembic
Trust model doc

Explicitly out of scope, but obvious next steps:

Actual runtime isolation (containerised execution, seccomp). The trust boundary is admin review right now. A future contributor could layer a container-per-plugin runner under the existing PluginRunner without breaking the API.
Plugin versioning. Approval is per upload today. Supporting v1 to v2 upgrades on the same plugin name is the natural next thing.
Community moderation (star, report, changelog). The data model has room, the UI does not surface it yet.
Dependency review. The validator is static and does not inspect third party packages a plugin might install.

What I got out of it

More than I thought I would. I came in thinking of GSoC as a code writing exercise. I am leaving thinking of it as a design and communication exercise where the code is just the output. Every hard bug I hit turned out to be a design problem in disguise. Every review taught me something I did not know I was missing.

Huge thanks to my mentors Abraham Aranguren and Viyat Bhalodia. Abraham gave me the room to make design choices and course-corrected me when they were off. Viyat pushed back on the sandbox in week two and probably saved the entire project from being a very nicely engineered piece of security theatre. Thanks to the OWASP Foundation and the OWTF maintainers for the trust, and to Google for running the program.

DEV Community