How to generate E2E tests from natural language without a single byte of your app leaving your machine — no API key, no cloud, no data retention
Built into the open-source ai-natural-language-tests platform, with a local fine-tuned model you can run offline.
The Problem Nobody Says Out Loud
Most AI test tools have a quiet catch.
To generate a test, they send your page HTML and your requirements to a third-party API.
Privacy-First AI Test Automation
For a public demo site, fine. For an internal admin panel, a banking flow, or a pre-release feature, that’s a leak.
Your DOM often carries field names, business logic, and sometimes real data. Shipping it to a vendor is exactly what security and compliance teams exist to prevent.
So the honest question is: can you get AI-generated tests without the cloud round-trip?
Yes. Here’s the setup.
What “Privacy-First” Actually Means
The phrase gets thrown around loosely. Here it means three concrete things.
Air-gapped. The generation step can run with no outbound network call at all.
Local model. The intelligence lives on your machine, not behind an API.
Zero-retention. Nothing is logged, stored, or sent anywhere you didn’t explicitly turn on.
If any one of those fails, it isn’t privacy-first — it’s privacy-flavored.
The Core Idea: Move the Model to the Data
Cloud AI works by moving your data to the model.
Privacy-first flips it: move the model to your data.
A small, fine-tuned model runs locally through Ollama. The platform points at it instead of OpenAI, Anthropic, or Google.
Same workflow — type a requirement, get a runnable Cypress or Playwright test — but the HTML and requirement never leave the laptop.
Step 1: Run the Model Locally
The fine-tuned model ships in GGUF format, so Ollama runs it with one command.
ollama run hf.co/aiqualitylab/ai-natural-language-tests
It downloads once (~1.6 GB), then runs fully offline.
No API key. No account. After the pull, you can disconnect from the internet entirely.
Step 2: Point the Platform at It
Tell the platform to use the local provider in your .env:
OLLAMA_MODEL=hf.co/aiqualitylab/ai-natural-language-tests
OLLAMA_BASE_URL=http://localhost:11434/v1
That localhost endpoint is the whole point. The request goes to a process on your own machine, not across the network.
Step 3: Generate — With No Cloud Call
python qa_automation.py "Test login with valid credentials" \
--url https://the-internet.herokuapp.com/login --framework playwright --run
The platform reads the page, matches past patterns locally, and generates the test.
Your DOM and your requirement stay on the box. Nothing is sent out.
The Part People Forget: Turn Off Telemetry
A local model is only half the story.
Observability can quietly leak too. The platform supports OpenTelemetry traces and Loki log shipping — genuinely useful in a team setting, but both send data to an external stack.
For a true air-gap, leave them unset.
# Leave these blank / unset for zero egress
OTEL_EXPORTER_OTLP_ENDPOINT=
OTEL_EXPORTER_OTLP_HEADERS=
GRAFANA_LOKI_URL=
GRAFANA_INSTANCE_ID=
GRAFANA_API_TOKEN=
If it isn’t configured, nothing ships. Privacy by default, not by promise.
Use Masked Data Too
One more habit worth keeping, even fully local.
Use synthetic or masked values in prompts, fixtures, and generated tests.
Real credentials and customer data don’t belong in a test asset — regardless of where the model runs. Local execution reduces the blast radius; masked data removes it.
Why This Beats “We Don’t Train on Your Data”
Cloud vendors offer zero-retention tiers and no-training promises. Those are policies.
A local model is an architecture.
The difference matters: a policy can change, be misconfigured, or fail silently. An air-gapped setup can’t leak what it never sends.
For regulated industries — finance, healthcare, defense — “the data physically cannot leave” is a far easier story to defend than “the vendor promised.”
The Honest Trade-Offs
Privacy-first isn’t free. Be clear-eyed about the costs.
Smaller model, narrower skill. A 1.5B local model handles its trained task — requirement to test — well, but it’s not a frontier model. For that narrow job, that’s fine.
You run the infrastructure. Ollama, the model, the environment — all yours to maintain. No managed endpoint doing it for you.
Setup effort up front. A cloud key is one line. A local stack takes a little more to stand up. You pay once, not per call.
For teams that can’t send their DOM to a vendor, these are easy trade-offs to accept.
When to Reach for This
Privacy-first is the right default when:
You’re testing internal, pre-release, or regulated applications.
Your DOM or fixtures could contain sensitive fields.
Compliance forbids sending app data to third-party APIs.
You need to work in an air-gapped or offline environment.
If you’re only testing public sites, a cloud model is fine. The moment the app is sensitive, move the model to the data.
The Takeaway
AI test automation and data privacy aren’t opposites.
The trick is architectural, not contractual: run a local fine-tuned model, keep the endpoint on localhost, leave telemetry off, and mask your data.
Do that, and you get natural-language test generation with a guarantee no policy can match — nothing left the machine, because nothing was ever sent.
Platform: github.com/aiqualitylab/ai-natural-language-tests
Model: huggingface.co/aiqualitylab/ai-natural-language-tests
Top comments (0)