DEV Community

Sandro
Sandro

Posted on

I Built Bug Hunter: A Self-Hosted Full-Stack Testing Tool for Web Apps and APIs

Modern web application failures rarely happen in one isolated place.

A user clicks a button, the frontend sends an API request, the backend returns an unexpected response, and the interface remains stuck in a loading state.

A health check may still be green. A unit test may still pass. The visible symptom may appear in the browser, while the actual cause is somewhere between the frontend, API, runtime environment, and application configuration.

I built Universal Bug Hunter to make those failures easier to detect, investigate, reproduce, and document.

It is a self-hosted testing tool that combines:

  • browser testing with Playwright and Chromium,
  • direct API smoke testing,
  • OpenAPI discovery,
  • optional AI-assisted test proposal generation,
  • deterministic bug detectors,
  • runtime evidence collection,
  • finding reproduction,
  • static analysis integrations,
  • and HTML/JSON reporting.

The goal is not to replace QA engineers, security testers, code review, or existing test suites.

The goal is more practical:

Execute realistic application flows, preserve useful evidence, and make failures easier to understand and reproduce.

Why I built it

Many automated checks answer only one narrow question:

Did the endpoint return HTTP 200?

That is useful, but it does not tell you what happened after the page loaded.

It does not tell you whether:

  • the browser logged a JavaScript error,
  • an API call returned HTTP 500,
  • the interface became stuck,
  • a click produced no visible state change,
  • the page overflowed horizontally,
  • a loading indicator never disappeared,
  • a test failed because of the application,
  • or the testing engine itself failed operationally.

I wanted one workflow that could test a web application from the outside, like a real user, while still collecting enough technical context for a developer or operator to investigate the result.

That idea became Universal Bug Hunter.

What Universal Bug Hunter does

The current workflow is:

Create project
→ Discover routes
→ Review and approve routes
→ Generate optional AI test proposals
→ Review and approve proposals
→ Compile proposals into scenarios
→ Execute browser and API scenarios
→ Detect findings
→ Reproduce supported findings
→ Generate HTML and JSON reports
Enter fullscreen mode Exit fullscreen mode

The approval steps are intentional.

Discovery does not automatically execute everything it finds.

AI does not approve or run its own proposals.

Compilation creates scenario files, but it does not automatically launch the browser.

The operator remains in control of what is allowed to run.

Browser testing with Playwright

Universal Bug Hunter uses Chromium through Playwright to test an application as a user would experience it.

Browser scenarios can navigate through the application, interact with visible elements, verify UI state, and capture screenshots.

A simple scenario can look like this:

steps:
  - navigate:
      path: /login

  - waitForVisible:
      target:
        label: Username

  - captureScreenshot:
      name: login-page
Enter fullscreen mode Exit fullscreen mode

During execution, Bug Hunter can collect:

  • browser console messages,
  • uncaught page errors,
  • failed network requests,
  • HTTP 4xx and 5xx responses,
  • screenshots,
  • DOM snapshots,
  • navigation information,
  • and other runtime evidence.

The current deterministic detector set includes checks such as:

  • console-error
  • page-error
  • failed-request
  • http-error
  • blank-page
  • horizontal-overflow
  • stuck-loader
  • no-state-change

These detectors do not depend on an LLM. They evaluate collected evidence using predictable rules.

Direct API testing

Browser testing only covers backend requests that are actually triggered by the frontend.

To improve backend coverage, I added direct HTTP scenarios.

For example:

steps:
  - httpGet:
      path: /ready
      target: api

  - assertHttpStatus:
      status: 200
Enter fullscreen mode Exit fullscreen mode

A POST scenario can look like this:

steps:
  - httpPost:
      path: /api/auth/login
      target: api
      jsonBody: {}

  - assertHttpStatus:
      status: 400
Enter fullscreen mode Exit fullscreen mode

Browser scenarios and HTTP-only scenarios use separate execution paths.

A composite orchestrator can still run both types as part of the same batch.

This keeps browser execution and direct API execution clearly separated while allowing them to contribute to the same project-level test workflow.

OpenAPI discovery and API smoke generation

For applications that expose an OpenAPI specification, Bug Hunter can import the specification and build an inventory of API operations.

The basic flow is:

OpenAPI specification
→ operation discovery
→ generated API smoke scenarios
→ scenario approval
→ execution
→ report
Enter fullscreen mode Exit fullscreen mode

The current generator supports basic GET and POST smoke coverage.

For example, an imported API specification can produce scenarios for:

GET /health
GET /ready
POST /api/auth/login
Enter fullscreen mode Exit fullscreen mode

This is intentionally conservative.

The current version does not yet attempt to generate complex destructive requests automatically.

Support for OpenAPI parameters, authentication profiles, request body generation, response schema validation, and negative API testing is planned for future versions.

AI-assisted exploration with human approval

AI exploration is optional.

When enabled, an LLM receives a controlled representation of:

  • approved routes,
  • discovered interactions,
  • existing scenarios,
  • and detected coverage gaps.

It can propose read-only test scenarios using a restricted action set such as:

  • navigate,
  • click,
  • assert visible,
  • assert URL,
  • capture screenshot.

Every proposal passes through a deterministic safety pipeline.

A proposal must:

  • target an approved route,
  • use only permitted actions,
  • remain read-only,
  • stay within configured step limits,
  • avoid credentials and sensitive values,
  • and conform to a strict schema.

Passing this validation does not mean the proposal is automatically approved.

The workflow remains:

AI-generated
≠ safe by default

Safety accepted
≠ human approved

Human approved
≠ automatically executed
Enter fullscreen mode Exit fullscreen mode

The operator reviews each accepted proposal and decides whether it should be approved, rejected, or compiled into a scenario.

Findings, evidence, and reproduction

A useful bug report should not only say:

A request failed.
Enter fullscreen mode Exit fullscreen mode

It should help answer:

  • Which scenario triggered the problem?
  • Which route was active?
  • Which action happened before the failure?
  • What did the browser log?
  • Which request failed?
  • Was the behavior reproduced?
  • Is there a screenshot?
  • Is there a DOM snapshot?
  • Is the finding eligible for reporting?

Bug Hunter stores evidence separately from project configuration.

A typical run can contain:

batch-report.json
batch-report.html
batch-state.json
runs/
  run-summary.json
  findings.json
  evidence/
  screenshots/
Enter fullscreen mode Exit fullscreen mode

When supported, a finding can be replayed in a fresh browser session.

This helps distinguish a reproducible application issue from one-time browser or environment noise.

Execution success is not the same as a passing test result

This distinction became one of the most important parts of the system.

A job can complete successfully from an infrastructure perspective while the tested application still produces findings.

For example:

Execution job: completed
Scenarios executed: 6
Passed: 3
With findings: 3
Operational failures: 0
Enter fullscreen mode Exit fullscreen mode

This means:

  • the execution engine completed,
  • all six scenarios were executed,
  • three scenarios produced no findings,
  • three scenarios produced detector findings,
  • and the runner itself did not fail operationally.

That is different from:

Operational failures: 3
Enter fullscreen mode Exit fullscreen mode

An operational failure may indicate:

  • the browser could not start,
  • navigation timed out,
  • a required element could not be found,
  • the scenario could not complete,
  • or the runtime environment failed.

I am continuing to improve this distinction in the operator dashboard so that Completed, Passed, With findings, and Operational failure cannot be confused with each other.

Testing localhost applications from Docker

One of my main requirements was testing applications before they are publicly deployed.

Bug Hunter can run locally through Docker while testing another application that is running:

  • directly on the Windows host,
  • inside another Docker Compose service,
  • on a private development address,
  • or on a regular staging URL.

From inside a container:

localhost
Enter fullscreen mode Exit fullscreen mode

refers to that container itself.

For an application running directly on the Windows host, Bug Hunter uses:

http://host.docker.internal:3000
Enter fullscreen mode Exit fullscreen mode

A local target can be configured like this:

target:
  baseUrl: http://host.docker.internal:3000
  environment: local
  allowedDomains:
    - host.docker.internal
Enter fullscreen mode Exit fullscreen mode

A full-stack project can define separate frontend and API targets:

project:
  type: fullstack

targets:
  frontend:
    baseUrl: http://host.docker.internal:3000

  api:
    baseUrl: http://host.docker.internal:8787
    healthPath: /ready
Enter fullscreen mode Exit fullscreen mode

Private and loopback targets are not enabled globally.

They must be explicitly allowed through the local environment and network policy configuration.

The same allowlist logic is applied to:

  • browser navigation,
  • redirects,
  • direct HTTP scenarios,
  • and OpenAPI retrieval.

This preserves the SSRF protection while still allowing intentional localhost testing.

Static analysis integrations

Bug Hunter also includes a static analysis layer with provider adapters for:

  • Semgrep for SAST,
  • Trivy for dependency and filesystem analysis.

The static analysis command writes normalized results into the project workspace.

If a configured tool is not installed, the pipeline reports that the provider is unavailable instead of crashing the entire workflow.

This is still an early integration.

The goal is not to replace dedicated Semgrep or Trivy workflows.

The longer-term goal is to present dynamic browser findings, API findings, dependency findings, and static analysis results within the same project context.

Architecture

Universal Bug Hunter is a TypeScript monorepo built with:

  • Node.js 22
  • TypeScript
  • pnpm workspaces
  • Fastify
  • Next.js
  • Playwright
  • Chromium
  • TanStack Query
  • Zod
  • Docker Compose
  • OpenAI as an optional exploration provider
  • Semgrep
  • Trivy
  • HTML and JSON reporters

The system is divided into focused packages for:

  • core domain models,
  • configuration and validation,
  • browser execution,
  • direct HTTP execution,
  • route discovery,
  • OpenAPI discovery,
  • AI exploration,
  • evidence storage,
  • deterministic detectors,
  • finding reproduction,
  • scenario execution,
  • reporting,
  • API services,
  • CLI commands,
  • and the web operator interface.

I intentionally avoided placing browser control, filesystem access, AI calls, HTTP testing, reporting, and security policy inside one large service.

Testing systems become difficult to maintain when every responsibility is tightly coupled to the runner.

The operator workflow

The web interface follows the project lifecycle:

Overview
→ Discovery
→ Exploration
→ Proposals
→ Scenarios
→ Runs
Enter fullscreen mode Exit fullscreen mode

The operator can:

  • create a project,
  • start route discovery,
  • approve or reject discovered routes,
  • start optional AI exploration,
  • review proposals,
  • approve or reject proposals,
  • compile approved proposals,
  • run all approved scenarios,
  • run selected scenarios,
  • inspect execution history,
  • review findings,
  • and download HTML or JSON reports.

The interface is deliberately operator-focused.

It does not expose target URL changes, browser configuration, credentials, or reporting configuration during scenario execution.

Those values remain controlled by the project manifest and backend policy.

Local verification

I tested the current full-stack workflow in two different ways.

Host execution

The first test executed directly from the host against the local web UI and API:

Selected: 6
Passed: 6
Operational failures: 0
Exit code: 0
Enter fullscreen mode Exit fullscreen mode

The batch contained:

  • three browser smoke scenarios,
  • three direct API smoke scenarios.

Docker-to-host execution

The second test ran from the Docker API container against the Windows host through host.docker.internal:

Selected: 6
Executed: 6
Passed: 3
With findings: 3
Operational failures: 0
Exit code: 1
Enter fullscreen mode Exit fullscreen mode

The three API scenarios passed.

The three browser scenarios completed operationally but produced console or network findings.

This demonstrated an important behavior:

The execution engine can complete correctly while the tested application still produces reportable findings.

An exit code of 1 does not necessarily mean the testing infrastructure crashed.

It can mean that relevant findings were detected.

Current limitations

Universal Bug Hunter is still evolving.

The current version has several known limitations:

  • job state is stored in memory and is lost after an API restart,
  • authentication currently supports one local admin account,
  • the execution queue runs inside one Node.js process,
  • API smoke generation has limited OpenAPI parameter and body support,
  • HTTP-only scenarios do not yet have the same reproduction flow as browser scenarios,
  • backend logs and distributed traces are not yet correlated with browser requests,
  • static analysis integrations are still basic,
  • artifact retention and cleanup need stronger lifecycle controls,
  • concurrency and browser resource limits need further hardening,
  • and deployment is currently focused on local self-hosting.

I prefer being clear about these limitations.

An automated testing tool should not create false confidence.

A successful run does not prove that an application has no bugs.

It only proves that the selected scenarios completed under the tested conditions without producing findings recognized by the enabled detectors.

What I am improving next

The next areas I am working on include:

  • job concurrency and queue limits,
  • browser session limits,
  • artifact retention and cleanup,
  • structured operational logging,
  • better runtime status visibility,
  • OpenAPI parameter support,
  • request body generation,
  • authentication profiles for API scenarios,
  • negative API test generation,
  • response schema assertions,
  • backend log and trace correlation,
  • and clearer result summaries in the operator dashboard.

One longer-term goal is to correlate the complete path:

Browser action
→ frontend state
→ API request
→ backend response
→ log or trace
→ one combined finding
Enter fullscreen mode Exit fullscreen mode

That would make Bug Hunter more useful than a collection of disconnected browser, API, and static checks.

Why this project matters to me

I built Universal Bug Hunter as a practical QA automation, DevOps, and DevSecOps project.

It combines several areas I regularly work with:

  • browser automation,
  • API testing,
  • TypeScript,
  • Docker,
  • application security,
  • evidence collection,
  • reporting,
  • CI/CD design,
  • and system architecture.

The most interesting part was not simply making Playwright click through a page.

The harder questions were:

  • What should the system be allowed to execute?
  • How do you prevent AI proposals from bypassing safety rules?
  • How do you distinguish a product bug from an environment problem?
  • How do you preserve useful evidence without leaking secrets?
  • How do you reproduce findings deterministically?
  • How do you test localhost applications safely from Docker?
  • How do you explain that a job completed even when tests found problems?

Those questions shaped the architecture more than the UI itself.

Current status

Universal Bug Hunter currently runs as a local, self-hosted testing environment.

I am testing it against local applications before considering a public deployment.

The complete current flow works from:

URL discovery
→ route approval
→ optional AI exploration
→ proposal review
→ scenario compilation
→ browser and API execution
→ findings
→ report download
Enter fullscreen mode Exit fullscreen mode

I am sharing the project to get feedback from:

  • QA engineers,
  • developers,
  • DevOps engineers,
  • platform engineers,
  • and security practitioners.

I would especially appreciate feedback on:

  • whether the workflow is understandable,
  • whether findings contain enough evidence,
  • whether operational failures and application findings are clearly separated,
  • which API checks would be most useful,
  • whether the approval workflow feels too strict or appropriately safe,
  • and what would make the tool useful in a real development workflow.

Universal Bug Hunter is still a work in progress, but the core idea is already clear:

Find problems through realistic application flows, preserve the evidence, and make the result easier to understand and reproduce.

Top comments (0)