DEV Community: Check My Vibe

What a Vibe Coding Security Scanner Can (and Cannot) Tell You

Check My Vibe — Wed, 15 Jul 2026 06:38:46 +0000

AI-assisted builders can take an idea from prompt to production in a weekend. That speed is useful, but it also compresses the part of the process where someone normally reviews deployment settings, browser-visible secrets, authorization boundaries, and recovery plans.

A public security scanner is a good first pass for that problem. It is also easy to misunderstand. A clean public scan does not mean an application is secure, and a warning does not always mean a vulnerability is exploitable.

The useful question is not “Did the scanner pass my app?” It is “What evidence could this scanner actually observe?”

Layer 1: the public deployment surface

A passive scanner can request the same resources that a normal visitor can reach. Depending on its scope, it may inspect:

HTTP security headers such as Content-Security-Policy and Strict-Transport-Security
HTTPS behavior and redirect consistency
Public JavaScript bundles for credential-shaped strings
Public source maps that expose original source structure
Common sensitive paths such as environment files or repository metadata
Cookie attributes and other response-level deployment signals

These checks are valuable because they test the deployed result, not the configuration you intended to ship.

For example, a repository may contain a CSP configuration while the CDN response does not. A source map may be disabled in one build configuration but still appear in production. A key may be stored safely on the server in most code paths while one client bundle accidentally contains a privileged token.

The deployed surface is where those mistakes become observable.

Layer 2: source-code review

A public URL cannot reveal every control behind an application. Source review or SAST can inspect code paths, configuration, data flow, and dangerous implementation patterns that never appear in a normal response.

This is where you can answer questions such as:

Is authorization enforced on the server?
Can a user change an object ID and read another tenant’s data?
Are database queries parameterized?
Are uploaded files validated before storage?
Do administrative actions check roles consistently?

Public scanning and source review are complementary. One checks what actually shipped; the other checks logic that may not be externally visible.

Layer 3: dependencies and supply chain

Dependency scanners answer a different question again. They compare package manifests, lockfiles, containers, or software bills of materials against known vulnerability data.

That can identify a vulnerable library version, but it cannot prove that your custom authorization is correct or that your CDN is serving the headers you expected.

For AI-built applications, it is worth combining dependency scanning with a manual review of newly introduced packages. Generated code can add unnecessary libraries, outdated examples, or packages whose names were never verified.

GitHub’s documentation on secret scanning and the OWASP Secure Coding with AI Cheat Sheet are useful starting points.

Layer 4: authenticated assessment

The deepest application risks often require test accounts and an agreed scope.

An authenticated assessment can compare user roles, tenant boundaries, protected workflows, payment states, and business-logic behavior. A public scanner should not attempt to log in, submit forms, bypass access controls, or simulate exploits without explicit authorization.

If your application handles payments, health information, private customer data, or privileged workflows, this layer is not optional.

Three findings that need careful interpretation

“Content-Security-Policy was not observed”

This means an important browser-side mitigation was missing from the checked response. It does not prove that cross-site scripting is present.

The next step is to inventory required scripts, styles, frames, and connections, then introduce a restrictive policy in report-only mode before enforcement. Mozilla’s HTTP Observatory is also useful for reviewing header posture.

“A source map is publicly reachable”

A production source map can reveal original file names, source structure, and implementation details. That is an exposure signal, not automatically a security vulnerability.

If the map is not required publicly, disable it or upload it privately to your error-monitoring service.

“A credential-shaped string was found”

Some browser keys are intentionally public. Others, including service-role tokens and private credentials, are not.

A scanner can identify and redact the pattern, but a human still needs to confirm the provider and privilege level. If the credential is private, rotate it and move it behind a server-side boundary.

A practical pre-launch sequence

For a small AI-built application, a reasonable minimum process is:

Run a passive check against the deployed URL.
Review the repository for secrets, authorization, input validation, and unsafe defaults.
Scan manifests and lockfiles for known vulnerable dependencies.
Verify database policies with separate low-privilege test accounts.
Test the production build from a clean browser and mobile device.
Record what was checked and what remains outside scope.

I built Check My Vibe as a free, no-account implementation of the first step. It checks a deliberately limited public surface and keeps its automated result separate from a 36-point manual checklist. It does not claim to replace source review, authenticated testing, or a professional assessment.

That boundary is the main thing I want builders to take away: security tools are most useful when their evidence and limitations are explicit.

A scanner is a starting signal. A secure release still requires human judgment across the layers the scanner cannot see.

A Practical Vibe Coding Security Checklist: What Passive Scanners Can and Cannot Prove

Check My Vibe — Mon, 13 Jul 2026 02:24:00 +0000

Vibe coding makes it possible to ship a useful web app in a weekend. It does not make the security model disappear.

A generated app can compile, deploy, and look polished while still exposing a source map, shipping a credential-shaped string in a JavaScript bundle, missing basic browser security headers, or relying on authorization rules nobody has tested.

The practical response is not to trust one score. It is to use the right scanner for each layer.

Disclosure: I am building Check My Vibe. I include it below alongside competing tools because the useful question is not which product has the loudest claim; it is which part of the system each one can actually inspect.

What a passive URL scan can actually prove

A passive scanner starts with the same public surface a normal visitor can reach. Without credentials or exploit attempts, it can provide useful evidence about:

HTTPS behavior, redirects, and certificate delivery
response headers such as Content-Security-Policy, HSTS, X-Content-Type-Options, and framing controls
cookies visible in public responses
public HTML and selected same-origin JavaScript assets
publicly reachable source maps or sensitive-looking files
credential-shaped strings that were shipped to the browser
CORS behavior that can be observed safely
technology and deployment clues exposed in responses

That evidence is valuable because these are real deployment mistakes. If a production bundle contains a private key, or a source map is publicly available, the problem is already observable from outside.

But wording matters. A passive scan reports what it observed at a particular URL and time. It does not prove the entire application is secure.

What it cannot prove

A public, unauthenticated scan normally cannot verify:

tenant isolation after login
whether user A can read or modify user B's records
the correctness of Supabase RLS policies that are not exposed through public behavior
server-side authorization on every API route
secret handling inside a private repository or CI system
vulnerable dependencies that never reveal their versions publicly
payment, webhook, password-reset, and invitation business logic
administrative paths that require legitimate test accounts
the absence of every vulnerability

A green public-surface result should therefore mean no issue was observed in the tested surface, not the app is safe in every respect.

Four tools, four different scopes

The feature descriptions below reflect the products' public sites in July 2026. They are not a benchmark result, and the tools can change.

Tool	Primary surface	Useful when	Important boundary
ScanVibe	Public URL	You want a fast outside-in scan, a grade, and fix prompts. Its site lists 18 analyzers covering SSL, headers, exposed secrets, libraries, files, cookies, and CORS.	Outside-in evidence cannot by itself prove private authorization, database isolation, or every server-side condition.
ShipSafe	GitHub repository and live URL	You want source-aware checks plus plain-English findings. Its site currently advertises 60+ checks and says uploaded code is deleted after scanning.	Repository access gives deeper visibility, but the result still depends on rule coverage, repository completeness, and runtime configuration.
SecureVibe	Cursor and VS Code	You want file-level feedback inside the IDE before deployment, with reports and remediation prompts. Its site says it does not collect or store code.	IDE analysis is strongest on code it can see; it may not observe the final CDN, proxy, DNS, header, or production configuration.
Check My Vibe	Passive public web surface plus a manual checklist	You want a free, no-account first pass over public deployment mistakes, followed by a separate 36-point review checklist.	It does not log in, exploit the target, recursively crawl the whole site, inspect a private repo, or certify backend policy correctness.

This is not a winner-takes-all category. The products above overlap, but they sit at different points in the delivery loop: editor, repository, deployed application, and manual review.

A practical layered workflow

For an AI-built app, I would use the following order.

1. Run a passive scan on the deployed URL

Do this before sharing the launch publicly. Fix obvious header gaps, exposed source maps, accidentally public files, browser-shipped secrets, and unsafe redirects. Re-run after the fix.

Only scan sites you own or are authorized to assess. Passive does not mean permissionless.

2. Scan the repository and dependency graph

A URL scanner cannot see most server code. Use repository-aware secret scanning, dependency analysis, and static analysis. Review findings rather than pasting every generated fix blindly; automated remediation can change behavior or introduce new bugs.

3. Add feedback inside the editor or CI pipeline

The cheapest issue is the one that never reaches production. IDE and CI checks can flag new secrets, unsafe patterns, and dependency changes close to the commit that introduced them.

4. Test authorization with real roles

Create a small role matrix: anonymous user, normal user, second tenant, support user, and administrator. For every sensitive action, record who should be able to read, create, update, and delete.

Then test the negative cases. Can user A request user B's object ID? Can a normal user call an admin API directly? Does hiding a button merely hide the action, or does the server enforce the rule?

5. Review the data layer separately

For Supabase, Firebase, or another backend-as-a-service, inspect policies in the provider console. Verify row-level rules, storage permissions, service-role key handling, backups, and logs. A public scanner should not claim to have proven private policy state it cannot access.

6. Exercise business logic

Password resets, email changes, invitations, checkout sessions, refunds, webhooks, and file uploads deserve explicit abuse cases. These failures are often specific to the product and are difficult for a generic scanner to infer.

7. Escalate when the risk justifies it

If the app handles money, health data, identity documents, regulated information, or meaningful volumes of personal data, automated checks are only preparation. A scoped professional review or penetration test is the appropriate deeper layer.

A compact pre-launch checklist

Before launch, I want clear answers to these questions:

Did any secret reach client-side code or version history?
Are production source maps intentionally public?
Are security headers configured at the final edge, not only in local code?
Does the server enforce authorization for every sensitive operation?
Have cross-tenant negative tests been run?
Are database and storage policies reviewed in the provider console?
Are dependencies and lockfiles scanned?
Are admin functions protected server-side?
Are rate limits and abuse controls present on expensive or sensitive endpoints?
Are logs, backups, incident contacts, and key-rotation steps documented?

You can use the free Check My Vibe passive scanner and 36-point checklist as the first step. Treat the automated score and the manual checklist as separate evidence, then add repository, IDE, authorization, and business-logic testing according to the risk of the application.

The responsible promise is not that a two-minute scan makes an app secure. It is that a well-scoped scan can reveal specific, fixable evidence — and tell you honestly what still has not been tested.