DEV Community: FlawPilot

Before You Deploy an AI-Built Website: A Developer's Launch Review

FlawPilot — Fri, 09 Oct 2026 07:55:15 +0000

AI coding tools can take a prompt to a working UI in an afternoon. The preview looks finished, the happy path works, and it is tempting to ship.

But a preview proves one thing: the intended path works in one controlled environment. It says nothing about whether user A can read user B's data, whether a key is sitting in your JavaScript bundle, or whether staging's noindex made it to production.

This is the review we run before deploying generated code. It is not a threat model, pen test, or compliance audit. It is a way to catch the common launch gaps while they are still cheap to fix.

Block the release if any of these is true:

A real secret reaches the browser or the repo history.
A normal user can reach another user's data or an admin action.
The core journey (signup, checkout, form) fails on the production URL.
Production inherited a staging restriction (noindex, wrong canonical, password wall).
Nobody can roll back or knows who is watching after launch.

Everything below is how to check these quickly.

1. Test the deployed journey, not the generated screens

Run the review against the production URL or an environment that mirrors it. Preview builds often differ in caching, asset paths, env vars, and domains.

Pick the one action the site exists for (contact form, signup, checkout, upload) and complete it end to end. A success toast is not proof. Confirm the data arrived, the email was delivered, and the right system received it.

Then break it on purpose:

Submit empty and malformed values
Reuse an expired token or old confirmation link
Reload mid-flow, then use the back button after completion
Upload a blocked or oversized file
Cancel or fail a payment in test mode

A failure should be clear and safe. It should never leak stack traces, internal paths, database errors, or tokens.

2. Test authentication and authorization separately

Authentication answers "who are you?" Authorization answers "what may you do?" A working login only proves the first.

Create at least two users with different roles, then try to cross the boundary:

# Logged in as user A, request a record owned by user B
curl -i -H "Authorization: Bearer $TOKEN\_A" \\
  https://example.com/api/invoices/<ID\_OWNED\_BY\_B>

# Expect 403 or 404. A 200 means broken access control.

Also try an admin endpoint as a normal user, change a role value in a request body, and attempt a write with a read-only account.

Hiding a button in the UI does nothing if the endpoint is open. Enforce ownership on the server, in the query itself:

app.get("/api/invoices/:id", requireAuth, async (req, res) => {
  const invoice = await db.invoice.findFirst({
    where: { id: req.params.id, ownerId: req.user.id },
  });
  if (!invoice) return res.status(404).end();
  res.json(invoice);
});

Multi-tenant apps need extra care: a user from one organization must not be able to read, search, or infer another organization's data.

Broken access control sits at the top of the OWASP Top 10, and generated code is a common place for it to hide.

3. Trace every secret from source to browser

If a value reaches the browser, assume anyone can read it. Frameworks differ on which env vars are bundled into client code, and generated examples do not always say.

Search your production build output, not just your source:

# Adjust the build folder for your framework (dist, build, .next/static, out)
grep -rEn "sk\_live|AKIA\[0-9A-Z]{16}|api\[\_-]?key|secret" dist/ 2>/dev/null

# Source maps can expose original code
find dist -name "\*.map"

# Was a key ever committed?
git log --all --oneline -S"sk\_live"

Also review network requests in DevTools, build logs, error messages, and example config files.

If a real credential was exposed, rotate or revoke it. Deleting it from the latest commit does not remove it from Git history, caches, logs, or deployment artifacts. Give each key the minimum permissions it needs and restrict it by origin, environment, or quota where the provider allows.

4. Prune dependencies and third-party scripts

Generated projects collect packages and scripts across prompts, and some outlive the feature that needed them.

npm audit --omit=dev
npx depcheck

Remove what is unused, review what is left, and apply upgrades in a branch with tests and a rollback path. Do not bulk-upgrade right before release.

For every third-party script on the deployed page (analytics, chat, ads, widgets), know who owns it, why it is there, and what data it receives.

5. Inspect the real production response

Your hosting platform, CDN, or proxy can change what the browser actually receives. Look at the live response:

# Headers on the final response
curl -sI https://example.com

# Redirect chain: http, www, and non-www should end on one preferred URL
curl -sIL http://example.com | grep -iE "^(HTTP|location)"
curl -sIL https://www.example.com | grep -iE "^(HTTP|location)"

Check:

HTTPS everywhere, no mixed content
One preferred host, with short redirect chains
Cookie flags (Secure, HttpOnly, SameSite)
CORS rules that match your actual clients
Security headers: Content-Security-Policy, Strict-Transport-Security, X-Content-Type-Options, Referrer-Policy, and framing controls

Do not paste a strict CSP from another site and deploy it. It can block your own scripts, fonts, payments, or auth provider. Start in report-only mode, watch what it flags, then enforce:

Content-Security-Policy-Report-Only: default-src 'self'; script-src 'self'; img-src 'self' data:; report-to csp-endpoint

The MDN CSP guide covers each directive.

If you use a CDN or proxy, also confirm the origin server is not directly reachable in a way that bypasses it.

6. Check what crawlers actually receive

Staging settings leak into production more often than you would expect.

# Robots directives and canonical on the live page
curl -s https://example.com | grep -iE "<meta\[^>]+robots|<link\[^>]+canonical"

# Header-level noindex
curl -sI https://example.com | grep -i x-robots-tag

curl -s https://example.com/robots.txt

Verify for key pages: correct status code, no accidental noindex, a canonical pointing to the preferred production URL, unique title and description, one h1, crawlable internal links, and a sitemap with live URLs only.

For client-rendered apps, inspect the rendered HTML. If essential copy, links, or metadata only appear after heavy JavaScript runs, a crawler may not see them. Google's indexing docs explain how rendering and canonicals are handled.

Also check social previews: a correct canonical can still ship with an Open Graph image from a builder preview domain.

7. Test performance and accessibility on the real build

Measure the deployed build on a mobile profile and a constrained network, not your laptop.

npx lighthouse https://example.com \\
  --only-categories=performance,accessibility,seo \\
  --view

Common causes of slow generated sites: oversized hero images, unused JavaScript, duplicate UI libraries, many font weights, animation packages for small effects, and blocking third-party widgets. Check loading, responsiveness, and layout stability, then complete the main task by hand. A good aggregate score can hide a form that still feels laggy.

Automated accessibility checks catch missing labels and contrast issues, but not the whole experience. The W3C WAI is explicit that tools cannot cover every requirement. Manually check:

Keyboard-only navigation, focus visibility, and focus order
Persistent form labels and readable error messages
Alt text on meaningful images
Zoom and reflow at small viewports
Reduced-motion behavior

8. Assign owners and make a release decision

A report fixes nothing by itself. Give every release-blocking issue one owner, one deadline, and one piece of evidence:

Critical journeys: product or QA, with a completed success and failure run
Access control and app behavior: developer, with role-based test results against the production build
Hosting, DNS, TLS, CDN: platform owner, with verified live configuration
Indexing and metadata: SEO or marketing, with live URL inspection
Analytics and forms: marketing ops, with confirmed events and delivery

One person can own several rows. What matters is that ownership is explicit.

Then sort findings by action rather than a scanner's severity label:

Block the release: exposed production credentials, broken authorization, public sensitive data, unsafe payment or account behavior, a critical journey that cannot be completed
Fix before public traffic: broken forms, accidental indexing blocks, wrong redirects, severe mobile problems, no monitoring on a critical service
Schedule and track: lower-impact hardening and cosmetic issues, each with an owner and a date
Investigate: unclear, duplicated, or environment-specific findings. Confirm the affected asset before changing production

After every fix, retest on the live environment. A merged PR does not prove your CDN, cache, DNS provider, or pipeline delivered the change.

9. Plan the first hours after deploy

Before release, decide who watches errors and uptime, who checks form and email delivery, who can roll back, and where launch issues are logged. Know how to restore from backup. Keep the domain registrar, DNS, hosting, repo, and email accounts under team control rather than one person's login. Re-run this review after any change to the domain, hosting, framework, auth, analytics, or integrations.

Automating part of this

Most of the checks above can be scripted or run with free tools: curl, Lighthouse, npm audit, OWASP ZAP, and axe. Several hosted scanners also cover the public-surface checks (security headers, performance, SEO, infrastructure) in one pass.

Disclosure: I work on one of them, FlawPilot. Whatever you use, keep the limits in mind. A public scan cannot sign in as each role, inspect private business logic, or judge accessibility in full, and source scanning does not replace runtime testing. Treat automated results as inputs to the release decision, not permission to deploy.

Final checklist

[ ] Production-like URL tested, primary journey works end to end
[ ] Failure and cancellation paths fail safely
[ ] Authentication and authorization tested separately, with multiple roles
[ ] No secrets in the bundle, source maps, logs, or Git history
[ ] Dependencies and third-party scripts reviewed
[ ] HTTPS, redirects, cookies, CORS, and headers checked on the live response
[ ] Important pages indexable, with correct canonicals and metadata
[ ] Performance tested on the production build under mobile conditions
[ ] Accessibility tested automatically and manually
[ ] Every open issue has an owner and a release decision
[ ] Monitoring and rollback responsibilities are clear
[ ] Critical fixes verified after deployment

AI changes how fast you can build. It does not change what production has to withstand.

What is the worst thing you have found in an AI-generated codebase before launch? Curious what others check that is missing from this list.

A Fast AI-Built App Can Still Miss Basic Security Controls

FlawPilot — Thu, 01 Oct 2026 09:13:06 +0000

An application can load quickly, pass a normal click-through test, and still be unready for production.

That is not a contradiction. Functional testing and security testing answer different questions.

A functional test asks whether a user can complete the intended journey. A security review asks what else the application, domain, browser, server, and deployment will allow.

An anonymized review of one AI-built web application made that difference clear. The app scored 100 out of 100 for performance, yet the broader review found 35 open issues:

1 critical
6 high severity
19 medium severity
9 low severity

The application was not visibly broken. It loaded fast, rendered correctly, and supported its main workflow. Most of the problems existed outside the path a builder would normally test in the browser.

What the review measured

The application was assessed across four areas:

Performance: 100 out of 100, with no findings
Security: 70 out of 100, with 16 findings
Infrastructure: 54 out of 100, with 10 findings
Search visibility: 46 out of 100, with 9 findings

The combined score was 67 out of 100.

The perfect performance result was valid. The app handled loading speed, caching, asset delivery, and rendering well. Modern frameworks and AI app builders often provide good defaults for these areas.

The security and infrastructure findings were also valid. They did not affect the normal user experience, so manual testing had not exposed them.

This is a common weakness in fast AI-assisted development. The builder checks whether the requested feature works. Nobody checks the surrounding controls unless the release process requires it.

Missing DMARC left the domain easier to impersonate

The domain had no DMARC policy.

DMARC works with SPF and DKIM to help receiving mail systems evaluate messages that claim to come from a domain. It also lets the domain owner specify what should happen when authentication checks fail.

Without an appropriate policy, attackers have more room to send email that appears connected to the company. That can support phishing against customers, employees, or partners.

Nothing about this problem appears in the app interface. A registration form, dashboard, or payment flow can work perfectly while the email domain remains poorly protected.

Teams should review email authentication whenever a new domain is prepared for production, even if the application itself does not send much email yet.

The browser had no Content Security Policy

The application did not send a Content Security Policy header.

CSP tells the browser which sources may provide scripts, styles, frames, images, and other resources. A well-designed policy can limit what malicious code is allowed to execute if an injection weakness or compromised third-party script reaches a page.

CSP is not a substitute for fixing cross-site scripting. It is an extra restriction that can reduce impact.

Adding it carelessly can break legitimate application behavior. Start by listing the resources the app needs, test the policy in a non-production environment, and review browser reports before enforcing it.

The important point is ownership. If nobody is responsible for response headers, the control may never enter the release checklist.

HTTPS was present, but HSTS was missing

The application used HTTPS but did not send a Strict-Transport-Security header.

HSTS tells compatible browsers to use HTTPS for future requests to the domain. It reduces the chance that a visitor's first connection will be downgraded to HTTP on an untrusted network.

A browser padlock confirms the current page uses HTTPS. It does not confirm that HSTS is configured correctly.

Before enabling HSTS, confirm that the domain and required subdomains work over HTTPS. A long duration or an includeSubDomains directive can create an outage if part of the domain still depends on HTTP.

This is a small configuration change with real consequences, so teams should test it instead of copying a header value without understanding its scope.

The app could be framed by another site

The review also found no effective clickjacking protection.

Without a framing restriction, another website may be able to load the application inside an invisible or misleading frame. An attacker can then try to persuade a user to click a real application control while believing they are interacting with something else.

The preferred modern control is the frame-ancestors directive in CSP. Some teams also retain X-Frame-Options for older browser support.

Applications that intentionally support embedding need a more precise policy. Blocking every frame is not appropriate when approved partner domains must embed the application.

Again, the correct setting depends on the product. The AI builder cannot infer that business rule from a generic request to create a dashboard.

Infrastructure findings sat outside the main code path

The review found several other gaps:

no CAA record restricting certificate issuance
no DKIM signature for outgoing email
server information exposed in responses
no robots.txt file for crawler guidance

These findings did not have equal security impact. A missing crawler file should not receive the same response as exposed credentials or broken authorization.

Their shared cause was a missing review step. The visible product had received attention. Domain, email, server, and deployment configuration had not.

This distinction matters because teams can waste time if they treat every automated finding as urgent. Severity is a starting point, not the complete decision.

Why vibe coding creates this blind spot

AI coding tools respond to the requirements in the prompt and the context available to them.

If someone requests a login page, dashboard, database integration, and billing flow, the tool focuses on those capabilities. It may not know the organization's email setup, deployment architecture, data classification, threat model, or access policies.

Many missing controls do not cause a build failure:

a missing security header does not stop the page from rendering
weak email authentication does not break a dashboard
excessive cloud permissions do not affect the happy path
a vulnerable package may continue to work normally
an exposed secret may remain unused by an attacker for weeks
a broken database policy may go unnoticed when testing with one account

This is why "the app works" cannot be used as evidence that the app is secure.

The issue is not that AI-generated code is automatically unsafe. The issue is that fast generation often removes the independent review that used to happen between implementation and release.

Teams need to put that review back into the workflow.

A public deployment review cannot inspect everything

A review of a live URL can examine publicly visible behavior and configuration, including:

TLS configuration
DNS and email authentication records
HTTP response headers
public files and browser assets
exposed server information
cross-origin behavior

It cannot inspect a private repository without authorized access.

Repository analysis answers different questions. It can look for insecure code patterns, secrets committed to Git history, vulnerable dependency versions, and code that is difficult to review safely.

Both views matter.

A source repository can appear clean while the deployed environment has weak headers or incorrect DNS records. A live site can appear well configured while its code contains an injection flaw, an outdated dependency, or a key committed in an older revision.

Treat repository checks and deployment checks as separate release controls.

Prioritize findings instead of chasing the total count

The number 35 sounds serious, but the total does not tell a team what to fix first.

Prioritization should consider at least five factors:

What an attacker could do
Whether the affected path is publicly reachable
Whether sensitive data or privileged actions are involved
How widely the issue affects the application
Whether the proposed fix can be tested and rolled back safely

Leaked production credentials, public database access, broken authorization, and reachable injection flaws normally take priority over low-impact information disclosure or missing search metadata.

Easy fixes can still be worth doing early when they reduce meaningful risk. The reviewed application could address several high-severity configuration gaps without rewriting application logic. That did not make every fix risk-free. Header and DNS changes still needed testing and verification.

A practical pre-release checklist

Before publishing an AI-built web application, verify the following:

Store production secrets outside source control.
Scan the complete Git history for previously committed credentials.
Rotate any exposed secret instead of only deleting the file.
Test authorization with multiple users and roles.
Confirm database policies prevent cross-account access.
Review direct and transitive dependencies for known vulnerabilities.
Restrict CORS to the origins, methods, and headers the application needs.
Configure relevant security headers and test them in staging.
Set up SPF, DKIM, and DMARC for domains that send email.
Remove unnecessary server and framework disclosures.
Review repository, CI/CD, cloud, and deployment permissions.
Scan a representative staging deployment before release.
Check the production URL after deployment.
Assign an owner and review date to every accepted risk.

Adjust the checklist to the application. A public portfolio has a different risk profile from a multi-tenant product that stores financial, health, or customer information.

Recheck after deployment

Security review is not complete when a developer merges a fix.

Confirm that the corrected configuration reached production. A reverse proxy, hosting platform, content delivery network, or environment variable can make the deployed result different from the local version.

Run checks again after changes to:

hosting providers
domains or DNS
authentication systems
database policies
third-party scripts
dependencies
deployment workflows
major application routes

Certificates expire, package vulnerabilities are disclosed, and configuration changes accumulate. A secure launch does not guarantee a secure application six months later.

The main lesson

The reviewed application was fast. It was functional. It also missed controls that mattered.

Those facts can all be true at the same time.

AI-assisted development reduces the time required to turn an idea into a working application. It does not remove the need to review code, dependencies, permissions, domain settings, deployment configuration, and the final production result.

Keep the speed advantage. Add a security checkpoint before launch, then verify the application again after it reaches production.

How to choose a SAST tool: 8 open-source options and a broader security platform

FlawPilot — Thu, 24 Sep 2026 12:02:03 +0000

Static application security testing helps developers find risky code without running the application. That definition is simple. Choosing a scanner is not.

A tool may support your programming language but miss the framework patterns that matter in your project. Another may produce useful findings but require more maintenance than your team can manage. A third may work well in continuous integration but provide little help when developers need to understand or fix an alert.

This guide compares eight open-source SAST tools and FlawPilot, a broader code and application security platform. The goal is to help you build a shortlist based on your codebase, workflow, and security needs.

Disclosure: The author is affiliated with FlawPilot. It is included in third position and assessed using the same practical criteria as the other options.

What SAST checks

SAST examines source code, bytecode, or build artifacts without sending test traffic to a running application. Depending on the engine and language, it may detect:

Unsafe input handling
Injection paths
Hardcoded credentials
Weak cryptographic choices
Unsafe deserialization
Path traversal risks
Cross-site scripting paths
Risky functions or insecure code patterns

Many SAST tools can run on a developer's machine, during a pull request, or in a continuous integration pipeline. They often identify the file and line connected to a finding, which gives the developer a place to begin the investigation.

That finding is not proof of exploitation. The scanner may not understand every validation step, authorization control, runtime condition, or business rule. A developer still needs to check whether the code is reachable, whether an attacker can control the input, and what the impact would be.

Why source scanning and live testing are different

SAST reads code. Dynamic or live application testing interacts with a running system.

Question	Source scanning	Live application scanning
What does it inspect?	Source code, bytecode, or build artifacts	A running website, API, or web application
When can it run?	During development and continuous integration	In a test environment, before launch, or after deployment
What context does it see?	Internal code paths and data flow	Public behavior, responses, services, and configuration
What can it miss?	Runtime settings and external exposure	Internal code that is not reachable during the test

One method cannot replace the other. A source scanner may identify an unsafe query construction. A live scanner may identify missing security headers, weak transport settings, or an exposed service. Teams that build internet-facing applications usually need both views.

Tool comparison

Tool	Best fit	Main focus
Semgrep Community Edition	Multi-language repositories and custom rules	Pattern-based source analysis
CodeQL	Projects hosted in GitHub workflows	Semantic analysis and code queries
FlawPilot	Teams that want source and deployed-application checks	Code scanning, live checks, and AI Fix prompts
Bandit	Python projects	Python security rules
Brakeman	Ruby on Rails applications	Rails-aware static analysis
Gosec	Go services and applications	Go-specific security checks
SpotBugs with Find Security Bugs	Java and JVM projects	Bytecode analysis with security detectors
MobSF	Mobile application assessment	Mobile static and dynamic analysis
PMD	Java-heavy projects with code-quality needs	General static rules with selected security checks

Capabilities, licenses, and supported languages can change. Confirm current details before adopting a tool across multiple repositories.

1. Semgrep Community Edition

Semgrep Community Edition scans source code with rules that describe insecure or unwanted patterns. It supports many languages, though the depth of support is not identical across all of them.

Its main advantage is flexibility. A team can use community rules, adjust existing rules, or write checks for patterns specific to its own code. Developers can run it locally and add it to a pull-request or continuous integration workflow.

That flexibility requires ownership. Someone needs to choose the rules, test them against the codebase, remove irrelevant checks, and maintain custom rules as frameworks change. Enabling a large rule collection without tuning can create noise.

Semgrep is worth considering when you have several languages, want editable rules, and have someone who can maintain the rule set.

2. CodeQL

CodeQL creates a database representation of a codebase and lets security queries analyze relationships and data flow. This approach can identify problems that simple pattern matching may not find.

It fits naturally into GitHub-based development. Findings can appear in code-scanning workflows and pull requests for supported languages. Security engineers can also write custom queries for organization-specific risks.

The tradeoff is complexity. Advanced query development requires knowledge of the CodeQL language and its libraries. Some compiled projects also need correct build configuration before analysis can work well.

CodeQL is a strong candidate when repositories already use GitHub and the team needs deeper semantic analysis. Check the current licensing and availability conditions for your repository type and plan.

3. FlawPilot

FlawPilot covers source code and the deployed application. Teams can connect a repository for code scanning and scan a website, web application, or SaaS product before or after launch.

The code scan looks for insecure code, exposed secrets, and vulnerable dependencies. The live scan checks publicly visible security, performance, infrastructure, and SEO signals. This combination helps a developer compare what exists in the repository with what the deployed environment exposes.

For supported findings, FlawPilot supplies an AI Fix prompt. A developer can copy that prompt into a coding assistant to receive finding-specific remediation guidance. The developer should review the resulting change, run the relevant tests, and scan again before deployment.

FlawPilot may suit founders and development teams that want prioritized explanations, repository checks, and live scanning in one workflow. It does not replace a manual penetration test. Automated checks can miss business-logic problems, complex authorization failures, authenticated paths, and attacks that depend on several weaknesses.

4. Bandit

Bandit scans Python abstract syntax trees and applies security-focused plugins. Its checks cover patterns such as hardcoded passwords, unsafe subprocess use, weak cryptography, and dangerous function calls.

The narrow focus is useful for Python teams. Bandit is easy to run locally or inside continuous integration, and it can produce structured output for other development tools.

Bandit still needs tuning. A rule may report code that is safe in the actual application context. Teams should document suppressions and review them instead of allowing an ignore list to grow without ownership.

Bandit does not check whether installed Python packages contain known vulnerabilities. Pair it with dependency analysis when assessing a Python project.

5. Brakeman

Brakeman is designed for Ruby on Rails. It understands Rails conventions and reviews application code without starting the application server.

Its framework knowledge helps it identify Rails-related injection, redirect, cross-site scripting, and configuration risks. It can run from the command line and in automated build workflows.

Brakeman is a focused choice, not a complete scanner for a mixed technology stack. A system with a Rails backend, a separate frontend, infrastructure files, and cloud services will require other checks.

Teams should tune warning confidence and review ignored findings after important code changes. A warning that seemed irrelevant in one release may matter after the surrounding logic changes.

6. Gosec

Gosec examines Go source code using security rules built around Go analysis packages. It can identify risky permissions, hardcoded credentials, weak cryptography, unsafe integer conversions, and other Go-specific patterns.

It fits into Go development workflows and can export results in formats used by code-scanning systems. That makes it practical for services that already run tests, formatting, and analysis in continuous integration.

Gosec is different from a general Go linter. A linter usually emphasizes correctness, consistency, and maintainability. Gosec adds security rules, but teams still need separate dependency analysis and testing of the running service.

Review each rule against the application. A command-line utility and a public API have different exposure and may need different rule settings.

7. SpotBugs with Find Security Bugs

SpotBugs analyzes Java bytecode for patterns associated with programming defects. The Find Security Bugs plugin adds security detectors for Java web and Android applications and can also support relevant patterns in other JVM languages.

This combination fits Maven and Gradle workflows. It is useful for teams that want general defect checks and security rules during the same build process.

Scan quality depends on build quality. Missing classes or an incomplete type hierarchy can reduce the analyzer's understanding of the application. Treat those build warnings as scan problems rather than harmless messages.

SpotBugs alone is not a complete application-security scanner. Make sure the security plugin is installed and configured if security analysis is the objective.

8. MobSF

Mobile Security Framework, commonly called MobSF, is designed for Android, iOS, and Windows mobile application assessment. It can perform static and dynamic analysis, depending on the application artifact and test setup.

MobSF can inspect permissions, certificates, application components, embedded files, and mobile platform configuration. That makes it different from scanners aimed at ordinary web application source code.

The broad mobile coverage requires more setup than a small language-specific scanner. Dynamic analysis needs a suitable environment, and the results need review by someone who understands the mobile platform and the application.

Mobile security also includes backend APIs, authentication, data handling, dependencies, and high-risk user flows. MobSF should form one part of that process.

9. PMD

PMD checks source code for programming problems, duplicated code, maintainability issues, and selected security patterns. It is strongly associated with Java, although it has modules for other languages.

It works well when a team wants one rule system for code quality and some security checks. Java projects can add it to common build workflows and create project-specific rules.

PMD is not as security-focused as Bandit, Brakeman, or Gosec. Treat its security rules as an extra layer. A Java application that needs deeper security analysis may pair PMD with a dedicated security tool.

Avoid enabling every rule at once. Select checks that apply to the codebase, then expand the rule set as the team learns how to handle the output.

How to build a useful shortlist

Tool selection should begin with your repository. Popularity and long feature lists are poor substitutes for testing a scanner against real project code.

Match the language and framework

Confirm that the scanner supports the language version and framework you use. Basic syntax support does not always mean deep framework-aware analysis.

For a mixed stack, decide whether one multi-language scanner provides enough coverage or whether each major language needs a focused tool. A focused scanner often gives better framework context, while a multi-language tool can reduce operational work.

Decide where feedback should appear

Choose when developers need results. Options include the editor, a local command, a pull request, a continuous integration job, or a scheduled scan.

Begin with reporting. Blocking every build on the first day can disrupt delivery if the scanner reports old issues or irrelevant patterns. After tuning the rules and reviewing the existing findings, block only new results that meet a clear severity and confidence threshold.

Test a representative repository

Run each shortlisted tool against code that reflects your normal architecture. Then review:

Whether the findings are relevant
Whether each result points to a useful code location
How often the team sees false positives
Whether the explanation helps a developer investigate
Whether the output works with existing systems
How much time the scan adds to the pipeline

Do not compare tools using the total number of alerts. A scanner that reports more issues is not automatically better. Review the accuracy and usefulness of those findings.

Check rule controls and suppressions

The team needs a way to disable irrelevant rules, exclude generated files, record accepted risks, and create project-specific checks when necessary.

Every suppression should have a reason and an owner. Review accepted risks after major architecture changes or framework upgrades. Otherwise, an old exception can hide a new problem.

Review maintenance and licensing

Check release activity, supported language versions, open issues, documentation, and the health of the contributor community. Also review the license for the scanner, plugins, and rule packs.

Open source does not mean there is no operating cost. Installation, upgrades, continuous integration jobs, rule maintenance, result triage, and developer support all require time.

A staged rollout for development teams

A gradual rollout gives the team time to understand the scanner before it controls releases.

Select one active repository with supported languages.
Run the scanner in reporting mode.
Review the first results with a developer who knows the code.
Disable irrelevant rules and document justified suppressions.
Fix high-confidence findings with meaningful impact.
Record a baseline for accepted existing findings.
Block new critical findings once the output is reliable.
Review tool versions, rules, and suppressions on a schedule.

This process separates existing security debt from newly introduced problems. The team can stop new high-risk issues while handling older findings through planned work.

What SAST does not cover

Static analysis cannot see every security problem. A complete testing process may also require:

Dependency analysis
Secret scanning
Infrastructure-as-code checks
Container image scanning
API security testing
Live application scanning
Manual testing of authorization and business logic

Choose the testing mix based on the application. A public content website, an authenticated SaaS product, and a financial mobile application do not have the same exposure or impact.

Frequently asked questions

Which open-source SAST tool should I choose?

Choose according to the codebase. Semgrep supports many languages and custom rules. Bandit focuses on Python, Brakeman on Ruby on Rails, Gosec on Go, and MobSF on mobile applications. Test shortlisted tools on your own repository before deciding.

Can SAST find every vulnerability?

No. SAST can find insecure patterns and some risky data flows, but it may miss runtime configuration, vulnerable dependencies, authorization failures, exposed services, and business-logic flaws.

When should a team run SAST?

Run it while developers write and review code, then repeat the scan in continuous integration. Start in reporting mode and introduce release blocking after the team has tuned the rules.

Is dependency scanning the same as SAST?

No. SAST analyzes code written for the application. Dependency scanning checks third-party packages and versions against known vulnerability information. Most projects need both.

Is SAST enough before launch?

No. Test secrets, dependencies, authentication, authorization, APIs, infrastructure configuration, and the deployed environment. High-risk systems may also need manual penetration testing.

Can SAST scan AI-generated code?

Yes, if the scanner supports the generated languages and frameworks. Review AI-generated code using the same security controls as human-written code. Developers must still confirm findings and test the deployed application.

Final guidance

The right SAST tool is the one your team can run, understand, tune, and maintain. Start with language support and workflow fit. Test the output against a real repository before creating release rules.

Static analysis gives developers visibility into code risks. Dependency checks, secret scanning, live testing, and manual review cover problems that source analysis cannot see. Use the combination that matches the application's exposure and risk.

A Developer-Friendly Workflow for Triaging Website Security Alerts

FlawPilot — Fri, 18 Sep 2026 07:40:14 +0000

A security scanner reports a missing header, an exposed key, an unsafe cookie, and a certificate warning. Which one should the team fix first?

The severity label helps, but it cannot answer that question by itself. A scanner does not know whether the affected asset is public, whether the credential is still valid, how many users depend on the service, or whether anyone is exploiting the weakness.

Developers need a repeatable way to turn scanner output into engineering work. The workflow below covers the full path: capture the evidence, validate it, decide whether the problem is an incident, assign an owner, make a safe change, and verify production.

Normalize the alert before discussing priority

Security tools describe findings in different ways. Before comparing them, put each alert into the same record format.

finding:
  title: "Public API key found in client-side JavaScript"
  asset: "https://example.com/assets/app.js"
  environment: "production"
  detected_at: "YYYY-MM-DD HH:MM UTC"
  scanner_severity: "high"
  evidence: "Key pattern and file location from the scan"

triage:
  publicly_reachable: true
  authentication_required: false
  sensitive_system_affected: "unknown"
  active_misuse_observed: "unknown"
  temporary_control_available: true

response:
  owner: "unassigned"
  status: "new"
  containment: "not started"
  permanent_fix: "not started"
  production_retest: "pending"

Do not place the full value of a secret, session token, or sensitive record in a general-purpose ticket. Record enough evidence to identify the issue, then store sensitive evidence in a restricted system.

This common format prevents several mistakes. It separates the scanner's assessment from the team's assessment. It also makes missing information visible instead of letting people assume that someone else checked it.

Validate the finding against the correct asset

Validation answers two questions:

Does the reported condition still exist?
Does the evidence point to the actual cause?

Start with the exact URL, hostname, file, port, certificate, DNS record, cookie, or response header named in the report. Confirm that it belongs to the intended environment.

Production and staging may behave differently because of:

CDN or reverse-proxy configuration
Environment-specific headers
Redirects between hostnames
Browser or edge caching
Different certificates
Feature flags
Old deployment assets

For an HTTP header finding, inspect the response returned by the public production URL. For a DNS finding, query the authoritative record rather than relying on an old screenshot. For a client-side exposure, inspect the deployed asset instead of checking only the current source branch.

Do not dismiss a finding because the page loads correctly. Missing cookie flags, loose cross-origin rules, and weak response headers often have no visible effect on the interface.

Separate a security finding from a security incident

A finding reports an unsafe or uncertain condition. An incident involves suspected or confirmed misuse, unauthorized access, data exposure, or service disruption.

A missing Content-Security-Policy header is normally a remediation task. A public credential with unexpected activity may require incident response.

Look for evidence such as:

Authentication events nobody recognizes
Requests made with an exposed credential
Unexpected administrative changes
Access to data outside an expected pattern
Unexplained traffic spikes or service disruption
Cloud, hosting, or repository alerts connected to the finding

If you see these signs, preserve relevant logs before making changes that could destroy evidence. Escalate according to the organization's incident process. An external website scan cannot determine what happened inside private accounts, logs, or authenticated workflows.

Prioritize with context, not severity alone

Use the scanner severity as one input. Then review four contextual factors.

Reachability

Can an unauthenticated person reach the affected asset from the public internet? A production endpoint that anyone can access usually deserves more attention than a private development service.

Impact

What could a successful misuse affect? Give more weight to findings connected to user sessions, privileged accounts, production credentials, payment paths, personal data, or administrative functions.

Preconditions

What must happen before the weakness can be used? A flaw that requires an authenticated administrator and an unusual configuration has different urgency from a working secret inside a public JavaScript bundle.

Current evidence

Is the risk theoretical, reproducible, or already being misused? Evidence of active misuse moves the work out of normal backlog prioritization.

After this review, place the finding into one of four action groups:

Contain now: suspected misuse, exposed credentials, unauthorized access, or active disruption
Fix next: public weakness with meaningful impact and a practical path to misuse
Schedule: missing defense or weak configuration with limited current exposure
Investigate: stale, duplicated, environment-specific, or unclear evidence

This classification is an engineering aid. It does not replace contractual, regulatory, or legal response requirements.

Assign the person who controls the affected system

An alert sent to an entire channel often has no owner. Assign one person to coordinate the next action and confirm that they accepted it.

Ownership usually follows the control:

Application developers own authentication logic, cookie settings, and application-level cross-origin behavior.
Platform or hosting owners handle response headers, redirects, certificates, public services, and edge configuration.
Domain or email administrators handle DNSSEC, CAA, SPF, DKIM, and DMARC.
The developer and service-account owner should work together when a credential is exposed.
A security or incident-response specialist should guide work when misuse or data exposure is possible.

The coordinator does not need to make every change. Their job is to keep the alert from becoming an unowned message thread.

Contain immediate risk before completing the repair

Containment reduces exposure while the team works on the root cause.

If a credential is public, revoke or rotate it. Deleting the value from the latest source file is not sufficient. Copies may remain in build artifacts, browser downloads, CDN caches, deployment history, logs, or Git history.

After rotation, restrict the replacement credential where possible:

Limit its permissions
Restrict allowed domains or IP addresses
Set usage limits
Separate production and development credentials
Enable provider-side monitoring

If a route or service appears to be under active attack, a temporary response may include disabling the route, restricting access, adding a narrow web application firewall rule, or rolling back a release.

Containment is not closure. Keep the task open until the team fixes the cause and verifies the live system.

Make the smallest safe change

Broad configuration changes can break authentication, payments, analytics, embedded tools, or third-party scripts.

Content Security Policy is a common example. Copying a strict policy from an unrelated site may block resources your application needs. Build the policy around the actual scripts, frames, styles, fonts, and connections used by your application. Test the affected user journeys before enforcing it in production.

Apply the same discipline to:

Cross-origin resource sharing rules
Cookie attributes
Redirect behavior
Cache controls
TLS and certificate changes
DNS records
Storage permissions

Use staging when it represents production accurately. Prepare a rollback path for changes that affect traffic, authentication, or external integrations.

Retest the public result after deployment

A merged pull request does not prove that the finding disappeared from production.

The application may still serve an old asset or cached response. The change may exist only in staging. A configuration file may not be connected to the active service.

Use this verification sequence:

Deploy the change through the normal release process.
Repeat the original check against the same production asset.
Confirm that the original evidence is gone.
Test the user journey that depends on the changed control.
Check for new warnings or broken behavior.
Attach the new evidence to the ticket.

Close the task only when the production result matches the intended state.

Use a reusable remediation issue template

The following Markdown template works in most issue trackers:

## Finding

- Title:
- Affected asset:
- Environment:
- Detected at:
- Scanner severity:
- Evidence location:

## Validation

- [ ] Correct asset and environment confirmed
- [ ] Finding reproduced or independently verified
- [ ] False-positive conditions reviewed
- [ ] Sensitive evidence stored securely

## Triage

- Publicly reachable: Yes / No / Unknown
- Authentication required: Yes / No / Unknown
- Potential impact:
- Preconditions:
- Signs of active misuse: Yes / No / Unknown
- Action group: Contain now / Fix next / Schedule / Investigate

## Ownership

- Coordinator:
- Technical owner:
- Target date:
- Escalation contact:

## Response

- Containment action:
- Root cause:
- Permanent fix:
- Rollback plan:

## Verification

- Production deployment:
- Original check repeated:
- User journey tested:
- New issues found:
- Evidence attached:
- Final status:

Adjust the fields to match the team's incident process and access controls. Avoid copying secrets or personal data into the issue.

Define the workflow before the next alert

Write down who reviews new findings, where the team records them, and which conditions start an incident response. Document who can rotate credentials, change DNS, update hosting configuration, and deploy emergency fixes.

The policy should also define how the team accepts a known risk and when the same asset will be checked again. A finding should not disappear simply because the current sprint ended.

NIST SP 800-61 Revision 3 connects preparation, detection, response, and recovery. A small team may combine several roles, but it still needs clear ownership and an escalation path.

Closing thought

Scanner output is evidence, not a completed decision. Validate the affected asset, add technical and business context, and assign a person who can move the work forward.

For urgent exposure, contain first. For normal remediation, make a controlled change and retest the public system. The alert is resolved only when production evidence confirms the fix.

This article is adapted from an original guide published on FlawPilot. It contains no affiliate links or paid placement.

Lovable App Security We Scanned a Real App and Found 29 Open Issues

FlawPilot — Fri, 14 Aug 2026 12:29:29 +0000

Quick answer: Lovable app security's most common failure isn't a missing header. It's an unset Supabase Row-Level Security policy. Skip it and the app still works, because the query still succeeds, just for anyone, not only its owner. We scanned a real Lovable app and found exactly that gap, live.

Picture the app you shipped last sprint. You described a dashboard, Lovable wired up the tables, the auth, the UI, and it all just worked the first time you clicked through it. That's the whole appeal. It's also exactly why nobody goes back and checks the one setting that doesn't announce itself when it's missing. So we did. We took a real, live Lovable app, built the way most Lovable apps get built (fast, from a prompt, shipped the same week), and ran it through FlawPilot, a free scan that checks a site's security, performance, infrastructure, and SEO in about two minutes, no login required. We won't name the app, but every number below is real, pulled straight from its report.

The scorecard

63 out of 100. "At risk." 252 points out of a possible 400, and 29 things sitting open: 1 critical, 5 high, 15 medium, 8 low.

Split out:

Security: 55/100. At risk. 13 findings.
Performance: 96/100. Excellent. 2 findings.
Infrastructure: 58/100. At risk. 9 findings.
SEO & Discoverability: 62/100. Needs attention. 5 findings.

Nothing was defaced. The app looked, and behaved, exactly like it was supposed to. That's the part worth sitting with: every screen worked, every click did the right thing, and the one critical finding on this report had never once shown up as a bug.

What it got right

96/100 on performance is close to a clean sweep. Lovable's scaffolding leans on Vite and React defaults that are already fast out of the box: quick first paint, sensible bundle size, nothing render-blocking that shouldn't be.

Which is exactly the trap. A fast, polished frontend is the thing you notice with your own eyes, so it's the thing that gets credited as "done." Everything below is the kind of gap that only shows up once something outside the team, a scanner, a bot, or someone who isn't supposed to, goes looking for it on purpose.

What was hiding underneath

A Supabase table had no Row-Level Security policy at all. Rated Critical, and it's the finding that matters most on this whole report. Lovable apps run on Supabase by default, and Supabase enforces access at the database layer through RLS policies, not through the app's own login screen. Turn RLS on for a table and the database itself checks who's allowed to see which row. Leave it off, and the table is exactly as open as the anon key that ships in every Lovable app's JS bundle: open to anyone who opens dev tools and watches a single network request. This app's login screen worked correctly the whole time. The database behind it never checked whether the person asking was allowed to see the data they were asking for.

The fix, in Supabase, is two SQL statements:

-- Enable Row-Level Security on the table
ALTER TABLE public.orders ENABLE ROW LEVEL SECURITY;

-- Scope reads to the row's own owner
CREATE POLICY "Users can view their own orders"
ON public.orders
FOR SELECT
USING (auth.uid() = user_id);

No rewrite, no data migration, just a policy that tells the database who's allowed to see which row instead of trusting the app's UI to gate it for you. Repeat per table, per operation (SELECT, INSERT, UPDATE, DELETE) as needed.

The rest of the report was more familiar territory:

No DMARC record: anyone could send email pretending to be this domain (High)
No Content-Security-Policy header: an injected script has nowhere to be stopped (High)
No Strict-Transport-Security header: first request can get downgraded to plain HTTP (High)
No clickjacking protection: the app can be framed invisibly inside another page (High)
No CAA record: any certificate authority could issue a cert for the domain (Medium)
No DKIM signature: outgoing mail at real risk of landing in spam (Medium)
Missing sitemap.xml, thin meta descriptions: dragged the SEO score to 62 (Medium)
Verbose error pages left on: handing back stack traces to anyone who triggers one (Medium)

None of this showed up while clicking through the app. All of it shows up the moment something outside the team, a scanner, a bot, an attacker, bothers to check.

Why this is the default, not a fluke

That combination, an unset RLS policy sitting next to missing headers and no DMARC, is close to the single most repeated pattern across Lovable apps specifically. Lovable isn't careless, it's optimizing for "the prompt works," and a Supabase table without an RLS policy still returns exactly the data you asked for in testing. The query succeeds either way. Nothing about that failure looks like a failure until someone sends a request the app wasn't expecting.

This isn't hypothetical. In March 2025, this exact gap, missing RLS in Lovable-generated apps, was catalogued as CVE-2025-48757 (CVSS base score 8.26, High), and left more than 170 live production apps with databases any unauthenticated visitor could read, write to, or delete from, just by inspecting the app's own network requests. Nobody shipped a bug that broke anything. The gap was a security control nobody had reason to think about, sitting quietly under apps that worked perfectly for months.

The broader research backs this up. Veracode's GenAI Code Security testing has run over 100 models across 80 coding tasks over four years, and the average pass rate on a security review has held at roughly 56%, meaning close to 44% of AI-generated code fails a review outright. A 2023 Stanford study (Perry et al., ACM CCS) found the same pattern at the human level: developers using an AI coding assistant wrote measurably less secure code than developers working without one, and felt more confident about it, not less. The role that used to catch an unset RLS policy, a senior engineer flagging it in code review, mostly doesn't exist in a Lovable workflow, because there's often no review step between describing the app and the app going live.

How to actually fix this

A severity label is only useful if it turns into something you actually do. Here's how FlawPilot handles that on this exact report.

Every finding gets grouped by area and dropped into a ranked "What to do next" list, not a raw dump of jargon. Item one was the RLS finding, and next to it, in plain English: enable Row-Level Security on this table and add a policy scoping rows to the authenticated user.

Four of the five High findings here (CSP, HSTS, clickjacking, DMARC) are pure configuration, no application logic involved. The RLS fix is a Supabase policy, not a rewrite. Clear those five and close out the CAA and DKIM gaps next to them, and a security score that opened at 55 is realistically headed for the 90s.

Try it on your own app

No setup, no coding, no passwords required. Drop in a URL, and in about two minutes you get a full scorecard: security, performance, infrastructure, and SEO, each out of 100, every finding in plain English, with a toggle between a plain-language view and the full technical one.

Scan your Lovable app for free →

FAQ

Is Lovable secure by default?
Not fully. Lovable handles HTTPS and gets you a working auth flow fast, but Row-Level Security on your Supabase tables is opt-in, and HTTP security headers, DMARC, and DNS hardening aren't configured automatically either.

What's the single most common Lovable security issue?
A Supabase table with no Row-Level Security policy, also the most severe, since it can leave underlying data readable or writable by anyone.

Does scanning my site give FlawPilot access to my Supabase project?
No. The scan only checks publicly accessible signals, the same things any browser or bot on the internet can already see. It never touches your Supabase dashboard, your database directly, or your credentials.

Can I fix this myself without hiring anyone?
Header, DNS, and email-authentication fixes usually live in your hosting or DNS provider's dashboard. The RLS fix lives in your Supabase project's SQL editor or policy UI and takes minutes once you know which table needs it.

How often should I actually check this?
Before anything you'd call a real release, and again any time you add a new table, since a new table means RLS is opt-in again.

The app worked the whole time. The database behind it never once did. That gap is the part worth remembering long after the specific header names and CVE numbers fade: "it works" and "it's secure" get checked by two different questions, and only one of them shows up on screen.