DEV Community: Jeremy Burgos

Compare two crawls to confirm your SEO fixes shipped (and catch what broke)

Jeremy Burgos — Tue, 07 Jul 2026 21:00:00 +0000

TL;DR: Making a fix is not proof it shipped. Crawl before you change anything, save that crawl, re-crawl after with identical settings, and compare the two. The comparison reports each issue as resolved, reduced, or unchanged, and surfaces anything your fix broke elsewhere. One real before-and-after below.

You shipped the fix and closed the ticket. That is not the same as proving the change is live, correct, and free of collateral damage. The proof is a second crawl, compared against the first.

A crawl comparison loads your re-crawl against a saved baseline and reports every issue as resolved, reduced, or unchanged, URL by URL. Here is the procedure, then a real worked result, then the part most people skip: using the comparison to catch what your fix broke somewhere else.

The procedure

Verify a technical SEO fix with a crawl comparison (Screaming Frog)

1. Set storage mode to Database. Required for saved, comparable crawls.
2. Crawl the site. Save it as your dated baseline.
3. Make the fixes. Deploy. Let the page cache or CDN clear.
4. Re-crawl with the IDENTICAL config and the IDENTICAL segments.
5. Switch to Compare mode and select the baseline and the re-crawl.
6. Read the Issues comparison. Filters: Added, New, Removed, Missing.

Crawl comparison needs database storage mode and a paid license. The free build is capped at 500 URLs with the config restricted, so this is a licensed-tool workflow from step one.

What it proved

This is a real cycle on a production WordPress property: a June 11 baseline against a June 22 re-crawl, identical settings, every delta taken from Screaming Frog's Compare mode.

Finding	Before	After	Result
Invalid HTML elements in head	39	0	Resolved
Rich result validation errors	34	0	Resolved
Rich result validation warnings	18	2	Reduced 89%
Internal redirect links	9	0	Resolved
External redirect links	20	0	Resolved
Internal client error (4xx)	1	0	Resolved
Duplicate H2 across pages	4	0	Resolved
Non-indexable internal URLs	14	4	Cleaner
External links resolving directly to 200	78	100	+22

Two of those rows need reading carefully rather than at face value. The four remaining non-indexable URLs are the XML sitemap files, which are non-indexable by design, not a problem to chase. And the 78 to 100 line is a side effect of the redirect cleanup, because internal links that pointed at redirecting legacy slugs were repointed to their final 200 destination, not a separate win.

The high-priority cluster, head validation, schema errors, and broken or redirecting links, closed in full. Those are the items that decide whether the rest of a page's signals are read correctly, which is why they go first.

The rule that makes the comparison mean anything

The baseline and the re-crawl have to use the same spider config, the same segments, and the same prompts if you run any. Change the measurement between runs and a later difference could come from the instrument instead of the work.

Treat the config as a ruler. You are measuring the same site twice and comparing the readings, so the difference is only attributable to your fixes if the ruler did not change. The failure mode to watch for is a fake win: you re-crawl with a tighter include rule, the non-indexable count drops, and it looks like your directive cleanup worked when you simply stopped crawling the URLs that carried the problem. Lock the config before the baseline and do not touch it until after the re-crawl.

Reading the comparison

The overview gives you current, previous, change, and change percentage on every total. The issues view is where verification happens: each finding from the baseline reads as resolved, reduced, or unchanged, with the high-priority items flagged so you can see at a glance whether the changes that matter actually closed. A number that did not move is telling you a fix did not land, which is exactly what you want to know before you call the work done.

The change detection runs on four filters: Added, New, Removed, Missing. That is what lets you see which specific URLs moved and in which direction, not just that a count went down.

The benefit most people skip: regression detection

A single audit snapshot tells you what is broken now. It cannot tell you what your last fix broke. That is the argument for re-crawling even when you are sure the change worked.

A comparison surfaces new and added issues alongside the resolved ones, and the new ones are where regressions hide:

A redirect fix orphans a page that was only reachable through the old URL.
A schema template change passes on the template you edited and breaks validation on a different template that shared the component.
A noindex directive leaks onto pages you meant to keep indexable.
An internal link repoint creates a fresh redirect chain you did not have before.

None of these show up if you only look at the issue you set out to fix.

Prove it without exposing private data

The worked cycle published zero Search Console figures and zero analytics figures and still proved every structural result. Response codes, the canonical profile, the directive profile, and schema validation all come from the crawl's own data. For client work that means you can show a fix shipped and nothing regressed without disclosing indexation counts or traffic numbers. The crawl-level evidence stands on its own.

When a fix does not show up in the re-crawl

Usually one of these:

A page cache or CDN is still serving the old version. Re-crawl after it clears, or verify against a direct origin pull.
The change deployed to staging and never reached production.
A segment rule built on a URL pattern silently skipped a slug that did not match the naming convention. Validate your segment counts before trusting per-segment diffs.
The re-crawl covered a different URL set than the baseline. Back to holding the config constant.
A rendering difference between the two runs, so the output you are comparing is not the same output.

Work that list before concluding a fix failed. More often the fix is fine and the measurement drifted.

Make it recurring

Once you have a saved baseline, a fixed config, and fixed segments, the whole thing becomes a loop: baseline, fix, re-crawl, compare. Because the config and segments are held constant, every new crawl compares cleanly against the last, drift gets caught early, and each fix stays measured rather than assumed to hold. Screaming Frog can even auto-compare the last two crawls on a scheduled run, which turns the loop into close to passive monitoring.

A crawl finds problems. A re-crawl compared against a saved baseline proves they were solved. The gap between those two is where most audits stop.

Originally published at https://www.techseoexperts.com/crawl-discovery-diagnostics/crawl-comparison-verify-seo-fixes/

JavaScript SEO is a proof problem: capture the evidence before you touch code

Jeremy Burgos — Tue, 30 Jun 2026 10:30:00 +0000

TL;DR: JS SEO failures are rarely "just JavaScript." Prove three states (response HTML, rendered DOM, indexable state) before changing code. The ten-minute triage below tells you which layer is actually failing.

The rule that saves the most wasted work:

Do not change code until you can show, in captured output,
what is missing or unstable.

The expensive mistake is rewriting templates to fix a rendering problem that was actually a noindex gate in the response, a canonical that flips after load, or links that only exist after a click. The fix was cheap; the misdiagnosis was not. JS SEO issues are a mismatch between what the browser shows and what search systems can reliably fetch, render, and interpret. The work is not to remove JavaScript. It is to prove whether JavaScript is hiding or changing the signals search engines need.

Three phases, not one moment

Crawling, rendering, and indexing are separate phases that produce different evidence. Crawling is retrieval: status codes, redirects, robots directives, canonical signals, and links in the response HTML. Rendering is execution: the DOM after scripts run, which may now contain content, links, metadata, or structured data that were not in the response HTML. Indexing is processing and storage of an eligible page. The separation is what lets you rule layers out: a page can be crawlable but not renderable, render fine but be non-indexable, or be indexable and still look thin because the indexed representation never included the rendered content you assumed was there.

The three states you prove

State 1  Response HTML    raw server HTML, before scripts. Proves app-shell vs real content,
                          fetch-time directives/canonical, links without execution.
State 2  Rendered DOM     after scripts run. Proves what appeared only after load,
                          and whether identity signals changed after load.
State 3  Indexable state  eligibility given status, directives, canonical consistency.

Critical rule: do not rely on JavaScript to fix indexability after load. If a directive is wrong at fetch time, treat it as wrong, because that is the version you cannot count on a search system to repair.

Ten-minute triage

Confirm indexability at fetch time: final status code, robots directives in response HTML, canonical in response HTML. If there is a noindex in the response or the canonical points somewhere unintended, stop and fix the gate. No rendering fix helps a page that is not eligible.
Compare response HTML to rendered DOM. If primary content, links, or metadata exist only after render, the site depends on execution, which raises the bar for stability and crawl-safe discovery.
Check interaction dependence. If main content or links appear only after scroll or click, or "Load more" hides crawlable URLs, assume discovery risk until you prove a crawl-safe pattern.
Validate metadata stability across both states: title, canonical, robots, structured data. A canonical or robots directive that changes after load causes consolidation problems even when content renders fine.

For diagnostics, the URL Inspection tool in Search Console shows the crawled versus rendered view, and Screaming Frog with JavaScript rendering enabled surfaces the same discrepancies across a whole site. Both are checks you run on your own property.

Rendering strategy sets the risk

Client-side rendering is highest risk when the response is mostly an app shell and the real content only appears after load. SSR lowers risk by delivering meaningful HTML at fetch time, but still needs validation for consistent canonicals and directives. Static generation tends to be most stable. Pre-rendering is a practical bridge when you need stable HTML fast but cannot rewrite the app yet. The evidence tells you which you need: critical elements missing from response HTML, or unstable head signals, is the signal.

Originally published at https://www.techseoexperts.com/rendering-javascript-diagnostics/javascript-seo-rendering-indexing-metadata/

Field vs lab data: why most Core Web Vitals arguments are dataset confusion

Jeremy Burgos — Tue, 23 Jun 2026 10:30:00 +0000

TL;DR: If you cannot name which dataset you are looking at, you are arguing, not diagnosing. Field answers "are real users failing at scale." Lab answers "what mechanism caused it." Sequence: field to scope, lab to diagnose, field to validate.

The sequence that resolves most stuck performance debates:

field  -> confirm the problem exists and find its scope
lab    -> isolate a cause you can test
field  -> validate the fix against the evidence you started with

Core Web Vitals work falls apart when two people argue from two datasets. One looks at real-user field results, the other at a lab simulation. Both valid, both useful, answering different questions. Name which one, or you are not diagnosing.

What they measure, and the 75th percentile

LCP (good: 2.5s or less) is loading, INP (good: 200ms or less) is responsiveness, CLS (good: 0.1 or less) is visual stability. They are outcome metrics: they tell you what users experienced, not why. The detail that ends most "but it is fast for me" disagreements: pass/fail is the 75th percentile, not an average. A site can feel fast for many users and still fail, because the 75th percentile forces you to care about weaker devices and slower networks.

Field and lab are not interchangeable

Field data is aggregated real-user experience across devices and networks, including variance you cannot simulate: slow hardware, cellular latency, packet loss, cache states. It answers whether users are actually failing at scale. Lab data is controlled and repeatable, and gives you the traces field cannot: render-blocking resources, long tasks, layout shift events, waterfalls. It answers what mechanism is likely causing the outcome. They diverge because field is a distribution across sessions and lab is a snapshot under one config. Do not average them. Sequence them. If you claim what users experienced, trust field. If you claim why, trust lab traces.

An evidence standard that stays falsifiable

Map every claim to an artifact that would look different if the claim were false:

Claim: real users are failing        -> Search Console CWV trend at URL-group level
Claim: it is template-based          -> repeated failures across same-type URLs
Claim: LCP delayed by server/cache   -> lab traces: late doc response, delayed render start
Claim: INP failing, main-thread      -> DevTools recording: long tasks overlapping input
Claim: a release caused a regression -> timing correlation with deploy + change log

Do not trust a single Lighthouse run as proof of field improvement, desktop-only testing to explain mobile failures, or "it feels faster" as a pass condition.

Triage

Name the failing dataset in one sentence. If you cannot, gather minimum evidence first. Search Console gives field status by URL group; PageSpeed Insights gives lab diagnostics and some field context.
Confirm scope. Treat it as template-based until proven otherwise. Template scope means high-impact fixes; narrow scope points at conditional triggers.
Classify the dominant constraint: server/delivery, render path, main thread, layout stability, third-party, or change-driven.
Pick the smallest test that could prove you wrong. Field failing but lab fine means expand representativeness. LCP failing means confirm the element first. Regressed after deploy means investigate the change before optimizing.

What actually moves the field number

A faster render path for primary content, less main-thread contention, reserved space to stop layout shift, more consistent delivery so cache and routing produce less variance, and removing or conditioning third-party work. Lab scores can improve without any of that moving, if you optimized the test setup instead of the experience. Use lab to prove a bottleneck is gone, then validate with the same field evidence you began with.

Originally published at https://www.techseoexperts.com/performance-page-experience-diagnostics/core-web-vitals-diagnostics-field-vs-lab/

Architecting a Wazuh SIEM with active response: the four layers that matter

Jeremy Burgos — Tue, 16 Jun 2026 10:30:00 +0000

TL;DR: A Wazuh SIEM and XDR deployment built across four layers (manager and agents, active response, network monitoring, traffic filtering) to move an environment from reactive log review to automated detection and response. Sanitized rules and decoders are public, linked at the end.

The architecture, in the order the layers depend on each other:

Layer 1  Manager + agents      -> broad endpoint coverage, secure agent comms, central dashboard
Layer 2  Active response       -> auto-block/mitigate brute force and unauthorized access
Layer 3  Network monitoring    -> real-time log/network watch, correlation across signals
Layer 4  Traffic filtering     -> block known-bad IPs, domains, signatures; updated on intel

Before this deployment, the environment looked like most: fragmented visibility, log review that depended on someone actually doing it, and no automated response for the threats that show up constantly, brute force, unauthorized access, malicious traffic. That means you find out late and every response starts cold. The goal was not to install a platform. It was to build a monitoring and response layer that acts on its own.

Layer 1: manager, agents, and the dashboard

The foundation is the Wazuh manager with agents deployed across endpoints for broad coverage, a dashboard centralizing alerting and rule management, and secure communication between manager and agents so the monitoring data itself cannot be tampered with in transit. Get this layer wrong and everything above it is built on unreliable signal.

Layer 2: active response, and where it earns its keep

Active response rules target higher-risk behavior, brute force, unauthorized access, and are designed to block or mitigate automatically. The part that separates a useful deployment from a noisy one is the refinement: the rules get tested and tuned to stay reliable while keeping false positives down. Active response that fires on legitimate activity gets ignored within a week, which is worse than no automation at all, because now nobody trusts the alerts either.

Layer 3: network and log monitoring

Real-time monitoring of system logs and network activity watches for the patterns that matter: unusual logins, unexpected outbound connections, privilege escalations. Existing monitoring signals are integrated so the system can correlate across sources instead of treating each in isolation. Correlation is where you catch the things a single log never shows you.

Layer 4: traffic filtering

Custom filtering rules detect and block unwanted patterns, known malicious IPs, suspicious domains, harmful signatures, and they get updated as threat intelligence evolves so the deployment stays current instead of going stale. Static filtering rules are a snapshot of last quarter's threats.

What changed

The environment moved from reactive to structured. Detection improved through real-time visibility into logs, traffic, and endpoint activity. Mitigation got faster because active response shrinks the window between a threat appearing and something being done about it. Network control tightened through filtering. And it integrated without disrupting operations, which is the part that makes it sustainable rather than a one-time project that decays.

The full deployment, with a video walkthrough of the actual setup and configuration, is at https://www.jeremyburgos.com/projects/wazuh-siem-and-xdr-deployment/. Sanitized detection rules and decoders are public at https://github.com/Jeremy-Burgos/wazuh-detection-engineering.

Originally published at https://www.jeremyburgos.com/projects/wazuh-siem-and-xdr-deployment/

Architecting isolated workspaces with Kasm: hardened, egress-controlled, disposable

Jeremy Burgos — Tue, 09 Jun 2026 10:30:00 +0000

TL;DR: A hardened Kasm Workspace deployment where every research session runs in an isolated, egress-controlled, throwaway container. Four layers (server hardening, VPN egress, instances and tools, monitoring), built so sensitive workloads never weaken the host.

The architecture:

Layer 1  Server hardening   -> locked-down base, secure headers, minimal mgmt surface
Layer 2  Egress VPN         -> all instance traffic through encrypted tunnels, no raw outbound
Layer 3  Instances + tools  -> AlmaLinux / Parrot / Ubuntu, Brave, SpiderFoot, Forensic OSINT
Layer 4  Monitoring         -> logging, access controls, periodic security review

Most people do their riskiest browsing in their most valuable environment, the same browser logged into client accounts, email, and banking. This deployment exists to separate the two: a controlled environment that can handle research, OSINT, and testing without weakening the machine or network underneath it. The design had to clear four bars at once, isolation, controlled egress, multi-OS support, and monitoring, which a standard server setup does not.

Layer 1: server hardening

The server is hardened from the ground up, with secure header controls and a locked-down management interface, so the environment can be administered without exposing extra attack surface. This is the base everything else sits on.

Layer 2: egress VPN

Instance traffic is routed through encrypted VPN tunnels rather than direct, unmanaged outbound paths, with authentication and encryption tightened so access and data flows stay protected. The containers are aligned to use secure outbound routing consistently, so nothing leaks out a side door.

Layer 3: instances and tools

Multiple operating systems, AlmaLinux, Parrot OS, Ubuntu, plus cloud browsers, Brave, Firefox, Chromium, with research tools including SpiderFoot and Forensic OSINT integrated for investigative work. Each session is its own container, so the work is sandboxed and disposable.

Layer 4: monitoring and review

Stricter access controls reduce exposure, and logging plus periodic security review keep the environment stable and trustworthy over time, rather than relying on the initial configuration holding forever.

What it changes about the work

The obvious win is privacy: research traffic moves through encrypted tunnels and a closed session leaves nothing behind. The less obvious win is cleaner observation. The moment your research environment is disposable and isolated, you stop confusing your own logged-in, ad-profiled browser with a neutral instrument, and you see closer to what a fresh visitor or a crawler sees. The discipline underneath it, separate sensitive work from disposable work, treat environments as ephemeral, assume new things are untrusted, carries well beyond research.

Honest tradeoff: this is not a five-minute setup. It runs on real infrastructure (a hardened DigitalOcean server in this case) and took deliberate work across hardening, VPN, instances, and monitoring. If you do not run your own infrastructure, a clean browser profile with no extensions and no signed-in accounts, on a separate device or VM, gets you a meaningful share of the benefit for far less effort.

The full build, with a video walkthrough of the setup and configuration, is at https://www.jeremyburgos.com/projects/secure-kasm-workspace-deployment/.

Originally published at https://www.jeremyburgos.com/projects/secure-kasm-workspace-deployment/

Four HTTP security headers every WordPress site should set

Jeremy Burgos — Tue, 02 Jun 2026 10:30:00 +0000

TL;DR: Four response headers, a few minutes of work, most of the header-level security gap closed. Exact values below, plus a one-line curl to check any site.

Run this against your own site first:

curl -I -s https://yoursite.com | grep -i -E 'strict-transport|x-content|x-frame|referrer'

Whatever does not come back is your to-do list. These four headers are public on every request and contain nothing sensitive, so you can check mine, I can check yours, and neither of us has to log into anything. Here is what each one is and the value I actually run in production.

Strict-Transport-Security

strict-transport-security: max-age=31536000; includeSubDomains; preload

Tells the browser to use HTTPS for your domain, full stop, for the max-age window. Once a browser has seen it, typing http:// does nothing; the browser refuses to send the insecure request. max-age=31536000 is one year. includeSubDomains pushes the rule to every subdomain, closing the gap where an attacker targets some forgotten staging host. preload is the part people skip, and it matters: without it, the very first request before the browser has ever seen your header can still go out over HTTP, and that first request is the attack window. Preloaded domains skip it because the browser ships already knowing your domain is HTTPS-only. Submit once at hstspreload.org, it is free, and inclusion rides the Chromium release train so it takes a few weeks.

X-Content-Type-Options

x-content-type-options: nosniff

Browsers used to guess at content types when the server was vague, which is exploitable. nosniff tells the browser to trust the declared Content-Type and stop guessing. There is no other value and nothing to tune. If your production site is missing this, you can fix it before you finish this article.

X-Frame-Options

x-frame-options: SAMEORIGIN

Your clickjacking defense. It stops someone loading your real, logged-in site in an invisible iframe and floating their own buttons over it. SAMEORIGIN allows only your own pages to frame your site. DENY blocks all framing including yours, which is wrong if your WordPress setup uses internal iframes (Elementor previews, some widgets). The modern successor is CSP's frame-ancestors; run both during the transition, X-Frame-Options for older clients, CSP for the rest.

Referrer-Policy

referrer-policy: strict-origin-when-cross-origin

Controls what your site leaks about users when they click away. strict-origin-when-cross-origin sends the full URL on same-origin requests so your analytics still work, only the origin on cross-origin HTTPS so you are not leaking that someone was on /account/billing, and nothing on cross-origin HTTP. Set it explicitly so you are not at the mercy of whatever default the next browser release ships.

Verifying

The curl above is the fastest check; all four lines should come back. In a browser, DevTools, Network tab, click the document request, read Response Headers. For a letter grade, securityheaders.com scores you against a known rubric. One quirk: these four alone land a B, and you reach A only once you add Content-Security-Policy.

These four are the floor. The next layer is Permissions-Policy and CSP in report-only mode. But if you only ever do these four, you have closed most of the gap, in minutes.

Originally published at https://www.webstackdefense.com/security-headers-production-websites/