DEV Community: Sandeep Singh

Automating Website Monitoring: What to Check Daily vs Weekly vs Monthly

Sandeep Singh — Sun, 18 Oct 2026 15:59:46 +0000

Most websites aren't monitored at all. Or they're "monitored" by whoever happens to notice something odd, which usually means a customer emailing to say the checkout doesn't work, a week after it stopped working.

The opposite problem is just as real. Teams that set up alerts for everything soon learn to ignore them, and an ignored alert is no better than no alert.

The fix is a schedule that matches the frequency of each check to two questions:

How fast can this break?
How much does it cost while it's broken?

A site going completely down can happen any minute and costs everything while it lasts, so you check it constantly. A domain renewal can only fail once a year and gives you weeks of warning, so you check it monthly. Everything else sits somewhere in between.

Here's a practical schedule for a small business, blog, store or SaaS marketing site, with what to automate at each level and a few small scripts to get started.

Continuous (every few minutes): is it up?

What: Does the homepage, and your most important page (checkout, booking, sign-up, login), return a 200 quickly?

Why this often: An outage is total. Every minute down means every visitor is lost, and it can happen at any time from a host failure, a bad deploy, an expired certificate or a DNS mistake.

How to automate: An uptime monitoring service that requests a URL every few minutes from outside your network and alerts you (email, SMS, chat) when it fails. Many offer free tiers for a handful of URLs. Check that the monitor:

Checks from outside your hosting provider (a monitor on the same server goes down with it)
Looks for expected text on the page, not just a status code, because an error page can still return 200
Requires a couple of consecutive failures before alerting, to avoid false alarms

Keep this list short: two to five URLs. This level is for "the site is down", not for every page.

Daily: is anything broken?

These are problems that don't take the site down but quietly cost visitors and sales. They can appear at any time (after an update, a content edit, a third-party change), and every day they go unnoticed costs you.

Broken links and missing images

Pages get edited, products get removed, external sites change, and links break. A daily crawl catches a broken "Buy now" button the day after someone deletes a product, not at the end of the quarter.

What to check:

Internal links returning 4xx/5xx
Missing images (404s on <img> sources)
New redirect chains or loops
Links to key conversion pages (cart, checkout, booking, pricing, contact) still resolving

I walked through doing this by hand for one platform in How to Find and Fix Broken Links on a Shopify Store. The daily version is the same checks, automated.

Certificate validity

Certificates are meant to renew automatically, but renewals do fail. Let's Encrypt certificates are valid for 90 days by default, and renewal is recommended every 60 days (Let's Encrypt FAQ). A daily check that warns when fewer than about 20 days remain gives you plenty of time to fix a stuck renewal:

#!/usr/bin/env bash
# cert-check.sh: warn if a certificate expires within N days
host="${1:?usage: cert-check.sh example.com [days]}"; days="${2:-20}"
cert=$(echo | openssl s_client -servername "$host" -connect "$host:443" 2>/dev/null)
end=$(openssl x509 -noout -enddate <<<"$cert" | cut -d= -f2)
if ! openssl x509 -noout -checkend $((days*86400)) <<<"$cert" >/dev/null; then
  echo "WARNING: $host certificate expires $end"; exit 1
fi
echo "OK: $host certificate valid until $end"

Backups completed

Not "are backups configured", but "did last night's backup finish?" Most backup tools can email on failure. Make sure that's turned on and goes to someone who reads it.

Key-page smoke test

A tiny script that requests your critical URLs and checks for the expected content:

#!/usr/bin/env bash
# smoke.sh: each line of urls.txt is "<url> <text that must appear>"
fail=0
while read -r url expect; do
  body=$(curl -sL --max-time 20 -w '\n%{http_code}' "$url")
  code=$(tail -n1 <<<"$body")
  if [[ "$code" != "200" ]] || ! grep -q "$expect" <<<"$body"; then
    echo "FAIL $code $url (missing: $expect)"; fail=1
  fi
done < urls.txt
exit $fail

One caution: don't automate form submissions or test purchases on a live site unless your platform provides a proper test mode. You'll fill your CRM with junk, trigger real emails or create real orders. Test forms by hand (weekly, below).

Running it on a schedule

Any scheduler works: cron on a server you control, your host's scheduled tasks, or a CI service. With GitHub Actions, for example:

name: daily-site-checks
on:
  schedule:
    - cron: "0 10 * * *"   # 10:00 UTC daily
  workflow_dispatch:
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: bash smoke.sh && bash cert-check.sh example.com

GitHub's documentation notes that scheduled workflows run on the default branch, can be delayed during periods of high load (particularly at the start of each hour), and in public repositories are automatically disabled after 60 days without repository activity (Events that trigger workflows). Pick an odd minute, and don't rely on this alone for time-critical alerts.

Weekly: the human checks

Some things need a person, or are slow-moving enough that weekly is fine.

Test every contact path by hand: submit the contact form, try the booking flow, tap the phone link on mobile. Contact forms often fail silently after plugin updates or email configuration changes.
Apply updates (CMS core, plugins, themes, dependencies), then re-run the daily checks and re-test forms.
Glance at Google Search Console. It emails about new critical issues, but a weekly look at Indexing → Pages for rising 404 or server-error counts catches slower trends.
Read the week's daily reports and fix anything still open.

Monthly: performance, content and access

Core Web Vitals. Search Console's Core Web Vitals report "shows how your pages perform, based on real world usage data" (Core Web Vitals report). Because it reflects real visitors over time, it moves slowly, so monthly is the right cadence. Google's targets are LCP within 2.5 s, INP under 200 ms and CLS under 0.1 (Core Web Vitals). Use PageSpeed Insights to diagnose any template that's slipping.
Content accuracy: prices, hours, staff, service areas, promotions that should have ended.
Outbound links on your most-visited pages, opened by hand, to check they still say what you linked them for. A status-code checker can't tell when a page changed content.
Access review: remove accounts for people who no longer need them (CMS, hosting, registrar, analytics).
Business listings: check that the website URL on your Google Business Profile and other directories still points at a working page.

Yearly: renewals and recovery

Domain renewal. Registrars of generic top-level domains must send at least two expiry reminders, about one month and about one week before expiry, under ICANN's Expired Registration Recovery Policy. That only helps if the contact email is current. Turn on auto-renew and confirm the payment card.
Restore test: restore a backup somewhere safe and confirm the site works.
Documentation: who owns the domain, hosting, DNS, email and each account, and when each renews.
Accessibility and SEO review of key templates.

For a non-technical version of this list, see The Website Maintenance Checklist for Small Business Websites.

Alerts versus reports

One last design decision makes the difference between monitoring people use and monitoring they mute:

Alerts (immediate, interruptive) are only for things that need action now: site down, checkout down, certificate expiring within days.
Reports (daily or weekly digests) are for everything else: broken links, missing images, redirect chains, slow templates. Nobody needs a 2 a.m. page about a broken link in a blog post from 2021, but they do need to see it in tomorrow's report.

If an alert fires and the right response is "I'll look at it later", it should have been a report.

Disclosure: I build Uchik, which handles the daily report layer. Uchik crawls your site once a day with read-only requests (it never submits forms or places orders) and emails a report of broken links, missing images, redirect chains, mixed content and basic SEO gaps, each tied to the exact page and HTML element. How it works explains the crawl, the features page lists the checks, and the sample report shows what arrives in your inbox. It isn't an uptime monitor and it doesn't measure Core Web Vitals, so pair it with the continuous and monthly checks above. Links added only by JavaScript may be missed. You can sign up for free.

The schedule on one page

Cadence	Check	Automate?
Every few minutes	Homepage + critical page up, expected text present	Yes: uptime monitor
Daily	Broken links, missing images, redirects	Yes: crawler
Daily	Certificate expiry > ~20 days	Yes: script or monitor
Daily	Backup succeeded	Yes: backup tool alerts
Daily	Critical URLs return expected content	Yes: smoke script
Weekly	Forms, booking, phone links	No: test by hand
Weekly	Updates applied, then re-tested	Partly
Weekly	Search Console indexing trends	Email alerts + quick look
Monthly	Core Web Vitals (field data)	Report; review by hand
Monthly	Content accuracy, outbound links, access, listings	No
Yearly	Domain renewal, restore test, documentation	Auto-renew + calendar

Start with the first two rows this week. Together they cover most of what quietly costs small sites visitors and sales.

Broken Links and Core Web Vitals: What Actually Affects Rankings

Sandeep Singh — Sun, 18 Oct 2026 15:59:30 +0000

Ask in any web forum whether broken links hurt SEO and you'll get confident answers in every direction. "Every 404 is a ranking penalty." "Google doesn't care about 404s at all." "Fix your Core Web Vitals or you'll never rank." "Core Web Vitals don't matter."

Most of these are partly true and mostly unhelpful. This article sticks to what Google's own documentation says, quotes it directly, and separates three different questions that usually get blurred together:

Do broken links affect rankings?
Do Core Web Vitals affect rankings?
What do both cost you regardless of rankings?

That third question is the one people skip, and it's often where the real losses are.

Part 1: What Google says about 404s and broken links

A 404 on its own is not a penalty

Google's Search Central blog answers this directly: "404 errors are a perfectly normal part of the web", and "If some URLs on your site 404, this fact alone does not hurt you or count against you in Google's search results." It goes further: the fact that some URLs return 404 "does not affect how your site's other URLs (the ones that return 200 (Success) status codes) perform in our search results" (Do 404 errors hurt my site?).

Google actually prefers a proper 404 or 410 for content you've removed, rather than a fake success page.

But a 404 on a page that matters is a real loss

The same post adds the important caveat: "if some of the pages that 404 are pages you actually care about, you should look into why." And Google's HTTP status documentation explains what happens to an indexed page that starts returning 4xx: "the indexing pipeline removes the URL from the index if it was previously indexed" (HTTP status codes).

So:

A random old URL returning 404 → normal, no penalty.
Your ranking product page, service page or popular post returning 404 → it drops out of search results. That isn't a "penalty", but the effect on your traffic is the same.

Internal links are how Google finds your pages

Broken internal links matter for a different reason: discovery. Google's link guidance says links are used to find new pages to crawl, and that "every page you care about should have a link from at least one other page on your site" (Link best practices). If the only link to a page is broken, or points through a JavaScript handler instead of a real <a href>, that page is harder for Google to discover.

Soft 404s and homepage redirects are worse than honest 404s

Google warns about two patterns: serving a "not found" page with a 200 status, and redirecting "any unknown URLs to their home page instead of returning 404 errors". Both "can have negative effects on our understanding and indexing" of your site (Do 404 errors hurt my site?).

Broken outbound links

I couldn't find any Google documentation saying broken links to other sites lower your rankings, so I won't claim they do. What they clearly do is make your content less useful and less trustworthy to readers, which is a good enough reason to fix them.

Part 2: What Google says about Core Web Vitals

The metrics and thresholds

Google's documentation defines three Core Web Vitals and the targets for a good experience (Core Web Vitals):

Metric	Measures	Good
Largest Contentful Paint (LCP)	Loading	within 2.5 seconds
Interaction to Next Paint (INP)	Responsiveness	less than 200 milliseconds
Cumulative Layout Shift (CLS)	Visual stability	less than 0.1

web.dev adds how these are assessed: a good threshold to measure is "the 75th percentile of page loads, segmented across mobile and desktop devices" (Web Vitals). In other words, most of your real visitors need a good experience, not just the developer on fast office Wi-Fi.

Yes, they're used in ranking, but they aren't decisive

Google's page experience documentation says: "Core Web Vitals are used by our ranking systems." In the same section it adds two important qualifications:

"Getting good results in reports like Search Console's Core Web Vitals report or third-party tools doesn't guarantee that your pages will rank at the top of Google Search results."
"Google Search always seeks to show the most relevant content, even if the page experience is sub-par. But for many queries, there is lots of helpful content available. Having a great page experience can contribute to success in Search, in such cases."

(Understanding page experience)

So the accurate summary is that Core Web Vitals are a real signal that can make a difference when competing pages are similarly relevant. They won't make an unhelpful page rank, and chasing a perfect score isn't worthwhile. Google says there is "no single signal" for page experience.

Lab scores versus field data

Lighthouse and PageSpeed Insights' lab results are diagnostic tools: one simulated load, on one simulated device. Search Console's Core Web Vitals report is different: it "shows how your pages perform, based on real world usage data (sometimes called field data)" (Core Web Vitals report). If you want to know how real visitors experience your site, the field data is what counts. Use Lighthouse to find out why.

The rest of page experience

Google's page experience page lists more than Core Web Vitals. It asks whether pages are "served in a secure fashion", display well on mobile, avoid excessive distracting ads and intrusive interstitials, and make the main content easy to distinguish. That "secure fashion" point is where HTTPS problems like mixed content come in. I covered fixing those in Mixed Content and Missing Images: A Practical Fix Guide for Store Owners.

Part 3: Where broken links and Core Web Vitals overlap

They're usually discussed separately, but several "link health" problems are also performance problems:

Redirect chains slow every page load

Every redirect is another network round trip before the page can even start loading. Lighthouse says this "can delay the loading of the resource by hundreds of milliseconds" and fails a page with two or more redirects (Avoid multiple page redirects). web.dev's LCP guide makes the connection explicit: Time to First Byte includes redirect time, a high TTFB "can make achieving a 2.5 second LCP challenging, or even impossible", and "a common cause of a slow TTFB for an otherwise fast site is visitors arriving through multiple redirects" (Optimize LCP). Visitors arriving from old internal links, ads or emails pay that cost. Updating internal links to point directly at final URLs is both a link fix and a speed fix.

Images without dimensions cause layout shift

web.dev lists "images without dimensions" first among the most common causes of poor CLS (Optimize CLS). Broken image references and slow-loading images often come together on older templates, so when you're fixing missing images, add width and height attributes (or CSS aspect-ratio) at the same time.

Broken resources waste requests

A page that requests a missing script, stylesheet, font or image still makes the request and waits for the 404. Removing references to resources that no longer exist is free performance.

Part 4: The costs that have nothing to do with rankings

Even where Google says something "does not hurt you" in search, it can still cost you:

A visitor who hits a 404 on a product, booking or sign-up page is a lost conversion today, whatever your rankings are.
A slow or jumpy page frustrates visitors who already found you. Core Web Vitals exist because these metrics reflect real user experience.
Broken links in content make readers trust your recommendations less.
Links from other sites to your dead pages stop sending referral visitors.

This is the strongest argument for fixing link problems: not that Google punishes them, but that people do.

Part 5: A sensible priority order

Based on all of the above, here's how I'd prioritise:

Important pages returning errors. Ranking pages, money pages and pages with backlinks that return 404/5xx. Restore them or 301 them to the correct replacement. (Direct loss of indexing and conversions.)
Broken internal links to live content. Fix at the source. (Discovery and user experience.)
Homepage-redirect catch-alls and soft 404s. Return honest 404s or targeted redirects. (Google explicitly warns about these.)
Redirect chains in internal links and navigation. Point links at final URLs. (Speed and crawl efficiency.)
Core Web Vitals issues in field data, starting with templates that affect many pages. (A ranking signal and a user-experience issue.)
Mixed content and HTTPS problems. (Part of page experience, plus browser warnings.)
Broken outbound links. (Trust and usefulness.)
Old, unlinked URLs returning 404. Usually nothing to do. That's normal.

Part 6: Monitoring both

The tools are complementary:

Search Console: Page indexing (404s Google sees), Core Web Vitals (field data), HTTPS report.
PageSpeed Insights / Lighthouse: diagnose why a page is slow.
A crawler (Screaming Frog's free tier handles up to 500 URLs; limits): every broken link, redirect and missing image, with the page it's on.

For personal and small sites, SEO Health Basics for Personal Websites and Portfolios turns this into a short routine.

Disclosure: I build Uchik, which covers the crawling side. Uchik checks your site daily and emails a report of broken links, missing images, redirect chains, mixed content and basic on-page SEO gaps, each tied to the exact page and HTML element. How it works explains the method, and the features page lists what it checks. It doesn't measure Core Web Vitals; use Search Console and PageSpeed Insights for that. The FAQ covers other limits, including that links added only by JavaScript may be missed. You can sign up for free.

Summary

A 404 on its own isn't a ranking penalty, but a 404 on a page that ranked removes it from the index.
Broken internal links hurt discovery, and homepage catch-all redirects are explicitly discouraged.
Core Web Vitals are used by ranking systems, but relevance comes first and good scores don't guarantee top rankings.
Redirect chains and images without dimensions are both link-health and performance problems.
The biggest cost of broken links is usually visitors, not rankings.

How to Audit Internal Links on a WordPress Site

Sandeep Singh — Sun, 18 Oct 2026 15:53:30 +0000

Internal links are the roads inside your website. They're how visitors move from a blog post to a product or service page, how they find related content, and how search engines discover and understand your pages. Google's own guidance says it plainly: "Every page you care about should have a link from at least one other page on your site" (Link best practices).

WordPress sites are especially prone to internal link problems because they grow over years. Posts get updated, slugs change, categories get reorganised, plugins come and go, themes get swapped, and the site gets moved between hosts or from staging to production. Each of those leaves some links pointing somewhere they shouldn't.

The result is lost traffic you never see: a reader following "read my full guide" hits a 404 and leaves, a high-converting landing page has no links pointing to it so nobody finds it, and search engines spend their time on redirects instead of your content.

This guide walks through a full internal link audit using free tools, then shows how to fix problems at the source rather than piling up redirects.

What you're looking for

An internal link audit checks for six things:

Broken internal links: links to pages on your own site that return 404 or another error.
Redirected internal links: links that work but go through one or more redirects first.
Orphan pages: published pages that no other page links to.
Deeply buried pages: important pages that take many clicks to reach from the homepage.
Weak anchor text: "click here" and "read more" instead of descriptive text.
Leftover URLs: links to your staging domain, http:// versions or old domain.

Step 1: Crawl the site

You need a full list of every internal link: the page it's on, where it points and what status that target returns. Some free options:

Screaming Frog SEO Spider (free up to 500 URLs; limits). Enter your homepage and crawl. Use Response Codes to filter internal URLs by status, and the Inlinks tab at the bottom to see every page that links to a selected URL, with anchor text.
The Broken Link Checker plugin (WordPress.org) checks links from inside WordPress and lets you edit or unlink from its dashboard. Plugin-based checkers use your server's resources, so check its settings on shared hosting.
Google Search Console's Links report shows your most internally linked pages from Google's point of view. It won't show broken links directly, but it's a useful sanity check.

A crawler that starts from your homepage only finds pages reachable by links. That's exactly what you want for steps 2–5, and it's why orphan pages need a separate check in Step 4.

Step 2: Fix broken internal links

Filter the crawl for internal URLs returning 4xx and 5xx. For each one, open Inlinks to see which posts or pages contain the bad link.

Common causes on WordPress:

A post slug was changed. WordPress core tries to rescue some of these: its wp_old_slug_redirect() function attempts to find a post's current slug from past slugs and redirect (WordPress developer reference). That doesn't cover every case, such as pages, custom post types on some setups, or posts that were deleted and recreated.
The permalink structure was changed in Settings → Permalinks (WordPress docs). Links hard-coded in old content keep the old format.
A page was deleted or unpublished.
A plugin that created pages was removed: shop, events, forums, landing page builders.
Category or tag base changed, breaking every category link in old posts.
Typos in manually entered links.

Fix the link in the content where it appears: edit the post, menu or widget and point it at the correct current URL. If the destination genuinely no longer exists, either link to the best replacement or remove the link.

Step 3: Update redirected internal links

Next, filter for internal links returning 3xx. These work, but every redirect is an extra request before the page loads. Lighthouse notes each redirect can delay a page "by hundreds of milliseconds" (Avoid multiple page redirects), and chains make it worse.

Typical WordPress sources:

Links to http:// versions of your own pages, redirected to https://
Links to www when the site lives on the bare domain (or the reverse)
Links missing the trailing slash, redirected to add it
Old slugs being rescued by a redirect plugin

Redirects are the right tool for external links and old bookmarks. For your own internal links, update the link to the final URL so visitors and crawlers get there directly. If you're seeing chains or loops, I covered fixing those in Redirect Chains and Loops: What They Cost an Ecommerce Site and How to Fix Them, and Mixed Content and Missing Images: A Practical Fix Guide for Store Owners covers the related http:// asset problem.

Bulk-fixing domain and protocol changes

If thousands of links point at an old domain or http://, editing posts by hand isn't realistic. WP-CLI's search-replace handles it, including serialized data in options and widgets:

# Always preview first, and take a database backup
wp search-replace 'http://example.com' 'https://example.com' --dry-run
wp search-replace 'https://staging.example.com' 'https://example.com' --dry-run

Remove --dry-run only after checking the counts look right. Many hosts and migration plugins offer a similar search-and-replace tool if you don't have shell access.

Step 4: Find orphan pages

An orphan page is published but not linked from anywhere on the site. A link-following crawler can't find it, and visitors are unlikely to either.

To find them, compare what you've published with what the crawler found:

Get the list of published URLs from your XML sitemap. Since WordPress 5.5, core generates a sitemap index at /wp-sitemap.xml (Make WordPress Core); SEO plugins often replace it with their own, usually at /sitemap_index.xml.
Export the list of HTML URLs your crawler found by following links.
Anything in the sitemap but not in the crawl is an orphan.

A spreadsheet works for this: paste both lists into separate columns and use a lookup to flag sitemap URLs missing from the crawl.

For each orphan, decide:

Still valuable? Link to it from relevant posts, a category page or a hub page.
Outdated or thin? Update it, merge it into a stronger page (and redirect), or unpublish it.
Intentionally hidden (a thank-you page or a campaign landing page)? Consider whether it should be in the sitemap at all, or whether it should carry noindex.

Step 5: Check click depth for important pages

Most crawlers report crawl depth: how many clicks from the homepage each page is. Your most important pages (key services, best guides, top products) shouldn't be buried at depth 5 behind pagination.

Ways to bring them closer:

Link to them from the main menu or a footer section
Create a hub or "start here" page linking to your best content
Add contextual links from related, popular posts
Use breadcrumbs (many themes and SEO plugins provide them)

Step 6: Improve anchor text

Google says good anchor text "is descriptive, reasonably concise, and relevant to the page that it's on and to the page it links to", and that it helps "both people and Google make sense of your site" (Link best practices).

In the crawl's Inlinks or All anchor text export, look for:

Generic text: "click here", "read more", "this post", "here"
Empty anchors, such as image links with no alt text
The same exact keyword phrase used for every link to a page, which reads awkwardly

Rewrite them as natural descriptions: "see our guide to [choosing a running shoe]" rather than "[click here] for our guide".

Step 7: Check your theme and page builder output

Two WordPress-specific issues to verify:

Links must be real <a href> elements. Google says it can generally only crawl links that are <a> elements with an href attribute (Link best practices). Some page builders and theme widgets render buttons or "clickable boxes" with JavaScript onclick handlers instead. View the source of a few pages and check that menus, buttons and cards are real links.
Related-posts and widget links. These are generated automatically and can point at drafts, trashed posts or attachment pages. Check them in the crawl, too.

Step 8: Make it a habit

A one-off audit fixes today's problems. WordPress sites keep changing, so links keep breaking. A sustainable routine:

After every significant change (theme switch, permalink change, plugin removal, migration): crawl and fix.
Monthly: a quick crawl filtered for 4xx and 3xx internal links.
Quarterly: the orphan-page and anchor-text review.

The Website Maintenance Checklist for Small Business Websites puts internal link checks in context alongside backups, updates and renewals.

Disclosure: I build Uchik, which automates the recurring part. Uchik crawls your site every day and emails a report of broken internal and external links, missing images, redirect chains and HTTPS problems, each with the source page and the exact HTML element to fix. It runs as an external crawler, so there's no plugin to install or keep updated. The features page lists every check, the sample report shows the output, and the FAQ explains its limits, including that links added only by JavaScript may be missed. It doesn't do the orphan-page comparison in Step 4; keep that in your quarterly review. You can sign up for free.

Checklist

[ ] Full crawl from the homepage
[ ] Broken internal links fixed at the source
[ ] Redirected internal links updated to final URLs
[ ] Old domain, staging and http:// links bulk-replaced (with backup and dry run)
[ ] Sitemap vs crawl compared; orphans linked, merged or removed
[ ] Important pages within a few clicks of the homepage
[ ] Generic anchor text rewritten
[ ] Menus, buttons and cards confirmed as real <a href> links
[ ] Recurring check scheduled or automated

Internal links are the one part of your site's linking you fully control. A few hours of auditing keeps readers and crawlers moving to the pages that matter.

404 Errors After a Site Migration or Redesign: A Recovery Checklist

Sandeep Singh — Sun, 18 Oct 2026 15:52:35 +0000

The new site launches. It looks better, it's faster, the team is happy. Then, a week or two later, someone notices: organic traffic is down, a few customers have emailed about "page not found" errors, and the Search Console Page indexing report shows a growing pile of 404s.

This is the most common way a redesign or platform migration goes wrong. The new pages are fine. The problem is that the old URLs, the ones Google, other websites, bookmarks, old newsletters and social posts all point to, now go nowhere. Every one of those is a visitor who arrived with intent and left with an error page.

The good news is that this is usually recoverable. Below is the checklist I'd work through, in order, whether you moved from WordPress to a headless setup, changed your Shopify theme and URL handles, or just reorganised your blog's categories.

First: what Google actually says about 404s

It helps to know what you're fixing, and what you're not.

Google's own Search Central blog is clear that 404s on their own aren't a penalty: "If some URLs on your site 404, this fact alone does not hurt you or count against you in Google's search results", and those 404s don't affect how your other working URLs perform (Do 404 errors hurt my site?). Google's HTTP status documentation adds that when a previously indexed URL returns a 4xx, "the indexing pipeline removes the URL from the index" (HTTP status codes).

Put those together and the real cost of migration 404s is clear:

Pages that used to rank drop out of the index, and the new equivalents have to earn their place from scratch instead of inheriting the old URL's signals.
Links from other sites now point to dead pages, so the value of those links, and the referral traffic they bring, is lost.
People who click hit a dead end. That's the immediate, measurable loss: leads, sales, sign-ups, readers.

So the goal isn't "zero 404s". It's every old URL that had value now lands on its correct new home in one step, and anything genuinely gone returns an honest 404 or 410.

Step 1: Rebuild the list of old URLs

You can't redirect what you can't list. If you didn't save a crawl before launch (most people don't), reconstruct it from several sources, because each one sees different URLs:

Search Console → Indexing → Pages → Not found (404). Export the examples. These are URLs Google knows about that now fail.
Search Console → Performance. Set the date range to before the migration and export the top pages. These are the URLs that were actually earning clicks, so they're your priority list.
Search Console → Links → Top linked pages. URLs other sites link to.
Your analytics tool. Landing pages from the months before launch.
The old sitemap, if you still have it, or the Wayback Machine's capture of it.
Server or CDN logs showing 404 responses since launch, if you have access.
Your own old content: newsletters, social posts, PDFs, documentation and email templates that hard-coded links.

Combine these into one spreadsheet, remove duplicates, and keep a column for where each URL came from.

Step 2: Check each old URL's current status

Run the whole list through a crawler in list mode. Screaming Frog SEO Spider does this in Mode → List (free up to 500 URLs; limits). For a quick script:

while read -r url; do
  printf '%s\t%s\n' "$url" "$(curl -s -o /dev/null -w '%{http_code} %{num_redirects} %{url_effective}' -L "$url")"
done < old-urls.txt > status.tsv

That gives you the final status, the number of redirect hops and the final URL for each one. Sort into buckets:

404/410: needs a redirect, or confirmation it's really gone
Redirects to the homepage: probably wrong (see below)
2+ hops: works, but slow and should be flattened
200 at the right place: done

Step 3: Map old URLs to new URLs

Google's site move guide calls this step out explicitly: "Prepare a URL mapping from the current URLs to their corresponding new format" (Site moves with URL changes). For each old URL, decide:

Same content, new address → 301 to the new URL.
Content merged into another page → 301 to the page that now covers it.
Content deliberately removed, no equivalent → let it return 404 or 410. Google's guide says that for deleted or merged content, make sure "those URLs correctly return an HTTP 404 or 410 error response code".

Look for patterns. Migrations usually change URLs systematically, such as /blog/2023/05/post-name/ becoming /posts/post-name, or /products/ becoming /shop/. One pattern-based rule can cover hundreds of URLs. Handle the exceptions individually.

Don't redirect everything to the homepage

It's tempting to catch every unknown URL and send it to /. Google warns against exactly this: a site that "redirects any unknown URLs to their home page instead of returning 404 errors" is treated as a soft 404 case, which "can have negative effects on our understanding and indexing" of the site (Do 404 errors hurt my site?). It's also bad for visitors, who land on the homepage with no idea why. Redirect to a genuinely relevant page or return an honest 404 with helpful navigation.

Step 4: Implement redirects properly

Use permanent, server-side redirects (301 or 308) for moved content. Google's redirect documentation explains that permanent redirects signal the target should be shown in search results, while temporary ones (302, 307) keep the old URL (Redirects and Google Search). Server-side redirects are the most reliably understood; JavaScript redirects are a last resort.
Redirect in one hop. If the old site already had redirects, update them to point straight at the final URL. Each hop adds a network round trip, which Lighthouse says can delay loading "by hundreds of milliseconds", and it fails pages with two or more redirects (Lighthouse: Avoid multiple page redirects).
Use your platform's redirect tool: Shopify's URL redirects, WordPress's Redirection plugin, _redirects files on static hosts, or rules at your web server or CDN.
Keep them. Google recommends keeping redirects "for as long as possible, generally at least 1 year", and suggests that from users' perspective you consider keeping them indefinitely.

If you moved to a new domain, also submit a Change of Address in Search Console for the old site. Google says this is only needed for domain or subdomain changes, not for HTTP→HTTPS or path changes within the same domain.

Step 5: Fix internal links at the source

Redirects rescue external traffic. Your own site shouldn't depend on them. After a migration it's common for menus, footers, in-content links, related-post widgets and image references to still point at old URLs, especially if content was imported with hard-coded absolute links.

Crawl the new site and check:

Internal links returning 4xx (fix the link)
Internal links that redirect (update them to the final URL)
Images returning 404 (re-upload or fix paths, as media paths often change in migrations)
Links still pointing at the staging domain

I walked through doing this on a store in How to Find and Fix Broken Links on a Shopify Store. The same approach works on any platform.

Step 6: Check the things migrations quietly break

noindex left on from staging. It's the classic launch-day mistake. View source on key templates.
robots.txt still blocking everything.
Canonical tags pointing at old or staging URLs.
HTTPS and mixed content: new templates loading old http:// assets.
Sitemap: make sure the new sitemap lists only new, working URLs, and submit it in Search Console.
Titles and meta descriptions that didn't migrate and fell back to a template default.

If your site is small, SEO Health Basics for Personal Websites and Portfolios covers how to check each of these quickly.

Step 7: Confirm the recovery

Re-run your old URL list from Step 2. Every URL should now be either a single-hop 301 to the right page or a deliberate 404/410.
Use URL Inspection in Search Console on your most important old and new URLs.
Watch the Page indexing report. The count of "Not found (404)" URLs Google reports may take a while to change because Google recrawls old URLs gradually. What matters is that the URLs you care about are redirected correctly.
Compare Performance data for your top pre-migration pages against their new URLs over the following weeks.
Watch referral traffic from sites that linked to you.

Be patient with search recovery. Google has to recrawl old URLs to see the redirects, and that happens over time, not overnight.

Step 8: Keep watching

The weeks after a migration are when new problems appear: a redirect rule gets overwritten by a plugin update, a CMS editor changes a slug, an image CDN path changes. A one-off audit catches what's broken today. Ongoing monitoring catches what breaks next week.

Disclosure: I build Uchik, which is meant for that ongoing part. Uchik crawls your site every day with read-only requests and emails a report of broken links, missing images, redirect chains and loops, and HTTPS problems, each with the page and HTML element that caused it. How it works explains how it follows redirects hop by hop, the features page lists every check, and the sample report shows what a redirect-chain finding looks like. It checks the HTML the server sends, so links added only by JavaScript may be missed, and it crawls the live site rather than your old URL list, so Step 2 still needs a list-mode check. You can sign up for free.

The checklist

[ ] Old URL list rebuilt (Search Console, analytics, old sitemap, logs, backlinks)
[ ] Every old URL's current status checked
[ ] Old → new URL map built, using patterns where possible
[ ] No blanket redirects to the homepage
[ ] Permanent, server-side, single-hop redirects in place
[ ] Change of Address submitted (domain moves only)
[ ] Internal links, images and menus updated to final URLs
[ ] No stray noindex, robots.txt blocks or staging canonicals
[ ] New sitemap submitted
[ ] Old URL list re-checked after the fix
[ ] Ongoing monitoring in place for the months after launch

A migration doesn't end at launch. It ends when every link people relied on still takes them somewhere useful.

The Website Maintenance Checklist for Small Business Websites

Sandeep Singh — Sun, 18 Oct 2026 15:41:15 +0000

For most small businesses, the website isn't the product. It's the front door. People find you on Google or Maps, open the site, check you're legitimate, and then call, book or fill in a form. If any step in that path is broken, the lead doesn't complain. It goes to the next business in the search results.

The tricky part is that websites rarely break all at once. A contact form stops sending after a plugin update. A "Book now" button points at a scheduling page the old booking tool deleted. A menu PDF moves and the link goes dead. The site still looks fine, so nobody notices until someone mentions that the phone has been quiet.

This checklist is for owners and office managers who aren't web developers. It's split into weekly, monthly and yearly tasks, and for each one says why it matters and how to check it with free tools.

Weekly (about 15 minutes)

â˜ 1. Test every way a customer can contact you

This is the most important check on the list.

Submit your own contact form using a personal email address. Did the confirmation appear? Did the message arrive in the business inbox, not spam? Contact forms fail silently more often than any other part of a small business site, usually because a plugin update, hosting change or email setting broke delivery.
Tap the phone number on your phone. Does it open the dialer with the right number? (On many sites the number is a tel: link, and a typo there doesn't show on screen.)
Click the email link, the WhatsApp or chat button, and the map/directions link.
Try the booking or quote tool through to the last step before submitting.

If anything fails, fix it before anything else on this list.

â˜ 2. Click through the pages that bring in business

Open the homepage, your main services pages, pricing (if you publish it), "About" and "Contact" in a private browser window, on desktop and on a phone. You're looking for:

Links or buttons that lead to error pages
Missing images (empty boxes or broken-image icons)
Out-of-date information: old hours, old prices, staff who've left, last year's holiday notice

â˜ 3. Check last night's backup

Log in to your hosting or backup tool and confirm a backup completed recently. If your site is on a hosted builder (Squarespace, Wix and similar), know what the platform backs up and what it doesn't, especially form submissions and uploaded files.

Monthly (about 45 minutes)

â˜ 4. Scan the whole site for broken links and missing images

Clicking around only covers the pages you think of. A crawler checks them all.

Screaming Frog SEO Spider is a free desktop app for up to 500 URLs (limits), which covers most small business sites. After the crawl, check Response Codes â†’ Client Error (4xx), then use the Inlinks tab to see which page contains each broken link.
WordPress sites can use a plugin such as Broken Link Checker.
Google Search Console (below) also lists URLs Google found broken.

Fix links at the source: edit the page or menu so it points to the right live page. If a page was deleted or renamed and other sites or old emails link to it, add a redirect from the old address to the closest current page.

â˜ 5. Look at Google Search Console

Google Search Console is free and is the only place Google tells you directly about problems it has with your site. Once a month, check:

Indexing â†’ Pages: a rise in "Not found (404)" or "Server error (5xx)" usually means something broke.
Security issues and Manual actions: both should say no issues.
Performance: which searches bring people to your site. A sudden drop for your main service plus your town ("plumber Leeds") is worth investigating.

â˜ 6. Check your Google Business Profile matches your website

For local businesses, the Business Profile on Search and Maps is often seen before the website. Confirm the website link goes to a working page (not an old domain or deleted page), and that your hours, phone and address match the site. Google's help page explains how to edit your Business Profile. Do the same for any other directories you rely on, such as industry listings or review sites.

â˜ 7. Apply updates, carefully

If your site runs on WordPress or another self-hosted system, apply core, theme and plugin updates monthly at least (security updates sooner). After updating, repeat check #1, because updates are the most common cause of broken forms. If your host offers a staging copy, update there first.

â˜ 8. Check speed and mobile experience

Run your homepage and main service page through PageSpeed Insights. You don't need a perfect score. Look for big new problems, such as a huge uncompressed image someone uploaded or a new chat widget slowing the page down.

â˜ 9. Review who has access

Remove logins for former staff, freelancers and agencies you no longer work with, on the website, hosting, domain registrar and Google accounts.

Yearly (an hour or two)

â˜ 10. Domain name renewal

An expired domain takes your website and your email offline. Registrars of generic domains (.com, .net and similar) are required by ICANN to send at least two reminders before expiry, one about a month before and one about a week before (ICANN Expired Registration Recovery Policy). Those reminders go to the contact email on the registration, which is often an old address or a former employee's. So:

Check the expiry date at your registrar.
Turn on auto-renew and make sure the payment card on file is current.
Update the registrant contact email to an address someone actually reads.

â˜ 11. SSL certificate

The padlock in the browser depends on a valid certificate. Many hosts use free certificates from Let's Encrypt, which are valid for 90 days by default and are meant to be renewed automatically (Let's Encrypt FAQ). Automatic renewal usually works, but not always, for example after DNS changes. Click the padlock and check the expiry date now and then. An expired certificate shows visitors a full-page security warning, and most will leave.

â˜ 12. Content review

Read every main page as if you were a new customer:

Are services, prices and service areas current?
Are team photos and bios up to date?
Are testimonials still accurate and still permitted by the people who gave them?
Do your page titles and meta descriptions describe each page clearly? These often appear as the headline and text in Google results. Google's guidance covers title links and snippets.

â˜ 13. Accessibility basics

Make sure images have meaningful alt text, text has enough contrast, buttons are labelled and the site works with a keyboard. The W3C's introduction to web accessibility is a clear starting point. Accessible sites work better for everyone, including older customers on small screens.

â˜ 14. Test a restore

Restore a backup to a test location once a year. A backup that has never been restored is a hope, not a plan.

â˜ 15. Write down who owns what

Keep one document listing the domain registrar, hosting provider, website platform, email provider, who has admin access, renewal dates and support contacts. When something breaks at 6pm on a Friday, this document saves hours.

What this protects

Every item above protects one of four things:

Risk	What it costs	Checks
A lead can't reach you	Lost enquiries and bookings, silently	1, 2, 4, 7
Customers don't trust the site	Visitors leave for a competitor	2, 8, 11, 12
Google can't find or show you	Fewer people find you at all	4, 5, 6, 12
The site goes offline	Website and email down	3, 10, 11, 14, 15

Making the routine stick

The checks that matter most (contact paths, broken links, missing images) are also the ones people skip when they're busy. Some ways to keep them from slipping:

Put it in the calendar as a recurring 15-minute appointment, and assign it to a named person.
Keep a log: a spreadsheet with the date, what was checked and what was fixed.
Use alerts where they exist: uptime monitors that email you when the site is down, your registrar's renewal emails, Search Console's email notifications.
Automate the crawl. Broken links and missing images can be checked by software every day, so a problem that appears on Tuesday is in your inbox on Wednesday, not discovered at the end of the month.

Disclosure: I build Uchik, which does that last part. Uchik crawls your website every day with read-only requests (it never submits your forms) and emails a short report of broken links, missing images, redirect problems, HTTPS warnings and basic SEO gaps such as missing page titles, each with the exact page and HTML element to fix. How it works explains the method in plain language, and the sample report shows what lands in your inbox. You can sign up for free, no credit card needed. It doesn't replace check #1: no crawler can confirm your contact form actually delivers email, so keep testing that by hand.

Printable summary

Weekly: test contact paths Â· click through key pages Â· confirm backup
Monthly: crawl for broken links/images Â· Search Console Â· Business Profile Â· updates (then re-test forms) Â· speed Â· access review
Yearly: domain renewal Â· SSL Â· content review Â· accessibility Â· restore test Â· ownership document

Fifteen minutes a week is a small price for making sure every customer who finds you can actually reach you.

Mixed Content and Missing Images: A Practical Fix Guide for Store Owners

Sandeep Singh — Sun, 18 Oct 2026 15:40:27 +0000

Two problems turn up on almost every store that has been around for a few years: mixed content (insecure http:// files loaded on a secure https:// page) and missing images (product photos, banners or icons that return an error, leaving an empty box or a broken-image icon).

They often share a root cause, usually a migration, an HTTPS switch, or media copied from another site, and they're fixed with similar tools, so it makes sense to tackle them together. This guide explains what's going on in plain terms and gives you fixes for Shopify, WooCommerce and everything else.

Part 1: Mixed content

What it is

Your page loads over HTTPS, which means the connection is encrypted. If that page then pulls in an image, script, stylesheet, font or iframe over plain http://, that file travels unencrypted. MDN's definition: mixed content means "securely loaded web pages that use resources to be fetched via HTTP or another insecure protocol" (MDN: Mixed content).

What browsers do about it today

This is where a lot of older advice is out of date. According to MDN, modern browsers:

Auto-upgrade images, video and audio (for example an <img src="http://â€¦">) to https://. If the file exists at the HTTPS address, it loads. If it doesn't, it fails and you get a missing image.
Block almost everything else outright: scripts, stylesheets, web fonts, iframes, fetch() calls, and images set through srcset or <picture>.
Block even upgradable types when the URL uses an IP address instead of a domain name.

For a store, that means:

An http:// script (an old tracking pixel, a reviews widget, a chat embed) is blocked, so that feature silently stops working.
An http:// stylesheet or font is blocked, so parts of the page lose their styling.
An http:// image on a server without HTTPS fails after the auto-upgrade, so a product photo disappears.
An http:// iframe (an embedded size chart, video or map) is blocked and shows a blank box.

So mixed content is no longer mainly a "Not secure" padlock warning. It's a broken features problem.

How to find it

In the browser (any platform): open the page, press F12, and look at the Console. Chrome and Firefox log a warning for every mixed-content request, saying whether it was upgraded or blocked, and with the exact URL. Check your homepage, a product page, a collection/category page, the cart and a blog post, since each uses a different template.

With Lighthouse (built into Chrome DevTools): run the Best practices category. The "Uses HTTPS" audit lists insecure requests it found on that page (Chrome for Developers).

Across the whole site: a crawler checks every page rather than the five you happened to open. Screaming Frog SEO Spider's free version (up to 500 URLs, limits) reports mixed content under its Security tab. Note the source page for each hit so you know where to fix it.

How to fix it

The rule is simple: every resource on your site should be requested over HTTPS. How you get there depends on where the http:// reference lives.

1. Your own files: change http://yourstore.com/... references to https:// or to relative paths (/wp-content/uploads/..., /cdn/shop/files/...).

2. Other people's files: change http:// to https:// if the provider supports it. Paste the HTTPS version into a browser tab to check it loads. If they don't support HTTPS, host the file yourself or find a different provider. An embed that can't load securely shouldn't be on a checkout-adjacent page anyway.

3. Where the references hide, by platform:

WooCommerce / WordPress

Most mixed content on WordPress comes from old URLs saved in the database before the switch to HTTPS, in post content, product descriptions, widgets and page-builder data.
First, check Settings â†’ General: both WordPress Address and Site Address should start with https:// (see WordPress's HTTPS guidance).
Then do a database search-and-replace from http://yourstore.com to https://yourstore.com with a tool that handles WordPress's serialized data, such as the free Better Search Replace plugin. Take a full backup first, and run it as a dry run before the real thing.
Check theme options, header/footer script boxes and any "custom HTML" widgets by hand. These often contain pasted embed codes.

Shopify

Shopify serves storefronts over HTTPS, so mixed content almost always comes from pasted code: old embed snippets in theme files, custom Liquid sections, product descriptions written in HTML view, or apps that inject http:// assets.
In Online Store â†’ Themes â†’ â€¦ â†’ Edit code, use the code editor's search to look for http:// across theme files.
In product descriptions and pages, switch the rich-text editor to HTML (<>) and search for http://.
If an uninstalled app left code behind, remove its snippet references. App developers usually document which files they touched.

Any platform

A Content-Security-Policy: upgrade-insecure-requests header tells browsers to upgrade all requests to HTTPS. It's a useful safety net, but it doesn't help if the HTTPS version of a file doesn't exist, so still fix the source.
If you use a CDN, its "automatic HTTPS rewrites" option can rewrite links on the fly. Again, treat that as a backstop.

Part 2: Missing images

Why images go missing

Deleted from the media library or Files while still used in a product description, banner or blog post.
Migrations where image paths changed (/images/ â†’ /wp-content/uploads/2026/10/ or Shopify's CDN) but old content still references the old path.
Hot-linked images pulled from a supplier's or another site's server, which later moved or blocked hot-linking.
Auto-upgrade failures, the mixed-content case above: http:// image, no HTTPS version, nothing loads.
Missing resized versions. WordPress generates several sizes for each upload and references them in srcset. After a theme change or migration, some sizes may not exist, so certain screen widths show a broken image while others look fine.
Case and character mismatches. Product-Photo.JPG and product-photo.jpg are different files on most Linux servers.
Lazy-loading or optimisation plugins that rewrite image URLs. If the plugin is removed or misconfigured, the rewritten URLs can point nowhere.

How to find them

Quick check: open DevTools â†’ Network, filter by Img, reload the page and sort by Status. Anything with 404 (or 403) is a missing image. Do this on desktop and in the device toolbar at a phone width, because srcset picks different files for different screens.

Whole site: in a crawler, look at image URLs with 4xx responses, then use the "inlinks" or "source" view to find which pages reference each one. Fix at the source page, not just the file.

Also worth checking while you're there: images with missing alt text. Alt text is how screen-reader users understand product photos, and Google uses it, along with surrounding content, to understand images (Google Search Central: image SEO). Describe the product plainly ("Navy linen shirt, front view"). Don't stuff keywords.

How to fix them

Re-upload the file to your own media library (WordPress) or Content â†’ Files (Shopify) and update the reference to the new URL. Don't hot-link.
Regenerate thumbnails on WordPress after a theme change, using a regenerate-thumbnails plugin or WP-CLI (wp media regenerate), so every srcset size exists.
Fix paths in bulk after a migration with a search-and-replace on the old path prefix (WordPress), or a bulk product CSV export, edit and import (Shopify).
Remove or replace images you no longer have rights to or can't recover. An empty space is better than a broken-image icon on a product page.
If an image URL was shared externally (Pinterest, ads, marketplaces), add a redirect from the old image URL to the new one where your platform allows it.

A 15-minute routine to keep both under control

After any theme change, migration, app removal or bulk import: open the homepage, one product, one collection, the cart and one blog post with the Console and Network tabs open.
Monthly: crawl the site and export mixed-content and image-4xx lists.
Before pasting any embed code, check that every URL in it starts with https://.
Keep product images in your own store's media, never on a supplier's server.

Automating the check

These problems usually appear right after a change, which is exactly when nobody is looking. A scheduled crawl catches them sooner.

Disclosure: I build Uchik. Uchik is a hosted monitor that crawls your store daily with read-only requests and emails a report of mixed content, missing images, broken links and redirect chains, each with the source page and the exact HTML tag. The features page explains how it reads src and srcset attributes on images, and the sample report includes an http:// image flagged on a checkout page. You can sign up for free. It reads the HTML your server sends, so images injected only by JavaScript still need the DevTools check above.

Summary

Symptom	Likely cause	Fix
Widget or embed stopped working	`http://` script or iframe blocked	Switch to HTTPS version or replace the provider
Page partly unstyled	`http://` stylesheet or font blocked	Update the reference to HTTPS/relative
Blank product photo	Image 404, or auto-upgrade to missing HTTPS file	Re-upload to your own media; update reference
Photo broken only on phones	Missing `srcset` size	Regenerate thumbnails
Console full of warnings after HTTPS move	Old `http://` URLs in database	Backup, then search-and-replace

Serve everything over HTTPS, host your own images, and check the console after every change. That covers almost every case.

SEO Health Basics for Personal Websites and Portfolios

Sandeep Singh — Sun, 18 Oct 2026 15:29:01 +0000

A personal website does a specific job. When a recruiter, client, conference organiser or potential collaborator searches your name, it should come up, look professional and lead them to your best work. You don't need to rank for competitive keywords. You need to be findable for your name and your niche, and you need the site to work when someone arrives.

That second part gets overlooked. Portfolios are built once, often in a weekend, and then left alone for years while the things they link to change underneath them: demo apps on free hosting tiers get shut down, repos get renamed or archived, client sites get redesigned, conference talk pages disappear. The person reviewing your work doesn't know any of that. They just see a broken link next to your name.

This guide is a health check, not a growth hack. It covers what to set up once, and what to check regularly so the site keeps doing its job.

Part 1: Can Google find and index your site?

Verify in Search Console

Add your site to Google Search Console. It's free, and it's the only direct source of truth about how Google sees your site. After verification:

Use URL Inspection on your homepage. It tells you whether the page is indexed and, if not, why.
Check Indexing â†’ Pages for pages excluded by noindex, redirects or errors.

Check you haven't hidden the site by accident

A common portfolio problem is a noindex left over from development, or a theme setting like WordPress's "Discourage search engines from indexing this site" left ticked. Google's documentation is explicit: when it finds the noindex rule, it will "drop that page entirely from Google Search results, regardless of whether other sites link to it" (Block Search indexing with noindex). View the page source and search for noindex, and check response headers for X-Robots-Tag:

curl -sI https://yourname.dev | grep -i x-robots-tag

Also check robots.txt isn't blocking everything (Disallow: /). Note that the same Google page warns that if robots.txt blocks a page, Google can't see its noindex rule at all, so don't combine the two.

Do you need a sitemap?

Probably not urgently. Google says you might not need a sitemap if your site is "small", about 500 pages or fewer, and comprehensively linked internally (Sitemaps overview). Most portfolios qualify. If your static site generator or CMS produces one automatically, though, submitting it in Search Console doesn't hurt.

Part 2: Make your links crawlable

This one catches a lot of developer portfolios, especially single-page apps.

Google's link guidance says that, generally, Google "can only crawl your link if it's an <a> HTML element ... with an href attribute", and that most other formats won't be parsed (Link best practices). So if your navigation is built from <div onClick> or <button> elements that change routes in JavaScript, crawlers may never discover your project pages.

<!-- Crawlable -->
<a href="/projects/weather-app">Weather app</a>

<!-- Not reliably crawlable -->
<span onclick="router.push('/projects/weather-app')">Weather app</span>
<button onclick="goTo('weather')">Weather app</button>

Most frameworks' router link components (<Link> in Next.js, <router-link> in Vue, and so on) render a real <a href>, so use those rather than click handlers. A quick check: disable JavaScript in your browser and see whether you can still navigate.

Part 3: Titles and descriptions

Page titles

The <title> is usually the clickable headline in search results. Google's guidance asks for titles that are descriptive and concise, unique to each page, and free of keyword stuffing (Title links). For a portfolio:

Homepage: Ana Rivera, Frontend Developer in Lisbon
Project page: Weather App (React, Mapbox) | Ana Rivera
Blog post: the post's actual title, plus your name if it fits

The commonest problem on portfolios is every page sharing the same title, like "Portfolio" or "Home", because the template was never changed.

Meta descriptions

Google says snippets are primarily created from the page content, but it "sometimes uses the meta description HTML element if it might give users a more accurate description of the page" (Snippets). Write a unique, plain-English description for at least the homepage and key projects. One or two sentences on who you are, what you do and what's on the page is enough.

Duplicate URLs

If your site responds on both www and non-www, or on both http and https, pick one version and redirect the others to it with a single 301. You can also add a rel="canonical" tag pointing at the preferred URL. Google treats canonical tags as a strong signal for choosing which duplicate to show, but not as a guarantee (Consolidate duplicate URLs).

Part 4: The links that actually matter, to your work

Here's where personal sites lose the most value. A portfolio is a set of claims ("I built this", "I spoke here", "I wrote this") backed by links. When the links die, the claims look unverifiable.

Link rot is not a fringe problem. Pew Research Center found that 38% of webpages that existed in 2013 were no longer accessible a decade later, and even of pages collected in 2021, about one in five were gone two years later (Pew Research Center, 2024). A portfolio that links out to many projects, employers and talks is exposed to all of that.

Common culprits:

Live demos on free hosting tiers that sleep, expire or were deleted when you cleaned up your account
GitHub repos that were renamed, made private or moved to an organisation (GitHub redirects renamed repos, but not deleted or private ones)
Client and employer sites that were redesigned, so the specific page you built no longer exists
Talks and articles on event or publication sites that reorganised their archives
Your own old URLs after you moved from one static site generator or host to another

How to check

Crawl your own site with Screaming Frog SEO Spider (free for up to 500 URLs; limits). Enable external link checking, then look at Response Codes â†’ Client Error (4xx) and Server Error (5xx), and use Inlinks to find which page holds each broken link.
Or script it. For a small site, a short script that pulls every href and requests each one is a fine weekend exercise. Treat timeouts and 5xx errors as "check again later" rather than definitely broken.
Look past status codes. A deleted demo on some hosts returns a 200 with a generic "app not found" page. Open your most important links by hand now and then.

How to fix

Update the link to the new location.
For a dead demo, replace it with a short screen recording or screenshots, and say the live version is offline.
For a deleted article or talk page, link to an archived copy in the Wayback Machine if one exists.
If you changed your own URL structure, redirect old paths to the new ones so links from other people's sites keep working.

Part 5: Images, HTTPS and speed

Images

Portfolios are image-heavy, and missing images are the most visible kind of breakage: a gallery of broken-image icons. After any move or rebuild, check every project page. For the images that work, add meaningful alt text. Google calls alt text "the most important attribute when it comes to providing more metadata for an image", and it also helps people using screen readers (Image SEO best practices).

HTTPS and mixed content

Serve everything over HTTPS. If an HTTPS page loads assets over plain HTTP, browsers treat that as mixed content. Some images, video and audio are automatically upgraded to HTTPS, and scripts, stylesheets and iframes are blocked outright (MDN: Mixed content). Old embeds and hard-coded http:// image URLs are the usual cause. Search your source for http://.

Speed

Run your homepage and a heavy project page through PageSpeed Insights or Lighthouse in Chrome DevTools. On portfolios the big wins are almost always oversized images (export at display size, use modern formats) and unnecessary third-party scripts.

Part 6: A maintenance routine that fits a side project

You won't audit your personal site every week, and you don't need to. A light routine:

After every change (deploy, redesign, host move):

Click through every page with JavaScript disabled
Check for noindex and robots.txt surprises
Confirm old URLs redirect in one hop

Monthly (10 minutes):

Glance at Search Console for new errors
Open your three most important project links by hand

Quarterly:

Full crawl with external link checking
Fix or replace dead demos, repos and article links
Update titles and descriptions if your role or focus has changed

Automate what's tedious. Link and image checks are the part that's easy to forget and easy for software to do. An automated daily check means a demo that dies the week before you apply for a job shows up as an email, not as a recruiter's silent impression.

Disclosure: I build Uchik, a tool for that last step. Uchik crawls your site every day and emails a short report of broken internal and external links, missing images, redirect chains, HTTPS problems and basic SEO gaps such as missing titles, meta descriptions and canonical tags, each with the page and HTML element to fix. The features page lists the checks, and the FAQ covers what it does and doesn't do. One limit is relevant here: links that only exist after JavaScript runs may be missed, which is one more reason to use real <a href> links. You can sign up for free, no credit card needed. Screaming Frog's free tier and a quarterly reminder in your calendar also work.

Quick checklist

[ ] Site verified in Search Console; homepage indexed
[ ] No stray noindex, X-Robots-Tag or Disallow: /
[ ] Navigation uses real <a href> links
[ ] Unique, descriptive <title> on every page
[ ] Meta descriptions on homepage and key projects
[ ] One canonical version of the domain (HTTPS, www or not)
[ ] No broken links to demos, repos, employers, talks or articles
[ ] No missing images; alt text on project images
[ ] No mixed content
[ ] Routine scheduled or automated

Your personal site is often the first piece of your work someone sees. Keeping it findable and unbroken is a small, regular job, and it's worth it.

How to Find and Fix Broken Links on a Shopify Store

Sandeep Singh — Sun, 18 Oct 2026 15:28:10 +0000

A broken link on a store costs more than one on a blog. A dead link in a blog post loses a reader. A dead link in your main menu, in a "Shop the look" block or on a cart upsell loses an order, and Shopify won't tell you about it. Products get deleted, collections get renamed, apps get uninstalled, themes get swapped, and every one of those changes can leave behind links that now lead to "Page not found."

This guide shows a repeatable way to find those links with free tools and fix them properly. You don't need a developer or a paid app.

Where broken links come from on Shopify

Knowing the usual causes tells you where to look first:

Deleted or drafted products. A product that is deleted, set to Draft or removed from the Online Store channel returns a 404. Every link to it from a collection description, blog post, menu or "related products" block now breaks.
Changed URL handles. If you rename a product or collection and edit its handle, the old URL stops working unless a redirect was created.
Menu items pointing at things that no longer exist. Menus (Content â†’ Menus in the Shopify admin) often link to seasonal collections or pages that were removed months ago.
Hard-coded links in theme sections and rich text. Banners, slideshows and "image with text" blocks often have full URLs pasted in by hand. Typos and old handles hide here.
External links. Links to suppliers, size guides on other sites, social profiles or affiliate partners break when their sites change.
Images. An image deleted from Files, or a hot-linked image on another server, leaves an empty box where a product photo should be.

Step 1: Check what Google has already found (Google Search Console)

If you haven't connected Google Search Console yet, verify your domain and submit your sitemap (yourstore.com/sitemap.xml; Shopify generates it automatically).

Then open Indexing â†’ Pages and look under "Why pages aren't indexed" for:

Not found (404). URLs Google tried to crawl that no longer exist.
Redirect error. Redirects that loop or go nowhere.

Click into each list to see example URLs. The URL Inspection tool tells you where Google first discovered a URL, which often points to the page holding the bad link.

Know the limits. Google crawls on its own schedule, so the report lags behind your changes. It only shows URLs Google cares to crawl, so it won't list a broken link in your footer if Google hasn't recrawled that page yet. And a 404 isn't always a problem: Google's documentation says URLs that return 4xx codes are dropped from the index over time (Google: how HTTP status codes affect crawling). That's fine for a product you truly retired. What you need to fix are the links that still point to it.

Step 2: Crawl your own store

A crawler visits your pages the way a visitor would and reports every link that returns an error, along with the page that contains it. That second part is what makes it useful.

Free options:

Screaming Frog SEO Spider (free version). A desktop app for Windows, macOS and Linux. The free version crawls up to 500 URLs per crawl, which covers many small stores. Saving crawls and most advanced settings need the paid licence (Screaming Frog pricing and limits). After a crawl, filter Response Codes â†’ Client Error (4xx), select a URL, and open the Inlinks tab at the bottom to see every page that links to it, with the anchor text.
Free online checkers such as Dead Link Checker or W3C's Link Checker are fine for a quick look at a small site. They usually report less about where the link lives.

Tips for crawling a Shopify store:

Start from your homepage and let the crawler follow internal links. Your main navigation and footer are on every page, so a broken menu link shows up on every page of the report. Fix it once and the whole list shrinks.
Exclude URLs you don't care about: /cart, /account, /search? and filter or sort parameters such as ?sort_by= and ?filter.. Those create many near-duplicate URLs and eat your 500-URL allowance.
Crawl gently. Most desktop crawlers let you limit requests per second. One or two is plenty for a small store.
Crawl after every big change: a theme switch, a bulk product import, an app removal or a menu rebuild.

Step 3: Check the pages that make money, by hand

Crawlers follow links in the HTML. Some Shopify themes and apps inject buttons, upsells or recommendations with JavaScript, and crawlers that read only server HTML can miss them. So once a month, walk the purchase path yourself:

Open your homepage in a private window.
Click each main menu item and each homepage banner.
Open a best-selling product, add it to the cart, and continue to the checkout page. Stop before paying.
Open the browser developer tools (F12) â†’ Network tab, reload, and sort by Status. Anything red (404, 500) is a resource or link that failed.
Do the same on your phone. Mobile menus are often separate markup.

This takes about ten minutes and catches problems no crawler sees.

Step 4: Fix each broken link the right way

For each broken URL you'll make one of three fixes.

Fix A: Update the link at its source (best)

If the link is under your control (a menu item, a banner, a product description), edit it to point straight at the right live URL. A direct link is always better than a link that relies on a redirect.

Menus: Content â†’ Menus â†’ open the menu â†’ click the broken item â†’ choose the new collection, product or page.
Theme sections: Online Store â†’ Themes â†’ Customize â†’ find the section â†’ update the link field.
Rich text (descriptions, pages, blog posts): open the item, select the link, and edit it. Switch to the HTML view (<>) if the link is buried.

Fix B: Add a URL redirect (for URLs other people link to)

If the old URL is in emails, ads, social posts or other sites' backlinks, you can't edit those links. Add a 301 redirect so visitors land somewhere useful:

Go to Content â†’ Menus â†’ View URL redirects â†’ Create URL redirect.
In Redirect from, enter the old path (e.g. /products/blue-summer-dress).
In Redirect to, enter the closest live match: the replacement product, or the parent collection. Avoid sending everything to the homepage; it frustrates visitors.

A few Shopify rules worth knowing, all from Shopify's URL redirect documentation:

A redirect only works if the "from" URL is actually broken. If a page still loads there, the redirect is ignored.
You can't redirect from reserved paths such as /cart, /orders or /apps.
When you hide a product or collection, then redirect its URL, and later make it visible again, Shopify deletes that redirect automatically.
Redirects can be exported and imported as CSV. That's handy for bulk fixes after a migration.

When you change a product or collection handle in the "Search engine listing" section, Shopify offers to create the redirect for you. Leave that box ticked.

Fix C: Remove the link

If the destination is gone for good and there is no sensible replacement (an expired partner offer, a discontinued external guide), remove the link or replace it with plain text.

Step 5: Fix external links and images

External links: open each one. If the page has moved, update to the new URL. If it's gone, link to a different source or remove it. For affiliate links, check that the link still goes to the right product, not just that it loads.
Images: in the crawler, filter by images with a 4xx response. Re-upload missing images to Content â†’ Files and update the reference. Don't hot-link images from other domains; when those servers change, your images break.

Step 6: Prevent the next batch

Most broken links are created during routine admin work, so a few habits prevent most of them:

Before deleting a product, search your store admin and theme for its handle, and decide where its URL should redirect.
Keep the redirect box ticked whenever you edit a handle.
Use relative links (/collections/sale) in theme settings instead of full URLs. They survive domain changes.
After uninstalling an app, check the pages where it showed widgets. Leftover blocks or snippets can point to app URLs that no longer exist.
Re-crawl on a schedule. Monthly for small stores, weekly if you publish or change products often.

Automating the check

The manual routine works, but it relies on you remembering to run it, and a broken link created on Tuesday can sit there for weeks. If you'd rather have it run on its own, there are scheduled options: Screaming Frog's paid licence can schedule crawls, several Shopify apps monitor 404s, and hosted crawlers will email you results.

Disclosure: I build Uchik, a hosted option of this kind. Uchik crawls a store daily with read-only GET requests and emails a list of broken links, missing images and redirect problems. The features page lists every check it runs, and each issue comes with the page and the exact HTML tag that caused it, as you can see in this sample ecommerce report. You can sign up for free, no credit card needed. It reads server-delivered HTML (the FAQ explains what that means for JavaScript-added links), so the manual checkout walk-through in Step 3 is still worth doing.

Quick checklist

[ ] Search Console connected, sitemap submitted
[ ] Indexing â†’ Pages â†’ "Not found (404)" reviewed
[ ] Store crawled with cart, account and filter URLs excluded
[ ] Every 4xx link traced to its source page via Inlinks
[ ] Links fixed at source; redirects added for externally shared URLs
[ ] Purchase path walked by hand on desktop and mobile
[ ] Next crawl date in the calendar

Broken links rarely appear all at once. They pile up one admin change at a time, and a short, regular check is the easiest way to keep them under control.

DEV Community: Sandeep Singh

Automating Website Monitoring: What to Check Daily vs Weekly vs Monthly

Continuous (every few minutes): is it up?

Daily: is anything broken?

Broken links and missing images

Certificate validity

Backups completed

Key-page smoke test

Running it on a schedule

Weekly: the human checks

Monthly: performance, content and access

Yearly: renewals and recovery

Alerts versus reports

The schedule on one page

Broken Links and Core Web Vitals: What Actually Affects Rankings

Part 1: What Google says about 404s and broken links

A 404 on its own is not a penalty

But a 404 on a page that matters is a real loss

Internal links are how Google finds your pages

Soft 404s and homepage redirects are worse than honest 404s

Broken outbound links

Part 2: What Google says about Core Web Vitals

The metrics and thresholds

Yes, they're used in ranking, but they aren't decisive

Lab scores versus field data

The rest of page experience

Part 3: Where broken links and Core Web Vitals overlap

Redirect chains slow every page load

Images without dimensions cause layout shift

Broken resources waste requests

Part 4: The costs that have nothing to do with rankings

Part 5: A sensible priority order

Part 6: Monitoring both

Summary

How to Audit Internal Links on a WordPress Site

What you're looking for

Step 1: Crawl the site

Step 2: Fix broken internal links

Step 3: Update redirected internal links

Bulk-fixing domain and protocol changes

Step 4: Find orphan pages

Step 5: Check click depth for important pages

Step 6: Improve anchor text

Step 7: Check your theme and page builder output

Step 8: Make it a habit

Checklist

404 Errors After a Site Migration or Redesign: A Recovery Checklist

First: what Google actually says about 404s

Step 1: Rebuild the list of old URLs

Step 2: Check each old URL's current status

Step 3: Map old URLs to new URLs

Don't redirect everything to the homepage

Step 4: Implement redirects properly

Step 5: Fix internal links at the source

Step 6: Check the things migrations quietly break

Step 7: Confirm the recovery

Step 8: Keep watching

The checklist

The Website Maintenance Checklist for Small Business Websites

Weekly (about 15 minutes)

â˜ 1. Test every way a customer can contact you

â˜ 2. Click through the pages that bring in business

â˜ 3. Check last night's backup

Monthly (about 45 minutes)

â˜ 4. Scan the whole site for broken links and missing images

â˜ 5. Look at Google Search Console

â˜ 6. Check your Google Business Profile matches your website

â˜ 7. Apply updates, carefully

â˜ 8. Check speed and mobile experience

â˜ 9. Review who has access

Yearly (an hour or two)

â˜ 10. Domain name renewal

â˜ 11. SSL certificate

â˜ 12. Content review

â˜ 13. Accessibility basics

â˜ 14. Test a restore

â˜ 15. Write down who owns what

What this protects

Making the routine stick

Printable summary

â˜ 1. Test every way a customer can contact you

â˜ 2. Click through the pages that bring in business

â˜ 3. Check last night's backup

â˜ 4. Scan the whole site for broken links and missing images

â˜ 5. Look at Google Search Console

â˜ 6. Check your Google Business Profile matches your website

â˜ 7. Apply updates, carefully

â˜ 8. Check speed and mobile experience

â˜ 9. Review who has access

â˜ 10. Domain name renewal

â˜ 11. SSL certificate

â˜ 12. Content review

â˜ 13. Accessibility basics

â˜ 14. Test a restore

â˜ 15. Write down who owns what