DEV Community: laing lii

How I Got 200+ Visitors in 20 Days: Refactoring a Pure PHP Toolbox & Adding Client-Side Security

laing lii — Sun, 08 Nov 2026 01:23:46 +0000

Hey DEV community! 👋

About three weeks ago, I launched Site Tools Box (https://sitetoolsbox.com)—a free, no-login toolbox for webmasters. It started as a simple side project, but after a surprising burst of traffic from Hacker News (200+ unique visitors in 20 days), I realized I needed to seriously level up the architecture.

Here is a breakdown of the latest technical updates, what I learned, and how I handled the "Zero-Login" challenge.

🚀 1. The Hybrid Architecture: Speed & Privacy First
When building a toolbox, you face a classic dilemma:

Option A: Do everything on the server (slower, heavy server load, privacy risks).

Option B: Do everything in the browser (faster, but limits complex queries).

I chose a Hybrid Architecture.
For tools like the JSON Formatter, the logic runs 100% in Vanilla JavaScript. Data never leaves the user's machine.
But for Bulk Domain Lookups, browsers can't query WHOIS/RDAP directly due to CORS and protocol limits. So, I built a server-side endpoint using native PHP's curl_multi. This allows parallel querying of up to 30 domains instantly, bypassing the sequential delays of traditional WHOIS lookups.

🔒 2. UI/UX Overhaul: Daily Domains
I recently added a "Daily Domains" feature, curating unregistered .com and .ai domains. I used to display them in clunky cards, but users got visual fatigue quickly.

I refactored the front-end to use a compact, single-row list grouped by date. It drastically reduced cognitive load. To handle the affiliate redirects cleanly, I pass the domain dynamically via URL parameters:
https://www.namecheap.com/domains/registration/results/?domain={domain}
🛡️ 3. The "Zero-Login" Security Challenge (MD5, SHA, Bcrypt)
Many users asked for a Password Generator. But as a developer, I know that generating a password is only half the battle—you need to hash it for storage.

I upgraded the tool to include:

MD5 / SHA-256 / Base64: Processed entirely via crypto-js in the browser.

Bcrypt Hashing: This was the tricky part. Bcrypt requires high CPU work (the "salt rounds"). Running it in JS is possible, but heavy.

The Tech Stack I used:

🎨 4. Avoiding the "Ugly System Alert"
One thing I hate about older web apps is the default alert() when you copy text. I wrote a tiny smooth-copy function that changes the button state to a green "✅ Copied!" and reverts after 1.5 seconds. Small details like this make a tool feel like a modern SaaS.

🚀 What's Next?
I'm currently focusing on SEO optimization and trying to get the tool indexed by more search engines. The recent jump in traffic from Japan (thanks to a tech media mention) has also motivated me to keep the UI language-neutral and fast.

If you want to check it out or give me some feedback on the architecture, here is the link:
Live Tool: https://sitetoolsbox.com
GitHub: https://github.com/kagouhui/site-tools-box

Thanks for reading! Happy coding! 💻

How I Built a Custom Anti-Spam Rate Limiter for My PHP Site (No Cloudflare

laing lii — Thu, 05 Nov 2026 04:51:46 +0000

If you run a free directory, the bots will find you. Within 48 hours of launching my free toolbox, I was getting hammered by automated scripts.

I decided to ditch heavy WAF rules and build a custom rate limiter using pure vanilla PHP 8 with flat JSON files. No Redis, no database overhead.

Here is how it works in 3 layers:

The Honeypot & Time Check
I added a hidden input field (bots fill it, humans don't). If filled, instant ban. I also record page load time. If a user submits in under 5 seconds, they are a bot.
Flat-File IP Rate Limiting
Instead of querying the database on every attempt, I store IP timestamps in a JSON file (ip_limits.json). If an IP submits more than 3 times in 24 hours, I block them before any SQL runs.
The Server-Side Math CAPTCHA
I generate two random numbers in a PHP session and ask the user to solve it (e.g., "What is 4 + 7?"). Scripts hate dynamic math.

The Result:
Spam submissions dropped to absolute zero. Real users don't notice a thing. A few lines of vanilla PHP saved me hours of headaches.

I'm currently building Site Tools Box, a free ad-free platform with 20+ developer tools and an AI Text Humanizer. You can see this anti-spam system in action here:
👉 sitetoolsbox.com

Have you ever fought a bot war on your own site? Let me know!

I built a free AI text humanizer with dual-engine fallback. Need some brutal feedback on the output.

laing lii — Wed, 04 Nov 2026 00:19:55 +0000

Hey everyone,

I got tired of "free" AI tools that actually charge you after 200 words. So I built my own over the weekend. It's a dual-engine setup (OpenRouter + SiliconFlow) written in pure PHP so it shouldn't crash when the API gets rate-limited.

I also added a strict daily limit (5 uses per IP) because a botnet attacked my endpoint 50 times right after I launched it.

I bundled it with 20+ other webmaster tools (WHOIS, DNS, JSON, etc). I'm mainly looking for feedback on the humanizer's output—does it actually sound natural to you?

If you want to test it out, just type "sitetoolsbox.com" into your browser, the humanizer is in the AI section.

What's the biggest pain point you usually hit with AI detectors? Would love to know what to improve.

The Silent Killer of Uptime: Why You Must Master DNS Lookups

laing lii — Tue, 03 Nov 2026 00:33:04 +0000

If you manage servers, deploy microservices, or just run a WordPress site, you've probably experienced it: A sudden, unexplainable outage. The server is running. The CPU is at 10%. The firewall is open. Yet, users are reporting that your site is completely unreachable.

Nine times out of ten, the problem isn't your code. It's the invisible routing layer of the internet: DNS.

Most developers treat DNS as a "set it and forget it" configuration—a one-time A record update in their domain registrar. But DNS is incredibly fragile. Here is why you need to actively monitor it, and how to diagnose it in seconds.

1. The CNAME Chain Reaction

Many of us use CNAME records to point subdomains to third-party services (like an AWS Load Balancer, a Shopify store, or a Mailchimp server). The danger here is the "chain." If that upstream provider updates their IP address and breaks the chain, your subdomain goes dark.

You can't just `ping` the domain; you need to query the exact record type. Our free DNS Lookup Tool allows you to instantly query the CNAME and trace where it's pointing. If it resolves to an IP that you don't recognize, you've found your culprit.

2. The "Propagation is Slow" Excuse

You changed your A record 3 hours ago, but half your users are still hitting the old server. You might think it's just "normal propagation." But if you are using Cloudflare or AWS Route 53, updates should be near-instant.

If you suspect slow propagation, run a query against the specific nameservers using our tool. You can check the NS records and the SOA records to see exactly what the authoritative name server is broadcasting. It gives you the hard truth before you blame the ISP.

3. The Silent Email Killer (MX & SPF)

Your website loads fine, but your contact form emails are bouncing, or customers aren't receiving password resets. This is a classic MX (Mail Exchanger) misconfiguration. Or worse, your TXT records (SPF/DKIM) are missing, causing Google and Microsoft to instantly trash your transactional emails.

A DNS Lookup isn't just for web traffic. Querying the MX and TXT records ensures your mail routing and domain verification are bulletproof.

How to Diagnose Any DNS Issue in 5 Seconds

Stop opening a terminal and typing `nslookup` or `dig` blindly. It's clunky, and if you're on a restricted network or VPN, the results might be cached locally and give you a false reading.

Use our web-based DNS Lookup Tool. It queries from our high-availability infrastructure, meaning you get a clean, uncached, real-world view of your DNS records.

Features you can use right now:

Check A and AAAA records to verify dual-stack (IPv4/IPv6) configurations.
Verify CNAME aliases to prevent routing loops.
Validate TXT records for SPF domain verification.

Whether you are a sysadmin fighting a 2 AM outage or a webmaster setting up a new domain, DNS is your first line of defense. Bookmark the tool, and save yourself hours of hair-pulling.

👉 Run a Free DNS Lookup Now →

Stop Using Slow APIs: The Right Way to Bulk Check HTTP Statuses (50+ URLs in Seconds)

laing lii — Sat, 31 Oct 2026 06:03:17 +0000

Last week, I found myself staring at a dead-links report for an old client project. Over 300 pages. I needed to know which URLs were returning 404s, which ones were silently redirecting, and which ones had just given up entirely.

My first instinct was to write a quick loop in Node.js using fetch.

I hit two massive walls immediately:

The Rate Limit / IP Ban wall: After about 15 requests, the target servers started dropping my connection.
The Timeout wall: My script hung for 5 minutes because three random URLs just never responded.

If you've ever tried to bulk check HTTP statuses at scale, you know the pain. So, let me save you a few hours of frustration.

Here is the breakdown of how this should work, and a free tool I started using to bypass the headache entirely.

The Naive Approach (And Why It Fails)

Here’s how most of us start writing a bulk HTTP checker:


javascript
// DON'T DO THIS FOR BULK CHECKS
async function checkUrls(urls) {
  for (const url of urls) {
    try {
      const res = await fetch(url, { method: 'HEAD' });
      console.log(url, res.status);
    } catch (e) {
      console.log(url, 'Error');
    }
  }
}

This is a disaster for a few reasons:

HEAD isn't always supported: Some servers block HEAD requests and only return a valid 200 for GET.

No Concurrency Control: If you use Promise.all(), you’ll fire 50 requests at once and your IP will get blocked by Cloudflare instantly.

No Timeouts: A single hanging URL will block your entire script.

The "Developer" Way to Fix It
To do this properly in a script, you need to implement:

Connection Pooling / Semaphores (to limit concurrency to ~5 requests at a time).

Custom AbortControllers (to kill requests after 5 seconds).

User-Agent spoofing (so it looks like a real browser).

It’s a fun technical exercise, but when you’re on a deadline and just need the data, writing this whole infrastructure is a waste of time.

The Alternative: Automate It Without the Code
When I realized I was spending more time debugging my scraper than actually auditing the site, I switched to using a bulk HTTP checker tool I found online.

I’ve been using Bulk HTTP Checker on Site Tools Box.

It’s a lightweight, backend-optimized tool that handles all the heavy lifting for you. You can paste up to 50 URLs, hit run, and it handles the concurrency, redirects, and timeouts instantly.

It returns a clean list of:

Status Codes (200, 301, 404, 500)

Redirect chains (where it actually goes)

Response times

And because it’s built for developers, it doesn't try to sell you a $29/month subscription for a simple check.

One Last Pro-Tip for SEO / Auditing
If you have a sitemap and want to find out which specific pages on your site are broken, you don't even need to copy-paste manually. Just use the Dead Link Checker on Site Tools Box. It scans a page, crawls the outgoing links, and highlights any broken ones in red. I used it to find 3 dead links in my footer that had been silently 404-ing for months.

Don't waste your weekends writing semaphores for a script you'll only use once. Optimize your workflow, and get back to writing actual code.

Have you ever accidentally DoS'd a server with a Promise.all() loop? Let me know in the comments. We've all been there.

Why I Dumped GA4 to Build a 2KB Privacy-First Stats System (and Caught a Fake "Big Tech" Company)

laing lii — Fri, 30 Oct 2026 00:07:28 +0000

When I launched my side project—a curated directory for webmaster and SEO tools—I did what everyone does: I slapped Google Analytics (GA4) on it.

But within a week, I was frustrated. GA4 felt bloated, it added heavy JS files to my tiny server (2-core, 2GB RAM), and the "bot filtering" was practically non-existent. I was getting hundreds of "visits" a day, but my PayPal account remained empty.

I realized I needed a brutal, no-nonsense way to see exactly who was hitting my site.

So, I deleted GA4 and built my own analytics system from scratch. Here is how it went, and the hilarious trap it helped me uncover.

The Architecture: Keep It Dead Simple

I didn't want to spin up a heavy MySQL database just for logging. I wanted something I could read with my own eyes.

I decided to use JSONL (JSON Lines) and a tiny PHP endpoint.

Here is how the data flows:

A lightweight JS fetch() on the main site sends a POST request to my stats subdomain.
The PHP API receives the data (URL, Referer, User-Agent, IP).
The Core Logic: The PHP script regex-matches the User-Agent against a list of known bot strings (googlebot, ahrefs, python, crawl).
If it's a bot, it's flagged as is_spider: 1.
The data is appended to a daily text file: log_2026-10-30.jsonl.

Here is what a raw log entry looks like:


json
{"time":"15:41:56","ip":"43.128.87.232","country":"France","url":"/pay.php","ref":"","ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X...)","is_spider":0}

The Moment It Paid Off: Catching the "Fake Big Tech"
Because I now have the raw IP and User-Agent, my dashboard separates the noise from the real people.

Yesterday, my stats showed a "Real Human" hitting my /pay.php page. I got excited—someone was actually looking to pay for my premium listing!

But then, I noticed something strange in the logs. A company called "Logipax" submitted their site. Their landing page was incredible. It claimed to be a top-tier passenger transport SaaS operating "Since 1995", complete with a London map background and enterprise pricing.

It looked like a massive, established European company. Why would they submit to a new, tiny directory?

Curiosity got the better of me. I pulled their IP from my JSONL log and ran a quick WHOIS lookup.

The result was hilarious.

The domain logipax.io was registered just 4 days ago.

They had built a highly polished UI, written fake enterprise copy, and spammed every directory on the web to build SEO backlinks. If I had been using GA4, I would have just seen a "session from the UK" and approved them without thinking.

(Insert your screenshot of the WHOIS lookup here)

The Takeaway for Indie Hackers
Building this tiny 2KB system changed how I view web traffic. It's no longer just a number on a chart; it's a real story.

GA4 is overkill for side projects. If you just want to know if real humans are visiting, build your own logger. It's a fun weekend project.

Bots are loud, humans are quiet. 70% of my daily traffic is just Ahrefs and Googlebot. Seeing this visually stopped me from stressing out over "zero traffic."

Data ownership is freedom. I can run custom SQL queries, filter by country, and spot fake traffic instantly, all without paying a monthly SaaS fee.

If you're building something small, don't be afraid to strip things down. The best stack is the one you actually understand.

(If you want to see the final result or find some cool dev tools, you can check out my project here: Site Tools Box.)

How I Built a Bulk Domain Availability Checker with RDAP and Automated Scoring

laing lii — Wed, 28 Oct 2026 02:23:31 +0000

I recently found myself needing to buy a short, brandable domain for a side project. I spent hours typing names into GoDaddy, only to get frustrated by the slow interface and the constant upsells.

As a developer, I thought: "Why not just query the RDAP registries directly and score the domains algorithmically?"

So I built a free tool to do exactly that. Here is how it works and why RDAP is the future of domain checking.

The Problem with Traditional WHOIS
Most domain tools still use standard WHOIS (Port 43). It is plain text, rate-limits aggressively, and often gets blocked by registrars.

RDAP (Registration Data Access Protocol) is the modern, ICANN-mandated replacement. It returns structured JSON and uses standard HTTPS status codes:

200 OK = Domain is registered.

404 Not Found = Domain is available.

This makes it incredibly easy for developers to query concurrently without getting blocked.

The Architecture
I built a tool that lets you paste up to 50 domains (one per line). Instead of firing 50 requests at once and getting rate-limited, the backend processes them in small concurrent batches (using curl_multi in PHP).

Here is what the result dashboard looks like:

The Domain Scoring Algorithm
Just knowing if a domain is available isn't enough. You need to know if it's actually good.

I wrote an algorithmic scoring system (0-100) that evaluates domains across 5 dimensions:

Extension (TLD): .com, .ai, .io score highest.

Length: Short names (4-5 letters) get exponential value.

Pronounceability: It analyzes vowel-consonant patterns (CVCV) to ensure it's easy to say.

Meaning: Checks against a built-in dictionary of common English words.

Brand Fit: Looks for industry prefixes like AI, tech, or e-commerce.

If a domain scores 80+, it's highlighted in red as a 🔥 High Value domain. You can copy all the high-value results to your clipboard with one click.

Why I Made It Free (For Now)
I'm currently offering single checks for free. Building this has taught me a lot about RDAP and concurrent API handling.

If you want to try it out, you can check it here:
👉 Site Tools Box - Domain Availability Check

(Note: The bulk check feature is currently locked behind a Pro tier while I optimize the server load, but single checks are 100% free).

Iterating on a Domain Checker: Adding Timestamps, Sources, and IANA Bootstrap

laing lii — Tue, 27 Oct 2026 03:20:23 +0000

A few days ago, I wrote a post about how I built a bulk domain availability checker using PHP and RDAP (instead of the traditional WHOIS protocol) over at Site Tools Box.

To my surprise, the post got some fantastic technical feedback from the community. One comment from a developer named Furqan completely nailed some of the pain points I was already struggling with in production.

Here are the three brilliant suggestions he made, and how I'm currently implementing them to make the tool more reliable.

1. The "Premium Domain" Illusion: Why a 404 is Just a Hint

In my previous post, I mentioned that querying RDAP returns a 404 Not Found if a domain is available, and 200 OK if it's registered.

The problem? Premium and reserved domains. Sometimes, a registry will return a 404 because it's technically unregistered, but it's actually a premium name that requires a 5-figure price tag at the registrar.

Furqan's suggestion: Flag 404s as "Likely Available" instead of a hard "Yes".

This is a brilliant UX change. I'm now updating the frontend to display a "Likely Available" badge with a tooltip explaining that premium domains can sometimes show as available but are locked behind a premium price wall.

2. Transparency: Adding "Last Checked" Timestamps and Sources

Right now, when you check a domain on Site Tools Box, you get the result, but you don't know when it was checked or where the data came from.

Furqan suggested adding a "last checked" timestamp and the source (RDAP vs. WHOIS).

This is crucial for debugging. When doing bulk checks of 100+ domains, network timeouts happen. By displaying the exact timestamp and the specific RDAP server endpoint used (e.g., rdap.verisign.com), users can immediately tell if a result is stale or if the query failed.

3. Ditching the Hardcoded Map for IANA Bootstrap

In my original PHP code, I had a massive switch statement and array mapping TLDs to their respective RDAP servers (e.g., .com -> Verisign, .io -> Identity Digital).

Furqan pointed out the obvious scaling issue: This hardcoded map is a nightmare to maintain. What happens when a new TLD like .xyz or .ai changes its registry?

His solution: Use IANA's Bootstrap file.

IANA provides a public JSON file (https://data.iana.org/rdap/dns.json) that automatically maps every TLD to its current RDAP endpoint. I'm currently rewriting my backend to fetch this file, cache it locally, and dynamically route queries. This means the tool will automatically support new TLDs without me lifting a finger.


php
// Pseudo-code for the new dynamic approach
$bootstrapUrl = 'https://data.iana.org/rdap/dns.json';
$bootstrapData = json_decode(file_get_contents($bootstrapUrl), true);

// Dynamically find the endpoint for .com
foreach ($bootstrapData['services'] as $service) {
    if (in_array('com', $service[0])) {
        $endpoint = $service[1][0];
        break;
    }
}



The Takeaway: Build in Public, Iterate Fast
This whole experience has been a masterclass in building in public. The feedback from the dev community is invaluable. What you think is a solid MVP might have fatal edge cases in the real world.

If you want to see the updated tool (and test the new features as they roll out), you can check it out here: Free Domain Availability Checker.

Have you ever built a tool that had to handle messy edge cases like premium domains or dynamic TLD routing? Let me know in the comments!

The Silent SEO Killer: How I Lost 30% of Organic Traffic on a Site Migration

laing lii — Sun, 25 Oct 2026 04:36:24 +0000

We’ve all been there. You migrate a site to a new server or change your permalinks, feeling confident because the new UI looks great. Then, a week later, you open Google Search Console and see a massive cliff: a 30% drop in organic traffic.

A migration is never just about moving files. It’s a complex DNS and routing operation. When it goes wrong, search engines don't forgive easily.

After spending three days debugging a client's botched migration, I realized that 90% of post-migration traffic drops are caused by four easily preventable technical issues. Here is the checklist I now strictly follow before and after any site move.

1. The Silent Fail: Broken Internal Links (404s)

When you change URLs, it's easy to miss internal links in old blog posts or documentation. If a crawler hits a 404 page, it stops traversing that path. Too many 404s, and your site's crawl budget takes a hit.

How to audit: Don't just click around manually. You need to extract all links from a page and check their HTTP response codes.
Instead of writing a complex Python script with requests and BeautifulSoup, I usually run a quick batch scan.

(Example: I use the bulk dead link scanner on Site Tools Box to scan the top 50 pages of the site. It crawls the DOM, sends HEAD requests to every anchor tag, and spits out a list of any 404s or 500s in seconds.)

2. The TTL Trap: DNS Propagation and AAAA Records

You updated the A record to the new server IP, but did you check the AAAA record? If you have an IPv6 record pointing to the old server, users on IPv6 networks (which is a huge percentage of mobile traffic) will still hit the dead server.

How to audit: Use a thorough DNS lookup. You need to verify A, CNAME, MX (don't forget emails!), and TXT records.
Before you switch the nameservers, lower your TTL (Time To Live) to 300 seconds at least 24 hours in advance. This ensures that when you make the switch, global ISPs pick up the change instantly rather than caching the old IP for 48 hours.

3. The Redirect Chain Nightmare

You moved from http://website.com/old-post to https://website.com/new-post.
If your redirects look like http -> https -> www -> new-url, you are burning server resources and leaking PageRank with every hop.

The rule: Every old URL should 301 redirect to its final destination in exactly one hop.

How to audit: Write down your 10 most valuable URLs (highest traffic pages). Check their status codes.
You can use an HTTP header/status checker to see the exact response and how long it takes the server to respond. If it's a 301, ensure Location headers point directly to the final HTTPS URL.

4. The Pre-Migration Domain Vetting

If you are migrating to a brand new domain, buying a clean domain is half the battle. If the previous owner was running a PBN (Private Blog Network) or was penalized by Google, your traffic will never recover.

How to audit: Before you even pay for a domain, do a deep Whois lookup. You need to check the creation date, the registrar, and most importantly, whether the domain has been dropped and re-registered multiple times.
A quick RDAP lookup is usually enough to reveal the registration history. If it was created 3 weeks ago but has 10,000 backlinks, run away.

Summary Checklist

Lower DNS TTL 24 hours prior.
Verify A, AAAA, and CNAME records.
Ensure single-hop 301 redirects for all legacy URLs.
Crawl the entire new site to find dead internal links.
Submit the new sitemap.xml to Google Search Console immediately.

The Takeaway
A successful migration isn't a visual success; it's a routing success. Take 30 minutes to run these technical diagnostics before you hit "deploy" in production. Your organic traffic (and your client) will thank you.

Stop Guessing: A Developer's Guide to Debugging HTTP Headers with cURL

laing lii — Fri, 23 Oct 2026 01:04:00 +0000

I've lost count of how many times I've spent hours debugging an API integration, only to realize the issue wasn't my code, but a silent 502 Bad Gateway or a misconfigured Content-Type header.

When a web application fails silently, the first thing you should do is look at the raw HTTP response. Browsers abstract this away from us, so as developers, we need to drop down to the network layer.

Here is my standard workflow for debugging APIs using cURL, and how to do it faster when you don't have terminal access.

The Basics: Fetching Headers Only When debugging, you usually don't care about the entire HTML body. You just want the headers. Using -I (or --head) tells cURL to only fetch the headers:

bash
curl -I https://api.example.com/v1/users
This will return the HTTP status code (200, 404, 500), the server type, cache control, and crucially, the Content-Type.

Following Redirects (301 vs 302) If you are migrating a site or testing an API endpoint, you need to know if the server is redirecting you. By default, cURL does not follow redirects.

If you see a 301 or 302 in the output, add the -L flag:

bash
curl -IL https://example.com/old-endpoint
This will follow the redirect chain and show you the final status code. If you see an infinite redirect loop, this is the fastest way to catch it.

Inspecting the Exact Request/Response Sometimes you need to see exactly what headers your client is sending. Maybe you forgot the Authorization token, or your Accept header is wrong.

The -v (verbose) flag is your best friend:

bash
curl -v -X POST https://api.example.com/v1/login \
-H "Content-Type: application/json" \
-d '{"user":"admin","pass":"secret"}'
The output will show lines starting with > (what cURL sent) and < (what the server returned). This instantly reveals if you are missing a bearer token or sending malformed JSON.

The Problem: When You Don't Have a Terminal As developers, we spend most of our time in the terminal. But what happens when you are debugging on a mobile device, or you just want a quick check without opening a new tab?

I got tired of trying to remember cURL flags and writing out long commands every time, so I built a couple of lightweight tools for my own workflow and put them on Site Tools Box (no sign-ups, no ads).

For single URLs: I use the HTTP Status Checker. It gives you the status code, response time, and headers in a clean UI.

For bulk checking: If you have a list of URLs (like checking a sitemap for 404s), use the Bulk HTTP Checker. It runs in your browser and checks up to 20 URLs at once via a CORS proxy.

Wrapping Up
Debugging is 90% observation and 10% fixing. Before you start tearing apart your frontend code, verify the HTTP layer. A quick cURL command or a fast online check can save you hours of frustration.

What's your go-to cURL trick for debugging? Let me know in the comments!

Stop Manually Checking Broken Links: A Developer's Guide to Automated 404 Hunting

laing lii — Thu, 22 Oct 2026 00:57:15 +0000

I used to have a really bad habit. After deploying a major update to a client's site or migrating a blog to a new CMS, I would click through a few pages, assume everything worked, and close my laptop.

Then, a week later, Google Search Console would light up with 404 errors, or a user would email me: "Hey, the link to your pricing page is broken."

Broken links are a silent killer. They destroy your SEO, frustrate users, and make your site look unmaintained. But here's the thing: Manually checking links is a waste of your time as a developer.

Here is how I stopped clicking links manually and started automating the 404 hunting process.

The Problem with "Simple" Link Checkers
If you Google "broken link checker," you'll find hundreds of tools. But most of them have fatal flaws for developers:

They require installing a massive desktop app (looking at you, Screaming Frog). It's great for deep audits, but I don't want to launch a heavy Java app just to check 20 links.

They are filled with ads and require sign-ups. I don't want to hand over my email address just to check an HTTP status code.

The free tier limits you to 5 URLs.

Approach 1: The cURL Bash Script (For Server-Side Checking)
If you have terminal access, you can write a quick Bash script to check a list of URLs using curl. Here is a simple script that reads a text file of URLs (urls.txt) and spits out the HTTP status code for each:

bash

!/bin/bash

Read from urls.txt

while read -r url; do
# -o /dev/null discards the HTML body
# -s silences the progress bar
# -w writes out the HTTP status code
# -I fetches only the headers
status=$(curl -o /dev/null -s -w "%{http_code}" -I "$url")

if [ "$status" -ne 200 ]; then
echo "❌ [$status] $url"
else
echo "✅ [200] $url"
fi
done < urls.txt
This is fast and doesn't require any third-party API. However, if you are checking URLs that are behind Cloudflare or have strict WAF rules, curl might get blocked (returning a 403 instead of the real status).

Approach 2: The Client-Side Browser Check (For Quick Audits)
Sometimes I just want to check a handful of links on the fly without opening a terminal. But doing this in the browser has a massive hurdle: CORS (Cross-Origin Resource Sharing). Browsers block JavaScript from making arbitrary HTTP requests to external domains.

To bypass this, you can use a public CORS proxy like allorigins.win to fetch the URL and return the status code via frontend JS:

javascript
async function checkLink(url) {
const proxyUrl = https://api.allorigins.win/get?url=${encodeURIComponent(url)};

try {
const response = await fetch(proxyUrl);
const data = await response.json();
const status = data.status?.http_code || 0;
return status;
} catch (err) {
return 'Error/Timeout';
}
}
Approach 3: Just Use a Fast, No-Signup Tool
Writing scripts is fun, but when I'm in the middle of a deployment, I just want a clean interface to do a quick check.

I got tired of navigating through ad-ridden sites, so I actually built a couple of lightweight tools for my own workflow and put them on Site Tools Box (a directory of tools with absolutely no sign-up required).

For checking a single page's assets (CSS, JS, images), I use their Dead Link Checker. (Link: https://sitetoolsbox.com/tool.php?slug=http_status)

For checking a batch of URLs at once (up to 20 URLs), I use their Bulk HTTP Checker. (Link: https://sitetoolsbox.com/tool.php?slug=http_status_batch)

Best Practices for Handling the Results
Once you have a list of 404s, don't just delete the links blindly.

If the page was deleted: Set up a 301 Redirect to the closest relevant page. This preserves your SEO juice.

If the page is temporarily down (500/502): Don't touch it yet. Wait a day and check again. It might just be a server hiccup.

If the external site is permanently dead: Remove the link and replace it with plain text, or find an alternative source.

Don't let broken links sit on your site. Write a script, run a weekly check, and keep your site's SEO healthy.

How do you handle broken links in your workflow? Do you use a CLI tool or a browser extension? Let me know in the comments below! 💬

Stop Guessing: A Developer's Guide to Demystifying HTTP Status Codes

laing lii — Tue, 20 Oct 2026 00:57:14 +0000

The 2xx Family (Success) These are the ones we like.

200 OK: The standard response for a successful HTTP request. The server did its job.

201 Created: Usually returned after a successful POST request (like creating a new user or posting a new article). The resource was successfully created.

If you are seeing these, your API is working perfectly. Go grab a coffee. ☕

The 3xx Family (Redirection) This is where SEO and infrastructure issues usually begin.

301 Moved Permanently: The resource has been permanently moved to a new URL. Pro Tip: Always use 301s when migrating websites to preserve your Google SEO ranking.

302 Found (Temporary Redirect): The resource is temporarily at a different URL. Don't use this for permanent moves, or search engines will get confused.

The 4xx Family (Client-Side Errors) This means the client (the browser or the developer) messed up.

400 Bad Request: The server cannot process the request due to a client error (e.g., malformed request syntax, invalid JSON).

401 Unauthorized: You are not authenticated. You haven't logged in, or your API token is missing/invalid.

403 Forbidden: You are authenticated, but you don't have permission to access this resource. (e.g., a normal user trying to access an admin panel).

404 Not Found: The classic. The server can't find the requested resource. It's either deleted or the URL is wrong.

The 5xx Family (Server-Side Errors) The server failed to fulfill a valid request. This is where backend developers start sweating.

500 Internal Server Error: The catch-all error. It means the server encountered an unexpected condition. You need to check your backend logs immediately.

502 Bad Gateway: Your reverse proxy (Nginx/Apache) tried to talk to your backend service (PHP-FPM, Node.js), but the backend gave an invalid response. Usually caused by the backend crashing or timing out.

504 Gateway Timeout: Similar to 502, but the backend simply took too long to respond. Your server might be overwhelmed or the script is running an infinite loop.

🛠️ How to Debug Status Codes Quickly (Without Terminal)
When you are building an API or migrating a server, your browser's developer tools are great. But what if you just want to check if a specific URL is returning the right status code right now without opening Postman or writing a cURL command?

I used to open my terminal, type curl -I https://example.com, and read the headers. It works, but it breaks my flow.

To speed this up, I recently started using a lightweight web toolset called Site Tools Box.

They have a free HTTP Status Checker that does exactly one thing, and does it well:

You paste the URL.

It instantly returns the Status Code, Response Headers, and Response Time.

(No, you don't need to sign up, and it doesn't throw 50 ads in your face.)

It’s incredibly useful when I need to quickly verify:

If my 301 redirects are properly configured.

If a website is actually down (is it a 502 or a 504?).

If an API endpoint requires auth (it will return a 401 immediately).

Wrap Up
Never underestimate the power of a status code. It is the fastest way to tell if the problem is in your frontend code, your network configuration, or your backend server.

Next time you get a 502, don't just restart the server blindly. Check the status code, read the headers, and use the right tools to isolate the issue.

Happy debugging! 🚀