<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: capsuloom</title>
    <description>The latest articles on DEV Community by capsuloom (@capsuloom).</description>
    <link>https://dev.to/capsuloom</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4253040%2F6a86c430-8191-49a6-893f-dea4938d2f8b.png</url>
      <title>DEV Community: capsuloom</title>
      <link>https://dev.to/capsuloom</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/capsuloom"/>
    <language>en</language>
    <item>
      <title>How to keep an offline copy of a website you can actually use</title>
      <dc:creator>capsuloom</dc:creator>
      <pubDate>Thu, 19 Nov 2026 05:49:00 +0000</pubDate>
      <link>https://dev.to/capsuloom/how-to-keep-an-offline-copy-of-a-website-you-can-actually-use-36b7</link>
      <guid>https://dev.to/capsuloom/how-to-keep-an-offline-copy-of-a-website-you-can-actually-use-36b7</guid>
      <description>&lt;p&gt;There is a difference between &lt;em&gt;having the files&lt;/em&gt; and &lt;em&gt;having a copy you can use&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;Most people find this out the hard way: you save a page, open it a week later from&lt;br&gt;
&lt;code&gt;file://&lt;/code&gt;, and get a white screen, broken fonts, and links that point at a domain that&lt;br&gt;
no longer resolves. The HTML is there. The site is not.&lt;/p&gt;

&lt;p&gt;This post is about the second thing — making an offline copy that behaves like the&lt;br&gt;
original, and knowing what you actually saved. It applies whether you use a crawler,&lt;br&gt;
a browser extension, or &lt;code&gt;wget&lt;/code&gt;.&lt;/p&gt;
&lt;h2&gt;
  
  
  Why "Save page as" is not enough
&lt;/h2&gt;

&lt;p&gt;A modern page is not one file. It is a small graph:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;the HTML document&lt;/li&gt;
&lt;li&gt;CSS files, which themselves reference fonts and background images via &lt;code&gt;url()&lt;/code&gt; and &lt;code&gt;@import&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;JavaScript files, sometimes loaded from a CDN&lt;/li&gt;
&lt;li&gt;images, some of which are lazy-loaded (&lt;code&gt;data-src&lt;/code&gt;, &lt;code&gt;data-srcset&lt;/code&gt;, IntersectionObserver)&lt;/li&gt;
&lt;li&gt;JSON for embedded data, SVG sprites, favicons, manifests&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;"Save page as → Webpage, Complete" gets some of this, but it flattens the directory&lt;br&gt;
structure into a &lt;code&gt;_files&lt;/code&gt; folder and rewrites links to point there. The result usually&lt;br&gt;
works for one page and falls apart the moment you click anything.&lt;/p&gt;
&lt;h2&gt;
  
  
  What a usable offline copy needs
&lt;/h2&gt;

&lt;p&gt;Five conditions, in order of how often they are skipped:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. Links resolve from &lt;code&gt;file://&lt;/code&gt;.&lt;/strong&gt;&lt;br&gt;
Absolute paths like &lt;code&gt;/assets/app.css&lt;/code&gt; break the moment the document is not served from&lt;br&gt;
a domain. They have to become relative paths, or the copy has to keep a directory&lt;br&gt;
structure that mirrors the original.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. CSS dependencies are followed, not just the stylesheet.&lt;/strong&gt;&lt;br&gt;
If &lt;code&gt;main.css&lt;/code&gt; contains &lt;code&gt;@import "print.css"&lt;/code&gt; or &lt;code&gt;url(../fonts/inter.woff2)&lt;/code&gt;, those are&lt;br&gt;
separate requests. A crawler that only downloads &lt;code&gt;&amp;lt;link rel="stylesheet"&amp;gt;&lt;/code&gt; hrefs will&lt;br&gt;
produce a page with missing fonts and broken icons.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Lazy-loaded images are materialised.&lt;/strong&gt;&lt;br&gt;
The usual pattern stores the real URL in a data attribute and copies it into &lt;code&gt;src&lt;/code&gt; when&lt;br&gt;
the element scrolls into view. An offline copy that keeps the data attribute and drops&lt;br&gt;
the JavaScript shows empty boxes. The fix is not to remove lazy loading from the copy;&lt;br&gt;
it is to keep a small local script that performs the same "set &lt;code&gt;src&lt;/code&gt; on scroll" step.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. External and protocol-relative URLs are left alone.&lt;/strong&gt;&lt;br&gt;
&lt;code&gt;//cdn.example.com/lib.js&lt;/code&gt; and &lt;code&gt;https://fonts.googleapis.com/...&lt;/code&gt; are &lt;em&gt;not&lt;/em&gt; part of the&lt;br&gt;
site you are copying. Rewriting them to local paths produces a copy that silently&lt;br&gt;
differs from the original, and it makes the diff useless later. Leave them external and&lt;br&gt;
accept that those pieces need a network.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;5. There is an index of what was saved.&lt;/strong&gt;&lt;br&gt;
This is the part everyone forgets. Two hundred files in a folder tell you nothing about&lt;br&gt;
which page had which title, which URL it came from, or which pages failed. Without an&lt;br&gt;
index, the archive is write-only.&lt;/p&gt;
&lt;h2&gt;
  
  
  Doing it by hand
&lt;/h2&gt;

&lt;p&gt;You do not need a tool to apply the same rules:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;1. Pick a scope: one section, a fixed depth, or a page list. Unbounded crawls of large
   sites fail halfway and leave you with a mess.
2. Fetch the HTML. If the page only renders with JavaScript, this step cannot work -
   see the limits below.
3. Parse the HTML and collect: &amp;lt;a href&amp;gt; (same origin only), &amp;lt;link rel=stylesheet&amp;gt;,
   &amp;lt;script src&amp;gt;, &amp;lt;img src|data-src|srcset&amp;gt;, &amp;lt;source srcset&amp;gt;, CSS url() and @import.
4. Download into a mirrored directory path, then rewrite same-origin absolute paths to
   relative ones. Keep external URLs untouched.
5. Record, for every file: original URL, page title, H1, meta description, canonical,
   lang, content type, byte size, local path.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Step 5 is what turns a folder into an archive. A CSV with those columns opens in Excel or&lt;br&gt;
Sheets and answers "what is this file, and where did it come from" without opening&lt;br&gt;
anything.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;url,title,h1,description,canonical,lang,kind,bytes,local_file
https://example.com/docs/,Docs - Example,Example API,Docs index,https://example.com/docs/,en,page,18422,example.com/docs/index.html
https://example.com/assets/app.css,,,,,en,asset,9123,example.com/assets/app.css
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;(Real exports usually also carry &lt;code&gt;keywords&lt;/code&gt; and a &lt;code&gt;kind&lt;/code&gt; column that distinguishes page&lt;br&gt;
from asset, which makes filtering in Excel trivial.)&lt;/p&gt;

&lt;p&gt;A flat CSV answers "what is this file". It does not answer "what shape is this site". The&lt;br&gt;
same data as a hierarchy sheet does: &lt;code&gt;TOP&lt;/code&gt;, &lt;code&gt;Level 2&lt;/code&gt; ... &lt;code&gt;Level 10&lt;/code&gt; columns with one row&lt;br&gt;
per folder or page, parent cells merged down their subtree, the CSV columns to the right,&lt;br&gt;
and the full URL as the last column so a row is still clickable. That is the layout people&lt;br&gt;
already use in site-structure documents, and it takes about 150 lines of Go to write an&lt;br&gt;
&lt;code&gt;.xlsx&lt;/code&gt; by hand (a zip with five small XML parts) - no Office install, no library.&lt;/p&gt;

&lt;p&gt;Two small details that save real time later:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Write the CSV as &lt;strong&gt;UTF-8 with BOM&lt;/strong&gt; if the audience uses Excel on Windows. Without the
BOM, Japanese and accented text opens as mojibake.&lt;/li&gt;
&lt;li&gt;Write the map &lt;strong&gt;even if the crawl is stopped early&lt;/strong&gt;. A partial map with a
&lt;code&gt;status=partial&lt;/code&gt; note is far more useful than nothing.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Where this breaks (say it before someone else does)
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Single-page apps.&lt;/strong&gt; If the content only exists after client-side routing and API
calls, a static copy cannot represent it. You can archive the shell, not the app.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Logged-in areas.&lt;/strong&gt; Anything behind auth is out of scope; a copier that logs in is
storing someone's session, which is a bad idea.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Infinite scroll.&lt;/strong&gt; There is no "end" to detect. Use explicit page lists.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Sites you do not own.&lt;/strong&gt; &lt;code&gt;robots.txt&lt;/code&gt; is the floor, not the ceiling. Permission comes
first; the technical part is easy.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Checking the result
&lt;/h2&gt;

&lt;p&gt;Before you trust an archive, do three checks:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Open the entry page from &lt;code&gt;file://&lt;/code&gt; with the network disabled. If it looks right, links
are correct.&lt;/li&gt;
&lt;li&gt;Open two deep pages, not just the home page. Deep pages are where relative-path bugs
live.&lt;/li&gt;
&lt;li&gt;Sort the CSV by &lt;code&gt;bytes&lt;/code&gt;. The largest files are usually videos or accidentally captured
downloads; the zero-byte rows are the failures worth retrying.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  The tool I ended up using
&lt;/h2&gt;

&lt;p&gt;I do this often enough (client sites before a redesign, documentation I have permission&lt;br&gt;
to keep) that I wrote a small Windows tool for it, &lt;strong&gt;SiteMirror&lt;/strong&gt;. It follows the five&lt;br&gt;
rules above: same-origin crawl with depth/page/asset limits, CSS &lt;code&gt;url()&lt;/code&gt; and &lt;code&gt;@import&lt;/code&gt;&lt;br&gt;
following, a local lazy-load shim, untouched external URLs, and after every crawl it&lt;br&gt;
writes the map as a filterable offline HTML tree, Markdown, the CSV described here, and&lt;br&gt;
an Excel hierarchy sheet (one row per folder or page, parent cells merged, full URL in&lt;br&gt;
the last column) that lists only pages which were actually saved - PDFs and downloads&lt;br&gt;
are detected by their bytes and stay in the CSV instead of filling the tree.&lt;/p&gt;

&lt;p&gt;It is a single &lt;code&gt;.exe&lt;/code&gt;, no installer, no account, no telemetry, and the output is plain&lt;br&gt;
files you can diff or archive. There is no free edition, so the honest advice is: try&lt;br&gt;
&lt;code&gt;wget --mirror&lt;/code&gt; first, and if the part you are missing is the index and the link&lt;br&gt;
rewriting, that is what it solves. $9 one-time, no subscription.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://capsuloom.itch.io/sitemirror?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=offline-copy" rel="noopener noreferrer"&gt;SiteMirror on itch.io&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I built the tool linked above. The rest of the post is the process I use&lt;br&gt;
regardless of which tool does the fetching.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>tooling</category>
      <category>productivity</category>
      <category>html</category>
    </item>
  </channel>
</rss>
