<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Ryan</title>
    <description>The latest articles on DEV Community by Ryan (@ryan_e200dd10ede43c8fc2e4).</description>
    <link>https://dev.to/ryan_e200dd10ede43c8fc2e4</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3818554%2F2d7b5e3d-0c0a-48a3-9ac8-ddbb9458f478.png</url>
      <title>DEV Community: Ryan</title>
      <link>https://dev.to/ryan_e200dd10ede43c8fc2e4</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/ryan_e200dd10ede43c8fc2e4"/>
    <language>en</language>
    <item>
      <title>How to Add 'Download All as ZIP' to Your SaaS in 30 Minutes</title>
      <dc:creator>Ryan</dc:creator>
      <pubDate>Tue, 17 Mar 2026 05:36:59 +0000</pubDate>
      <link>https://dev.to/ryan_e200dd10ede43c8fc2e4/how-to-add-download-all-as-zip-to-your-saas-in-30-minutes-5a</link>
      <guid>https://dev.to/ryan_e200dd10ede43c8fc2e4/how-to-add-download-all-as-zip-to-your-saas-in-30-minutes-5a</guid>
      <description>&lt;p&gt;Your users uploaded 200 photos. Now they want to download them all. What do you do?&lt;/p&gt;

&lt;p&gt;The naive approach — loop through files, zip them on your server, serve the result — falls apart fast. Memory spikes with large files. Egress fees add up. You need temp storage, cleanup jobs, and error handling for partial failures.&lt;/p&gt;

&lt;p&gt;I hit this exact wall building a file-sharing service that's now processed &lt;strong&gt;550K+ files and 10TB+&lt;/strong&gt; of archives. After weeks of wrestling with ZIP64, streaming, and Cloudflare Workers' 128MB memory limit, I turned my solution into an API. Here's how you can skip that pain entirely.&lt;/p&gt;




&lt;h2&gt;
  
  
  The 30-minute version
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Step 1: Get an API key
&lt;/h3&gt;

&lt;p&gt;Sign up at &lt;a href="https://eazip.io?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=download-all" rel="noopener noreferrer"&gt;eazip.io&lt;/a&gt; (free tier, no credit card). Grab your API key from the dashboard.&lt;/p&gt;

&lt;h3&gt;
  
  
  Step 2: Collect your file URLs
&lt;/h3&gt;

&lt;p&gt;You already have these — they're in your database. S3 presigned URLs, R2 public URLs, any HTTPS endpoint that returns a file.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;fileUrls&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;query&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;SELECT file_url, original_filename FROM uploads WHERE project_id = ?&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;projectId&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Step 3: One API call
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;https://api.eazip.io/jobs&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;method&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;POST&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;X-API-Key&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;process&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;env&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;EAZIP_API_KEY&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Content-Type&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;application/json&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="na"&gt;body&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;JSON&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;stringify&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
    &lt;span class="na"&gt;files&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;fileUrls&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;map&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;f&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;({&lt;/span&gt;
      &lt;span class="na"&gt;url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;f&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;file_url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="na"&gt;filename&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;f&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;original_filename&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;})),&lt;/span&gt;
  &lt;span class="p"&gt;}),&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;job_id&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Step 4: Poll for completion and redirect
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;waitForZip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;jobId&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;MAX_ATTEMPTS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;60&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="nx"&gt;MAX_ATTEMPTS&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;res&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="err"&gt;\&lt;/span&gt;&lt;span class="s2"&gt;`https://api.eazip.io/jobs/&lt;/span&gt;&lt;span class="se"&gt;\$&lt;/span&gt;&lt;span class="s2"&gt;{jobId}&lt;/span&gt;&lt;span class="se"&gt;\`&lt;/span&gt;&lt;span class="s2"&gt;, {
      headers: { 'X-API-Key': process.env.EAZIP_API_KEY },
    });
    const { job } = await res.json();

    if (job.status === 'completed') return job.download_url;
    if (job.status === 'failed') throw new Error('ZIP job failed');

    await new Promise(r =&amp;gt; setTimeout(r, 2000)); // wait 2s
  }
  throw new Error('ZIP job timed out');
}

// In your route handler:
const downloadUrl = await waitForZip(job_id);
res.redirect(downloadUrl);
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's it. Your users click "Download All", get a ZIP a few seconds later.&lt;/p&gt;




&lt;h2&gt;
  
  
  What you didn't have to build
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;ZIP64 support&lt;/strong&gt; — files over 4GB just work&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Streaming&lt;/strong&gt; — constant memory regardless of archive size&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Error recovery&lt;/strong&gt; — if file #500 fails, the job retries from a checkpoint&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Temp storage cleanup&lt;/strong&gt; — signed URLs expire automatically&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Egress optimization&lt;/strong&gt; — zero egress fees on Cloudflare's network&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  When this makes sense
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;SaaS with user-uploaded files&lt;/strong&gt; — "Download all attachments" in a support ticket, bulk photo export from a gallery&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;E-commerce&lt;/strong&gt; — product image packs, digital goods delivery&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Internal tools&lt;/strong&gt; — compliance teams exporting 6 months of audit logs, database backups&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  When it doesn't
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;If you only serve a handful of small files — just zip them in memory&lt;/li&gt;
&lt;li&gt;If you need real-time streaming to the browser — this is async (job → download URL)&lt;/li&gt;
&lt;li&gt;If you need custom compression settings — eazip uses STORE (no compression) for speed&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://eazip.io?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=download-all" rel="noopener noreferrer"&gt;eazip.io&lt;/a&gt; — free tier: 60 GB-days/month, no credit card.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Building an export feature? I'd love to hear about your use case — drop a comment or reach out.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>tutorial</category>
      <category>javascript</category>
      <category>node</category>
    </item>
    <item>
      <title>3 Ways to ZIP Files Stored on Cloudflare R2</title>
      <dc:creator>Ryan</dc:creator>
      <pubDate>Sun, 15 Mar 2026 05:05:18 +0000</pubDate>
      <link>https://dev.to/ryan_e200dd10ede43c8fc2e4/3-ways-to-zip-files-stored-on-cloudflare-r2-2b7d</link>
      <guid>https://dev.to/ryan_e200dd10ede43c8fc2e4/3-ways-to-zip-files-stored-on-cloudflare-r2-2b7d</guid>
      <description>&lt;p&gt;You have files sitting in Cloudflare R2 and a user just clicked "Download All." Now what?&lt;/p&gt;

&lt;p&gt;R2 doesn't have a built-in "zip these objects" operation. You need to figure it out yourself. After building a file processing API that has archived &lt;strong&gt;550K+ files and 10TB+&lt;/strong&gt; on R2, here are the three approaches I've found — each with very different trade-offs.&lt;/p&gt;




&lt;h2&gt;
  
  
  Approach 1: Pull to a Server and Use the &lt;code&gt;zip&lt;/code&gt; Command
&lt;/h2&gt;

&lt;p&gt;The most straightforward approach. Spin up a container (Fargate, Cloud Run, EC2, etc.), pull the files from R2, and run the good old &lt;code&gt;zip&lt;/code&gt; command.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Pull files from R2 and zip them&lt;/span&gt;
aws s3 &lt;span class="nb"&gt;sync &lt;/span&gt;s3://your-r2-bucket/files/ /tmp/files/ &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--endpoint-url&lt;/span&gt; https://&amp;lt;account-id&amp;gt;.r2.cloudflarestorage.com
zip &lt;span class="nt"&gt;-r&lt;/span&gt; /tmp/archive.zip /tmp/files/
&lt;span class="c"&gt;# Upload the archive back to R2 or serve it directly&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Dead simple.&lt;/strong&gt; &lt;code&gt;zip&lt;/code&gt; is battle-tested and handles everything — compression, large files, edge cases&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No ZIP implementation needed.&lt;/strong&gt; You're not writing any ZIP logic yourself&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Full control&lt;/strong&gt; over the server environment, compression level, file structure&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Disk and memory bound.&lt;/strong&gt; You need enough disk space to hold all the files + the archive. For large archives (10GB+), this means provisioning beefy instances&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Egress costs if your server isn't on Cloudflare.&lt;/strong&gt; Pulling files from R2 is free (R2 has zero egress fees), but once the ZIP lives on your AWS/GCP server, &lt;em&gt;serving it to the user&lt;/em&gt; or &lt;em&gt;uploading it back to R2&lt;/em&gt; means paying your cloud provider's egress fees&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Infrastructure overhead.&lt;/strong&gt; You need to manage containers, queues, autoscaling, and cleanup. It's no longer "just zip these files" — it's a whole pipeline&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Not real-time.&lt;/strong&gt; The user has to wait for the entire download + zip + upload cycle before they can start downloading&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Best for:&lt;/strong&gt; Batch processing, internal tooling, or when you already have server infrastructure and egress costs aren't a concern.&lt;/p&gt;




&lt;h2&gt;
  
  
  Approach 2: Stream a ZIP in a Cloudflare Worker
&lt;/h2&gt;

&lt;p&gt;Instead of pulling files to a server, you can stream a ZIP archive directly from a Cloudflare Worker. Libraries like &lt;a href="https://stuk.github.io/jszip/" rel="noopener noreferrer"&gt;JSZip&lt;/a&gt; and &lt;a href="https://github.com/101arrowz/fflate" rel="noopener noreferrer"&gt;fflate&lt;/a&gt; support streaming, so you can pipe R2 objects through them without buffering entire files.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Using a streaming ZIP library in a Worker&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;ZipWriter&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;some-streaming-zip-lib&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;export&lt;/span&gt; &lt;span class="k"&gt;default&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="nf"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;request&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;env&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;keys&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;file1.pdf&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;file2.jpg&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;file3.csv&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;];&lt;/span&gt;

    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;readable&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;writable&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;TransformStream&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;zipWriter&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;ZipWriter&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;writable&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;async &lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;key&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;keys&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;obj&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;env&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;BUCKET&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;key&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;zipWriter&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;addStream&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;obj&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
      &lt;span class="p"&gt;}&lt;/span&gt;
      &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;zipWriter&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;close&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
    &lt;span class="p"&gt;})();&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Response&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;readable&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="na"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Content-Type&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;application/zip&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;});&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This works well for simple cases. But things get complicated fast when you need production-level reliability.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Constant memory usage.&lt;/strong&gt; Only one file chunk in memory at a time&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Zero egress fees.&lt;/strong&gt; R2 → Worker → client, all within Cloudflare's network&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Streaming.&lt;/strong&gt; Client starts downloading immediately, no waiting for the full archive&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Horizontal scaling.&lt;/strong&gt; Workers handle many concurrent requests naturally — high throughput isn't a problem&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Per-archive size is limited.&lt;/strong&gt; Workers have a 15-minute wall clock limit and a subrequest cap per invocation, so large archives (tens of GB+) won't complete in a single run&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Error handling is brutal.&lt;/strong&gt; If file #500 of 1000 fails mid-stream, you've already sent 499 files to the client. The HTTP response is in-flight — you can't restart or send an error code. The client just gets a truncated ZIP&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Checkpoint/resume requires a custom ZIP implementation.&lt;/strong&gt; To work around the wall clock limit, you'd need to serialize mid-stream state — CRC32 computations, byte offsets, multipart upload progress — and resume exactly where you left off. At that point, off-the-shelf libraries won't cut it, and you're deep in the ZIP spec implementing local file headers, data descriptors, central directory, and ZIP64 extensions yourself&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Best for:&lt;/strong&gt; Small-to-medium archives where the 15-minute wall clock limit isn't a concern. For anything larger, you'll need either serious engineering investment or a different approach.&lt;/p&gt;




&lt;h2&gt;
  
  
  Approach 3: Use a ZIP API Service
&lt;/h2&gt;

&lt;p&gt;Instead of building and maintaining streaming ZIP infrastructure yourself, use an API that handles it for you. You send a list of R2 URLs (or presigned URLs), and get back a ZIP.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-X&lt;/span&gt; POST https://api.eazip.io/jobs &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s2"&gt;"Authorization: Bearer YOUR_API_KEY"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s2"&gt;"Content-Type: application/json"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{
    "files": [
      { "url": "https://your-bucket.r2.dev/file1.pdf" },
      { "url": "https://your-bucket.r2.dev/file2.jpg" },
      { "url": "https://your-bucket.r2.dev/file3.csv" }
    ]
  }'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The service handles streaming, CRC32, ZIP64, error recovery, and checkpoint/resume — all the hard parts — so you don't have to.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;One API call.&lt;/strong&gt; No ZIP implementation to build or maintain&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Handles edge cases&lt;/strong&gt; you don't want to think about (ZIP64 for large files, Data Descriptors, checkpoint/resume for failures)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Zero egress&lt;/strong&gt; if the service also runs on Cloudflare's network&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Scales to 5,000+ files per archive, up to 50GB&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Third-party dependency.&lt;/strong&gt; You're relying on an external service&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cost.&lt;/strong&gt; Free tier exists but large-scale usage has costs&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Less control&lt;/strong&gt; over ZIP structure details&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Best for:&lt;/strong&gt; Teams that want ZIP functionality without building ZIP infrastructure. Ship in an afternoon instead of a sprint.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Full disclosure: I built &lt;a href="https://eazip.io?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=3ways-r2" rel="noopener noreferrer"&gt;Eazip&lt;/a&gt; because I went through Approach 2 myself and realized most teams shouldn't have to.&lt;/em&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  Comparison
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Server + zip&lt;/th&gt;
&lt;th&gt;Stream in Worker&lt;/th&gt;
&lt;th&gt;ZIP API&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Memory&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;O(total size)&lt;/td&gt;
&lt;td&gt;O(chunk size)&lt;/td&gt;
&lt;td&gt;N/A&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Egress cost&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Depends on server location&lt;/td&gt;
&lt;td&gt;$0&lt;/td&gt;
&lt;td&gt;$0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Max archive size&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Limited by disk&lt;/td&gt;
&lt;td&gt;Limited by wall clock (15 min)&lt;/td&gt;
&lt;td&gt;50GB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Implementation time&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Hours&lt;/td&gt;
&lt;td&gt;Hours–Weeks&lt;/td&gt;
&lt;td&gt;Minutes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Maintenance&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Medium (infra)&lt;/td&gt;
&lt;td&gt;High (ZIP spec edge cases)&lt;/td&gt;
&lt;td&gt;None&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Error recovery&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Easy (retry all)&lt;/td&gt;
&lt;td&gt;Hard (mid-stream failures)&lt;/td&gt;
&lt;td&gt;Built-in&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  Which Should You Pick?
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Have server infrastructure and don't mind egress costs?&lt;/strong&gt; → Approach 1. Pull the files, run &lt;code&gt;zip&lt;/code&gt;, and move on. Just keep in mind that egress fees add up fast at scale — especially if you're on AWS or GCP.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Want to stay on Cloudflare's network?&lt;/strong&gt; → Approach 2 works great for small-to-medium archives. But once you hit the wall clock limit or need error recovery, complexity escalates quickly.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Want to ship the feature and move on?&lt;/strong&gt; → Approach 3. One API call, zero infrastructure, zero egress. You can be done in an afternoon.&lt;/p&gt;

&lt;p&gt;The reality is that ZIP archiving looks simple until it isn't. What starts as "just zip these files" turns into managing disk space, egress bills, wall clock limits, or mid-stream error recovery — depending on which approach you choose. I learned this the hard way after archiving 10TB+ of files.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;What's your approach? Have you tried something different? Let me know in the comments.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>cloudflare</category>
      <category>r2</category>
      <category>webdev</category>
      <category>serverless</category>
    </item>
    <item>
      <title>Zero Egress: Why I Chose Cloudflare Workers + R2 Over AWS for a File Processing API</title>
      <dc:creator>Ryan</dc:creator>
      <pubDate>Thu, 12 Mar 2026 08:24:10 +0000</pubDate>
      <link>https://dev.to/ryan_e200dd10ede43c8fc2e4/zero-egress-why-i-chose-cloudflare-workers-r2-over-aws-for-a-file-processing-api-2640</link>
      <guid>https://dev.to/ryan_e200dd10ede43c8fc2e4/zero-egress-why-i-chose-cloudflare-workers-r2-over-aws-for-a-file-processing-api-2640</guid>
      <description>&lt;p&gt;Most cloud cost calculators focus on compute and storage. They rarely mention the line item that dominated my bill: &lt;strong&gt;egress fees&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;I run &lt;a href="https://eazip.io?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=zero-egress" rel="noopener noreferrer"&gt;eazip.io&lt;/a&gt;, an API that takes an array of URLs and returns a ZIP archive. The service fetches remote files, streams them into a ZIP64 archive, and stores the result for download. In production, it has processed 550K+ files totaling over 10 TB of archived data.&lt;/p&gt;

&lt;p&gt;When I was choosing where to build this, I modeled the costs across AWS, GCP, Azure, and Cloudflare. The results changed my entire architecture.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Workload
&lt;/h2&gt;

&lt;p&gt;Here's what happens on every API call:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Fetch&lt;/strong&gt; remote files (1 to 5,000 URLs per job)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Stream&lt;/strong&gt; them into a ZIP64 archive (up to 50 GB per job)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Store&lt;/strong&gt; the archive for the customer to download&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Serve&lt;/strong&gt; the download (this is where egress hits)&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The critical insight: every byte of the final archive gets transferred &lt;strong&gt;twice&lt;/strong&gt; — once during creation (fetch → process → store) and once during download (store → customer). For a file processing API, egress isn't a rounding error. It's the dominant cost.&lt;/p&gt;

&lt;h2&gt;
  
  
  Egress Pricing Comparison
&lt;/h2&gt;

&lt;p&gt;Here's what the major providers charge for data transfer out to the internet (as of early 2026, standard pricing tiers):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Provider&lt;/th&gt;
&lt;th&gt;Egress Cost (per GB)&lt;/th&gt;
&lt;th&gt;Free Tier&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;AWS S3&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;$0.09/GB (first 10 TB)&lt;/td&gt;
&lt;td&gt;100 GB/month&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;GCP Cloud Storage&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;$0.12/GB (first 1 TB)&lt;/td&gt;
&lt;td&gt;Free tier egress limited&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Azure Blob Storage&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;$0.087/GB (first 5 GB free)&lt;/td&gt;
&lt;td&gt;5 GB/month&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Cloudflare R2&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;$0.00/GB&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Unlimited&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;That's not a typo. R2 charges zero egress. You pay only for storage ($0.015/GB/month) and operations (Class A: $4.50/million, Class B: $0.36/million).&lt;/p&gt;

&lt;h2&gt;
  
  
  Scenario 1: The Single Large Job
&lt;/h2&gt;

&lt;p&gt;Let's model a realistic scenario: a customer creates a 10 GB ZIP archive and downloads it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;AWS S3:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Storage: negligible (temporary)&lt;/li&gt;
&lt;li&gt;Egress: 10 GB × $0.09 = &lt;strong&gt;$0.90&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;If they download it 10 times (sharing with team): &lt;strong&gt;$9.00&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Cloudflare R2:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Storage: negligible&lt;/li&gt;
&lt;li&gt;Egress: 10 GB × $0.00 = &lt;strong&gt;$0.00&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;10 downloads: still &lt;strong&gt;$0.00&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;One job, one customer, $9 difference. Now multiply that across hundreds of customers.&lt;/p&gt;

&lt;h2&gt;
  
  
  Scenario 2: Monthly Sustained Usage
&lt;/h2&gt;

&lt;p&gt;A more realistic model: 1,000 jobs per month, average 1 GB per archive, each downloaded twice.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Cost Component&lt;/th&gt;
&lt;th&gt;AWS S3&lt;/th&gt;
&lt;th&gt;Cloudflare R2&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Storage (temp, ~100 GB avg)&lt;/td&gt;
&lt;td&gt;$2.30&lt;/td&gt;
&lt;td&gt;$1.50&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Egress (2 TB out)&lt;/td&gt;
&lt;td&gt;$180.00&lt;/td&gt;
&lt;td&gt;$0.00&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Operations&lt;/td&gt;
&lt;td&gt;~$0.50&lt;/td&gt;
&lt;td&gt;~$2.00&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Monthly Total&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;$182.80&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;$3.50&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Egress is &lt;strong&gt;98.5%&lt;/strong&gt; of the AWS bill. The actual compute and storage costs are almost identical between providers. The entire cost difference comes from one line item.&lt;/p&gt;

&lt;h2&gt;
  
  
  Scenario 3: What Happens at Scale
&lt;/h2&gt;

&lt;p&gt;This is where it gets scary. Let's say the service grows to 10,000 jobs/month (20 TB egress):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Scale&lt;/th&gt;
&lt;th&gt;AWS Egress&lt;/th&gt;
&lt;th&gt;R2 Egress&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;2 TB/month&lt;/td&gt;
&lt;td&gt;$180&lt;/td&gt;
&lt;td&gt;$0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;20 TB/month&lt;/td&gt;
&lt;td&gt;$1,740&lt;/td&gt;
&lt;td&gt;$0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;200 TB/month&lt;/td&gt;
&lt;td&gt;$16,200&lt;/td&gt;
&lt;td&gt;$0&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;On AWS, egress grows linearly with success. Every new customer, every additional download directly increases your largest cost. On R2, success doesn't punish you.&lt;/p&gt;

&lt;p&gt;For an API business charging $9-29/month, an egress bill of $1,740 makes the unit economics impossible.&lt;/p&gt;

&lt;h2&gt;
  
  
  "Just Put a CDN in Front of S3"
&lt;/h2&gt;

&lt;p&gt;This is the first objection everyone raises. And for many workloads, it's the right answer. But for a ZIP archive API, it doesn't work.&lt;/p&gt;

&lt;p&gt;Every archive is &lt;strong&gt;unique&lt;/strong&gt;. Customer A's ZIP contains different files than Customer B's. The cache hit rate for unique, per-customer archives is effectively &lt;strong&gt;zero&lt;/strong&gt;. CloudFront, Fastly, or any CDN will just pass through to S3, and you still pay S3 egress to the CDN.&lt;/p&gt;

&lt;p&gt;CDN caching works when many users request the same content. When every response is unique — which is the case for any file processing, transformation, or generation API — the CDN is just an expensive proxy.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Hidden Costs You Don't See Coming
&lt;/h2&gt;

&lt;p&gt;Egress isn't the only surprise on an AWS bill for this kind of workload:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;NAT Gateway fees&lt;/strong&gt;: If your processing runs in a VPC (ECS, Lambda in VPC), data passing through a NAT Gateway costs $0.045/GB on top of regular egress. For a service processing terabytes, this doubles your data transfer costs.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Cross-region transfer&lt;/strong&gt;: If your compute is in us-east-1 but your customer's files are elsewhere, you pay $0.01-0.02/GB for inter-region transfer — before any processing even begins.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Temporary storage I/O&lt;/strong&gt;: EBS volumes have I/O costs. If you're writing temporary files during ZIP creation, those IOPS add up.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Lambda execution time&lt;/strong&gt;: A 10 GB ZIP takes minutes to create. At Lambda pricing, long-running file processing jobs are expensive. And Lambda has a 15-minute timeout, so you need step functions or ECS for large jobs.&lt;/p&gt;

&lt;p&gt;On Cloudflare:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Workers → R2: &lt;strong&gt;free&lt;/strong&gt; (same network)&lt;/li&gt;
&lt;li&gt;R2 → customer: &lt;strong&gt;free&lt;/strong&gt; (zero egress)&lt;/li&gt;
&lt;li&gt;No NAT Gateway (Workers run at edge)&lt;/li&gt;
&lt;li&gt;No cross-region transfer (R2 is globally distributed)&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The Architecture That Emerged
&lt;/h2&gt;

&lt;p&gt;The zero-egress model didn't just save money — it shaped the architecture:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Customer Request
      ↓
  [Cloudflare Worker]
      ↓ (fetch remote files)
  [Stream → ZIP64]
      ↓ (multipart upload, free)
  [Cloudflare R2]
      ↓ (download, free)
  Customer
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Every data transfer in this pipeline is free. The only costs are:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Workers compute&lt;/strong&gt;: $0.30 per million requests + CPU time&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;R2 storage&lt;/strong&gt;: $0.015/GB/month&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;R2 operations&lt;/strong&gt;: pennies per thousand&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Total monthly bill for 550K+ files processed and 10 TB+ archived: &lt;strong&gt;single-digit dollars&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The same workload on AWS would be estimated at $200-400/month minimum, scaling to $1,000+ as usage grows. And that's before the NAT Gateway surprise.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Tradeoffs
&lt;/h2&gt;

&lt;p&gt;Cloudflare Workers aren't free of constraints. I want to be honest about what you give up:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;128 MB memory limit&lt;/strong&gt;: You can't buffer large files in memory. I had to build a fully streaming ZIP64 implementation. (I wrote about the technical details in my &lt;a href="https://dev.to/ryan_e200dd10ede43c8fc2e4/how-i-built-streaming-zip64-on-cloudflare-workers-128mb-ram-no-filesystem-3aaf"&gt;previous article&lt;/a&gt;.)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;CPU time limits&lt;/strong&gt;: Workers get seconds of CPU time, not minutes. For large jobs, I built a checkpoint/resume system that serializes state to R2 and picks up where it left off.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;No filesystem&lt;/strong&gt;: V8 isolates don't have &lt;code&gt;fs&lt;/code&gt;. Everything must be streamed or held in memory.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Vendor lock-in&lt;/strong&gt;: Your code is tied to Cloudflare's runtime. Workers are not standard Node.js — they're a subset of Web APIs. Migrating to AWS Lambda would require significant rewriting.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Smaller ecosystem&lt;/strong&gt;: Fewer libraries, fewer examples, smaller community compared to AWS Lambda.&lt;/p&gt;

&lt;p&gt;For my use case — a data-heavy API where egress dominates costs — these tradeoffs were worth it. For a CRUD app with minimal data transfer, the cost difference wouldn't matter, and AWS/GCP's richer ecosystem might be more valuable.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Decision Framework
&lt;/h2&gt;

&lt;p&gt;Here's how I'd think about it for your workload:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Choose Cloudflare Workers + R2 when:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Data transfer is a significant portion of your workload&lt;/li&gt;
&lt;li&gt;You serve large files or many downloads&lt;/li&gt;
&lt;li&gt;Each response is unique (low CDN cache hit rate)&lt;/li&gt;
&lt;li&gt;You need global low-latency without multi-region setup&lt;/li&gt;
&lt;li&gt;Cost predictability matters (no surprise egress bills)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Stay with AWS/GCP when:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;You need specific managed services (RDS, SQS, etc.)&lt;/li&gt;
&lt;li&gt;Your workload is compute-heavy, not data-heavy&lt;/li&gt;
&lt;li&gt;Egress is a small fraction of total cost&lt;/li&gt;
&lt;li&gt;You need the mature ecosystem and tooling&lt;/li&gt;
&lt;li&gt;Your team already has deep AWS/GCP expertise&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Quick Cost Calculator
&lt;/h2&gt;

&lt;p&gt;For a rough estimate of your potential savings, here's a simple formula:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Monthly egress savings = Monthly data out (GB) × $0.09

If you also use NAT Gateway:
Add: Monthly data through NAT (GB) × $0.045
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If that number is more than your compute costs, egress is your dominant expense and it's worth evaluating R2.&lt;/p&gt;

&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;

&lt;p&gt;After 6+ months in production:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;550K+ files processed&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;10 TB+ total data archived&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Monthly infrastructure cost: single-digit dollars&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Estimated AWS equivalent: $200-400/month&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Zero egress bills, ever&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The service handles jobs ranging from 1 file to 5,000 files, with archives up to 50 GB. The checkpoint/resume system means large jobs complete reliably despite Worker timeout limits. And the cost stays flat regardless of how many times customers download their archives.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;

&lt;p&gt;If you're building something that needs to create ZIP archives from remote URLs, &lt;a href="https://eazip.io?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=zero-egress" rel="noopener noreferrer"&gt;eazip.io&lt;/a&gt; wraps this entire architecture into a single API call. Free tier available, no credit card required.&lt;/p&gt;

&lt;p&gt;But even if you don't need a ZIP API — if you're building any data-heavy service, model your egress costs before choosing a provider. It might be the most important line item on your bill.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Questions about the cost analysis or the Cloudflare Workers architecture? Drop a comment — happy to share more specific numbers.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>aws</category>
      <category>serverless</category>
      <category>webdev</category>
      <category>cloudflarechallenge</category>
    </item>
    <item>
      <title>How I Built Streaming ZIP64 on Cloudflare Workers (128MB RAM, No Filesystem)</title>
      <dc:creator>Ryan</dc:creator>
      <pubDate>Wed, 11 Mar 2026 14:08:08 +0000</pubDate>
      <link>https://dev.to/ryan_e200dd10ede43c8fc2e4/how-i-built-streaming-zip64-on-cloudflare-workers-128mb-ram-no-filesystem-3aaf</link>
      <guid>https://dev.to/ryan_e200dd10ede43c8fc2e4/how-i-built-streaming-zip64-on-cloudflare-workers-128mb-ram-no-filesystem-3aaf</guid>
      <description>&lt;p&gt;I needed to ZIP 1,000+ files totaling 10 GB stored in Cloudflare R2. The catch: I had to do it on Cloudflare Workers -- 128 MB memory, no filesystem, no long-running processes. Every existing solution I tried failed. So I built my own streaming ZIP64 archiver from scratch.&lt;/p&gt;

&lt;p&gt;This is the story of how that archiver became &lt;a href="https://eazip.io?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=zip64" rel="noopener noreferrer"&gt;eazip.io&lt;/a&gt;, and the technical decisions behind it.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Problem
&lt;/h2&gt;

&lt;p&gt;I run a file-sharing service. Users upload files to Cloudflare R2, and sometimes they want to download hundreds or thousands of files at once. The obvious answer is a ZIP file. The not-so-obvious part is where to create it.&lt;/p&gt;

&lt;p&gt;The constraints were brutal:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;128 MB memory limit&lt;/strong&gt; per Worker invocation&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No filesystem&lt;/strong&gt; -- Workers run in V8 isolates, not containers&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Subrequest limits&lt;/strong&gt; -- every &lt;code&gt;fetch()&lt;/code&gt;, R2 read, and database call counts toward a per-invocation cap&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;CPU time limits&lt;/strong&gt; -- you get seconds, not minutes&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Files up to 4 GB+&lt;/strong&gt; -- meaning ZIP64 is mandatory, not optional&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;And one more thing: I wanted to avoid egress fees entirely. If I sent R2 data through AWS ECS to build the ZIP, I'd pay AWS egress on every byte. Cloudflare Workers talking to Cloudflare R2 costs nothing in bandwidth. That economic constraint shaped the entire architecture.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Existing Solutions Failed
&lt;/h2&gt;

&lt;p&gt;I evaluated every ZIP library I could find for the Workers environment.&lt;/p&gt;

&lt;h3&gt;
  
  
  zip.js -- Almost Perfect
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://github.com/gildas-lormeau/zip.js" rel="noopener noreferrer"&gt;zip.js&lt;/a&gt; supports ZIP64 and streaming. It looked ideal. But internally it chains &lt;code&gt;TransformStream&lt;/code&gt; instances using &lt;code&gt;pipeTo()&lt;/code&gt;, and Cloudflare's &lt;code&gt;workerd&lt;/code&gt; runtime doesn't implement inter-TransformStream &lt;code&gt;pipeTo()&lt;/code&gt;. I tried shimming &lt;code&gt;globalThis.TransformStream&lt;/code&gt; with &lt;code&gt;IdentityTransformStream&lt;/code&gt; and manually implementing &lt;code&gt;pipeTo()&lt;/code&gt; in a wrapper. Memory usage exploded. The fundamental issue is that Workers' &lt;code&gt;TransformStream&lt;/code&gt; lacks backpressure propagation across chained transforms.&lt;/p&gt;

&lt;h3&gt;
  
  
  JSZip -- Wrong Model
&lt;/h3&gt;

&lt;p&gt;JSZip buffers the entire archive in memory before producing output. With a 128 MB limit and 10 GB of input files, that's a non-starter. It also doesn't support ZIP64 writes.&lt;/p&gt;

&lt;h3&gt;
  
  
  fflate -- No ZIP64
&lt;/h3&gt;

&lt;p&gt;fflate is fast and Workers-compatible, but it doesn't support ZIP64. Any archive over 4 GB (or with entries over 4 GB) is out.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Decision
&lt;/h3&gt;

&lt;p&gt;None of them worked. I needed to write a ZIP64 archiver from scratch, designed specifically for the Workers constraint model.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architecture: Streaming ZIP64 Without a Filesystem
&lt;/h2&gt;

&lt;p&gt;The key insight is that the ZIP format is sequential enough to stream -- if you make the right tradeoffs.&lt;/p&gt;

&lt;h3&gt;
  
  
  How ZIP Files Work (The Short Version)
&lt;/h3&gt;

&lt;p&gt;A ZIP file is structured like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[Local File Header + File Data] x N
[Central Directory]
[End of Central Directory]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The Central Directory at the end is an index referencing all files. This is why ZIP files can be read without scanning the entire archive -- readers jump to the end first.&lt;/p&gt;

&lt;p&gt;For our purposes, this structure is actually a gift: we can &lt;strong&gt;stream Local File Headers and file data sequentially&lt;/strong&gt;, accumulate metadata in memory, then write the Central Directory at the end. The metadata per file is small (filename, offset, CRC32, sizes), so even 5,000 files fit comfortably in memory.&lt;/p&gt;

&lt;h3&gt;
  
  
  Data Descriptors: Write Now, Fill In Later
&lt;/h3&gt;

&lt;p&gt;Normally, a Local File Header must contain the CRC32 and compressed size &lt;em&gt;before&lt;/em&gt; the file data. That means you'd need to read the entire file first to compute these values, then write the header. With a 4 GB file, that's obviously impossible in 128 MB of RAM.&lt;/p&gt;

&lt;p&gt;The solution: &lt;strong&gt;Data Descriptors&lt;/strong&gt; (GPBF bit 3). This ZIP feature lets you write placeholder values in the header and append the real CRC32 and sizes &lt;em&gt;after&lt;/em&gt; the file data:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[Local File Header (CRC=0, size=0)]  -- write immediately
[File Data stream]                    -- stream through
[Data Descriptor (real CRC, size)]   -- write after streaming
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Here's the conceptual flow:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nf"&gt;streamZipEntry&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="nx"&gt;filename&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nx"&gt;sourceStream&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;ReadableStream&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nb"&gt;Uint8Array&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;
&lt;span class="p"&gt;):&lt;/span&gt; &lt;span class="nx"&gt;AsyncGenerator&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nb"&gt;Uint8Array&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="c1"&gt;// Write local file header with zeros for CRC/size&lt;/span&gt;
  &lt;span class="c1"&gt;// GPBF bit 3 signals "data descriptor follows"&lt;/span&gt;
  &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nf"&gt;buildLocalFileHeader&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;filename&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;gpbf&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mh"&gt;0x0008&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;crc32&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;compressedSize&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;uncompressedSize&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="p"&gt;});&lt;/span&gt;

  &lt;span class="c1"&gt;// Stream file data, computing CRC32 on the fly&lt;/span&gt;
  &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;crc&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;size&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="nx"&gt;n&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;reader&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;sourceStream&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getReader&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;

  &lt;span class="k"&gt;while &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;value&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;done&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;reader&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;read&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;done&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;break&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nx"&gt;crc&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;updateCrc32&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;crc&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;value&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="nx"&gt;size&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="nc"&gt;BigInt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;value&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;byteLength&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nx"&gt;value&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c1"&gt;// Pass through without buffering&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="c1"&gt;// Write ZIP64 data descriptor with real values&lt;/span&gt;
  &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nf"&gt;buildZip64DataDescriptor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;crc&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;size&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;size&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;We use STORE mode (no compression). The data passes through the Worker byte-by-byte with zero buffering beyond the current chunk. Peak memory stays well under 128 MB regardless of file size.&lt;/p&gt;

&lt;h3&gt;
  
  
  CRC32 Mid-Computation Serialization
&lt;/h3&gt;

&lt;p&gt;CRC32 is computed incrementally as data flows through. But what happens when a Worker hits its CPU time limit mid-file?&lt;/p&gt;

&lt;p&gt;CRC32 state is just a 32-bit integer. We serialize it along with the byte offset and resume from exactly where we left off:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kr"&gt;interface&lt;/span&gt; &lt;span class="nx"&gt;ZipCheckpoint&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="c1"&gt;// ZIP state&lt;/span&gt;
  &lt;span class="nl"&gt;currentFileIndex&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;number&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nl"&gt;currentFileOffset&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;bigint&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nl"&gt;crc32State&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;number&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c1"&gt;// Just a uint32&lt;/span&gt;
  &lt;span class="nl"&gt;entryMetadata&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;EntryMeta&lt;/span&gt;&lt;span class="p"&gt;[];&lt;/span&gt;

  &lt;span class="c1"&gt;// R2 multipart state&lt;/span&gt;
  &lt;span class="nl"&gt;uploadId&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nl"&gt;uploadedParts&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;partNumber&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;number&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nl"&gt;etag&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt; &lt;span class="p"&gt;}[];&lt;/span&gt;
  &lt;span class="nl"&gt;tailBuffer&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;Uint8Array&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c1"&gt;// &amp;lt; 5 MB leftover&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This checkpoint gets serialized to R2 as JSON (with the tail buffer stored separately). When a new Worker invocation picks up the job, it deserializes the checkpoint and resumes streaming from the exact byte where the previous invocation stopped.&lt;/p&gt;

&lt;h3&gt;
  
  
  5 MB Boundary Buffering for R2 Multipart
&lt;/h3&gt;

&lt;p&gt;R2 (and S3) multipart uploads require each part to be at least 5 MB (except the last one). But streaming ZIP data doesn't naturally align to 5 MB boundaries. A Local File Header might be 120 bytes. A tiny file might be 2 KB.&lt;/p&gt;

&lt;p&gt;The solution: a tail buffer. We accumulate bytes until we hit 5 MB, then flush a part:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;flushToR2&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="nx"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;Uint8Array&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;MultipartState&lt;/span&gt;
&lt;span class="p"&gt;):&lt;/span&gt; &lt;span class="nb"&gt;Promise&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="k"&gt;void&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="c1"&gt;// Append to tail buffer&lt;/span&gt;
  &lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;tail&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;concat&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;tail&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;chunk&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

  &lt;span class="c1"&gt;// Flush when we have enough for an R2 part&lt;/span&gt;
  &lt;span class="k"&gt;while &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;tail&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;byteLength&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="nx"&gt;PART_SIZE_MIN&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;part&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;tail&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;PART_SIZE_MIN&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;tail&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;tail&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;PART_SIZE_MIN&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;uploaded&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;r2&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;uploadPart&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
      &lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;uploadId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;nextPartNumber&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nx"&gt;part&lt;/span&gt;
    &lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;uploadedParts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
      &lt;span class="na"&gt;partNumber&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;nextPartNumber&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="na"&gt;etag&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;uploaded&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;etag&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;});&lt;/span&gt;
    &lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;nextPartNumber&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The tail buffer is always under 5 MB, so memory stays bounded.&lt;/p&gt;

&lt;h3&gt;
  
  
  Checkpoint/Resume: State Serialization to R2
&lt;/h3&gt;

&lt;p&gt;Workers can die at any point -- CPU limit, subrequest limit, or infrastructure issues. The system is designed around this assumption.&lt;/p&gt;

&lt;p&gt;The "Checkpoint Authoritative" model:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Stream data, flush R2 parts, accumulate metadata&lt;/li&gt;
&lt;li&gt;At a safe stopping point (e.g., every 128 MB of data processed), serialize the full state to R2&lt;/li&gt;
&lt;li&gt;Update the database with the checkpoint reference&lt;/li&gt;
&lt;li&gt;If the Worker dies before a checkpoint, we replay from the last checkpoint&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The checkpoint is the single source of truth. Any R2 parts uploaded after the last checkpoint are treated as non-existent on resume. This makes the system crash-safe without distributed transactions.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Worker Instance 1:
  [stream 128MB] → checkpoint A → [stream 128MB] → checkpoint B → [dies]

Worker Instance 2 (resume):
  [load checkpoint B] → [stream from where B left off] → ...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A Cron-based monitor watches for stalled jobs (Worker died without updating status) and spawns new Worker instances to resume them.&lt;/p&gt;

&lt;h3&gt;
  
  
  Central Directory: The Finale
&lt;/h3&gt;

&lt;p&gt;After all files are streamed, we write the Central Directory and ZIP64 End of Central Directory records using the metadata we accumulated:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nf"&gt;finalize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="nx"&gt;entries&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;EntryMeta&lt;/span&gt;&lt;span class="p"&gt;[]&lt;/span&gt;
&lt;span class="p"&gt;):&lt;/span&gt; &lt;span class="nx"&gt;AsyncGenerator&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nb"&gt;Uint8Array&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;cdOffset&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;currentOffset&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;entry&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;entries&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nf"&gt;buildCentralDirectoryHeader&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;entry&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;cdSize&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;currentOffset&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="nx"&gt;cdOffset&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="c1"&gt;// ZIP64 End of Central Directory&lt;/span&gt;
  &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nf"&gt;buildZip64EndOfCentralDirectory&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;entries&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;cdSize&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;cdOffset&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nf"&gt;buildZip64Locator&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;cdOffset&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="nx"&gt;cdSize&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="nf"&gt;buildEndOfCentralDirectory&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;entries&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;cdSize&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;cdOffset&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The whole ZIP never exists in memory at once. It flows from source URLs, through the Worker, into R2 parts, and gets assembled into a complete file -- all without ever exceeding a few megabytes of RAM.&lt;/p&gt;

&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;

&lt;p&gt;This architecture has been running in production as the backend for &lt;a href="https://eazip.io?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=zip64" rel="noopener noreferrer"&gt;eazip.io&lt;/a&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;550K+ files processed&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;10 TB+ total data archived&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Up to 5,000 files and 50 GB per job&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Zero egress fees&lt;/strong&gt; (Workers + R2 = no bandwidth charges)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Peak memory well under 128 MB&lt;/strong&gt; regardless of job size&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The checkpoint/resume mechanism means jobs survive Worker restarts gracefully. A 50 GB ZIP that takes many Worker invocations to complete will checkpoint and resume automatically, with no manual intervention.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Learned
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;The ZIP format is more streaming-friendly than it looks.&lt;/strong&gt; Data Descriptors were designed in 1993 for tape drives, but they solve exactly the same problem we have in serverless: you can't seek backward.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Serializable state is the key to serverless resilience.&lt;/strong&gt; If you can serialize your entire computation state to a few kilobytes of JSON plus a small buffer, you can resume anywhere, on any instance, after any failure.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Constraints breed creativity.&lt;/strong&gt; I never would have built this architecture if I had a 16 GB VM with a filesystem. The Workers limitations forced a design that's actually more resilient and cost-efficient than the traditional approach.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Egress fees are a hidden tax on cloud architectures.&lt;/strong&gt; Keeping compute and storage on the same provider's network isn't just a performance optimization -- it's an economic one. A 10 GB ZIP downloaded 100 times would cost roughly $90 in AWS egress. On Workers + R2, it costs $0.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;

&lt;p&gt;If you need to create ZIP files from remote URLs without managing servers, temp files, or cleanup jobs, &lt;a href="https://eazip.io?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=zip64" rel="noopener noreferrer"&gt;eazip.io&lt;/a&gt; wraps all of this into a single API call:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-X&lt;/span&gt; POST https://api.eazip.io/jobs &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s2"&gt;"X-API-Key: &lt;/span&gt;&lt;span class="nv"&gt;$API_KEY&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s2"&gt;"Content-Type: application/json"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{
    "files": [
      {"url": "https://cdn.example.com/report-q1.pdf"},
      {"url": "https://cdn.example.com/report-q2.pdf", "filename": "Q2.pdf"}
    ]
  }'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;There's a free tier (no credit card required) and the &lt;a href="https://eazip.io/getting-started/quickstart/?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=zip64" rel="noopener noreferrer"&gt;documentation&lt;/a&gt; covers the full API.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Have questions about the ZIP format internals or the Workers architecture? Drop a comment -- happy to go deeper on any part of this.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>serverless</category>
      <category>webdev</category>
      <category>architecture</category>
    </item>
  </channel>
</rss>
