Unlocking Seamless Image Delivery with Airtable, Vercel & Cloudflare Images

Summary

As engineers, we're always on the lookout for ways to optimize our workflows and streamline processes. One challenge that arises while working with Airtable is the attachment type refresh known-issue, particularly the issue is that Airtable changes the attachment url every two-hours. Imagine that you are using Next.js and static site generation then you can end up having stale not working images. Also even if you are not using static site generation you can also surpass the Vercel image optimization limits since the urls will change frequently. In this article, I'll share how we successfully tackled this problem by migrating our attachments from Airtable to Cloudflare Images, maintaining Next.js image optimization techniques while optimizing costs with Vercel.

Understanding the Airtable Attachment Refresh Issue

Airtable is a fantastic tool for managing data, including attachments such as images and documents. However, one limitation is the attachment field type that refreshes every two hours. This can be problematic, especially in applications where static generations or incremental static regenerations take place, more info here. To address this issue effectively, we needed a solution that would provide static image urls without compromising performance.

Migrating to Cloudflare Images

To circumvent the Airtable attachment refresh issue, we decided to migrate all our attachments to Cloudflare Images. Cloudflare Images offers a robust content delivery network (CDN) with instant cache purging capabilities. By storing our attachments on Cloudflare, we could ensure that urls will remain static, and any updates would be reflected immediately without any caching delays.

Strategic Migration Approach

In our quest to resolve the Airtable attachment refresh issue, we adopted a strategic approach to seamlessly transition our attachments to Cloudflare Images while maintaining data integrity and optimizing performance. Here's how we executed the migration process:

• Fetching Images from Airtable

We began by fetching all the images stored in Airtable using its API. This ensured that we had a comprehensive dataset of attachments ready for migration.

• Utilizing Cloudflare Images library

To streamline the migration process, initially we tried to use the Cloudflare api. But some functionalities were missing like (removing duplicates and deleting multiple images), so we created & published a wrapper around it. The cloudflare-images-client npm library. This library provided us with the necessary tools and functionalities to upload all images to Cloudflare's storage infrastructure seamlessly. On top of that this library gives you the ability to delete multiple images and to remove duplicate images if necessary.

• Mapping Metadata to Airtable Records

During the migration process, we attached metadata to each image in Cloudflare Images, including the record ID from Airtable. This strategic decision allowed us to maintain a 1-1 relationship between the images stored in Cloudflare and their corresponding records in Airtable. That way we ensured that data integrity was preserved throughout the migration process. This strategic mapping enabled us to seamlessly link images to their corresponding records, facilitating easy retrieval and updates in the future.

---

Migration script example

import { CloudflareImagesClient } from "cloudflare-images-client";

const cloudflareImagesClient = new CloudflareImagesClient({
  accountId: process.env.CLOUDFLARE_ACCOUNT_ID || "",
  apiToken: process.env.CLOUDFLARE_API_TOKEN || "",
});

async function migrateImages() {
  await cloudflareImagesClient.deleteDuplicateImages();

  const airtableImages = await findImages();

  for (const image of airtableImages) {
    const response = await cloudflareImagesClient.uploadImageFromUrl({
      url: image?.image?.[0]?.url,
      metadata: {
        uniqueImageId: image.uniqueImageId,
        recordId: image.recordId,
      },
    });

    const CDNimageUrl = response?.result?.variants?.[0];

    await getTable("airtableImages").update([
      {
        id: image.recordId,
        fields: {
          cloudflareImage: CDNimageUrl,
        },
      },
    ]);
  }
}

Optimizing Costs with Vercel

While Cloudflare Images provided static urls for our images, we also wanted to optimize costs associated with image optimization. Vercel, our chosen deployment platform, offers image optimization services as part of its billing model. However, by utilizing Cloudflare Images for storage and delivery, we were able to avoid additional charges for unnecessary new urls while making frequent new builds & deployments, ultimately saving on hosting costs.

Conclusion

By migrating our attachments from Airtable to Cloudflare Images and maintaining Next.js capabilities for optimized images, we successfully addressed the attachment refresh issue while optimizing costs with Vercel. This approach ensured efficient delivery of static content without incurring unnecessary expenses.

For developers facing similar challenges with Airtable attachments, consider exploring alternatives like Cloudflare Images API and integrating them seamlessly with platforms like Next.js and Vercel. By embracing innovative solutions and leveraging the power of modern web technologies, you can overcome obstacles and deliver exceptional user experiences.

If you want to support cloudflare-images-client, give it a ⭐️ and/or open issue/pull request to extend it. Find it on Github, Npm

See you until the next challenge 🚀

Feel free to connect on:

Github
LinkedIn