DEV Community: Merbin J Anselm

🤝 Collaborative & 📸 PhotoGrid Collage Maker - Built on Netlify Primitives ⚡️

Merbin J Anselm — Sun, 12 May 2024 16:51:44 +0000

This is a submission for the Netlify Dynamic Site Challenge: Visual Feast, Build with Blobs, Clever Caching.

What I Built

Recently, I embarked on a rejuvenating journey to the picturesque Thekkady region of South India, seeking respite from the sizzling summer heat waves. Little did I know that this trip would inspire a unique project for the Netlify Dev Challenge. Upon my return, I found myself tasked with creating a collage of the captivating photos I had captured during my adventure. Instead of settling for the usual approach of using another online 'collage maker', I decided to take up the challenge and craft something useful – the Collaborative Photo Collage Maker. Yes, I know, this is something bit too much for a trivial task. But it's fun to do so ;).

I set out to create the collaborative PhotoGrid Collage Maker, a dynamic web application that allows users to visually build photo collages together with friends, even asynchronously. The project is a perfect fit for the Netlify Dynamic Site Challenge, showcasing the power of Netlify's platform primitives, including Netlify Blobs, Netlify Image CDN, and Netlify Cache-Control. At its core, the PhotoGrid Collage Maker is a blank grid where users can select from various grid layouts, ranging from 2 to 8 images. With a simple drag-and-drop interface, users can upload their photos and arrange them within the chosen layout. The beauty of this application lies in its collaborative nature – users can share a unique link with friends, allowing them to contribute to the collage one image at a time.

But that's not all! The app also boasts intelligent conflict resolution, ensuring that even if multiple users make changes simultaneously, their edits are seamlessly merged. Thanks to Netlify Blobs' 'strong' consistency mode, the app keeps track of any conflicts and notifies users if reconciliation is required. Once the masterpiece is complete, users can effortlessly download their collage as a high-quality PNG image with a single click. And let's not forget about performance – the Netlify Image CDN optimizes all uploaded images, ensuring they're served at the perfect resolution, while Netlify's granular Cache Control (using Netlify-CDN-Cache-Control & Netlify-Vary headers) keeps the site lightning-fast.

I wrote the whole application in AstroJS with Svelte Islands. Though I initially started in SvelteKit but was hit with this infamous issue, but later switched to AstroJS as the Netlify adapter already supports Netlify Functions v2. I know, switching from SvelteKit used as an SPA/SSR framework to AstroJS as an MPA framework does not make sense at first sight, but I realised once after this implementation, that was the right choice to demonstrate the platform primitives for the challenge. And for the effort they both have a lot of similarities, and AstroJS can support multiple UI frameworks as islands, the porting was a lot easier.

anselm94 / netlify-challenge-collagemaker

A Photo Collage Maker for Netlify Dev Challenge

Collaborative PhotoGrid Maker

A collaborative PhotoGrid collage maker written in AstroJS, submitted for the Dev.to Netlify Challenge.

Get Started

Clone this repository
Install dependencies

npm install

Start local server via Netlify CLI

npm run dev

License

MIT License

View on GitHub

Demo

Try Demo - photogrider.netlify.app

Let me walkthrough you for a short demo of user-facing functionalities and how Netlify Platform Primitives enables them with its technical capabilities using GIFs.

a. Open the website. You'll be greeted with a homepage

b. Then click on 'Create a Photogrid' action to start creating a photogrid

c. You are presented with an empty grid with drag-drop zones for 2 images. For adding more images, select a layout from the list to add even more upto 8 images in different layouts

d. You can drag and drop images into these drag-drop zones to upload them

e. Accidentally selected the wrong image? Fear not, just delete it

f. Next to the fun part, you can share your photogrid with your friends. What if your friend made an edit while you just edited? Sorry your changes will be lost, but no worries, you can always try again at the now synced version

g. Happy with your edit? You can now download your photogrid collage as a PNG image

Platform Primitives

Hope you enjoyed the demo. Now, onto the technical details. As said in the introduction, this submission focuses on all 3 prompts - Visual Feast, Build with Blobs, Clever Caching. Let me walk you through how I leveraged the Netlify Platform Primitives.

1. Netlify Blobs

Usecase 1: Data structure - Metadata + Images

At the heart of this application lies Netlify Blobs storage solution, to persist both the metadata and images associated with each photogrid. For every photogrid created, a metadata object and up to seven image entries are stored in Netlify Blobs. The keys are K-sortable UIDs enabling UIDs to be sorted in chronological order.

key	value
`{ksuid}/metadata`	{metadata}
`{ksuid}/image-0`	{image-file-0}
`ksuid}/image-1`	{image-file-1}
...	...

Usecase 2: Wiki-style collaboration

The metadata object contains essential details such as the grid ID, last modified timestamp, layout, and an array of image IDs.

type GridMetadata = {
  id: string; // grid id - ksuid based
  lastModified: number; // to enable collaboration
  layout:
    | "grid-2"
    | "grid-3"
    | "grid-4"
    | "grid-5"
    | "grid-6"
    | "grid-7"
    | "grid-8";
  images: Array<string | null>; // array containing image-ids. Initially contains 8 null values.
};

Netlify Blobs' 'strong' consistency mode plays a crucial role in enabling real-time collaboration. After every action, the changes are committed back to the Netlify Blob storage, along with the lastModified timestamp. For every client-side action, the last known lastModified timestamp is sent from the web client and compared with the entry in the storage. If the client's lastModified is older, the operation is discarded; otherwise, the entry is updated, ensuring seamless synchronization across multiple users.

2. Netlify Image CDN

The photogrid is a CSS Image Grid. For each of the images in the grid, the width & height of the image differs between the selected layout. The absolute image size is required along with how the image is spanned across in the grid. Below is how the layout info looks like

type LAYOUTS = Record<
  string, // grid-2, grid-3, etc.
  {
    cells: Array<{
      width: number; // width of image in the cell in px
      height: number; // height of image in the cell in px
      colspan: number; // css grid colspan of the cell in the grid
      rowspan: number; // css grid rowspan of the cell in the grid
    }>;
  }

View full layout metadata

export const LAYOUTS: Record<
  string,
  {
    cells: Array<{
      width: number;
      height: number;
      colspan: number;
      rowspan: number;
    }>;
  }
> = {
  "grid-2": {
    cells: [
      {
        width: 350,
        height: 700,
        colspan: 4,
        rowspan: 8,
      },
      {
        width: 350,
        height: 700,
        colspan: 4,
        rowspan: 8,
      },
    ],
  },
  "grid-3": {
    cells: [
      {
        width: 700,
        height: 350,
        colspan: 8,
        rowspan: 4,
      },
      {
        width: 350,
        height: 700,
        colspan: 4,
        rowspan: 8,
      },
      {
        width: 350,
        height: 700,
        colspan: 4,
        rowspan: 8,
      },
    ],
  },
  "grid-4": {
    cells: [
      {
        width: 350,
        height: 350,
        colspan: 4,
        rowspan: 4,
      },
      {
        width: 350,
        height: 350,
        colspan: 4,
        rowspan: 4,
      },
      {
        width: 350,
        height: 350,
        colspan: 4,
        rowspan: 4,
      },
      {
        width: 350,
        height: 350,
        colspan: 4,
        rowspan: 4,
      },
    ],
  },
  "grid-5": {
    cells: [
      {
        width: 350,
        height: 350,
        colspan: 4,
        rowspan: 4,
      },
      {
        width: 350,
        height: 350,
        colspan: 4,
        rowspan: 4,
      },
      {
        width: 175,
        height: 350,
        colspan: 2,
        rowspan: 4,
      },
      {
        width: 350,
        height: 350,
        colspan: 4,
        rowspan: 4,
      },
      {
        width: 175,
        height: 350,
        colspan: 2,
        rowspan: 4,
      },
    ],
  },
  "grid-6": {
    cells: [
      {
        width: 350,
        height: 350,
        colspan: 4,
        rowspan: 4,
      },
      {
        width: 350,
        height: 350,
        colspan: 4,
        rowspan: 4,
      },
      {
        width: 175,
        height: 350,
        colspan: 2,
        rowspan: 4,
      },
      {
        width: 175,
        height: 350,
        colspan: 2,
        rowspan: 4,
      },
      {
        width: 350,
        height: 175,
        colspan: 4,
        rowspan: 2,
      },
      {
        width: 350,
        height: 175,
        colspan: 4,
        rowspan: 2,
      },
    ],
  },
  "grid-7": {
    cells: [
      {
        width: 350,
        height: 175,
        colspan: 4,
        rowspan: 2,
      },
      {
        width: 350,
        height: 175,
        colspan: 4,
        rowspan: 2,
      },
      {
        width: 175,
        height: 350,
        colspan: 2,
        rowspan: 4,
      },
      {
        width: 350,
        height: 350,
        colspan: 4,
        rowspan: 4,
      },
      {
        width: 175,
        height: 350,
        colspan: 2,
        rowspan: 4,
      },
      {
        width: 350,
        height: 175,
        colspan: 4,
        rowspan: 2,
      },
      {
        width: 350,
        height: 175,
        colspan: 4,
        rowspan: 2,
      },
    ],
  },
  "grid-8": {
    cells: [
      {
        width: 550,
        height: 550,
        colspan: 6,
        rowspan: 6,
      },
      {
        width: 175,
        height: 175,
        colspan: 2,
        rowspan: 2,
      },
      {
        width: 175,
        height: 175,
        colspan: 2,
        rowspan: 2,
      },
      {
        width: 175,
        height: 175,
        colspan: 2,
        rowspan: 2,
      },
      {
        width: 175,
        height: 175,
        colspan: 2,
        rowspan: 2,
      },
      {
        width: 175,
        height: 175,
        colspan: 2,
        rowspan: 2,
      },
      {
        width: 175,
        height: 175,
        colspan: 2,
        rowspan: 2,
      },
      {
        width: 175,
        height: 175,
        colspan: 2,
        rowspan: 2,
      },
    ],
  },
};

To leverage the power of the Netlify Image CDN to deliver optimized and responsive images tailored to each user's device and selected layout, the application utilizes the unpic library, which has out-of-the-box support for Netlify Image CDN and generates responsive image tags with ease.

Actual Unpic Svelte configuration:

<Image
  class="h-full w-full"
  src={images[i] ?? ""} // parameter - image url -> /.netlify/images?url=/{gridId}/image/{imageId}
  priority={true} // should be loaded with high priority and interactive on load
  layout="constrained" // parameter - layout
  width={cell.width} // parameter - max width
  height={cell.height} // parameter - max height
  alt="Photogrid image - {i}"
/>

Rendered HTML code:

<img
  class="h-full w-full"
  src="/.netlify/images?w=700&h=350&fit=cover&url=/{gridId}/image/{imageId}"
  priority="true"
  layout="constrained"
  alt="Photogrid image - 2"
  loading="eager"
  fetchpriority="high"
  srcset="
    /.netlify/images?w=640&h=320&fit=cover&url=/{gridId}/image/{imageId}   640w,
    /.netlify/images?w=700&h=350&fit=cover&url=/{gridId}/image/{imageId}   700w,
    /.netlify/images?w=750&h=375&fit=cover&url=/{gridId}/image/{imageId}   750w,
    /.netlify/images?w=828&h=414&fit=cover&url=/{gridId}/image/{imageId}   828w,
    /.netlify/images?w=960&h=480&fit=cover&url=/{gridId}/image/{imageId}   960w,
    /.netlify/images?w=1080&h=540&fit=cover&url=/{gridId}/image/{imageId} 1080w,
    /.netlify/images?w=1280&h=640&fit=cover&url=/{gridId}/image/{imageId} 1280w,
    /.netlify/images?w=1400&h=700&fit=cover&url=/{gridId}/image/{imageId} 1400w
  "
  sizes="(min-width: 700px) 700px, 100vw"
  style="object-fit: cover; max-width: 700px; max-height: 350px; aspect-ratio: 2 / 1; width: 100%;"
/>

This approach ensures that each image in the grid is rendered at the appropriate size and resolution, optimizing performance and providing a seamless visual experience for users across various devices and screen sizes.

3. Netlify Cache Control

To further enhance performance, I implemented Netlify's granular Cache Control. The application has 1 GET API endpoints serving the images by reading from the Netlify Blob and 2 server-side rendered pages (home page and dynamic page for each photogrid), which are cached for 24 hours on the CDN, improving load times and reducing server load.

Astro.response.headers.set(
  "Netlify-CDN-Cache-Control",
  "public, max-age=0, s-maxage=86400, must-revalidate" // cache for 24 hours in the cdn, but not in the browser.
);

However, there is a catch. Since the photogrid image is a server-side rendered page that changes with every update (i.e. on uploading an image, deleting an image or selecting a layout), a more granular approach is required. This is where Netlify's granular cache control comes into play. The cache objects are key-values cached in the edge networks. For every action, the URL includes query parameters ?lastmodified=${lastModified}&hasconflict={true|false}. By utilizing the Netlify-Vary header to instruct the CDN to vary the cache object key based on these query parameters, the application ensures that the CDN cache is used to its fullest potential while still serving the most up-to-date content.

Astro.response.headers.set("Netlify-Vary", "query=lastmodified|hasconflict");

Additional Netlify Platform Primitives

In addition to the core primitives mentioned above, the PhotoGrid Collage Maker leverages several other Netlify offerings:

Netlify Functions: The project is built using the AstroJS framework, which compiles the application to Netlify Functions, ensuring seamless deployment and serverless execution.
Netlify CLI: The Netlify CLI simplified the entire development workflow, enabling local testing and integration of Netlify Blobs and the Netlify Image CDN, providing an exceptional developer experience.
Netlify Site Deploys: The deployment process is streamlined with a simple git push command, leveraging Netlify's seamless CI/CD workflow.

Fun, Beautiful, Safe, Printable 'Story Cards' for Kids with Cloudflare AI

Merbin J Anselm — Fri, 12 Apr 2024 19:40:23 +0000

This is a submission for the Cloudflare AI Challenge.

What I Built

An interactive AI-powered "Story Cards" Maker specifically designed for children (we were once a kid too 😉). This fun and user-friendly web app helps kids craft their own narrative adventures in the size of a postcard, complete with beautiful illustrations. The Story Card can be translated, re-styled with illustrations with a variety of children's storybook themes and even can be downloaded and printed. In the age of AI, the safety of generated content is a priority - the illustrations and the story generated are vetted for safety moderation using AI.

What are 'Story Cards'?

Story Card = Story in a Post Card

For my newborn boy, I recently bought illustrated storybooks with 365 stories each. I really loved how the stories were written with a precise theme in a small paragraph with just one illustration. I was wondering at the same time, if I can have each of the stories printed on a card, which I can shuffle later and not read all those stories in order. Thus, the idea of 'Story Card' was born.

Here's how it works:

🎒 Choose Your Adventure: Kids can select from a range of captivating story elements, including genre (fairy tale, superhero, mystery), character (unicorn, robot, prince), setting (castle, candy land, jungle), tone (funny, adventurous, magical), and theme (friendship, bravery, imagination).
🌟 Surprise Me!: Click the "Surprise Me" button for a completely random story combination, leading to unexpected and delightful narrative twists!
📇 Bring Your Story to Life: Once the elements are chosen, the AI conjures up a captivating story title, narrative content, and even an eye-catching illustration that perfectly complements the adventure.
🌏 Translate for the World: Translate your story title and content into various languages, broadening the child's horizons and sparking an interest in foreign languages.
💅 Style the Adventure: Restyle the illustrations in a variety of storybook themes. Choose from a variety of options to create a truly personalized story card.
🦺 Safe for Kids: The illustration and the story content are vetted for safety classification using AI
💖 Download and Share: Once you're happy with your masterpiece, download the story card as a beautiful postcard-style image, ready to print, share with friends, or treasure forever!

Demo

AI-Powered Story Card Maker - Demo

User Journey 1: Create a 'Story Card'

User Journey 2: Translate the story

User Journey 3: Re-style the illustration

Other features

a. Surprise Me!

b. Safety Moderation

c. Download Story Card

d. Link to Story Card

The story cards once generated, will have a unique link to access and download and will be available for 24 hours.

My Code

This AI-powered Story Card Maker is built as a SvelteKit application with Typescript. Using Flowbite Svelte component library, the whole application was laid out. The layout for the Story Card (emulating the size of a postcard - 4" x 3") is created as an HTML Canvas using Fabric.js.

Also, the SvelteKit app is deployed in Cloudflare Pages with Cloudflare R2 (for storing illustrations), Cloudflare KV (for storing story metadata) and Cloudflare AI (obvious choice for LLM and other GenAI models)

Finally, this is my first experience building a full web app with SvelteKit and deploying it on Cloudflare Pages

The complete source is available here under the MIT license

anselm94 / cf-challenge-ai-storycard

Create Stories in a card with illustrations using AI. Created as part of Cloudflare AI Dev Challenge

AI Story Card (#CloudflareAIDevChallenge)

Create Stories in a card with illustrations using AI. Created as part of Cloudflare AI Dev Challenge.

Blog Post - https://dev.to/anselm94/fun-beautiful-printable-story-cards-for-kids-with-cloudflare-ai-3fbf

Note

This project won Cloudflare AI Challenge under 'Multiple Models' prize category winner - Dev.to Announcement Post 😉

Demo - https://cf-challenge-ai-storycard.pages.dev

Features

Create a Story Card
Translate the Story
Restyle the illustration
Surprise Me
Download Story Card
Safe for Kids If any inappropriate content is detected, the story card will be marked as "unsafe" with a blurred preview image
Collaborate with friends The url for a storycard is valid for a day. Comeback, edit and download it anytime within 24 hours.

Cloudflare AI Models used:

Text Generation - mistral-7b-instruct-v0.2
Image Generation - stable-diffusion-xl-base-1.0
Translation - m2m100-1.2b
Image to Text - uform-gen2-qwen-500m
Text Content Moderation - llamaguard-7b-awq

Developing

Once you've created a project and installed dependencies with npm install (or pnpm install or…

View on GitHub

Journey

The Cloudflare Dev AI Challenge sparked a creative fire! Inspired by my child's illustrated storybook, I envisioned a collection of story cards – short, engaging narratives that could be printed, shuffled and read in any order.

Here's how I brought this idea to life:

Inspiration & Design:

Cloudflare's open AI models, powerful for focused tasks, seemed perfect for crafting short stories. Contrasting them with larger LLMs, I saw their potential for creative writing. Thus, generating short stories with illustrations, presented as printable story cards, became my challenge entry.

First, I sketched the story card layout using Affinity Designer, focusing on child-friendly fonts and a layout reminiscent of a storybook.

Building the Web App:

SvelteKit, with its smooth developer experience, was a joy to use. Since I envisioned multiple features, Cloudflare KV and R2 became natural choices for data storage, with SvelteKit's adapters making integration effortless. Finally, Wrangler CLI helped me deploy the application with ease.

Harnessing AI Power:

The story card maker utilizes five core AI tasks:

Text Generation: The mistral-7b-instruct-v0.2 LLM model generates the story title, content, and illustration caption.
Image Generation: The stable-diffusion-xl-base-1.0 model creates illustrations and allows for style variations.
Translation: The m2m100-1.2b model translates the story content into various languages (limited to Latin character sets due to font usage constraints).
Image to Text: The uform-gen2-qwen-500m model writes a detailed precise description of the generated illustration.
Text Content Moderation: The llamaguard-7b-awq model moderates the story content as well as the illustration description, to be 'safe' or 'unsafe' for children.

Key Takeaways and Future Aspirations:

This challenge was a fantastic learning experience, helping me transform an idea into a working application in a short timeframe. Looking ahead, I'm excited to expand this project with new features and generate even more story card varieties for my son and others. On the technical side, I identified limitations with certain models, like character limits for translation. Exploring solutions like breaking down content for translation in chunks could be a future avenue.

What I'm Proud Of:

Overall, I'm most proud of creating a user-friendly and engaging application that combines the power of AI with the magic of storytelling.

Multiple Models and/or Triple Task Types

The application currently fits into both types.

Multiple models per task: The first journey - 'Create a story card' utilises the following 2 models for 3 steps in order

a. mistral-7b-instruct-v0.2 - Generates the story title and content for the provided theme, genre, character, location etc.

b. mistral-7b-instruct-v0.2 (same model) - Based on the generated story, identifies one primary scene, for which the illustration is to be generated.

c. stable-diffusion-xl-base-1.0 - Based on the scene description, a text-to-image prompt is constructed to generate the illustration in a particular style.

d. uform-gen2-qwen-500m - Then, then a detailed description of the image is generated.

e. llamaguard-7b-awq - Finally, both the story content (title + content) as well as the illustration description are moderated for safety.
Triple Task: The AI models across 5 task types were utilized.
1. Text Generation - mistral-7b-instruct-v0.2 - For story title and content generation, text-to-image prompt (scene) generation
2. Image Generation - stable-diffusion-xl-base-1.0 - For illustration generation
3. Translation - m2m100-1.2b - For translating story title and content into multiple languages
4. Image to Text - uform-gen2-qwen-500m - For understanding the content in the generated illustration, to be vetted by an LLM.
5. Text Content Moderation - llamaguard-7b-awq - For moderating the illustration description and the story title & content.

Edit:

Updated mistral-7b-instruct-v0.1 model to mistral-7b-instruct-v0.2, since I was facing issue soon after the post was published
Added the 'Safety Classification' feature, since this is essential for AI-generated text and image content. Moreover, for an application focussed on generating content for kids, it's an absolute need. Add the usage of uform-gen2-qwen-500m & llamaguard-7b-awq for content moderation.