DEV Community

Cover image for Building a Short Story Assistant using Cloudflare AI
Jonathan ARNAULT
Jonathan ARNAULT

Posted on

2 1 1

Building a Short Story Assistant using Cloudflare AI

This is a submission for the Cloudflare AI Challenge.

What I Built

Short Stories Assistant is a simple tool that helps writers generate short stories of about 2000 words using Text Generation models.

An example of story generation flow would be:

  • Providing a prompt using an audio file (optional) ;
  • Generation a story using one of the available models (e.g. Mistral 7b) ;
  • Editing and saving the story (this will automatically generate the story title, summary, and cover image) ;
  • Improving the story title and summary ;
  • View previously generated stories.

The application acts as a human assistant, and, therefore, allows the user to customize the story, the title, and the summary at any step of the flow.

Demo

The application is live at https://demo.short-stories-assistant.pages.dev/

Image description

My Code

Code available at https://github.com/jonathanarnault/short-stories-assistant

Technologies used:

Journey

I never used Cloudflare workers before this challenge, but I was looking to test it on a side project since it seemed to be a powerful IaaS. This challenge was an opportunity to test the service on a simple project.

Since this challenge was focused on Workers AI, I wanted to build a Proof of Concept with basic features for a Short Story Assistant. Thus, I decided to only store data client-side and focus on using as many task types as possible that can fit in the app.

For the Short Story Assistant, I used the following task types:

  • Speech Recognition (whisper), for prompting the service using audio files ;
  • Text Generation (mistral-7b-instruct-v0.1, falcon-7b-instruct, llama-2-13b-chat-awq), for the story generation ;
  • Text Summarization (bart-large-cnn) for the story abstract and title ;
  • Text to Image (dreamshaper-8-lcm) for the story cover image generation.

During the development, I was impressed by the ease of integration of Workers AI inside a project. I was able in a few hours to implement a fully functional Proof of Concept with many task types. The Cloudflare SDK makes it easy to call an AI service and get results back.

Despite the ease of integration, I faced some technical challenges with the AI services integration:

  • First of all, when I tried to implement text generation, the service usually timed out with partial text. I solved this issue using an EventSource to stream data and avoid timeout ;
  • The second challenge that I experienced was to find a text generation model that does not ignore my prompts while following system prompts. While Mistral 7b was working well with my prompt, I had to skip Llama 2 7b and Phi-2 as they were not answering my prompts correctly ;
  • Finally, while the Remix integration with Cloudflare Pages is great, especially the deployment, I did not find an easy way to call Workers AI in dev mode. To solve this, I simply ran the application in preview mode locally to test the integration.

In conclusion, this was a fun project to implement and I recommend using Cloudflare Workers AI in projects that do not need custom models. I will also probably test more features of Cloudflare Workers in the future as they provide a great set of services to build web applications.

API Trace View

How I Cut 22.3 Seconds Off an API Call with Sentry πŸ‘€

Struggling with slow API calls? Dan Mindru walks through how he used Sentry's new Trace View feature to shave off 22.3 seconds from an API call.

Get a practical walkthrough of how to identify bottlenecks, split tasks into multiple parallel tasks, identify slow AI model calls, and more.

Read more β†’

Top comments (0)

Sentry image

See why 4M developers consider Sentry, β€œnot bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more

πŸ‘‹ Kindness is contagious

Please leave a ❀️ or a friendly comment on this post if you found it helpful!

Okay