This is a submission for the Cloudflare AI Challenge.
What I Built
Short Stories Assistant is a simple tool that helps writers generate short stories of about 2000 words using Text Generation models.
An example of story generation flow would be:
- Providing a prompt using an audio file (optional) ;
- Generation a story using one of the available models (e.g. Mistral 7b) ;
- Editing and saving the story (this will automatically generate the story title, summary, and cover image) ;
- Improving the story title and summary ;
- View previously generated stories.
The application acts as a human assistant, and, therefore, allows the user to customize the story, the title, and the summary at any step of the flow.
Demo
The application is live at https://demo.short-stories-assistant.pages.dev/
My Code
Code available at https://github.com/jonathanarnault/short-stories-assistant
Technologies used:
- Remix ;
- TailwindCSS and DaisyUI ;
- Cloudflare Pages ;
- Cloudflare Workers AI ;
- IndexedDB.
Journey
I never used Cloudflare workers before this challenge, but I was looking to test it on a side project since it seemed to be a powerful IaaS. This challenge was an opportunity to test the service on a simple project.
Since this challenge was focused on Workers AI, I wanted to build a Proof of Concept with basic features for a Short Story Assistant. Thus, I decided to only store data client-side and focus on using as many task types as possible that can fit in the app.
For the Short Story Assistant, I used the following task types:
- Speech Recognition (whisper), for prompting the service using audio files ;
- Text Generation (mistral-7b-instruct-v0.1, falcon-7b-instruct, llama-2-13b-chat-awq), for the story generation ;
- Text Summarization (bart-large-cnn) for the story abstract and title ;
- Text to Image (dreamshaper-8-lcm) for the story cover image generation.
During the development, I was impressed by the ease of integration of Workers AI inside a project. I was able in a few hours to implement a fully functional Proof of Concept with many task types. The Cloudflare SDK makes it easy to call an AI service and get results back.
Despite the ease of integration, I faced some technical challenges with the AI services integration:
- First of all, when I tried to implement text generation, the service usually timed out with partial text. I solved this issue using an EventSource to stream data and avoid timeout ;
- The second challenge that I experienced was to find a text generation model that does not ignore my prompts while following system prompts. While Mistral 7b was working well with my prompt, I had to skip Llama 2 7b and Phi-2 as they were not answering my prompts correctly ;
- Finally, while the Remix integration with Cloudflare Pages is great, especially the deployment, I did not find an easy way to call Workers AI in dev mode. To solve this, I simply ran the application in preview mode locally to test the integration.
In conclusion, this was a fun project to implement and I recommend using Cloudflare Workers AI in projects that do not need custom models. I will also probably test more features of Cloudflare Workers in the future as they provide a great set of services to build web applications.
Top comments (0)