Personas - an Ai Assistant

#cloudflarechallenge #devchallenge #ai

This is a submission for the Cloudflare AI Challenge.

What I Built and What it does

Personas is a AI assistant - a Web app demonstrating the ease with which one can build app using Cloudflare Pages Cloudflare Worker combined with Cloudflare AI which provides plethora of AI models for a wide range of tasks.

Try Persona
A 2 minute video demo

PERSONAS Showcases 3 personas

Story Writer that is kidSafe
Coding Assistant that can help a developer
Sentiment Analysis & Reasoning

The above personas can take voice or text as input and will respond based on Handrails provided by the system prompt for the respective models.

Story Writer generates stories appropriate for Kids under 10 years with the option of translating to German. One can also analyse sentiment of the input prompt and seek the reasoning behind the evaluation. [tasks: Text Generation, Transation, Voice to text]

Coding Assistant is a useful for generating small code snippets in most programming languages, including syntax highlighting. You can also ask it to explain how a particular piece of code works. I used it do some of the grunt work for this project including getting ideas for react components.[tasks: Code Generation, and voice Voice to text]

Sentiment Analysis & Reasoning This was to demonstrate a good system prompt can be leverage an existing model and reduce reliance on specialist models for sentiment analysis. Here we see the the same text generation model used on Story Writer producing structured output as a JSON - which will be pretty useful for decision making in a chain of tasks.[tasks: Structured text Generation, Voice to text]

Demo

Try Persona
A 2 minute video demo

My Code

Code on Github

Journey

I came to know about the challenge on 4th April. Initially, I wasn't planning on doing much considering I was new to Cloudflare and learning about it seemed a daunting task.

This introductory video and the the Cloufare Workers AI got me started in no time.

Development process

Tested ideas for system prompt using Cloudflare AI playground
User examples in AI tutortials to build end points
tested the end points using httpie and sometime curl
Built the frontend with REMIX-Cloudflare

Multiple Models and/or Triple Task Types

Four Models were used to achieve many tasks Story Generation, Translation, Code Generation and Sentiment analysis:

@hf/mistral/mistral-7b-instruct-v0.2 two tasks:
Story Generation,
Sentiment Analysis
@cf/thebloke/discolm-german-7b-v1-awq for language translation
@hf/thebloke/codellama-7b-instruct-awq for Code Generation
@cf/openai/whisper for audio to text transcription.

Some unsung heros (commands/tools/libraries) without which this would have been impossible:

wrangler npx wrangler pages deployment tail <deployment id>
https from httpie
zod, zodix js modules to do validation and parsing
and last but not the least lodash my fav goto for anything that you think is impossible.

Future Plans:

Extend Story Writer to be hands free - voice commands for stories : hey Monk, tell me a story a picture story
Do Sentiment Analysis before generating stories.
Create Custom Persona
Extend the App to leverage Local open source LLMs from ollama.ai
Exploring Cloudflare LORA support for training to improve Personas further

Learning Lots of things learnt over last 7/8 days. From not knowing what is Cloudflare to deploying - I am pretty happy with the outcome. I was proud of two components I implement
1) Component to catch streaming data
2) the audio component. Both were challenging

Many Thanks to Cloudflare and dev.to for giving us this opportunity.