Originally posted at https://blog.hadenes.io/post/chloe-ai-assistant/

Chloe AI Assistant

Chloe is a powerful AI Assistant written in Go that leverages OpenAI technologies (ChatGPT, Whisper, and DALL-E) along with Google’s Text-to-Speech engine to provide versatile and comprehensive assistance.

It offers multiple interfaces and utilizes the Chain of Thought approach to understand and respond to complex instructions.

Features

• Use Chain of Thought to determine actions, falling back to standard completion if no action is found
• Scrape websites to have them on its context
• Search Google for information
• Perform calculations
• Use Google’s Text-to-Speech engine to speak
• Use OpenAI’s DALL-E to generate images

Due to the Chain of Thought approach, Chloe can also be extended with additional capabilities by simply adding new actions.

Installation

1. Clone the repository

git clone https://github.com/kamushadenes/chloe.git

1. Change directory to the project folder

cd chloe

1. Compile

go build -o chloe cmd/chloe/main.go

1. Setup the required environment variables

export TELEGRAM_TOKEN="your_telegram_bot_token"
export OPENAI_API_KEY="your_openai_api_key"
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/credentials.json"

Usage

Running the chloe binary will start the bot.

./chloe

You can use Chloe in multiple ways.

Telegram

Chloe can be used as a Telegram bot. To use it, you need to create a bot using the BotFather and set the TELEGRAM_TOKEN environment variable to the token provided by the BotFather.

1. Start a conversation with your bot
2. Just chat.

Chloe should automatically detect what you want to do and respond accordingly, including images and voice messages. You can also send voice messages and it will convert them to text and respond accordingly.

The following commands are available:

• /listmodes - Retrieve the list of all available bot modes or personalities
• /forget - Wipe all context and reset the conversation with the bot
• /mode - Instruct the bot to switch to a different mode or personality
• /generate - Generate an image using DALL-E
• /tts - Converts text to speech

Examples

Here, Chloe understands that I want an image of a cat and responds with one. It also understands when I ask it to give the cat a cute yellow hat, and it preserves the context of the previous image.

You can see how Chloe reasons about the context and understands what I want to do.

Here, Chloe says “Beware of the dog, the cat is shady too”. I then ask (in a voice message) for it to show me a picture of what she just said, and she generates relevant images.

HTTP

Chloe can also be used as an HTTP server. By default, it will listen on port 8080. You can change this by setting the PORT environment variable.

The following endpoints are available:

POST /api/complete

This endpoint will complete the given text using OpenAI’s ChatGPT.

Request

{
"content": "Hello, Chloe!"
}

Response

Text

If the response is a text (the default), you’ll receive a streamed response. You can use curl’s -Nflag to receive the text as it’s generated.

curl -N -X POST -H "Content-Type: application/json" -d '{"content": "Hello, Chloe!"}' http://localhost:8080/api/complete



Hello Henrique! How can I assist you today?

Image

In case the Chain of Thought detects the user wants to generate an image, you’ll receive the PNG image as a response.

curl -N -X POST -H "Content-Type: application/json" -d '{"content": "show me a picture of a cat"}' http://localhost:8080/api/complete

Audio

In case the Chain of Thought detects the user wants to convert text to speech, you’ll receive the MP3 audio as a response.

curl -N -X POST -H "Content-Type: application/json" -d '{"content": "Say out loud: Hello, my name is Chloe!"}' http://localhost:8080/api/complete

Command Line (CLI)

Chloe can also be used as a command line interface. This is useful for testing purposes.

./chloe \[command\]

The following commands are available:

• complete [args] - Complete the given text using OpenAI’s ChatGPT
• generate [args] - Generate an image using DALL-E
• tts [args]- Converts text to speech
• forget - Wipe all context and reset the conversation with the bot

If you call complete without any arguments, it will start a REPL.

./chloe complete

Contributing

We welcome contributions! If you would like to improve Chloe, please follow these steps:

1. Fork the repository
2. Make your changes
3. Open a pull request

License

Chloe is licensed under the MIT License.

References

• A simple Python implementation of the ReAct pattern for LLMs

Acknowledgements

• sashabaranov for the Go OpenAI SDK
• awesome-chatgpt-prompts for the personalities