Poem to Post — Turning Poetry Into Visual Stories

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend.

What I Built

I built Poem to Post, an AI-powered tool that turns a poem into a complete visual social-media post.

I built it for a friend who writes poetry but finds it difficult to turn her poems into visual content. Instead of starting with an image and trying to fit a poem around it, I wanted the poem itself to drive the creative process.

The app takes a poem and generates:

• its dominant mood
• an Instagram caption
• relevant hashtags
• a detailed visual concept
• an AI-generated background
• a final visual post containing the original poem

The visual interpretation is not restricted to landscapes. Depending on the poem, it can become an interior, urban scene, symbolic composition, surreal image, or another visual representation that fits the poem.

Demo

I recorded a short demo showing the application running through its Gradio interface: poem in → AI-generated visual post out.

Code

GitHub repository:

https://github.com/kavyachandak07-bot/poem-to-post

How I Built It

The project uses two open-weight AI models for different parts of the creative pipeline.

Gemma 3 4B runs locally through Ollama. It reads the poem and generates the dominant mood, caption, hashtags, and a detailed visual prompt.

That visual prompt is then passed to FLUX.1-schnell through Hugging Face Inference Providers to generate the artwork.

Finally, Pillow combines the generated artwork with the original poem, and Gradio provides the web interface.

The pipeline is:

Poem
  ↓
Gemma 3 4B
  ↓
Mood + Caption + Hashtags + Visual Prompt
  ↓
FLUX.1-schnell
  ↓
AI-generated Background
  ↓
Pillow
  ↓
Final Visual Post

The language-model part runs locally, while image generation uses hosted inference.

Why Does Open Innovation Matter?

Open-weight AI made this project possible to build as a collection of interchangeable pieces rather than depending on one closed AI system.

Gemma runs locally on my machine through Ollama. That means the poem analysis and creative text generation can happen locally without sending the poem to a hosted language-model API.

It also made experimentation easier. Gemma can focus on understanding the poem and creating the visual concept, while a separate image model can turn that concept into artwork.

For this project, the most important part of open innovation is that the models can be used as building blocks. I can experiment with the pipeline, change models, and decide which part of the process should run locally or remotely.

That flexibility helped me build something around a real person's creative workflow rather than simply building another generic AI chatbot.

My Agent Session

I used DevRelay while working on the project.

Prize Categories

Best Use of Gemma

Poem to Post uses Gemma 3 4B as a core part of its creative pipeline.

Gemma runs locally through Ollama and handles poem understanding, mood detection, caption generation, hashtag generation, and creation of the visual prompt that drives the image-generation stage.

Best Use of DigitalOcean

I am not entering this category because DigitalOcean is not currently part of the application's runtime.

---

Built for a friend, with open-weight AI.