DEV Community: Nikisha

Dinner Wasn't Missing Ingredients. It Was Missing Coordination: Building New Dish with Local Open-Source AI

Nikisha — Sun, 18 Oct 2026 21:17:14 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

The idea arrived during one of those completely ordinary dinner moments.

I opened the fridge and found tomatoes, some cooked rice, half an onion, and several other ingredients that were all technically food but did not yet feel like dinner. A one of my friend had spinach and chickpeas and the other had carrots and yogurt.

Individually, none of us had an exciting plan. Together, we almost had a meal.

That thought became New Dish: a local-first community kitchen that turns ingredients from one to three households into one real, safe recipe.

It works for one person too. You can type what you have, photograph an ingredient list, or take a picture of your fridge. New Dish proposes ingredients, asks you to confirm them, applies the group's allergies and preferences, and matches everything against a documented recipe.

The people I imagine paying for this are roommates, shared apartments, hostels, small community kitchens, and sustainability groups trying to reduce food waste. A paid version could add a shared weekly pantry, expiry reminders, shopping lists, and private household groups. The core version stays useful without an account or an API bill.

How the idea changed

My first version was called CrowdChef. It imagined nearby households discovering one another and pooling leftovers.

It sounded fun, but it was too large. Location, strangers, trust, scheduling, and food safety all arrived before anyone had cooked a single thing.

So I reduced the idea to a small, real moment:

One person should be able to use it.
A few known neighbors or roommates can join.
Everyone's allergies must be treated as group-wide boundaries.
The result has to be an actual edible recipe, not an AI improvisation.
Nutrition must come from a source, not from a model guessing confidently.

That smaller idea was much better. The cartoon interface and the animated pepper guide, Peper, came later because dinner planning should feel friendly—not like filling out tax software.

The full New Dish workspace: kitchens on the left, a verified recipe on the right.

Demo

🚀 Live demo: New Dish

The quickest way to try it:

Choose one, two, or three kitchens.
Rename the neighbor kitchens if you want.
Add ingredients manually or scan a photo.
Select allergies and cooking preferences.
Create the dish and review the source recipe, missing ingredients, instructions, and per-serving nutrition.
Ask Peper to read the recipe aloud.

The app is responsive, so the same flow works on a phone:

Code

The complete project is open on GitHub:

nikisha77 / New-Dish

New Dish

New Dish is a cheerful, local-first recipe matcher for one person or a group. Kitchens can type ingredients or confirm suggestions from an ingredient-list/fridge photo, choose group-wide dietary boundaries, and get a documented edible recipe with source nutrition per serving.

Local AI

Verified recipes: UniTools Recipes v1 (CC BY-SA 4.0), including quantities, detailed method, timings, licensed photos, and source-provided calories/protein/fat/carbs per serving.
OCR: Tesseract.js with local image preprocessing.
Fridge suggestions: Xenova/detr-resnet-50 through Transformers.js. Detections always require confirmation.
Optional wording helper: HuggingFaceTB/SmolLM2-360M-Instruct, falling back to onnx-community/Qwen2.5-0.5B-Instruct.
Voice: onnx-community/Kokoro-82M-v1.0-ONNX through Kokoro.js.
Models run in browser workers. Model files are downloaded once and cached by the browser.
The language model never creates recipe facts or nutrition. If no safe source recipe matches, the app says so instead of fabricating one.

Run locally

python3 -m http.server 4173 --directory dist

Then open http://127.0.0.1:4173.

or try here - https://nikisha77.github.io/New-Dish/#recipe

View on GitHub

Repository: github.com/nikisha77/New-Dish

There is no backend and no secret API key. The application is static HTML, CSS, JavaScript, Web Workers, open models, and an open recipe dataset.

How I Built It

The architecture

New Dish deliberately separates recognition, recipe truth, and presentation.

Fridge photo / ingredient label
              ↓
   Local OCR + object detection
              ↓
      Human confirmation
              ↓
 Allergies + dietary preferences
              ↓
   Verified open recipe matching
              ↓
 Recipe, licensed photo, instructions
 and sourced nutrition per serving
              ↓
       Peper's local voice

1. Reading a fridge without uploading it

For printed ingredient lists, I use Tesseract.js in the browser. Before OCR, the image is resized and processed using grayscale, contrast enhancement, and a second high-contrast pass when the first result is weak.

For fridge photographs, a DETR ResNet-50 object-detection model runs through Hugging Face Transformers.js in a Web Worker. The model never receives an uploaded URL from a server; it works with the image selected in the browser.

The most important part is what happens after inference: the app does not automatically trust it.

My first fridge test was humbling. The recognizer found one correct word—apple—surrounded by a parade of meaningless fragments. The pipeline had technically succeeded, but the result was useless.

I fixed that by:

matching OCR output against a vocabulary built from the recipe dataset;
rejecting random text fragments;
deduplicating singular and plural ingredient names;
requiring higher confidence for fridge detections;
preselecting only reliable suggestions; and
requiring the person using the app to confirm every ingredient.

That failure changed the product. A confident model is not the same thing as a correct model.

2. Refusing to let a language model invent dinner

The early prototype asked a small language model to create a recipe. That was exciting until I asked a more important question: Would I actually cook and serve this?

A plausible sentence is not evidence that a recipe exists. The same problem applies to nutrition. A model can produce wonderfully formatted calories and macros that have no relationship to the actual meal.

So the final architecture uses UniTools Recipes v1, an open CC BY-SA 4.0 collection of documented recipes. Each record includes ingredient quantities, scaling behavior, detailed steps, timings, a licensed image, and source-provided calories, protein, carbohydrates, and fat per serving.

The browser matches confirmed pantry ingredients against that data, removes recipes conflicting with selected allergies, applies preferences such as vegan, vegetarian, gluten-free, dairy-free, high-protein, easy, and cooking-time limits, then scales the ingredient quantities for the requested number of people.

If no safe recipe matches, New Dish says so. It does not manufacture one to avoid an empty screen.

3. Open models with graceful fallbacks

The optional local language helper uses SmolLM2 360M Instruct as its primary open-weight model and Qwen2.5 0.5B Instruct as the fallback. Both run through Transformers.js, using WebGPU when available and WASM otherwise.

They are intentionally not responsible for recipe facts or nutrition. Their role is limited to presentation help, where a wording error is inconvenient rather than unsafe.

Peper's natural voice uses the open Kokoro 82M model locally. If that model cannot load, the app falls back to the browser's device voice. Peper's face animates while speaking, which is a small detail but makes the interface feel alive.

4. Safety and privacy in the interface

Allergy exclusions apply to the complete group. The current list includes peanuts, tree nuts, dairy, eggs, gluten, soy, fish, shellfish, sesame, mustard, celery, lupin, and sulphites.

Typed ingredients that match a selected allergen are removed from recipe matching, and recipe records are checked too. The interface still reminds people to verify packaging and cross-contact because software cannot inspect a kitchen.

Neighbor names are chosen by the user and saved only in browser storage. Images and ingredients stay on the device. No account is required.

5. Shipping it

New Dish is a buildless static application hosted on Github pages. That keeps deployment simple.

Why Does Open Innovation Matter?

Open innovation is not decoration in this project. It changes whether the idea is practical.

With closed APIs, every photo scan, recipe request, and spoken instruction could become a metered request. That would make a small household tool expensive before it had its first paying customer. It would also require sending photographs of people's kitchens and food habits to servers they did not choose.

Using open models locally gave me four important freedoms:

Privacy: fridge photographs and ingredient lists stay on the device.
Low operating cost: using the app does not create a growing inference bill.
Replaceability: SmolLM2, Qwen, DETR, Tesseract, and Kokoro can be updated or exchanged without redesigning the product around one vendor.
Honesty: model output can be constrained to what models do well, while recipe and nutrition facts come from inspectable open data.

The most valuable freedom is the ability to disagree with the model. I can put confidence thresholds around it, add a food vocabulary, require human confirmation, or remove it entirely from a safety-sensitive step. A closed black-box response would give me fewer places to intervene.

Open did not simply make New Dish cheaper. It made the product's privacy and trust model possible.

My First User

My first user was me, standing in front of a fridge and expecting the camera feature to understand what I could already see.

It did not work beautifully on the first attempt. It returned one correct apple and a lot of nonsense. That was actually the most useful feedback I could have received because it exposed a dangerous assumption: I was treating “the model returned something” as “the model understood the image.”

After adding food-vocabulary validation and confidence thresholds, I tested the exact failed output again. Only apple survived.

No one has paid for New Dish yet. I would rather say that honestly than invent traction for a submission. But the product already received its first useful requests: support a single person, show verified nutrition per serving, let users set their neighbors' names, and never fabricate a recipe. Each request changed the app.

The next real test is putting it in front of roommates or a shared household for a week and measuring whether it helps them rescue at least one dinner they would otherwise have abandoned.

Open-source building blocks

Transformers.js for in-browser model inference
SmolLM2-360M-Instruct
Qwen2.5-0.5B-Instruct ONNX
DETR ResNet-50 for Transformers.js
Tesseract.js
Kokoro 82M ONNX
UniTools Recipes v1, CC BY-SA 4.0

Thanks for reading. If your fridge has ever contained plenty of ingredients but somehow no dinner, New Dish is for you.

The Next Chapter Is Outside: Building an Open-Source AI Storybook with Streamlit

Nikisha — Sun, 11 Oct 2026 14:09:12 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass.

The next chapter is outside

Most digital stories ask us to sit still.

We read, tap “next,” watch another animation, and stay exactly where we are. When I saw the Touch Grass prompt, I wanted to reverse that relationship. What if a story could begin on a screen, send you outdoors, and refuse to continue until you came back with something you had discovered?

That idea became Outside the Lines: a Python and Streamlit story experience in which the player chooses a comic adventure, reads one short chapter, receives a real-world mission, and photographs something they find outside. An open-weight vision model interprets the discovery, and the object becomes part of the next chapter.

The screen is not the destination. It is the doorway.

What I Built

Outside the Lines is an interactive outdoor storybook for children, families, and anyone who needs a playful excuse to step away from a screen.

The player chooses one of three stories:

Barnaby’s Great Bush Disaster — an overconfident orange cat attempts an heroic squirrel chase and discovers gravity.
Leo and the Lunch-Lover — a peaceful mountain picnic becomes a very fast argument with a hungry goat.
Maya’s Flying Masterpiece — a gust of wind turns an artist’s canvas into a kite and the hillside into a rescue mission.

Each story has three chapters and three outdoor missions. A mission might ask the player to find a leaf that looks brave, a natural shape that points toward an escape route, or a color that belongs in Maya’s recovered painting.

The interaction loop is intentionally simple:

Choose a story
      ↓
Watch and listen to a short chapter
      ↓
Receive a safe outdoor mission
      ↓
Step away from the screen
      ↓
Photograph a natural object
      ↓
OpenCLIP suggests what it might be
      ↓
The player confirms or changes the answer
      ↓
That discovery appears in the next chapter

The most important product decision was that the AI does not replace the player’s imagination. It gives a suggestion, but the player can edit it. If the model thinks a seed pod is a stone, the child does not have to accept the machine’s answer. The discovery still belongs to the person who found it.

The missions also avoid collecting, damaging, or feeding wildlife. The player can photograph an object where it is and leave it outside.

Demo

Try the Streamlit app here:

👉 Launch Outside the Lines

The first use of the vision or narration features can take longer because the open model weights need to be downloaded and cached. The story itself remains playable while those optional features load.

Deployment note: the Streamlit Community Cloud app must be set to public before submitting, otherwise visitors will see a sign-in page.

Code

The complete Python project is open source:

nikisha77 / outside-the-lines

Outside the Lines

A Streamlit adventure that turns three real-world discoveries into a moving hand-drawn story. The application stack is Python from end to end.

Open models

OpenCLIP ViT-B/32 interprets trail photos in Python.
Kokoro-82M performs the narration with a natural voice in Python.
Pillow creates chapter-specific animated WebP scenes in Python.

No Gemini, proprietary model API, JavaScript application, Node build, account, or location tracker is required. Model weights are fetched once and cached.

Run

cd outside-the-lines
python3 -m venv .venv
.venv/bin/python -m pip install -r requirements.txt
.venv/bin/streamlit run app.py

Open http://localhost:8501. The story itself works immediately. OpenCLIP and Kokoro download their model weights the first time their features are used.

Structure

app.py — Streamlit screens, navigation, session state, and UI
story_data.py — all three stories and their outdoor missions
ai_services.py — animation, photo understanding, and narration
assets/ — the supplied comic reference art and its story panels
requirements.txt…

View on GitHub

Repository: github.com/nikisha77/outside-the-lines

The project is deliberately small enough to understand:

outside-the-lines/
├── app.py              # Streamlit UI, navigation, and session state
├── story_data.py       # Three stories and nine outdoor missions
├── ai_services.py      # Animation, vision, and narration
├── requirements.txt    # Reproducible Python dependencies
└── assets/             # Comic artwork and story panels

How I Built It

1. I started with the story loop, not the model

The first version was a static web story. It looked polished at a glance, but it had the wrong kind of motion: leaves floated, objects pulsed, and the page moved without the characters actually doing anything.

That exposed an important difference. An animated interface is not the same thing as an animated scene.

I rewrote the experience around character actions: Barnaby crouches, launches, and crashes; Leo and Billy race around the boulder; Maya chases a canvas down the hill. The movement needed to support the comedy and make each chapter feel like a moment in a comic, not a collection of decorative effects.

I also expanded the stories. Instead of a single shallow scene, each theme now has anticipation, action, impact, and an ending affected by what the player finds outside.

2. I moved the complete application to Python and Streamlit

I wanted one readable stack for the interface, story state, model inference, audio, and animation. Streamlit made that possible without maintaining a separate JavaScript application.

app.py controls the screens and stores the current story, chapter, discoveries, and audio state in Streamlit session state. story_data.py keeps the narrative separate from the UI, so a story can be edited, translated, or replaced without changing the application logic.

Expensive results are cached:

@st.cache_data(show_spinner=False)
def scene_bytes(panel: int, chapter: int) -> bytes:
    return animated_scene(panel, chapter)

This matters because model-backed interactions feel much better when rerunning the Streamlit script does not regenerate the same media every time.

3. OpenCLIP turns a trail photo into a story ingredient

For image understanding, I used OpenCLIP ViT-B/32 with LAION pretrained weights.

A mission supplies a small set of relevant labels such as “a leaf,” “a twig,” “a stone,” “tree bark,” or “a seed pod.” OpenCLIP embeds the uploaded image and each candidate label into the same feature space. The app then chooses the label with the highest similarity.

The core is compact:

model, _, preprocess = open_clip.create_model_and_transforms(
    "ViT-B-32",
    pretrained="laion2b_s34b_b79k",
)
tokenizer = open_clip.get_tokenizer("ViT-B-32")

image_features = model.encode_image(image)
text_features = model.encode_text(text)

image_features /= image_features.norm(dim=-1, keepdim=True)
text_features /= text_features.norm(dim=-1, keepdim=True)

scores = (100 * image_features @ text_features.T).softmax(dim=-1)
best_label = candidates[int(scores.argmax())]

I chose constrained candidate labels on purpose. This is not a general “tell me everything in this image” chatbot. It is a small, understandable classification step tied to the current mission. That makes it faster, easier to explain, and easier for the player to correct.

If the model cannot load, the app falls back to a neutral “outdoor discovery” instead of breaking the story. AI should enhance the adventure, not hold it hostage.

In the hosted version, the uploaded photo is processed by the Streamlit server. It is not sent to a separate proprietary vision API, and the app does not request location data. For the strongest privacy boundary, the same repository can be run locally so inference stays under the player’s control.

4. Kokoro gives the stories a warmer voice

The first narration attempt sounded like a utility reading a notification. That did not fit a mischievous cat, a terrified hiker, or a painter sprinting downhill.

I replaced it with Kokoro-82M, an open-weight text-to-speech model. Narration is produced as a WAV file and cached using a hash of the voice and story text:

pipeline = KPipeline(lang_code="a")
chunks = [
    audio
    for _, _, audio in pipeline(
        text,
        voice="af_heart",
        speed=0.95,
    )
]

audio = np.concatenate(chunks)
sf.write(cached_path, audio, 24000, format="WAV")

The slightly slower pace makes the voice feel more like story narration than a screen reader. Because the voice layer is independent, another Kokoro voice—or another open TTS model—can be substituted without rewriting the story system.

5. Pillow creates motion without a video-generation bill

The moving scenes are generated in Python with Pillow and NumPy. Each chapter uses a different camera rhythm:

anticipation uses a gentle push-in,
chase scenes pan and add motion arcs,
impacts use a quick zoom, shake, and comic burst.

Twenty frames are rendered into a looping animated WebP. That is not full character animation, and I do not pretend it is. It is a lightweight comic-motion system that runs predictably, costs nothing per scene, and works with the supplied hand-drawn panels.

The architecture also leaves a clean seam for future work: generated or hand-animated frames can replace the source panel without changing the Streamlit story flow.

6. Deployment taught me to respect the Python version

The app worked locally, but the first Streamlit deployment failed while building blis, a dependency pulled in by spaCy. The cloud environment had selected Python 3.14, while the dependency chain expected a version with compatible wheels.

The fix was not an elaborate code change. I redeployed with Python 3.11 and kept the dependencies explicit in requirements.txt.

That debugging moment was a good reminder: an open-source project is only useful when another machine can actually install it. The model choice gets attention, but reproducible environments are part of the product.

Why Does Open Innovation Matter?

Open models are not decoration in this project. They are the parts that make the interaction possible.

The project does not pay for every discovery.

There is no per-photo vision request and no per-sentence narration bill. That matters for a playful app that may be used repeatedly by a family, classroom, or community group.

The models can be inspected and replaced.

OpenCLIP and Kokoro sit behind small Python functions. A developer can swap the vision backbone, change the label set, choose a different voice, or run quantized models on smaller hardware. The application is not designed around one company’s response format.

The local path is real.

The public demo runs on Streamlit Community Cloud for convenience, but the same code can run on a laptop without sending photos to a third-party AI API. Once weights are downloaded and cached, the model layer can be controlled by the person or organization running the app.

Failure does not end the story.

Open model weights can be large, networks can disappear, and low-memory machines exist. The narrative and missions continue even if vision or narration is unavailable. The human activity remains the primary feature.

The player stays in charge of meaning.

A model prediction is shown as an editable suggestion. This is especially important for a children’s experience: the software can notice patterns, but it should not teach the player that an AI label is unquestionable truth.

A closed multimodal API could probably produce a more fluent description with less setup. But it would also introduce a remote dependency, unpredictable usage cost, less control over model behavior, and a harder privacy conversation around children’s outdoor photographs. For this project, the flexibility of open components is a better fit than maximum model size.

What I Learned

The hardest part was not connecting a model. It was deciding where AI belonged.

My first instinct was to add more generation: more animation, more description, more automatic storytelling. The better version uses AI in two narrow places—recognizing a discovery and narrating a chapter—while the player supplies the movement, attention, and imagination.

I also learned that “get people outside” cannot be solved with outdoor-colored pixels. The product needs a reason to close the laptop or put down the phone. In Outside the Lines, progress pauses until the player looks at the real world.

There are still limitations. OpenCLIP chooses from a constrained list and can be wrong. Kokoro’s first load can be slow. The animated WebP scenes are expressive camera motion, not fully articulated character animation. Streamlit Community Cloud also means the hosted version is not an offline trail app.

Those are useful boundaries, not hidden defects. The next steps are to add a downloadable offline mode, more accessible mission alternatives, multilingual stories, and community-authored story packs.

The part I care about most

I do not want Outside the Lines to become another app that rewards a child for staring at it longer.

The ideal session is a few minutes of story, a much longer stretch of noticing leaves, stones, shadows, wind, bark, and paths, and then a quick return to see how that discovery changed the adventure.

If the player remembers the squirrel, the goat, or the flying canvas, that is lovely.

If they remember the strange leaf they found outside, that is the real ending.

Prize Categories

I am entering the overall Hacktoberfest Open-Source AI Challenge: Week 1 — Touch Grass category. The current build does not claim a partner-technology prize category.

Thanks for reading—and please close this tab for a minute when you are done. The next chapter is outside.