DEV Community: Shah_Dev

Echoes of Earth 🌿: Exploring the Sounds of Nature with Open-Source AI

Shah_Dev — Mon, 12 Oct 2026 11:46:47 +0000

Echoes of Earth 🌿: Turning Everyday Sounds into Outdoor Exploration with Open-Source AI

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

What if your phone helped you pay more attention to the world instead of keeping you glued to the screen?

That's the idea behind Echoes of Earth — an open-source AI-powered environmental sound exploration tool that encourages people to step outside, listen to their surroundings, and discover the sounds they might otherwise overlook.

Think about sitting in a garden, walking along a quiet trail, or spending time outdoors. You might hear birds calling, rain falling, footsteps, or the sound of a passing vehicle. We often hear these sounds without really paying attention to them.

Echoes of Earth turns that everyday experience into an opportunity to explore.

How it works

🌳 Step outside. Find a place where you can listen to the environment around you.
🎙️ Record or upload a sound. Capture a short environmental audio clip using the browser's microphone or upload an existing audio file.
🧠 Let AI analyze it. YAMNet identifies potential sound categories, while the Evidence Engine organizes detections into timestamped events.
🔎 Explore the soundscape. Review detected labels, approximate timestamps, and model scores to understand what the audio may contain.

The application currently supports environmental sound analysis through a working web interface and FastAPI backend.

The goal isn't to replace the experience of being outdoors with another screen. It's to make the screen useful for a short moment, so curiosity can take over and the real world becomes the main experience.

Echoes of Earth is an early-stage MVP. Its detections can be incorrect or overlap, and its model scores are not calibrated probabilities. The results should be treated as AI-generated evidence, not absolute truth.

Demo

Source code: Echoes of Earth on GitHub

The current version runs locally. To try it, follow the setup instructions in the repository's README, start the FastAPI application, and open the local web interface in your browser.

A short demo should show the complete workflow: recording a sound, previewing it, submitting it for analysis, and exploring the resulting sound labels and timestamps.

Code

🔗 GitHub repository: https://github.com/Codie-ds/Echoes-Of-Earth

The project is built around a small, modular architecture:

backend/audio_analyzer.py — Loads and runs YAMNet for environmental sound classification.
backend/evidence_engine.py — Converts frame-level predictions into timestamped sound events using configurable score thresholds and persistence rules.
backend/main.py — Provides FastAPI endpoints for health checks and audio analysis, including upload validation and temporary-file cleanup.
backend/static/index.html — Provides the browser interface for recording, uploading, previewing, and analyzing audio.
backend/tests/ — Contains automated tests for event detection, API behavior, validation, cleanup, and reliability.

The reported test suite has 21 passing tests, including tests that exercise real YAMNet inference.

How I Built It

I built Echoes of Earth incrementally, focusing on making the core experience work before adding more complicated AI features.

1. Environmental sound classification with YAMNet

I used YAMNet, a pretrained sound-classification model available through TensorFlow Hub, to analyze environmental audio.

Rather than training a new model from scratch, I started with a pretrained model and tested it against six short audio clips from the ESC-50 environmental sound dataset.

This helped establish that the model could identify potentially relevant categories such as birds, rain, footsteps, and vehicles, while also revealing that some predictions are ambiguous.

2. An Evidence Engine for timestamped detections

A list of sound labels alone isn't enough to help someone explore an audio clip.

I built an Evidence Engine that processes frame-level predictions and groups sustained detections into events with:

Sound labels and broader categories
Approximate start and end timestamps
Peak and mean model scores
Configurable score thresholds and persistence requirements

This makes the output more useful than a simple list of the top predicted classes.

3. FastAPI and a browser interface

I used FastAPI to expose the audio-analysis functionality through an API and built a lightweight HTML, CSS, and JavaScript frontend.

The interface supports audio uploads and in-browser recording. Browser recordings are encoded as PCM WAV, allowing the application to analyze recordings without requiring FFmpeg for that workflow.

The backend also validates uploads, limits audio duration, handles decoding errors, and cleans up temporary audio files.

4. Privacy and reliability

I spent additional time hardening the application rather than rushing into more features.

The latest implementation includes bounded upload handling, filename sanitization, generic client-facing error messages, temporary-file cleanup, and startup cleanup for leftover files.

The reported tests cover both normal operation and failure scenarios.

Technology stack

AI: YAMNet
ML framework: TensorFlow
Audio processing: librosa and SoundFile
Backend: Python and FastAPI
Frontend: HTML, CSS, and JavaScript
Testing: pytest and FastAPI TestClient
Dataset for initial experiments: ESC-50

I deliberately kept the architecture simple. At this stage, I haven't added a database, an LLM-based explanation layer, or a cloud deployment.

Why Does Open Innovation Matter?

For a project like Echoes of Earth, open innovation is more than a way to reduce development costs. It gives us more control over how AI interacts with people's data and how the system can evolve.

Privacy and control

Once the model assets are downloaded and cached, the current inference workflow can run locally on the machine. Audio recordings don't need to be sent to a third-party AI API for classification.

The initial model download does require an internet connection, and some external audio formats may require additional decoding software. The current implementation is therefore not entirely offline from the first launch.

Transparency and experimentation

Using an accessible pretrained model made it possible to experiment with sound classification without building an entire training pipeline.

The Evidence Engine also lets me inspect how thresholds and persistence rules influence which detections become events. That makes it easier to test the behavior and understand its limitations.

Room to evolve

Because the model, analysis logic, API, and interface are separate components, the project can evolve without requiring a complete rewrite.

Future work could explore local language models to explain detected sound events, better evaluation against labeled audio, and improvements to the outdoor discovery experience.

Most importantly, an open approach makes it easier for other developers to inspect the implementation, reproduce the experiments, and contribute improvements.

My Agent Session

The implementation involved iterative coding, debugging, testing, and code review with an AI coding agent.

DevRelay session: [Add your saved DevRelay agent-session link here, if available.]

The session is optional, but sharing it would help readers understand the development process and how the project evolved through testing and refinement.

Prize Categories

This submission is intended for the overall Hacktoberfest Open-Source AI Challenge.

I am not claiming a partner-specific prize category in this version because the current implementation does not meaningfully integrate a listed partner's technology. If a qualifying partner integration is added and verified, the relevant category can be listed accurately.

What's Next?

The current milestone is a working MVP with a tested audio-analysis pipeline.

The next steps are to improve the quality of the detections, evaluate the system more systematically, and explore whether an optional local AI model can generate useful explanations grounded in the actual detected events.

But the larger goal remains simple:

Use AI to make the world outside more interesting, not to give us another reason to stare at a screen.

I'd love to hear your thoughts: What would make you curious enough to stop, listen, and explore the sounds around you?

🌿 Explore the code: Echoes of Earth on GitHub

devchallenge #hf26challenge #hacktoberfest #opensource #machinelearning

I Built FriendOS: An Adaptive AI Learning Companion for a Friend

Shah_Dev — Mon, 05 Oct 2026 06:16:41 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

FriendOS — An AI That Learns How You Learn

Most learning apps ask:

"What do you want to learn?"

I wanted to build something that asks a different question:

"How do you actually learn?"

So I built FriendOS, an adaptive AI learning companion for my friend.

Instead of simply generating explanations and questions, FriendOS builds a learning profile from what the learner actually does inside the application and uses that evidence to decide what should happen next.

What I Built

FriendOS is an AI-powered adaptive learning companion.

A learner starts by providing:

Their name
Their learning goal
Topics they already know

They can then upload their own learning material, such as a PDF.

FriendOS analyzes the material and creates a learning structure from it.

The learner then takes a diagnostic assessment, practices questions, and interacts with the system.

The important part is what happens next.

FriendOS observes measurable learning behavior such as:

Correct and incorrect answers
Number of attempts
Time spent answering
Hints requested
Questions skipped
Topic performance
Difficulty

It then uses this evidence to adapt the next learning recommendation.

For example:

A learner who is struggling with a topic, taking longer to answer, and repeatedly requesting hints may receive:

Review Concept

While a learner who is consistently answering correctly without hints may receive:

Move to Next Topic

The goal is simple:

Don't just teach the learner. Learn how the learner is learning.

The Core Loop

FriendOS follows this loop:

Plan → Act → Observe → Learn → Adapt

1. Plan

The learner provides their goal and learning material.

2. Act

They answer questions and practice inside FriendOS.

3. Observe

The system records measurable behavior.

4. Learn

The system updates the learner's skill and behavior profile.

5. Adapt

The AI recommends the next learning action.

This creates a continuous feedback loop instead of a static question generator.

Demo

🚀 Live Demo: https://friend-os-rho.vercel.app

Try the complete flow:

Create a learner profile
Upload learning material
Take the diagnostic
Practice questions
Submit answers
See the adaptive recommendation

The backend is deployed separately and communicates with the frontend through the API.

Code

The complete project is open source:

GitHub: https://github.com/Codie-ds/FriendOS

The project contains:

Next.js / React frontend
FastAPI backend
MongoDB Atlas database
Gemma-powered AI services
PDF learning-material processing
Diagnostic assessment
Behavior tracking
Adaptive recommendation engine

How I Built It

Frontend

React
TypeScript
Vite
Tailwind CSS

Backend

Python
FastAPI
Pydantic
PyMuPDF

Database

MongoDB Atlas

AI

The core AI model is Gemma.

Gemma is used for tasks such as:

Extracting learning topics from uploaded material
Generating diagnostic questions
Generating adaptive learning recommendations

However, FriendOS does not blindly ask the LLM to calculate everything.

Deterministic calculations such as:

Accuracy
Attempts
Average response time
Hint count
Skip count
Topic performance

are calculated by Python.

Gemma then receives this structured evidence and makes the higher-level learning recommendation.

This separation makes the system more predictable and prevents the AI from inventing behavioral evidence.

From PDF to Adaptive Learning

The complete pipeline looks like this:

PDF

↓

Topic Extraction

↓

Diagnostic Questions

↓

Initial Skill Profile

↓

Learning Session

↓

Behavior Events

↓

Behavior + Skill Analysis

↓

Gemma Recommendation

↓

Next Learning Action

This means the PDF tells FriendOS what should be learned, while the learner's actual interaction tells FriendOS how the learner is performing.

Why Does Open Innovation Matter?

Open innovation made it possible for me to build the core intelligence of FriendOS without treating AI as a black box.

Using Gemma allowed me to build an AI layer that I could integrate directly into my own application architecture.

More importantly, the project combines open AI technology with transparent application logic.

The system doesn't simply say:

"The AI thinks you are weak at Arrays."

Instead, it can ground the recommendation in observable evidence such as:

Low accuracy
Multiple attempts
Frequent hints
Longer response times

That makes the adaptation easier to understand and reason about.

Open innovation also allowed me to combine different open technologies into one system rather than depending on a single closed platform.

What Makes FriendOS Different?

A lot of AI learning tools focus on generating content.

FriendOS focuses on the learning loop.

It doesn't just ask:

"What answer did the student give?"

It also asks:

"What happened while they were learning?"

The goal is to eventually make the system increasingly personalized based on real interaction rather than only a user's initial prompt.

My Agent Session

I used AI coding agents during the development of FriendOS for implementation, debugging, testing, and production-readiness work.

The development process was divided into phases:

Backend foundation
Learning-material processing
Diagnostic assessment
Behavior tracking
Adaptive learning engine
Practice system
Full end-to-end testing
UI/UX refinement
Production security audit
Deployment

I also used automated testing throughout development to make sure the adaptive learning flow remained stable as new features were added.

Production

FriendOS is deployed as:

Frontend: Vercel

Backend: Render

Database: MongoDB Atlas

The application is fully connected end-to-end, including:

User onboarding
PDF processing
AI topic extraction
Diagnostic generation
Skill profiling
Practice
Behavior tracking
Adaptive recommendations

Prize Categories

I'm entering FriendOS for the categories that match the technologies actually used in the project:

Gemma
MongoDB Atlas
Render

What's Next?

FriendOS is currently focused on the core adaptive-learning loop.

Future improvements could include:

Voice-based learning with ElevenLabs
Better learning-resource discovery
More sophisticated long-term learner modeling
Additional learning formats
Spaced repetition
More granular topic mastery
Personalized learning plans

But the core idea is already working:

FriendOS doesn't just teach you. It learns from how you learn.

Final Thoughts

I built FriendOS because I wanted to explore a simple question:

What if an AI learning companion could adapt to your actual behavior instead of treating every learner the same?

The result is FriendOS — a small experiment in building learning software around an adaptive feedback loop rather than just an AI chatbot.

Thanks for reading!

If you try the demo, I'd love to hear what you think.