DEV Community: Umang Shukla

🦥 Slow and Steady: Building AI That Does the Work So You Don't Have To | Hacktoberfest 2026

Umang Shukla — Sun, 25 Oct 2026 18:14:38 +0000

*🦥 Slow and Steady: Building AI That Does the Work So You Don't Have To | Hacktoberfest 2026

What if AI didn't just help you work faster, but helped you work less?

We're surrounded by tools that promise productivity, yet we still spend hours on repetitive tasks, managing digital clutter, and doing things that could be automated. Sometimes, the smartest solution isn't doing more. It's building something that takes the unnecessary work off your hands.

That's the idea behind my project for the Hacktoberfest Open-Source AI Challenge: Week 3 — Slow and Steady.

🔗 GitHub Repository: https://github.com/UMANG-SH941/Dev_R

🦥 The Idea: Let AI Handle the Repetitive Work

The theme this week is simple: build something with open-source AI that rewards laziness.

And honestly, I think that's a great engineering philosophy.

We shouldn't have to spend our time on every small, repetitive task when we can build systems that handle them for us. Instead, we should focus our energy on things that actually require human creativity, decision-making, and attention.

My project explores this idea by bringing AI and automation together, with the goal of making everyday digital work less tedious.

The broader vision is simple: build once, automate intelligently, and spend less time doing the same things over and over again.

🤖 Why Open-Source AI?

For me, open-source AI isn't just about using a different model. It's about having greater control over the technology we build with.

Here's why that matters:

🔐 Privacy and control

AI-powered tools can involve personal information and sensitive data. Open models and local inference can help developers build privacy-conscious applications without automatically sending everything to an external AI service.

💸 Lower barriers to experimentation

Building AI projects shouldn't always require expensive API subscriptions. Open-weight models and self-hosted inference give developers more flexibility to experiment within their available hardware and budget.

🛠️ Freedom to customize

Open-source AI makes it easier to experiment with different models, modify workflows, and adapt the system to specific needs instead of being restricted to a single provider's ecosystem.

🌍 Innovation through openness

Open tools let developers learn from existing implementations, build on shared knowledge, and create solutions that others can inspect, improve, and extend.

That's what makes open innovation exciting: the ability to turn an idea into something useful without needing permission from a closed ecosystem.

⚙️ Built for Practicality

I wanted this project to be more than just another demonstration of what AI can do.

The interesting engineering challenge is making AI useful in a practical workflow: connecting model capabilities with automation, keeping the experience straightforward, and reducing the amount of manual effort required from the user.

A good automation tool shouldn't introduce another complicated process that people have to manage. It should make the existing process simpler.

That's the standard I believe AI-powered productivity tools should aim for.

🌱 What I Learned From the Slow and Steady Theme

This challenge reminded me that innovation doesn't always mean building the biggest model or the most complicated agent.

Sometimes, a small tool that eliminates a frustrating daily task is more valuable than a sophisticated system that nobody actually uses.

As developers, we often focus on optimizing performance, adding features, and pushing technical boundaries. But we should also ask a simpler question:

Can we build something that gives people their time back?

That's the kind of problem I find worth exploring.

🚀 What's Next?

I'd like to continue exploring how open-source AI can power practical, accessible automation tools.

Some directions worth exploring include:

More autonomous workflows for repetitive digital tasks.
Greater privacy through local or self-hosted inference, where feasible.
Flexible model integration instead of dependence on a single provider.
Simpler interfaces that make AI useful even for people who aren't technical.

The long-term goal is to make AI feel less like another tool demanding our attention and more like a quiet assistant working in the background.

🦥 Final Thoughts

The best technology doesn't always ask us to do more.

Sometimes, it helps us realize how much we never needed to do manually in the first place.

That's my takeaway from Hacktoberfest 2026 — Week 3: Slow and Steady.

Build thoughtfully. Automate the boring stuff. Keep learning. And occasionally, embrace your inner sloth. 🦥

🔗 Explore the project on GitHub: https://github.com/UMANG-SH941/Dev_R

I'd love to hear your thoughts!

What's one repetitive task you'd happily hand over to an open-source AI assistant?

🏷️ Tags

hacktoberfest #opensource #ai #productivity is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady*

What I Built

Demo

Code

How I Built It

Why Does Open Innovation Matter?

My Agent Session

The Lazy Payoff

Prize Categories

SAM3 + Gemma 4 on CUDA: Chaining Candidate Segmentation Masks with Local Multimodal Reasoning

Umang Shukla — Sun, 18 Oct 2026 17:03:18 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Ask any freelance video editor, VFX artist, e-commerce product photographer, or boutique AI data-labeling contractor what eats up the most unpaid hours in their week, and they will give you the same answer: isolating specific objects across messy images and video timelines.

When a client hands an editor raw footage and asks to "track and highlight only the specific target items in the scene," or an e-commerce seller needs to isolate products from cluttered tabletop photos alongside structured catalog metadata, standard tools fall apart:

Generic background removers blindly strip out everything in the foreground with zero semantic awareness.
Manual frame-by-frame rotoscoping in editing suites takes hours of tedious keyframing.
Cloud-based AI video masking tools lock freelancers into expensive monthly subscriptions, charge steep per-minute rendering credits, and require uploading gigabytes of confidential client footage to third-party servers.

To turn this tedious bottleneck into a zero-marginal-cost service, I built SAM3-Gemma4-CUDA—an interactive, GPU-accelerated computer vision and multimodal reasoning studio. By leveraging SAM3 to generate robust candidate segmentation masks and using Gemma 4 to intelligently filter and reason about these proposals based on user prompts, the tool delivers highly accurate, context-aware visual grounding right on a local machine.

Wrapped in a custom-styled "Steel Blue" Gradio interface, the application provides three core commercial workflows that freelancers, creators, and small studios would gladly pay for:

Image Detection with AI Filtering: Utilizes SAM3 to generate candidate regions from a text prompt. Gemma 4 then analyzes these proposals and filters them to match the user's exact request—providing a clean visual output alongside a natural-language explanation and structured JSON breakdown of its selections.
Automated Video Segmentation: Processes raw video files (.mp4) using SAM3's video tracking model. Users simply enter a text prompt to automatically segment and track target objects across frames, rendering either pure colored mask overlays or comprehensive bounding boxes and contours in seconds rather than hours.
Interactive Click Segmentation: Features an interactive canvas for pixel-perfect client revisions where users click directly on target objects, and the SAM3 tracker instantly generates and overlays precise segmentation masks based on cumulative point data.

Demo

🔗 Live Demo / Deployed Space: [Will update shortly]

🎥 Video Walkthrough / Screenshots: [Will update shortly]

Code

UMANG-SH941 / SAM3---Gemma-4---Segmentation

🔗 Direct Repository Link: https://github.com/UMANG-SH941/SAM3---Gemma-4---Segmentation

Repository Structure


text
├── examples/
│   ├── 1.jpg
│   ├── 1V.mp4
│   ├── 2.jpg
│   └── 3.jpg
├── app.py
├── LICENSE.txt
├── pre-requirements.txt
├── README.md
└── requirements.txt

# OmniScan AR: Turning the Real World into an Interactive AI Learning Experience 🌍📱

Umang Shukla — Sat, 10 Oct 2026 16:23:18 +0000

OmniScan AR: Turning the Real World into an Interactive AI Learning Experience 🌍📱

This is my submission for the Hacktoberfest Open-Source AI Challenge — Week 1: Touch Grass.

What I Built

OmniScan AR (EvoSnap) is an AI-powered, camera-first application that encourages people to step away from their screens and explore the world around them.

Instead of searching the internet every time we encounter something interesting, what if we could simply point our camera at it and start learning?

A plastic bottle, a leaf, an electronic device, or an everyday object can become the starting point for an interactive learning experience.

With OmniScan AR, users can:

🔍 Explore the world around them: Identify everyday objects using AI-powered visual analysis.
🌱 Understand environmental impact: Learn about materials, recycling, sustainability, and the lifecycle of the things we use.
♻️ Discover the hidden story of objects: Explore how materials degrade over time and understand the environmental consequences of waste.
🔮 Visualize the future: Generate illustrative visualizations of how an object might look after 10, 25, 50, 100, or even 500 years of environmental exposure.
💬 Ask questions and learn: Interact with an AI knowledge companion to explore an object's properties, uses, and environmental significance.
📷 Experience live object scanning: Use a camera-based interface with bounding boxes and tracking to make exploration feel more interactive.

The idea behind OmniScan AR is simple: technology should encourage us to experience the world, not just consume content about it.

It is designed for students, curious explorers, educators, and anyone who wants to understand the everyday world through a more interactive and environmentally conscious lens.

Demo

🚀 GitHub Repository: https://github.com/UMANG-SH941/omniscan-ar

The repository contains the application source code, backend services, research artifacts, and material-classification model.

I'm sharing the repository as the primary project link. A deployed demo or walkthrough video is attached.

Code

Explore the complete project on GitHub:

👉 UMANG-SH941/omniscan-ar

The project is organized into a few major components:

Frontend: React Native, Expo, TypeScript, and Expo Router.
Backend: FastAPI and Python.
Vision and language: A provider-agnostic layer for visual understanding and conversational explanations.
Object detection: AI-generated bounding boxes combined with client-side tracking and smoothing.
Material classification: A MobileNetV3-Small model trained using TrashNet and exported to ONNX.
Future visualizations: An image-generation pipeline with configurable providers.
Offline support: A local scan vault that keeps previously scanned objects accessible on supported native devices.

The architecture is modular, so individual AI components can be replaced or improved without rebuilding the entire application.

How I Built It

I wanted to build something that combines practical AI engineering with a tangible, real-world purpose.

1. Camera-first exploration

The application captures objects from the user's surroundings and sends images through the visual-analysis pipeline. The resulting information is presented through an interactive mobile interface.

For live detection, the backend currently uses a vision-language model to identify visible objects and estimate their bounding boxes. The frontend then matches detections between frames using Intersection over Union (IoU) and exponential smoothing to reduce visual flickering.

2. An open-source material classifier

One of the important components is a MobileNetV3-Small material-classification model, fine-tuned using the TrashNet dataset and exported to ONNX for local inference.

The checked-in evaluation reports 88.93% validation accuracy on the project's validation split. This is an initial result rather than a claim of equivalent accuracy on real-world camera images.

The classifier provides a complementary material prediction alongside the vision-language analysis, connecting practical machine learning with waste awareness.

3. A modular AI backend

The backend uses FastAPI and separates visual analysis, knowledge generation, detection, storage, and image generation into independent services.

The LLM integration supports configurable providers, including Gemini, OpenAI, and locally hosted Ollama models. Image generation also supports different providers, including self-hosted Stable Diffusion WebUI.

This makes it possible to experiment with different models and deployment strategies rather than tying the entire application to a single AI service.

An important distinction: the application is built around an extensible AI architecture, but not every current component runs locally or uses open weights. The material classifier is an open-source ML component, while the actual inference path depends on the configured provider.

4. Visualizing long-term environmental impact

The Future Vision feature combines material-aware prompts with image generation to create illustrative visualizations of environmental degradation over time.

These images are intended to communicate environmental possibilities, not scientifically validated predictions of an individual object's exact future.

That distinction matters: AI can make abstract environmental timescales easier to understand, but generated imagery should not be confused with scientific evidence.

Why Does Open Innovation Matter?

Open innovation makes projects like OmniScan AR possible for independent developers and student builders.

I didn't need to train a vision model from scratch or build every infrastructure component myself. I could combine existing open-source frameworks, a pretrained MobileNetV3 model, ONNX Runtime, and modular AI services to build a complete application around a new idea.

It also gives the project room to evolve.

Reproducibility: Other developers can inspect the implementation and understand how the components fit together.
Experimentation: Different open-weight vision-language models and local inference backends can be tested without redesigning the application.
Accessibility: Open-source tools reduce the initial cost of experimentation and make advanced ML workflows more approachable.
Community collaboration: Contributors can improve detection, optimize inference, expand environmental knowledge sources, or make the application more accessible.
User control: Local inference and self-hosted services offer potential paths toward better privacy, lower recurring costs, and reduced dependence on external providers.

Open innovation is not just about using free tools. It is about making the underlying technology inspectable, adaptable, and improvable by a wider community.

My Agent Session

I used AI-assisted development to help explore implementation choices, structure services, and iterate on the application.

Agent session: I'll add a DevRelay session link here once I have a shareable session URL.

Prize Categories

Open Innovation Open to any and MongoDB for I used it in my database management.

What's Next?

OmniScan AR is an ongoing project, and there is plenty of room to improve it.

Some of the next steps I'd like to explore include:

Replacing remote live-frame detection with an efficient on-device object detector.
Integrating more open-weight vision-language models through local inference.
Improving material classification with real-world camera images and stronger evaluation.
Grounding environmental estimates in reliable, cited datasets and sources.
Making the application easier to use outdoors, with better offline workflows and privacy controls.

The larger vision is to make learning more experiential: walk outside, discover something ordinary, point your camera at it, and leave with a better understanding of the world.

Let's build AI that helps us touch grass — and understand the planet we're standing on. 🌍

Hacktoberfest #OpenSourceAI #TouchGrass #MachineLearning #ComputerVision #OpenSource #Sustainability #ReactNative #AI