This is a submission for the DEV Weekend Challenge: Community
The Community
This targets the AI/ML community in general, especially those who want to get their feet wet into creating their own model. I notice when I start learning AI, there aren’t a lot of resources (to my knowledge) that allow you to create your own model and have the freedom to download the files to import it to your project without having to sign up and possibly pay to use their services (I might be wrong but that’s based on my experience). The lack of free tools pretty much restricted me since it is not accessible and I do not enjoy signing up and paying for their services just to train my model. Even if it is free, it is quite limited in some way.
Additionally, when I use Pre-Trained models, sometimes it is not as accurate as I needed it to and sometimes it goes back to the same problem: Signing up, either pay or not, and use their API key (which is good in some cases, but when it comes to creating a Chrome Extension for example, it’s hard to hide).
What I Built
I remember creating the ASL project that I shared on Dev.to where I mentioned I created another project that is dedicated to creating my own model using Hand Pose. I thought it would be convenient to create a free tool that allows you to train your own model, download it, and use it to your project.
I created a website called “HandTracker” where you can create your own model using the ml5.js Hand Pose feature. This allows you to create any gestures you want and classify them to your liking. For example, you can create a hand gesture that means “Hello” or create the Gojo’s signature gesture for Domain Expansion: Infinite Void.
Demo
✨ Live Demo: https://francistralt.github.io/HandTracker/
This is what the website looks like when you first visit:
Note: Make sure to enable your camera on. Otherwise, it would not work!
These are the settings:
- Define Classes: You can create class that will correlate with the gesture you made. For example, if it is for ASL, you can do the “A” gesture, and it will output the class “A”.
- Collection: You can modify how many data samples you would like to capture for each class. It is recommended that for all classes, it needs to be the same amount.
- Training Parameters: You can modify the training parameters for your liking depending on the sampling size you decide to train.
- Once you train all the classes and modify your training parameters, then you can click “Begin Training” and let the magic happens!
Note: During training, the training is collecting data if the hand is visible. If the hand is not visible, it pauses.
After you train the model, you can test out the features as shown here:
You also have the ability to download the model as a zip folder. The contents in that folder includes
Model.json
Model.weights.bin
Model_meta.json
How I Built It
This is built using Vanilla HTML, CSS, and JS. Additionally, I uses ml5.js to perform Machine Learning related stuff and p5.js library. This is fully built using Google Gemini and the code is 100% coded by Gemini.
Code
Feel free to see the code here: https://github.com/FrancisTRAlt/HandTracker
If you are interested in contributing, feel free! The whole goal is making this tool free to use and I want to expand features where it is not just the Hand Pose. Imagine using it to train models that does Image Classification, Text generation, etc!
Any questions/comments? I would love to hear from you!
Top comments (16)
Wow, great project Francis! and even though I'm still trying to wrap my head around it. From what I've learned, it appears to be a game-changer for those who want to train hand pose models without having to sign up or pay for API keys. I'm really curious to see how this project will shape up in the future - it has the potential to make a significant impact on the community.
Hey Aryan! Thanks for reading! I appreciate the comment. I think I could have word it better. My apologies if it didn't make sense from reading, but glad you understood it!
Hey Francis, really cool project!
I've actually been wanting to try making one of these hand gesture projects myself, especially after seeing a similar one that used Naruto's shadow clone gesture. And I couldn't agree more about the struggle of not having many free resources for creating your own models, and that whole 'sign up here, pay here' dance, truly hate it. So, it's awesome that you're providing this kind of freedom.
I'm definitely curious about trying it out and maybe even contributing!
Thanks Viv! Glad you liked it! Note that this is an MVP (since it was coded with Gemini), so there is a lot to work with and I might need to add a sample code in that website in order for the model to work in your project (since it is not simply a drag and drop to your project of choice). Again, thanks for reading! Let me know if anything! :D
This is really cool — the fact that you can download the trained model files and use them offline is huge. Most tools lock you into their ecosystem and that's exactly why a lot of hobby projects never get past the prototype stage.
Quick question: how does the model handle variations in lighting and hand distance from the camera? I've worked with MediaPipe hand tracking before and the landmark detection can get pretty noisy when lighting conditions change. Did you find ml5.js handles that well enough, or did you have to do any preprocessing/normalization on the landmark data before training?
Also love the Domain Expansion reference lol. Now I want to build a gesture-controlled Jujutsu Kaisen soundboard.
Hey Kai! Hope you are well! I appreciate you enjoyed it!
For your question, it handles lighting pretty well. If it was image classification, then it won't since it captures the whole image. ml5.js is built on top of Tensorflow.js and it allows to use machine learning on the web quite easier since I believe it handles most of the conditions you mentioned. I did test it out when all the lights are off and it seems to detect it just fine. I did have to normalize the landmark data, which is expected since you don't really want to mess with raw data.
I link you the documentation for ml5.js and mediapipe for reference just in case, even though you have experience already. Just in case :)
docs.ml5js.org/#/reference/handpose
github.com/google-ai-edge/mediapip...
Hope this helps and thanks for reading!!
“HandTracker” is not just another demo — it actually solves a real problem that many beginners in AI/ML face. When people start learning machine learning, they often run into platforms that require sign-ups, API keys, limited free tiers, or paid subscriptions. That can be discouraging, especially when you just want to experiment and learn.
Here you learn more Cheats and Codm Injectors
The fact that you built a completely free, client-side tool where users can train their own hand pose model and download it without depending on external APIs is incredibly valuable for the community.
I especially love that you built this using and with vanilla HTML, CSS, and JavaScript. That makes it approachable for web developers who are curious about AI but might feel intimidated by Python-heavy ML ecosystems. It really lowers the barrier to entry.
Some standout features that I found particularly powerful:
The ability to define your own custom classes
Controlling how many samples are collected per class
Adjustable training parameters
Automatic pause when the hand is not visible (great for dataset quality!)
Downloadable model files, model.weights.bin, model
That last point is huge. Giving users full ownership of their trained model instead of locking them into a hosted service is empowering. It also makes this tool perfect for projects like Chrome extensions or fully client-side apps where hiding API keys can be difficult.
I also appreciate how creative this can get. It’s not limited to something formal like ASL — users can create any gesture they want, from simple “Hello” signs to fun pop-culture-inspired poses. That mix of creativity and machine learning makes learning more engaging and memorable.
Another thing worth highlighting is your vision for expansion — turning this into a broader ML playground for image classification, text generation, and more. If you continue building in that direction, this could evolve into a powerful open, community-driven ML training platform for beginners.
Overall, this feels bigger than just a weekend challenge submission. It’s a thoughtful, practical tool that makes AI more accessible and hands-on. Huge respect for building something that removes barriers instead of creating them.
Hey Deshawn! Hope you are well. Thanks for taking the time to read and reflect! I appreciated it! :D
Hi dude, I've got actually a question. I'm not coming from AI field, so could you tell me what for have u built such project ? I mean are you collecting the data on finger points and movements, so that you could build a robotic hardware hand, or what is it for ?
Hey Luftie! Thanks for reading.
The MVP I have is essentially collecting the whole hand (as you mentioned) and can use that ML model for other projects. From the example, I was doing ASL (As I demonstrated doing the ABC since I named 3 of my classes A, B, and C). ml5.js is more towards the creative coding side, but can be used for other applications.
It can be use for robotics like you mentioned, but I don't know the exact use case where you will be needing a camera that captures the hand and the robotic hand moves along with it (Unless you are building a remote robotic suite, sure). You might need to do more since I am not an expert in the hardware side of development. Here is the documentation I followed:
github.com/google-ai-edge/mediapip...
docs.ml5js.org/#/reference/handpose
Overall, it can be use for anything as long as you are using the Hand pose model. I hope this makes sense. Let me know if it doesn't or if there is any other questions. Thanks for reading man!
Hey, cool project!
Thanks Matsync! Glad you enjoyed it!
Wow, it's great!!! It really cool..😍😍
Thanks meherab! Glad you liked it! :D
Some comments may only be visible to logged-in visitors. Sign in to view all comments.