DEV Community

Cover image for Facial Detection in JavaScript using Tracking.js
Ali Spittel
Ali Spittel

Posted on • Updated on

Facial Detection in JavaScript using Tracking.js

Earlier this week, I saw a really cool article about how to create Snapchat like filters in Python. I was curious whether similar opensource technology existed for the frontend. I found a couple resources:

I was really excited that this data-sciencey technology existed on the front-end instead of just in more traditional data science languages like Python or R.

I thought all of the above projects were very interesting, and they all had relatively similar star-level's on GitHub. I decided to use Tracking.js because the documentation was really good, and there were a bunch of examples which, for me is the easiest way to learn! I do wish there was better documentation on what was going on behind the scenes with this library -- I'm not sure what statistics are going on or how the tool works past implementation.

That being said, it was very easy to implement. I could easily extend the face camera example for the purposes of the app I ended up building!

The Learning Process

Since Tracking.js is a smaller library with less of a community behind it than I normally end up using, my learning was pretty limited to looking at the examples on their website. I did search Codepen, but the few pens on there seemed to be incomplete or very similar to the examples.

The Final Project

Warning: I definitely learned throughout this project that webcam selfies are super awkward! I have no idea how I had a webcam selfie as my profile pic on Facebook back in high school!

I started off by copying the face camera example on the tracking.js site. I ended up getting it working locally, with a few tweaks, and with downloading the tracking library locally.

the example code

Then, I found a couple PNG's online to layer on top to be the filters. I just used Google Image search and then copied them locally. I then implemented a simple algorithm for adding the filter to the face -- I just hard-coded the dimensions based on where the user's face was. It ended up looking like this:

I also tried to implement the dog face that Snapchat has, but the math turned out to be too specific to the person's face for something as quick as this project!

I then generalized the math and added the ability for the user to switch between the filters. The math was relatively easy -- the event listener that fires when the person moves on the screen returns an array of the coordinates of the faces within view. Then I would draw the filter on top using my improved coordinates. The key code looked like this:

    context.drawImage(img, rect.x + (filterX * rect.width),
      rect.y + (filterY * rect.height),
      rect.width * filterWidth,
      rect.height * filterHeight
Enter fullscreen mode Exit fullscreen mode

After that, I polished up the CSS, which was really hard! The way this ended up working was layering a HTML Canvas over a video element, so getting the grid system to align everything was really tricky. I ended up plugging in CSS Grid for the first time to see if that would work. It ended up working, but the way I did it felt hacky. I will have to keep looking at CSS Grid in the future!

The app ended up looking like this:

My final output seems okay, it is by no means perfect, but to get it better I would probably have to create my own library or my own filters. I also really struggled to get more complex filters to work. I would have had to break them up into their parts -- like each ear and the nose -- and then figure out the math to add these to each face. I seemed to have issues linking the different facial features back together if I wanted to track the eyes or mouth separately, rather than the face as a whole.

If I wanted to put more into this project, I would probably have also tried to add some smoothing so that the filter jumps less when a person moved a tiny bit. Overall, the project probably could have been better but I achieved what I wanted to in order to learn the library.

Next Steps

Tracking.js is really cool and well documented! I think its a great library for doing something simple like this app. If I were to do this for a job or something larger, I would probably have to improve a lot of the functionality in the app. I also found the webcam API to be shockingly difficult to use -- I struggled to find examples of styling and picture taking online. I would also like to look into it more in the future. Overall, I had fun with this! It isn't perfect, but it was a good quick project.


Part of my On Learning New Things Series

Top comments (18)

z0al profile image

Come on! How do know what I want to learn next? You are usually a step ahead of me ;)

Great post, thanks

aspittel profile image
Ali Spittel

Haha thanks!

ioedeveloper profile image
David Disu

You are awesome!!!

kwabenberko profile image
Kwabena Bio Berko


frontend_fairy profile image
Freyr H

Fantastic work! You could try implementing lerping to the position of the overlay to combat the jagged movements!

semosem profile image


goatsaretasty profile image

This is super duper cool

kidsclimatecha1 profile image

dog And CatAndSkeletonAndLionAndTigerThat's it
Can you please add those filters inThat website
Thank you

michael profile image
Michael Lee πŸ•

@juantalon hahahaha!!! Awesome :)

aspittel profile image
Ali Spittel

Ah! That's super exciting!

annarankin profile image
Anna Rankin

This is so cool! πŸ’― Did you have any issues with the library's speed at all?

enzoftware profile image
Enzo Lizama Paredes

Just amazing <3

saiky93 profile image

Hi, I want to know how we can integrate tracking.js into angular 2 application?
I want to use tracking.js to detect the faces from the webcam and do face recognition.

ananthavijay profile image

how to add take photo command while using this?

ananthavijay profile image

How to add take photo command? so that user can take photo while using it?

michael profile image
Michael Lee πŸ•

OMG! My 2yo LOOOOOVVVVVEEEESSSSS bunnies! I'm going to show her this and she is going to flip out love this!!! Thanks Ali!!!! This is awesome :)