Make the whole web your playground

For some time I have dabbled in user scripts and user styles. Whenever I wanted to test something I read about or some idea I had I just wrote a simple script. Some of the cool things about user scripts are that I can get started right away and that I always have some basis for my ideas.

In this post, we are going to explore a little bit of what user scripts are capable of and how you can get started with using them too. And to get a glimpse of what I can do I have put together a little example at the end.

Getting started

To get started we have to be able to execute our user scripts. My preferred method is Sprinkles, although it is only available through the Mac App Store for now. However, any user script web extension will do, like Greasemonkey, Tampermonkey or the like.
If you do not use Sprinkles, you might want some extension that can apply your styles to web pages, like Stylus or stylish.

Note: You should generally be careful about user scripts, especially those you did not write yourself.

Creating something

Well, you have added an extension that lets you write and execute user scripts, now what? We create a basic HTML DOM element and append it to the body of a website to show the webpage who the boss is

const buttonElement = document.createElement("button");
buttonElement.innerHTML = "Hello world";
buttonElement.className = "hello-world__button";

document.body.appendChild(buttonElement);

And add some styling in a user style such that the button is nicely placed in the middle of a webpage

.hello-world__button {
 position: absolute;
 left: 50%;
 top: 50%;
 transform: translate(-50%, -50%);
}

With our newly created "hello world"-button we are ready to make modifications to any webpage.

You can do anything

You do not require any library to do what you want. Everything is possible. Libraries and frameworks make things easier, but when using any library or framework like Angular or React, it is important to remember that it all boils down to regular HTML, CSS, and javascript in the end. This means that even though it feels like it, our power has not been limited just because we only use plain javascript.

Doing Something Useful

So what should we do with all that power? Well, why not hook up a hotkey to add googly eyes to all faces on the page you are looking at?

Introducing face detection in your browser (coming soon)

For now, face detection is a part of the 'Experimental Web Platform features' that you can enable on Chrome and Opera. Getting started with the Face Detection API, we do the following to initialize the FaceDetector

const faceDetector = new FaceDetector({
 maxDetectedFaces: 5,
 fastMode: false
});

Note: A little more information is found here

We are pretty much ready to go after that. We start by listening for a hotkey combination on a keydown event and inside this event is where all the magic is going to happen.

const onKeyDownEvent = (event) => {
 if (event.code === "KeyG" && event.altKey && event.ctrlKey) {
 // Do magic here
 }
};
document.addEventListener("keydown", onKeyDownEvent);

When making something small I always like to note down what the intended order of events should be.

The order of events in this situation, when the right key combination is being pressed, should be

1. Get all images on the page.
2. Detect all the faces on each image.
3. Calculate the x and y-position for each eye found.
4. Draw a googly eye for each found eye placed at the calculated position

My implementation

First of all, here is my implementation

const faceDetector = new FaceDetector({ maxFacesDetected: 1, fastMode: false });

const placeEye = (x, y) => {
 const eye = document.createElement("div");
 const innerEye = document.createElement("div");
 eye.appendChild(innerEye);
 eye.classList.add("eye");
 innerEye.classList.add("inner-eye");
 eye.style.left = x + "px";
 eye.style.top = y + "px";
 innerEye.style.left = 10 + Math.random() * 80 + "%";
 innerEye.style.top = 10 + Math.random() * 80 + "%";

 return eye;
};

document.addEventListener("keydown", (event) => {
 if (event.code === "KeyG" && event.altKey && event.ctrlKey) {
 const images = Object.values(document.getElementsByTagName("img"));
 images.forEach(async (image) => {
 const faces = await faceDetector.detect(image);
 faces.forEach((face) => {
 face.landmarks.forEach((landmark) => {
 if (landmark.type === "eye") {
 const averageX =
 landmark.locations.reduce((prev, curr) => prev + curr.x, 0) /
 landmark.locations.length;
 const averageY =
 landmark.locations.reduce((prev, curr) => prev + curr.y, 0) /
 landmark.locations.length;
 const eye = placeEye(
 averageX + image.offsetLeft,
 averageY + image.offsetTop
 );
 image.offsetParent.appendChild(eye);
 }
 });
 });
 });
 }
});

With some styling

.eye {
 background-color: white;
 width: 15px;
 height: 15px;
 border-radius: 15px;

 position: absolute;
 overflow: hidden;

 z-index: 100;
 transform: translate(-50%, -50%);
}

.inner-eye {
 position: absolute;
 background-color: black;
 width: 8px;
 height: 8px;

 transform: translate(-50%, -50%);

 border-radius: 8px;
}

For clarity, I am going to explain a little bit about it down below.

const images = Object.values(document.getElementsByTagName("img"));

It might be somewhat illogical that we have to wrap document.getElementsByTagName("img") in Object.values(...), but the reason for this is that otherwise we are left with a HTMLCollection which is not traversable. By treating the HTMlCollection like an object and only caring about its values, we get a list of 'img'-elements that we can traverse.

images.forEach(async (image) => {
 const faces = await faceDetector.detect(image);
 ...
}

the ´detect´ method from faceDetector returns aPromisewhich returns its result when resolved. This is why the function is an async arrow function and theawait` keyword is prepended the method call such that it waits for the promise to resolve.

javascript
faces.forEach((face) => {
face.landmarks.forEach((landmark) => {
if (landmark.type === "eye") {
...
}
...
}
...
}


Here we traverse through the faces discovered. Each face has a boundingBox that encapsulates the area of the face detected and some landmarks. These landmarks tell us where the eyes, the mouth, and the nose is placed. Each of these landmarks has a type, eye, mouth or nose, and some locations for each. An example can be seen here.

javascript
...
const averageX = landmark.locations.reduce((prev, curr) => prev + curr.x, 0) / landmark.locations.length;
const averageY = landmark.locations.reduce((prev, curr) => prev + curr.y, 0) / landmark.locations.length;
...


As of this example, I just find the average of the locations as there is not a lot of information about these for now.

javascript
const eye = placeEye(averageX + image.offsetLeft, averageY + image.offsetTop);
image.offsetParent.appendChild(eye);


I append the immediate parent of the image with my newly created googly eye. To get the correct position for the eye inside the parent element, the offset to the left and top of the image in relation to the parent element has to be added to the x and y respectively.

The placeEye function is pretty straight forward, as it creates two div-elements and nests one inside the other, gives them both class names such that we can style them, and then sets the outer element's position to the given position and places the inner div in a random position inside the outer element.

Pressing the right key combination on any webpage now results in googly eyes galore.

Closing remarks

This is just a quirky example of what can be done relatively simply by user scripts and user styles. The implementation is not anywhere good and could easily be improved, but I think it is good enough as an example of what can be done with just a little bit of javascript, CSS, and creativity.

Comments

[GrahamTheDev]

It would be interesting to see if this works on my website klu.io as I am not sure if my content security policy would block you running sprinkle inserted scripts or not, does it have an equivalent of unsafeWindow like tamper monkey?.

[Christian Hundahl]

As of right now, it does not seem like Sprinkles has an equivalent of unsafeWindow. The unsafeWindow does also somewhat defeat the purpose of content security policies in the first place. A website having a strict content security policy hopefully has a valid reason for this and as such if one feels the need for user scripts, the website in question might have chosen the wrong content security policy.