DEV Community: Blue Tomato Dev

For fun and learning: playing around with JavaScript face detection and html5 canvas

Markus Tiefenbacher — Fri, 10 Apr 2020 11:36:52 +0000

It’s easter and wanted to make some fun for my coworkers and what should be better than some Covid-19 fun in this surreal times.

I had the idea to put every model in our webshop a surgical mask on their face.

But how should I solve this?

Face Detection!
This was a topic I ever wanted to discover and so it was a perfect match for this. I started reading the face-api.js documentation and after some hours of playing around with it I was ready to ship the easter egg on our testing server.

Result:

Try it yourself:

I made also a version where you can try it yourself: https://tiefenb.github.io/use-a-mask/

... and here are the source code: https://github.com/tiefenb/use-a-mask

Reactions so far:

Learnings:

Loading images for HTML5 canvas is only possible with images with crossorigin attribute
crossorigin attribute has to be set on the image tag before the image is loaded - it’s not possible afterwards. That means also, if you create the image element with JavaScript you have to add the crossorigin-attribute before the src-attribute.
Rotating an single image in html5 canvas is not possible in the way most of us think. you have to rotate the canvas-context, than you put on the canvas the image and you rotate the canvas-context back to your normal angle
Putting an image on a canvas is only possible after the image is full loaded - so you have to listen in the image.onload event
face-api.js tensorflow.js-models are quite big - better to make detection it on the server not the client

Thank's for reading and happy easter!

Dear appear.js, I have met someone new. Her name is IntersectionObserver.

Markus Tiefenbacher — Tue, 14 May 2019 13:19:10 +0000

Dear appear.js, I have met someone new. It was more or less a good time with you. But I want to break up and you have to leave your stack.

The new one? Her name's IntersectionObserver. She is nice to me. She is smart, pretty syntax and very fast in doing things. I think I will have a good time with her.

Enough with joking.
At blue-tomato.com, we used some jQuery based image lazy loading library and appear.js for lazyloading things and doing effect based on elements in the viewpoint.

Both libraries listen on the scroll event and looking if the actual scroll position is in the range of the given elements position.

This technic was the default for long time, but since IntersectionObserver comes up, it was clear to switch sometimes to this. The main problem with the old method was the listing on the scroll-event was very expensive to browser performance.

Example

Making a navigation bar sticky with appear.js

In this example, after the .main-content became visible, the .navigation bar will get the CSS class .navigation--sticky and will become sticky. When .main-content disappers (e.g. by scrolling back to the top of the page) the .navigation bar will become static by removing the .navigation--sticky class.

appear({
  elements: function elements() {
    return document.querySelectorAll('.main-content');
  },
  appear: function appear() {
    $(".navigation").addClass('.navigation--sticky');
  },
  disappear: function appear() {
    $(".navigation").removeClass('.navigation--sticky');
  }
});

Making a navigation bar sticky with InteractionObserver

With InteractionObserver this would work in this way:

var element = document.querySelector('.main-content');
if(element) {
  let observer = new IntersectionObserver(entries => {
    entries.forEach(element => {
      if(element.intersectionRatio > 0) {
        $(".navigation").addClass('.navigation--sticky');
      } else {
        $(".navigation").removeClass('.navigation--sticky');
      }
    });
  }, { rootMargin: '0px 0px', threshold: 0.00 });

  observer.observe(element);
}

Whats that rootMargin and thresold?

Appear.js has only some "bounds" option to let it fire some pixels before the element is really in the viewport. IntersectionObserver has a really powerful and flexible method to control this via the root/rootMargin and thresold options. Have an eye on this article from Smashing Magazine where Denys Mishonnuc explains this very detailed: https://www.smashingmagazine.com/2018/01/deferring-lazy-loading-intersection-observer-api/#intersectionobserver-initialization

src/.../styled.js - stairway to Styled-Component's CSS-in-JS heaven

Manuel Penaloza — Tue, 08 Jan 2019 10:30:39 +0000

Using a fabulous open source library like Styled-Components often comes with the nice effect of appreciating what you can do with it more and more and more. If you are like me, at some point you get curious about what happens behind the (library-)scenes. I recently checked the Styled-Components Github repository to know better what's going on when using the library. As you can imagine the library does A LOT of cool things, so in the first attempt I limited my investigations to the library's practical entry point.

Remember, creating a styled component with the Styled-Components library is as simple as a that...

const Button = styled.a`
    font-size: 14px;
`;

...or as that

const BorderedButton = styled(Button)`
  border: 2px solid white;
`;

Please pay special attention to the differences of how the styled function is used in the examples above.

we call a as a method of the styled object passing it css as an argument
we call the styled function passing it the previously created styled component as an argument and add css as a next argument

So how does the library provide that exposed possibilities for us? Let's take a look.

A CSS-in-JS holy grail named styled

Theoretically and taking into account Styled-Component's package.json information, the technical runtime entry point to the library is the src/index.js file. But that said, the practical entry point enabling us to do all the cool things is the styled.js file being part of the /src/constructors directory. Next see what this file looks like (status 01/2019, branch master):

In a nutshell this file does the following:

some internal imports (lines 2-4)
one const styled declaration referencing a function (line 8)
one forEach loop (lines 11-13)
finally the export of our holy grail styled 🎉 (line 15)

The styled function and its function object methods

Confused of that headline? Remember, in Javascript functions are objects! As a result developers can make use of this by i.e. adding methods to that function object. Styled-Components and the real entry point returned by styled.js makes heavy use of this. Line 8 shows a const styled declaration referencing a function, but more on that later. For now just keep in mind that a function referenced by the styled const was created. Now take a special look at the lines 11 to 13.

We see a forEach loop iterating an array of defined domElements. Each iteration attaches a method with the name of the iterated dom element to the styled function object. And voilá! There it is, the first hard-working component factory attaching dom element methods to the styled function object like a boss. The body of each attached method is nothing else than the result of a styled function call returning a properly built component constructor for each dom node. Having reached this point at runtime we can already call such an attached method like this in our own codebase:

const Button = styled.a`
    ...this will be the method argument in the form of css to the styled.a call...
`;
const Container = styled.div`
    ...this will be the method argument in the form of css to the styled.div call...
`;

So far we know (some kind of) what's going on when we use styled.a or styled.div in our codebases using Styled-Components. That said we've gone "half the way" to demystify what styled.js exports for us. Remember when I wrote "Line 8 shows a const styled declaration referencing a function, but more on that later."? It is that function we need to talk about more right now to demystify the other half. Here we go.

The styled function: how it's constructed and distributed

Constructed and distributed? Yes!
As you can see in line 8, styled itself is declared as a function. It is either called directly by us developers using the distributed default export of the styled function or by calling the function object method which requires internal construction as described above. So in concrete this means for us:

This...

const Container = styled('div')` ...css... `;

and this

const Container = styled.div` ...css... `

returnes exactly the same. But remember: we can conveniently use the second option in our codebase, as Styled-Components takes care of building the proper component constructor executing first option internally to add the function object method (here: .div).

The styled function takes a tag as a parameter and returns the result of calling the function constructWithOptions (line 8) which receives two arguments.

a StyledComponent function - Have you noticed I never mentioned React in this post so far? Well, here we go. Let's not forget Styled-Components is part of the React ecosystem. As a result this StyledComponent argument is used to actually create a React component giving access to React EventHandlers and all the other cool things React does for us.
the tag we've passed to the styled function - The argument is of type Target which is either a dom node string or an already styled component. Check this line to find the origin of how the Target type is defined using Flow. The tag argument is simply passed through to constructWithOptions without any modifications.

Receiving these two arguments, the constructWithOptions() function execution returns another function that let's us create a styled component already considering the tag we want to use in our codebase. Using the returned function in the most basic form, the only thing to we need to pass it is pure css within a tagged template literal argument. Check out this post of Max Stoiber in case you want to go into detail about how this part of Javascript works.

So to close the cycle of styled.js... Now as we know what the styled function returns and how it is distributed or used for internal construction, we also know better what the different method bodies of the styled function object methods do. styled.a works as the styled(domElement) execution in line 12 of styled.js distributes a "tag-predefined" styled component constructor to it! 🎉

Thx for reading!

As you can see, styled provides a lot of things. On the one hand a function, on the other hand it acts as an object providing predefined methods for each dom node to build styled components. In my opinion this approach the library provides us to style our components is pure gold.

I hope this post helps you to better understand what styled.js does and how it exposes stuff we can use. Please note all information is based on the Styled-Component's master branch by Jan 2019. In case there'll be changes I don't notice, feel free to contact me in order to update this post.

This post was originally posted here.

What are those PeerDependencies in a NodeJS project?

Manuel Penaloza — Fri, 30 Nov 2018 20:07:16 +0000

PeerDependencies, one of the terms that brought up confusion at least to me when I got a PeerDependency warning in my terminal like the following:

Recent happenings about a malicious code attack in a node package that heavily include the topic of PeerDependencies finally made me that curious about this topic to start some deeper investigation about how PeerDependencies work. In this blog post I'll write down what I found out about NodeJS PeerDependencies in a way that also might help you to better understand this topic.

Searching for "What are peer dependencies" using Google - of course - returns some results. Nevertheless none of the main references Google returned made me understand PeerDependencies in way I was satisfied with. After some time I found this Stackoverflow Page including a great PeerDependency explanation of Stijn De Witt. His explanation came quite close to a version that made me understand the basics of PeerDependencies and brought up some imaginary "Aha!" moments (Thank you Stijn!). But somehow and as I am more a visual learning type Stijn's "text-driven" Stackoverflow explanation did not bring up that imaginary last-mile satisfaction for me in terms of understanding PeerDependencies. As a result I draw some code around his explanation (you can see quoted below) and suddenly things became more clear to me.

What's the problem?

Upfront: in the upcoming example, JillsModule will be the tricky part (subsequently the PeerDependency) of the process. That's why I added fictional version appends (@1.0, @2.0) when using it.

Let's say we are building OurCoolProject and are using JacksModule and JillsModule@2.0. Also let's suppose that JacksModule also depends on JillsModule, but on a different version, say JillsModule@1.0. As long as those 2 versions don't meet, there is no problem. The fact that JacksModule is using JillsModule below the surface is just an implementation detail. We are bundling JillsModule (as the code uses 2 different versions, but not related to each other (!)) twice, but that's a small price to pay when we get stable software out of the box.

In code this means something like

// OurCoolProcject.js

import JacksModule from 'jacksmodule';
import JillsModule(@2.0) from 'jillsmodule(@2.0)';

const OurCoolProcject = () => {
    // do some stuff with JacksModule
    // do some stuff with JillsModule(@2.0). stuff won't break as we have the compatible @2.0 version of JillsModule available in this scope.
}

export default OurCoolProject;

// jacksmodule.js (an npm module)

import JillsModule(@1.0) from 'jillsmodule(@1.0)';

const JacksModule = () => {
    // do some stuff with JillsModule(@1.0). stuff won't break as we have the compatible @1.0 version of JillsModule available in this scope.
}

export default JacksModule;

But next this dependency relationship gets more tricky.

Now let's suppose that JacksModule exposes its dependency on JillsModule in some way. It accepts an object instanceof JillsClass for example... What happens when we create a new JillsClass using version 2.0 of the library and pass it along to jacksFunction (that excepts the 1.0 version as we know!)? All hell will break loose! Simple things like jillsObject instanceof JillsClass will suddenly return false, because jillsObject is actually an instance of another JillsClass, the 2.0 version.

In code this means something like this:

// OurCoolProcject.js

import jacksFunction from 'jacksmodule';
import JillsModule(@2.0) from 'jillsmodule(@2.0)'; // node resolves to OUR dependency of JillsModule which is 2.0!

const OurCoolProcject = () => {    
    const jillsObject = new JillsModule(@2.0).JillsClass;

    // next the beginning of all evil, we'll pass a jillsObject of version 2.0
    // to jacksFunction (that would expect jillsObject of version 1.0 🤦‍♀️)
    jacksFunction(jillsObject); 
}

export default OurCoolProject;

// jacksmodule.js (an npm module)

import JillsModule(@1.0) from 'jillsmodule(@1.0)';

const jacksFunction = (jillsObject) => {
    // make sure jillsObject is compatible for further usage in this function
    const jillsObjectRocks = jillsObject instanceOf JillsModule(@1.0).JillsClass;
            // └─> 🔥🔥🔥 `jillsObjectRocks` will be a big, fat FALSE
            // as the JillsModule dependencies actively used in this function and
            // passed to this function differ in versions (1.0 vs. 2.0) 🤦‍♀️
    ...
}

export default jacksFunction;

Do you notice what's going on here? jacksFunction receives an incompatible jillsObject as the object was constructed from JillsModule(2.0) and not from JillsModule(1.0) JacksModule is be compatible with. So far, this only shows the problem that in the worst case, leads to non-working software.

How PeerDependencies solve this problem

Luckily npm has some built-in intelligence trying to solve this. If JacksModule declares JillsModule(@1.0) as a PeerDependency, npm can warn the user about this when installing dependencies of your project. So JacksModule's package.json should include this declaration:

{
  "name": "JacksModule",
  ...
  "peerDependencies": {
    "JillsModule": "1.x"
  },
  ...
}

So npm's PeerDepenedency intelligence basically triggers a console output notifying us developers with a warning saying this:

"Hey, this is JacksModule speaking here. Let me tell you: I need this specific package of JillsModule, but I really need the version that is part of my JacksModule project and listed in my package.json file. So please make sure it's installed and make sure it's not some other version of JillsModule you might have installed for your own usage somewhere else in your application."

So in the end - thinking this further - depending on npm packages that require PeerDependencies can be tricky. In case you need a new version of the package X for separated usage in your application this might lead to problems if another dependency you use in your application has a PeerDependency on another version of package X. If this pops up - and in the worst case also leads to problems with your software - you are on your own to decide which package to use or which code maybe needs refactoring to meet all requirements.

I hope those explanations and code examples made sense for you and closed the last thought gap you've had about PeerDependencies. If you've questions or want to suggest some article optimization, feel free to contact me or leave a comment.

This post was originally posted here.

SVG-based image lazyloading as a matter of enjoyable UX

Manuel Penaloza — Tue, 06 Feb 2018 11:57:05 +0000

There was that day somewhere in November 2017, when our team came across this article by José M. Pérez including tons of new-to-us stuff regarding image placeholding and lazyloading. Watching the examples mentioned in the article compared to how we handled it that day was impressing. So impressing that we decided to give it a shot and implement the technique of image lazyloading using inline SVGs on our blue-tomato.com content section named Blue World.

pretty much the transition we've strived for

Apart from the impressing "magic-happens" moment when the SVG resolves to the high-resolution image, we encountered even more advantages that improve user experience:

immediate visual response when DOM is ready, as SVG placeholders are inlined and thus part of html source code
reduce amount of requests to get all small-pixeled lazyloading initial placeholders

Mentioned upfront: we managed to implement it and have already deployed it, so check it out in our Blue World Section or take a quick look in the video up next to see the implementation.

Implementation - slowmo showcase

The development journey went along with technical challenges, conceptual tasks and finally some rewrites of the implementation. So in case you like SVG-based image lazyloading and want to implement it on your own, check our upcoming hints and implementation notes so you can do this too (without those annoying rewrites). The main stack we used to accomplish it consists of the following awesome libraries:

Primitive: Golang tool reproducing images with geometric primitives
Svgo: Nodejs tool for optimizing SVG vector graphics files
appear.js: execute callbacks when DOM elements appear in and out of view
Processwire: PHP-based cms; no need to go with that cms, choose whatever cms you want following the concepts and solutions we suggest up next

Implementation details SVG-based image lazyloading

We won’t go through every line of code we shipped to our repository to make SVG-based image lazyloading happen. Our code is tightly coupled to our internal systems (primarily Processwire cms) and it wouldn’t make sense in terms of giving you a good overview of how we solved the implementation. Instead we will equip you with our roadmap we’d consider today if we’d have to do it again from scratch. Given below you'll be guided through these main steps:

Generate and save SVG placeholders
Check for SVG availability and ship it
Proper SVG lazyload placeholder frontend handling

1st Generate and save SVG placeholders

Challenges
Our first approach was to generate the SVGs during the upload process in the admin panel of our cms. The first problem we faced was that the upload process of an image would take longer due to the converting process we triggered using the excellent Primitive library. But this was the smaller of the two problems. The second issue was that our image assets management system (Adobe Scene7) created all required image formats on the fly. So initially we created only one SVG using the format of the uploaded image without considering other formats handled by the image assets management system.

Different image formats require different SVG formats

Example
Uploaded image format:
15:4

BUT formats used in our templates:
skateboard-image.jpg?format=16:9
skateboard-image.jpg?format=4:3
skateboard-image.jpg?format=1:1

As we've only had that one SVG based on the original upload image width/height ratio that sometimes led to awkward looking effects transitioning from that one SVG to the original images once those were loaded from the server.

Our second approach was to generate the SVG file on the fly and cache the whole generated html files having all required SVGs inlined. We knew the initial process of producing and caching the whole markup file including all SVGs would take long. But in the end it would not affect user experience. So was our assumption :). Well, finally we ran into http timeout issues our editors would struggle with when saving pages that require a long SVG generation loop.

Solution
Our third approach led to our current working solution. If an editor uploads an image and the SVG is not yet produced in our network file system we automatically create a "svg-to-be-produced-soon.json" file. This file includes an information object about the SVG (dimensions, image server source, hashed name, etc.) that needs to be produced.

{  
   "jpgTmpFilePath":"/root_path/assets/primitives/tmp/kwleklsl090309f0g0.jpg",
   "svgFilePath":"/root_path/assets/primitives/svg/kwleklsl090309f0g0.svg",
   "svgMinFilePath":"/root_path/assets/primitives/svg/kwleklsl090309f0g0.min.svg",
   "bgColor":null,
   "inputUrl":"http://s7w2p3.scene7.com/is/image/bluetomato/ugc/2-1508153542.tif?$mc1$",
   "inputUrlHash":"kwleklsl090309f0g0",
   "env":"production"
}

Example of a svg-to-be-produced-soon.json file

As you can see, this svg-to-be-produced-soon object is quite "hash-intense". We use that hash - being taken from the inputUrlHash property - to match the image and the SVG to have a proper asset identification process starting at the SVG creation up to the usage in our template files.

So each new svg-to-be-produced-soon.json file is part of a queue a cronjob will take care of and trigger on an hourly base to generate the SVGs. Let's see what it does.

#!/bin/bash

for jsonFile in /root_path/primitives/queue/*.json
do
  echo "Processing $jsonFile"

  primitiveBinary="${root_path}tools/primitive" 

  # primitive image
  goPrimitiveCommand="$primitiveBinary -i $jpgTmpFilePath -o $svgFilePath -n 64 -m 1 -r 256 -bg='$bgColor'"
  eval $goPrimitiveCommand

  # svgo
  svgoBinary="${root_path}node_modules/svgo/bin/svgo"
  svgoCommand="$svgoBinary $svgFilePath -o $svgMinFilePath"
  eval $svgoCommand

  rm -f -- $jsonFile
done

echo "All SVGs created"
exit

The simplified cronjob (we removed our system-related code that'd lead to confusion) does four main things.

loop over and get each new svg-to-be-produced-soon.json file
get the primitive binary and execute it to create the SVG
get the svgo binary and execute it to optimize the created SVG code
delete the svg-to-be-produced-soon.json file

The time between uploading the images and publishing the page is usually more than one hour. As a delightful result mostly the already available SVGs are shipped to our users instead of our (meantime internally uncool ;)) lazyload fallback option.

So we finally got it! We have our created and saved SVGs. Let's go the next step and ship fancy and performant placeholder images to our users.

2nd Check for SVG availability and ship it

Once the SVGs creation is done things get easier and more straight forward. We've defined a createPrimitiveSvg function in our code base that takes an image and its format as arguments and returns the final SVG code packed into a base64-based data-uri.

function createPrimitiveSvg($image, $format) {
  if(inSvgCache($image)) {
    $svgFromCache = loadFromSvgCache($image); // svg already available, get it
    return makeBase64DataUri($svgFromCache);
  } else {
    pushToSvgGenerationQueue($image, $format); // svg not available, make sure it will be made
    return getBlurryImageUrl($image); // ship old lazyload fallback
  }
}

Our PHP createPrimitiveSvg function processing SVGs for our templates

Further on using this function and passing the image's root data from our cms to it looks like this.

<div class="imageContainer">
  <img
    class="js-lazyload"
    src='{$image->url|createPrimitiveSvg:"16:9"}'
  />
</div>

<div class="imageContainer">
  <img
    class="js-lazyload"
    src='{$image->url|createPrimitiveSvg:"4:3"}'
  />
</div>

3rd Proper SVG lazyload placeholder frontend handling

appear.js does a great job checking whether a targeted asset in the DOM has reached the viewport. Using it to do so we can manage lazyloading on the frontend using pretty much lightweight code.

appear({
  elements: () => document.querySelectorAll("img.js-lazyload"),
  appear: (element) => {
    const img = $(`<img class='${element.className} image--fadeIn' alt='${element.alt}' />`)
      .attr('src', element.dataset.original)
      .bind('load', () => {
        img.insertAfter(element);
        window.setTimeout(() => {
          $(element).remove();
        }, 2000);
      });
  }
});

Let's see what this does:

Call appear(config: Object) initializing all config object properties appear.js requires
When the lazyload SVG image has reached the users's viewport, the appear callback handles the high-resolution image tag creation
Once the high-resolution image has loaded, it will be inserted behind the already displayed SVG
The SVG image will fade out using CSS3 and its blur transition options
Removal of the lazyload SVG image from the DOM after 2 seconds

That's it!

Conclusion

We are super happy with the results and gains we profit from having SVG-based lazyloading implemented in our Blue World. In case you like the perceived user experience at least a little bit. Give it a try. It makes so much more fun seeing all of this happening in your own projects. Happy coding, happy lazyloading!