Lottie under the hood (part1)

Stepan Mikhailiuk — Wed, 14 Jun 2023 05:54:12 +0000

Before you begin reading this article, please take 15 seconds to visit the what is lottie? page. It's an excellent starting point for exploring Lottie.

Why do we need to understand what is under the hood?

Lottie is everywhere.

Currently, there are numerous articles available on the topic of Lottie. You can explore them through various platforms:

Each of these sources provides valuable insights into why and how to utilize Lottie in different environments.

Lottie is highly regarded by many for the following reasons:

It boasts a small file size.
It offers infinite scalability.
It enjoys support across various platforms.
Animations created with Lottie can be interactive.

Originally, Lottie was designed as a tool to convert Adobe After Effects animations into a format compatible with specific platforms. However, it has evolved into an independent specification and is no longer bound by Adobe After Effects terminology. You can find the Lottie specification

Recently, LottieFiles introduced a plugin for figma, further expanding the popularity and potential growth of Lottie. This development signifies that Lottie is gradually becoming an industry standard, which is fantastic news for the community.

Nearly every article discussing Lottie highlights its ecosystem and praises its ease of use. It is indeed a straightforward tool that effectively accomplishes its purpose. However, let's shift our focus to the challenges that may arise.

My personal experience with Lottie pertains specifically to web development, so these observations may not be relevant to other environments.

If you intend to incorporate Lottie animations into a web project, the official documentation will direct you to lottie-web.

Community

Lets look at list of top contributors:

In the first place with 1602 contributions is bodymovin, the creator of the "bodymovin/lottie". It's not surprising that the top contributor is the library's creator.
Coming in second place with 250 contributions is Hernan-dev It's likely the same individual.
In the third place with 15 contributions is knekne who created a helpful introductory video about using lottie on the web

Out of the remaining contributors, only 80 commits were made, which is less than 5% of the total commits. Most of these commits may not be directly related to animation playback, seeking, applying transformations, or optimizing drawing. While these commits are undoubtedly important, it appears that the main bulk of the code is primarily maintained by bodymovin

Issues

On one hand, the maintainers of Lottie are doing an excellent job by closing numerous issues. However, it's worth noting that the presence of 600 open issues does indicate that there are ongoing challenges and areas that require attention. This suggests that there is still work to be done to address and resolve these outstanding issues.

Tests

As of my knowledge cutoff in May 2023, Lottie-web did not have any tests available on its GitHub repository. It is possible that the tests were implemented within Airbnb's internal codebase and not publicly visible. However, it's important to note that since I started writing this article, the maintainers have added some tests, which is a fortunate coincidence. This highlights the fact that contributing to Lottie's internals was challenging in the past due to the lack of validation mechanisms. It is encouraging to see that steps have been taken to address this issue with the inclusion of tests.

Features

Lottie-web does not support all features described in lottie format.

Different renderers with different problems

Lottie-web offers support for rendering animations to 2D canvas context, SVG, and HTML.

Each of these renderers presents its own set of challenges. For instance, when using the canvas renderer, it is possible to optimize animation playback by moving it to a web worker through the use of transferControlToOffscreen. This can significantly enhance the performance of the animation logic. However, SVG renderer does not support this capability since it is not possible to access the DOM from within a web worker.

On the other hand, the SVG renderer provides features such as hardware-accelerated blur filters, making updating SVG elements relatively fast. Canvas renderer, on the other hand, lacks built-in support for simple blur filters. Additionally, if an SVG contains numerous objects, it can impact performance. Therefore, when playing an animation, it is essential to strike a balance between performance and the desired features, taking into consideration the capabilities and limitations of each renderer.

You can find different articles about that for example

Or even service which optimizes your lottie,.
and this, lol

It is indeed noticeable that many articles focus on optimizing Lottie files and their structure, employing various techniques. In some cases, these optimizations may necessitate modifications to the original After Effects file.

Readme says

More optimizations are on their way, but try not to use huge shapes in AE only to mask a small part of it.
Too many nodes will also affect performance.

But why playing animation in web can be a problem?...

There is no renderer to `webgl`, `webgpu`

Imagine you're creating a 2D game using WebGL. It would be great to incorporate animations created in After Effects into your game. Although you can draw animations using the canvas element, it lacks certain features. Another option is to render animations as SVG, but to use them in the WebGL context, you need to serialize the SVG, convert it into an image, and then upload that image to WebGL. Unfortunately, at present, there is no ideal solution for this.

Actually, that's not entirely accurate. The lottie-web
library itself doesn't support rendering to WebGL. However, there is a package called canvaskit-wasm that wraps Skia (a graphics engine) with WebAssembly (wasm). This package includes a module called skottie which supports rendering animations into a WebGL surface. However, there is a drawback with this approach: using wasm requires loading a relatively large package, and it's uncertain whether all features are supported correctly, as the official compatibility table that tracks lottie support on different platforms does not include skottie.

History

I often wonder why this technology was only introduced in 2015. The concept seems quite straightforward, doesn't it? Well, perhaps not. My guess is that there weren't many developers with experience in this particular type of software. Anything related to graphics can often feel like a mysterious realm, almost like black magic, especially for someone like me.

What we should do as a community?

It's evident that the lottie-web project is in need of assistance.

How we can help?

Donate Contributing financially is one way to support the project. Even a small monthly donation can make a difference. Personally, I contribute $5 each month, and I hope that bodymovin considers the issues I've opened while enjoying a cup of coffee funded by my contribution.
contribute If you have the knowledge and skills, contributing to the project directly is another valuable way to assist. However, keep in mind that contributing can be challenging if you're not familiar with animation basics and the lottie format. That's precisely what this article aims to address!

Demystifying `lottie`

I did a workshop, where I tried to build lottie player from scratch.
Another video can be interesting for you, it is about generating video, but the same time it overlaps with information I'm going to share.

Our plan:

Create simple animation in after effects
Export animation to json using bodymovin
Explore content of json file and it's structure
Create demo where we use lottie-web to render video
Replace lottie-web with our custom renderer to webgl
Compare our renderer design with design of lottie-web

Project in after effects

Let's create a simple animation in After Effects

Create project in after effects
Set up composition

A composition is a framework for a movie. Each composition has its own timeline. A typical composition includes multiple layers representing components such as video and audio footage items, animated text and vector graphics, still images, and lights

Create Solid layer

Here you can find an overview of different layers you can create with after affects.

Let's create just basic, solid color layer.

Here we can specify it's color, width and height.

Ok, this is our animation:

But there is no animation!!!

Is animation the process of transforming something on a screen?
What options do we have for modification?
When you click on a solid layer, you can observe that it has the transform feature available.

You can change position, rotation, opacity, scale, anchor point of layer. Let's change someting:

Actually, this doesn't quite qualify as an animation since we're only altering the position for the initial frame.

Now, it's time to introduce keyframes:
Let's shift our rectangle to the bottom right position.

What does this entail?
For the initial keyframe at time (0), we have specified that the layer should be positioned at (0, 0).
For the second keyframe at time (2s), we have declared the position as (1920, 1080). During the time interval between 0 and 2 seconds, the position will be interpolated using a linear function.

Now, how can we create a non-linear animation?
To achieve that, we can utilize bezier curves. These curves, named after Pierre Bézier, allow for more complex and customizable animations. You can learn more about them here.

In this screencast, we have opened the curve editor and separated the curve for the position (we will delve into this topic in more detail later on).

Therefore, we have the ability to govern the changes in the x and y coordinates over time by employing a cubic bezier.

What exactly is a cubic bezier? You can find a comprehensive explanation on Wikipedia, which provides a valuable resource on this topic. Here's the link:

In order to define such a curve, we only need to specify two points, commonly referred to as in and out control points.

For instance, take a look at this curve It is composed of two control points:

The first control point has coordinates x=0, y=1.
The second control point has coordinates x=1, y=0.

It's worth noting that we can even create a linear animation using a cubic bezier curve. Simply set the control points to be on the diagonal

In order to simplify our first animation, let's roll back to linear animation

Ok, lets store our animation to json, for that we use bodymovin
Follow plugin installation steps

Lottie structure

Let's look at saved file:

{
    "v": "5.10.2",
    "fr": 29.9700012207031,
    "ip": 0,
    "op": 900.000036657751,
    "w": 1920,
    "h": 1080,
    "nm": "Moving square",
    "ddd": 0,
    "assets": [],
    "layers": [
        {
            "ddd": 0,
            "ind": 1,
            "ty": 1,
            "nm": "Red Solid 1",
            "sr": 1,
            "ks": {
                "o": {
                    "a": 0,
                    "k": 100,
                    "ix": 11
                },
                "r": {
                    "a": 0,
                    "k": 0,
                    "ix": 10
                },
                "p": {
                    "a": 1,
                    "k": [
                        {
                            "i": {
                                "x": 0.833,
                                "y": 0.833
                            },
                            "o": {
                                "x": 0.167,
                                "y": 0.167
                            },
                            "t": 0,
                            "s": [
                                0,
                                0,
                                0
                            ],
                            "to": [
                                0,
                                0,
                                0
                            ],
                            "ti": [
                                0,
                                0,
                                0
                            ]
                        },
                        {
                            "t": 15.0000006109625,
                            "s": [
                                1920,
                                1080,
                                0
                            ]
                        }
                    ],
                    "ix": 2,
                    "l": 2
                },
                "a": {
                    "a": 0,
                    "k": [
                        150,
                        150,
                        0
                    ],
                    "ix": 1,
                    "l": 2
                },
                "s": {
                    "a": 0,
                    "k": [
                        100,
                        100,
                        100
                    ],
                    "ix": 6,
                    "l": 2
                }
            },
            "ao": 0,
            "sw": 300,
            "sh": 300,
            "sc": "#ef0c0c",
            "ip": 0,
            "op": 900.000036657751,
            "st": 0,
            "bm": 0
        }
    ],
    "markers": []
}

It contains a lot of one-two symbol named fields. It's already zipped 😃!

In order to understand meaning of all fields we can use this lottie schema
Another thing which can be convenient is json editor
Actually, lottie has very nice documentation, you can read it and skip this article 😄

Root level of lottie file called animation in specification.

Top level object, describing the animation

Let's focus on it

{
    "v": "5.10.2",
    "fr": 29.9700012207031,
    "ip": 0,
    "op": 900.000036657751,
    "w": 1920,
    "h": 1080,
    "nm": "Moving square",
    "ddd": 0,
    "assets": [...],
    "layers": [...],
    "markers": [...]
}

v - version. It used for versioning data, according semver
fr - Framerate in frames per second
ip - "In Point", which frame the animation starts at (usually 0)
op - "Out Point", which frame the animation stops/loops at, which makes this the duration in frames when ip is 0"
w - Width of the animation
h - Height of the animation
nm - Name, as seen from editors and the like
ddd - Whether the animation has 3D layers
layers - Layers

So, I'm not going to copy paste the whole specification into this article, main idea, use it to understand content of lottie.

Some IDE support schemas
for example, in vscode you can add vscode/settings.json

{
    "json.schemas": [

        {
            "fileMatch": [
                "src/animations/*.json"
            ],
            "url": "https://lottiefiles.github.io/lottie-docs/schema/lottie.schema.json"
        }
    ]
}

And you will get description of of every property of your json right inside IDE.

Everything clear with top level object.

What is inside layers?
Just one solid layer:

{
    "ddd": 0,
    "ind": 1,
    "ty": 1,
    "nm": "Red Solid 1",
    "sr": 1,
    "ao": 0,
    "sw": 300,
    "sh": 300,
    "sc": "#ef0c0c",
    "ip": 0,
    "op": 900.000036657751,
    "st": 0,
    "bm": 0,
    "ks": {
        "o": {
            "a": 0,
            "k": 100,
            "ix": 11
        },
        "r": {
            "a": 0,
            "k": 0,
            "ix": 10
        },
        "p": {
            "a": 1,
            "k": [
                {
                    "i": {
                        "x": 0.833,
                        "y": 0.833
                    },
                    "o": {
                        "x": 0.167,
                        "y": 0.167
                    },
                    "t": 0,
                    "s": [
                        0,
                        0,
                        0
                    ],
                    "to": [
                        0,
                        0,
                        0
                    ],
                    "ti": [
                        0,
                        0,
                        0
                    ]
                },
                {
                    "t": 15.0000006109625,
                    "s": [
                        1920,
                        1080,
                        0
                    ]
                }
            ],
            "ix": 2,
            "l": 2
        },
        "a": {
            "a": 0,
            "k": [
                150,
                150,
                0
            ],
            "ix": 1,
            "l": 2
        },
        "s": {
            "a": 0,
            "k": [
                100,
                100,
                100
            ],
            "ix": 6,
            "l": 2
        }
    }
}

ty - Layer Type, in our case it is 1, which means solid layer according schema
sw - With
sh - Heigh
sc - color, in out case it's #ef0c0c which is not exactly red! Ok, after effects, I got you.

Let's skip other properties for now, let's focus on important thing - ks

ks - is a Transform object which is responsible for animating this layer.

{
    "o": {
        "a": 0,
        "k": 100,
        "ix": 11
    },
    "r": {
        "a": 0,
        "k": 0,
        "ix": 10
    },
    "p": {
        "a": 1,
        "k": [
            {
                "i": {
                    "x": 0.833,
                    "y": 0.833
                },
                "o": {
                    "x": 0.167,
                    "y": 0.167
                },
                "t": 0,
                "s": [
                    0,
                    0,
                    0
                ],
                "to": [
                    0,
                    0,
                    0
                ],
                "ti": [
                    0,
                    0,
                    0
                ]
            },
            {
                "t": 15.0000006109625,
                "s": [
                    1920,
                    1080,
                    0
                ]
            }
        ],
        "ix": 2,
        "l": 2
    },
    "a": {
        "a": 0,
        "k": [
            150,
            150,
            0
        ],
        "ix": 1,
        "l": 2
    },
    "s": {
        "a": 0,
        "k": [
            100,
            100,
            100
        ],
        "ix": 6,
        "l": 2
    }
}

o - opacity
r - rotation
p - position
a - anchor point
s - scale

All properties which we saw in after effects transform object.

Every of them contains

a property, which says is it animated property or not.
If it's not an animated property it means there is no keyframes for that, value is static.

In our example only position is an animated property.
If property is not animated k contains static value.

so, in our example

opacity = 100
rotation = 0
anchor point = [150, 150, 0]
scale = [100, 100, 100]

position is animated, so k contains keyframes.

[
    {
        "i": {
            "x": 0.833,
            "y": 0.833
        },
        "o": {
            "x": 0.167,
            "y": 0.167
        },
        "t": 0,
        "s": [
            0,
            0,
            0
        ],
        "to": [
            0,
            0,
            0
        ],
        "ti": [
            0,
            0,
            0
        ]
    },
    {
        "t": 15.0000006109625,
        "s": [
            1920,
            1080,
            0
        ]
    }
]

it describes 2 key frames.

for t (time) = 0, s (state/value) is [0,0] means our layer in top-left corner of scene
for t (time) = 15.000000610962, (state/value) is [1920,1080] means our layer in bottom-right corner of scene

Pretty easy, right?

It's more complex than you think

First keyframe contains other properties: i, o, to, ti

What does it mean?

As I mentioned earlier, after effects support two different modes of position.

splitted and combined (this is not an after effect term).

with splitted mode x, y coordinates has separated keyframes definitions.

in this case we would have p (position) object like this:

{
    "s": true,
    "x": {
        "a": 1,
        "k": [
        ],
        "ix": 3
    },
    "y": {
        "a": 1,
        "k": [
        ],
        "ix": 4
    }
}

s means splitted.

let's consider p.x (position.x)

[
    {
        "i": {
            "x": [
                0.833
            ],
            "y": [
                0.833
            ]
        },
        "o": {
            "x": [
                0.167
            ],
            "y": [
                0.167
            ]
        },
        "t": 0,
        "s": [
            0
        ]
    },
    {
        "t": 15.0000006109625,
        "s": [
            1920
        ]
    }
]

Everything is simple here
for t = 0 value = 0
for t = 15.0000006109625, value = 1920

{
    "i": {
        "x": [
            0.833
        ],
        "y": [
            0.833
        ]
    },
    "o": {
        "x": [
            0.167
        ],
        "y": [
            0.167
        ]
    }
}

i - means in control point of cubic-bezier
o - means out control point of cubic-bezier

We've previously discussed the meaning of these terms.

When these values are used, the animation becomes linear.

Now, let's calculate the value for frameNumber = 7. Here's how it can be done.

const bezier = new cubicBezier({in: [0.83, 0.83], out = [0.167, 0.167]});

// progress between 2 frames is a value between 0 and 1;
const progress = frameNumber / (15.0000006109625 - 0);

const x = bezier.get(progress) * (1920 - 0);

The same for y.
Notice that the same logic of interpolation is applicable to opacity, rotation, scale and etc.

let's go back to our initial case with combined position

What is i, o, to, ti?

In this mode after effects exposes bezier curve which describes curve for position, and it's not the same as interpolation cubic bezier curve.

In transform.position interpolation settings we can select type of interpolation we want.

It does not change the way how it will be exported to lottie, because, as I mentioned before, line can be represented with bezier curve too.

Let's consider this example:

In exported json

you can see that ti, and to describe in and out control points of bezier 2d path.

{
    "to": [
        0,
        1080,
        0
    ],
    "ti": [
        0,
        -540, // notice that it's relative to second keyframe value
        0
    ]
}

With this mode, ti, to describe only path, but how animation player should understand how fast we should move layer on that path?

Lottie stores that information inside i and o properties

In after effects it can be changed in keyframe velocity settings

We'll address the interpolation of position in this scenario later on.

Now that we understand the significance of all the key properties in our Lottie file, we have a complete grasp on the top-level object, layer storage, transform object functionality, and the organization of keyframes. We also know how to interpolate values using this data.

I believe we are prepared to build our custom Lottie renderer from the ground up.

Let's proceed with that in the next article.

DEV Community: Stepan Mikhailiuk