DEV Community: Bharat

How to write a custom ESLint rule in 19 lines of code

Bharat — Sat, 23 Dec 2023 22:46:08 +0000

I don't like it that JavaScript does not have named parameters. In python for example, I can do function_name(a=1, b=2), but in javascript this does not work. Hence what we do is pass in the parameters as an object and later de-structure it in the function. Which works great, since JS has a great de-structuring syntax.

But sometimes engineers write a function which will take in four strings as normal arguments. This can backfire. Since all arguments are strings, the caller can jumble arguments and there might be unexpected issues. Catching this stuff in code review is also very hard and the only way to find it out is when testing.

We can do better.

Why not enforce that every function which has more than two arguments should always take in parameters as an object so that the caller can be explicit?

ESLint has a rule for this, It's called max-params, setting it to 2 ensures that we do not allow any function to have more than two arguments. So if the function needs to have three arguments, it means that it has to be converted to an object. And then it becomes a single argument. So switching it on should solve our problem, right?

All the typescript servers we write use NestJS as the framework. NestJS does dependency injection out of the box, which means that the dependencies are declared at a class constructor level and they are populated by NestJS during runtime. The max-params rule throws an error here as well, stating that the constructor has more than 2 parameters. That's a problem, since we cannot fix this as its not an issue in the first place. So we'll need to configure max-params to exclude constructors. I could not find one, so let's write it.

How does a linter work?

Linter reads the code and checks it against a set of rules. If any of these rules do not pass, it'll raise a alert stating that the rule. The way linters "read" code is by going through the Abstract Syntax Tree of the codebase. Or more colloquially known as AST.

What is an Abstract Syntax Tree?

Code is converted from plain text to a tree structure for by the parser. This tree is later parsed by the interpreter to run the code. Since this is a data structure to represent the abstract syntactic representation of the program, this can be parsed for patterns!

Here is the AST of function hello() {}

{
    "type": "Program",
    "start": 0,
    "end": 19,
    "loc": {
        "start": { "line": 1, "column": 0 },
        "end": { "line": 1, "column": 19 }
    },
    "comments": [],
    "range": [0, 19],
    "sourceType": "module",
    "body": [
        {
            "type": "FunctionDeclaration",
            "start": 0,
            "end": 19,
            "loc": {
                "start": { "line": 1, "column": 0 },
                "end": { "line": 1, "column": 19 }
            },
            "id": {
                "type": "Identifier",
                "start": 9,
                "end": 14,
                "loc": {
                    "start": { "line": 1, "column": 9 },
                    "end": { "line": 1, "column": 14 },
                    "identifierName": "hello"
                },
                "name": "hello",
                "range": [9, 14],
                "_babelType": "Identifier"
            },
            "generator": false,
            "expression": false,
            "async": false,
            "params": [],
            "body": {
                "type": "BlockStatement",
                "start": 17,
                "end": 19,
                "loc": {
                    "start": { "line": 1, "column": 17 },
                    "end": { "line": 1, "column": 19 }
                },
                "body": [],
                "range": [17, 19],
                "_babelType": "BlockStatement"
            },
            "range": [0, 19],
            "_babelType": "FunctionDeclaration",
            "defaults": []
        }
    ]
}

Spend some time analyzing the above AST, its pretty interesting. ESLint works off this AST to validate rules.

Open AST explorer. On the top right third menu bar element, select babel-eslint and write some code on the first pane. You'll see the tree changing on the side pane. The side pane is showcasing the AST of code you have written.

Explore the tree a bit, there is a clear notation in which code gets translated into the tree. For example, in the above example FunctionDeclaration has an id and a body as a BlockStatement.

Writing our own rule

Start by looking at max-params rule itself on github.

Let's briefly understand the structure of an ESLint rule. The js file is expected to export a function called create. this returns back an object of which the keys are the nodes they are targeting and values are the functions to run on that node.

In the max-params rule, the rule is run on three different types of nodes

FunctionDeclaration (const arrowFnExample = (a, b, c, d) => {})
ArrowFunctionExpression (const arrowFnExample = (a, b, c, d) => {})
FunctionExpression (functions inside classes)

This makes sense, and then the checkFunction is also very clean and simple. A more stripped version of the same code

function checkFunction(node) {
    if (node.params.length > numParams) {
        context.report({ node, message: 'y u do dis?' });
    }
}

At this point, the only change we'll need to make to this is to recognise if a method type is a constructor. If yes, then ignore that method. After a bit of console.log - ing of node contents, we can find that there is a pointer to the parent in node.parent. And a constructor has a special "kind" called "constructor".

Great! Plugging that in, the final rule becomes

let numParams = 2;
module.exports = {
    create(context) {
        function checkFunction(node) {
            if (node.parent.kind !== 'constructor' && node.params.length > numParams) {
                context.report({ node, message: 'y u do dis?' });
            }
        }

        return {
            FunctionDeclaration: checkFunction,
            ArrowFunctionExpression: checkFunction,
            FunctionExpression: checkFunction,
        };
    },
};

There you go, 19 lines of code and your first ESLint rule.

Running this on an existing code base

Create a folder in your existing code base, I called it eslint-rules. Create a file inside that and name it max-params-exclude-constructor.js.

Now change package.json 's lint command to run eslint with an extra option called --rulesdir and pass the folder as an argument. An updated command might look like

eslint \"{src,apps,libs,test}/**/*.ts\" --fix --rulesdir ./eslint-rules

And in the eslint config, add one more rule

"max-params-exclude-constructor": "error"

and now you should have a custom functional ESLint rule!

Making it better

The messaging around the rule can be much more eloquent. context.report is very powerful and can be used to share more context of the code.

Ideally the options should come from the user, context also has a key for this called options which can be parsed and used in the rule (example from max-params)

Some ESLint rules also have a --fix option. The way it works is by mutating the AST to a desired structure.

But you say, this is something others would have run into, right?

You're right.

Why is a base 64 encoded file 33% larger than the original?

Bharat — Sat, 11 Mar 2023 13:59:15 +0000

Base 64 is a binary to text encoding scheme which represents binary data in ASCII string format as six, four bit strings. All base 64 characters are between A-Z, a-z & 0-9.

How does base64 work?

Let’s take an example of a simple string cat

In binary, cat is 01100011 01100001 01110100. A byte is a collection 8 bits. A bit is a 0 or 1. So, cat is essentially 3 bytes.

But for base64, the rules are different. Binary should be broken down into 4 chunks of 6 bits each.

01100011 01100001 01110100 becomes 011000 110110 000101 110100

Six bits can represent 64 states (2 ^ 6). In base64, this is A-Z , a-z, 0-9, +, / and = (for padding). That is 26 capitals + 26 smalls + 9 digits + 1 + + 1 /, that’s 63 states. Finally = completes this set to 64. The base64 RFC has a handy table for all these states

 Value Encoding  Value Encoding  Value Encoding  Value Encoding
     0 A            17 R            34 i            51 z
     1 B            18 S            35 j            52 0
     2 C            19 T            36 k            53 1
     3 D            20 U            37 l            54 2
     4 E            21 V            38 m            55 3
     5 F            22 W            39 n            56 4
     6 G            23 X            40 o            57 5
     7 H            24 Y            41 p            58 6
     8 I            25 Z            42 q            59 7
     9 J            26 a            43 r            60 8
    10 K            27 b            44 s            61 9
    11 L            28 c            45 t            62 +
    12 M            29 d            46 u            63 /
    13 N            30 e            47 v
    14 O            31 f            48 w         (pad) =
    15 P            32 g            49 x
    16 Q            33 h            50 y

Let’s go back to our cat example, now that the basics are out of the way.

Using the table as a reference, cat would be

011000 110110 000101 110100
|      |      |      |
Y      2      F      0

(Since 011000 in decimal is 24 and 24 maps to Y and so on)

cat being 24 bits was no coincidence ;) If the data is not 24 bits, then = is used as padding to make it into a multiple of 24 bits. So ca encoded in base64 would be

011000 110110 000100 (padding)
|      |      |
Y      2      E      =

Reason behind increase in size

Instead of one character representing 256 (2^8) as it does in normal encoding, each character represents only 64 (2^6) states in base64.

To send three bytes of information (cat for example) we would have to actually send out four bytes in base64 (Y2F0). The base64 version is 4/3rd larger.

So, 33 % more storage is used than normal.

Why is base64 used then?

Base64 is used when there was a need to encode binary data so that it can be stored and transferred over mediums that primarily designed to deal with ASCII text. E-Mail attachments are sent out as base64 encoded strings.

Base64 is useful to send over any kind of data over the wire as a plain text string. API endpoints which have JSON payloads sometimes take the file as base64 in the JSON payload.

Data sent over the wire, between the server and the browser is usually compressed with algorithms like gzip, brotli etc. The encoding size overhead is not that significant (typically around 2-5%) between the compressed original and compressed encoded files. So, the space overhead is not a deal-breaker for sending files over web.

The base64 implementation shown above covers the core concept. To dive deeper into the details of the specification, refer RFC4648

Let us make an audio visualizer using typescript

Bharat — Tue, 08 Dec 2020 16:17:37 +0000

Making an audio visualizer has always been a dream for me. Ever since I saw those winamp visualizations.Recently I thought I will give it another shot. Apparently it is not hard as it looks. Web Audio API simplifies a lot of the hard parts for us.

What we will be building by the end

Click here to see what the end result looks like

The entire source code can be found at https://github.com/BharatKalluri/bharatkalluri.com/blob/main/components/AudioVisualizer.tsx

The Web Audio API

What we are building is called a frequency bar graph.

For every frame paint in the visualization, we get the frequencies of the audio at that time as integers. We use those integers to plot the bars on the graph. Frequencies are retrieved by doing a Fast Fourier Transform (FFT) on the song's signal.

This is where it gets complicated. Fourier transform is one of the most beautiful mathematical equations. But I do not have enough intuition about it to explain just as yet. Maybe one day when it clicks, I will have a clear writeup on it. For now the gist of it is that, it takes a complex signal and breaks it into simpler signals.From which we can extract frequencies. (This is how my understanding is, I might not be a hundred percent accurate here).

How do we get these frequencies? The web audio API has a utility called analyzer. From the analyzer, we can get the frequency data. We need to set the size of the FFT (Fast Fourier Transform) and we will get back those many frequencies. So if the FFT size is set to 1024, we get back an array of 1024 numbers denoting the frequencies. One condition here is that it must be a power of 2 between 32 and 32768.

One more concept to understand is that WebAudio works in terms of nodes. These individual nodes form a graph and this graph is executed. For example, the web audio API has a node called the Gain Node. If we set the gain value to 1, that means it is the loudest. 0 means silence. A media element can be created and a Gain node can be attached to it. And then the gain node is connected to the destination (which is the speaker). So when the program is run, the audio flows through all the nodes before reaching the destination.

Analyser is also a node. One advantage of such an approach is that a node can have multiple nodes which are its children.,One media element can route to a destination and an analyzer at the same time. This is an important idea to digest.

The actual visualization is not that hard. All the values will be plotted as rectangles on a canvas with a color. That's it. One function to be aware of here is the requestAnimationFrame.,this method tells the browser that we wish to perform an animation and requests the browser to call a specified function. Typically this matches the refresh rate of the display.

Lets code!

Now that the theory is out of the way. Let us first start by creating a template in HTML of what we want.

<input
    type="file"
    id="audioPicker"
    accept="audio/*"
/>
<audio>No support for audio</audio>
<canvas
    id="canvas"
></canvas>

A simple input, to pick up the audio file. A audio tag to actually play the audio and a canvas element to draw the frequency bar graph. Lets jump into the typescript piece now.

const audioContext = new (window.AudioContext || window.webkitAudioContext)();
const audioElement = document.getElementById("audio");
const audioPickerElement = document.getElementById("audioPicker");
const canvasElement = document.getElementById(
    "canvas"
) as HTMLCanvasElement;
canvasElement.width = window.innerWidth;
canvasElement.height = window.innerHeight;
const canvasCtx = canvasElement.getContext("2d");

Some basic declarations, we get the AudioContext first. This interface represents the audio processing graph built ,from different nodes. Then we use the getElementById to get the audioElement, audioPickerElement and canvasElement. We will be mutating them soon. Then we set the canvas height and width. And finally get the canvas context, using which we can draw on the canvas.

audioPickerElement.onchange = function () {
    // @ts-ignore
    const files = this.files;
    audioElement.src = URL.createObjectURL(files[0]);
    audioElement.load();
    audioElement.play();
}

Staring simple, on file change. The first file is retrieved and set as the source of the audio element. Then the audioElement is loaded and played. At this point, you should be able to pick a song and should here the audio playing. Now let us involve the WebAudio piece.

// ...from now the code you see here is a part of the onchange function of audiopicker in the above example
const track = audioContext.createMediaElementSource(
    audioElement
);
track.connect(audioContext.destination);

// Analyzer node
const analyser = audioContext.createAnalyser();
analyser.fftSize = 128;
track.connect(analyser);

This is the graph construction. The base element of the graph is the createMediaElementSource, which is the actual audio source. That is connected to a destination (audio out/speaker) on one end and a analyzer. This is what the illustration was showing earlier. We skipped the gain node for simplicity. An analyzer node can provide realtime frequency and time domain analysis. We will be needing the frequency data from this later on. The FFT size is set to 128, so that we only get 128 values, which we can plot. Too many values,means the bars will be much thinner.

// Creating the array to store the frequency data
const bufferLength = analyser.frequencyBinCount;
const dataArray = new Uint8Array(bufferLength);

From here, an Uint8Array of the fftSize needs to be created to store all the frequency data will keep flowing to.

// Some useful constants
const WIDTH = canvasElement.width;
const HEIGHT = canvasElement.height;
const barWidth = (WIDTH / bufferLength) * 2.5;
let barHeight;
let x = 0;

// Colors used for plotting
const MATTE_BLACK = "#1A202C";
const WHITE = "#FFFFFF";

// The function which will get called on each repaint
function draw() {
    requestAnimationFrame(draw);
    if (canvasCtx !== null) {
        x = 0;
        analyser.getByteFrequencyData(dataArray);
        canvasCtx.fillStyle = WHITE;
        canvasCtx.fillRect(0, 0, WIDTH, HEIGHT);
        for (let i = 0; i < bufferLength; i++) {
            barHeight = dataArray[i];
            canvasCtx.fillStyle = MATTE_BLACK;
            canvasCtx.fillRect(x, 0, barWidth, barHeight);
            x += barWidth + 3;
        }
    }
}
draw();

Some constants such as width, height, barWidth (width is multiplied by 2.5 just to make the bars look bigger, try without the ,2.5 as well to see why we do this) and some colors are defined for convenience. Now to the important part.,The draw function is the function that actually paints the canvas. On each invocation, we first call the ,requestAnimationFrame with the same function as input so that it calls the draw function around 60 times per second (if ,you have a 60 hertz display). Remember that all of this is happening async.

Inside the function, we start with x=0, Which is (0,0) in the canvas. Then we use the getByteFrequencyData function ,of the analyzer to populate the frequency data to the dataArray we declared earlier. I suggest you to have a look at the ,data array to get a taste of what actually is being populated (caution: be careful while doing this, if a song is loaded and you have,a console log in the draw function, huge arrays of size 128 will be logged in the console >30 times per second, this will crash,the browser or really slow it down).

Now that the data is in an array, set the canvas background to white. And for each element in the array, plot a rectangle. ,The coordinates will be from (x, y, width, height) => (0, 0, barWidth, barHeight). Fill this rectangle with MATTE_BLACK,and increment the x by barWidth + 3. So assuming the bar width is 50px, then the second rectangle will have the following,co-ordinates (53, 0, barWidth, barHeight). This keeps going on for each element in the array. That is how one frame is drawn.

This repeats >30 (based on the refresh rate of the display) times in one second to give you a feel like it is a smooth continuous moving graph. 🤯

The entire source code can be found at https://github.com/BharatKalluri/bharatkalluri.com/blob/main/components/AudioVisualizer.tsx

We just scratched the surface here, there are a lot of awesome visualizations out there. This is one of my favorites.

This article was inspired by MDN's article about visualizations with the Web Audio API.

Understanding python decorators with step by step examples

Bharat — Fri, 30 Oct 2020 15:52:02 +0000

Decorators are used to add functionality to an existing code. They allow you to wrap a function within another function, thereby adding functionality.
Let us look at some examples to get a sense of what decorators are. Before that, let us get some concepts cleared out about functions in python

Functions are first class objects in python

A function can be used like a value in python. For example

def printer():
    print("Printing")
a = printer
a()
# printing

Here we assigned the function to a variable and now the variable holds the function.
Functions can also be passed around as arguments between functions (since they just act like values). Another popular example is map. The first argument of the function is a function (Callable) and the second argument of the function is an Iterable(List, tuple etc.). Then map runs over the Iterable and executes the function.
Let us write our own simple map implementation

from typing import Callable, Iterable

def mapper(func: Callable, list_to_iterate: Iterable):
    return [func(el) for el in list_to_iterate]

def double(number: int):
    return number * 2
mapper(double, [1,2])
# [2, 4]

Functions can also be nested inside one another. For example

def outer_func():
    def inner_func():
        print("Inner func")
    print("Outer func")
outer_func()
# Outer func
# Inner func

Armed with this information, let us create our first decorator

A simple decorator

Let us create a function that counts the number of seconds a function took to execute. A very similar example to the mapperfunction.

from time import time, sleep
from typing import Callable

def timer_decorator(func: Callable) -> Callable:
    def wrapper():
        start_time = time()
        func()
        end_time = time()
        print(end_time-start_time)
    return wrapper

def sleeper() -> None:
    sleep(5)

timed_sleeper: Callable = timer_decorator(sleeper)

timed_sleeper()
# 5.00

The above example pretty much combines everything we have learned up until now. The timer_decoratorreturns a wrapper function. The function starts up a timer, runs the input function, and then ends the timer and calculates the difference to print it out. sleeperjust sleeps for 5 seconds.
When timer_decorator is called with sleeper as an argument. We get back a function which wraps the sleeper func with the time difference calculation logic. Now, if timed_sleepis called, it runs and prints out the number of seconds it took to run the sleeperfunc.
Takes some time and make sure you understand what is happening. This is what decorators are. Python gives us some syntactic sugar over the same idea. Using decorators syntax, the sleeper function would look like this

@timer_decorator
def sleeper():
    sleep(5)

sleeper()
# 5.00

That is it! Decorators wrap a function and modify its functionality/behaviour

Decorating functions with arguments

The current code does not work well for functions with arguments, let us see why

from time import time, sleep
from typing import Callable

def timer_decorator(func: Callable) -> Callable:
    def wrapper():
        start_time = time()
        func()
        end_time = time()
        print(end_time-start_time)
    return wrapper

@timer_decorator
def sleeper(sec: int):
    sleep(sec)

sleeper(5)

# ---> 16 sleeper(5)

# TypeError: wrapper() takes 0 positional arguments but 1 was given

The error is pretty clear, wrapper does not take any arguments. So now we should have a mechanism to cascade arguments to child functions. Python's *args and **kwargscome in handy here. Let us modify the decorator so that it cascades all arguments to child functions.

from time import time, sleep
from typing import Callable

def timer_decorator(func: Callable) -> Callable:
    def wrapper(*args, **kwargs):
        start_time = time()
        func(*args, **kwargs)
        end_time = time()
        print(end_time-start_time)
    return wrapper

@timer_decorator
def sleeper(sec: int):
    sleep(sec)

sleeper(5)
# 5.00

And, it works as expected!

Returning values from decorated functions

Give it a thought, what would happen if sleeper were to return the number of seconds it was sleeping. Who would have the return value?
According to the current implementation, the decorator is not giving back anything. We just call the funcbut are not returning anything from the wrapper function. The returns also need to cascade upwards so that the caller get back the expected value.

from time import time, sleep
from typing import Callable

def timer_decorator(func: Callable) -> Callable:
    def wrapper(*args, **kwargs):
        start_time = time()
        ret_val = func(*args, **kwargs)
        end_time = time()
        print(end_time-start_time)
        return ret_val
    return wrapper

@timer_decorator
def sleeper(sec: int):
    sleep(sec)
    return sec

sleep_time = sleeper(5)
print(sleep_time)
# 5.00 (This print is from the decorator)
# 5 (This is from the print statement)

Passing arguments to decorators

What if we want to have a timer_decoratorthat takes in an argument, say in_minwhich can be true or false. And if true, then the time should be printed in minutes instead of seconds.
Remember that the only requirement from a decorator is that it should return a function that takes in a function as an argument.

from time import time, sleep
from typing import Callable


def timer_decorator(in_min: bool = False):
    def decorator(func: Callable) -> Callable:
        def wrapper(*args, **kwargs):
            start_time = time()
            ret_val = func(*args, **kwargs)
            end_time = time()
            time_consumed = end_time - start_time
            print(time_consumed if in_min is False else time_consumed / 60)
            return ret_val
        return wrapper
    return decorator


@timer_decorator(in_min=True)
def sleeper(sec: int):
    sleep(sec)
    return sec


sleep_time = sleeper(5)
# 0.08336979548136393 (time consumed in minutes)

In this implementation, timer_decoratortakes in an argument and gives back a decorator (!). This decorator already has access to the variable in_min and will use it. This might be the most complicated example of the post. Take your time and make sure you understand everything clearly.

Practical use cases of decorators

Flask uses decorators for routing, authentication guards etc..: It makes the code much more clean and readable
Decorators can be used for profiling: Like the example we just wrote
Retry decorator on certian exceptions: An @retrydecorator which takes in a list of exception types as arguments and retry's the function if the function exceptions out (Celery does this)

And many more...

Congratulations on coming this far! Decorators are quite tricky in the beggining, but once you understand them. They are really powerful and useful.