What is Transpilation?

Introduction

So there was this ruckus in my head about some words like babel, webpack, transpilation and these are some words that you hear a lot if you are a web developer.

The kind of words that make you pause, and wonder, "Am I supposed to know what these mean?" I mean yeah you can skip it, it won't really hurt your development, but it's good to know that what's happening underneath.

Today we'll try to understand these words, No fancy jargon, just plain talk about what these things actually are.

Transpilation

As programmers, we like to use the most recent features, I mean who doesn't like using a spread operator which is a feature introduced in ES6, but on the other hand, how to make our modern code work on older engines that don't understand ES6 or JSX or Typescript?

Browser's JS engines are not configured to understand all of this, they only understand Javascript and yeah most of the browsers these days do support ES6 but it still does not solves the problem of them not understanding JSX or typescript. That's where Transpilation comes in.

Transpilation, short for "source-to-source compilation," is the process of converting source code written in one programming language to equivalent code in another language or another version of the same language.

source to source compilation? We are talking about transpilation, right? Where did this compilation come from all of a sudden?

Well, Transpilation is a subset of compilation, there is also a lot of debate about whether there should even be a separate word for this process.

Well after reading a lot of debate in my opinion Transpilation feels like a word of choice, you like to have some distinction? Call Conversion of code written in one language to another or equivalent but different version as Transpilation, You think it's too similar to be given a different name call it a subset of a compiler.

Even Babel, the tool which is the most used tool for transpilation addresses itself as a compiler.

Now that we have made that clear lets continue talking more about transpilation,

So by now, we have understood that what a transpiler does, it takes my source code and converts it to JS engine compatible JS code. But how does it do it?

If you know how the JS engine works it will be a little easier for you, for others do not worry,

The process typically involves the following key steps:

Lexical Analysis
The transpiler starts by breaking down the source code into individual tokens, such as keywords, operators, and identifiers.
This phase is known as lexical analysis and involves creating a stream of tokens from the source code.

Here is an example -
let x = 10 + 5;

Its tokenized form will look like - "let", "x", "=", "10", "+", "5", ";"

Abstract Syntax Tree (AST) Creation
The stream of tokens is then used to build an Abstract Syntax Tree (AST). The AST represents the hierarchical structure of the code, making it easier to analyze and manipulate.

Transformation
The transpiler applies transformations to the AST. This step involves converting code written in the source language (e.g., ES6) into an equivalent representation in the target language (e.g., ES5).

Code Generation
The transformed AST is used to generate the final transpiled code. This code is designed to be compatible with the target environment or runtime.

Now this is a very high-level understanding of how transpilation works and not exactly how transpilation works.

Transpilation of let and const

Ok, we know that how transpilation works, but while reading about this a question struck me, what about let and const? I mean these keywords were introduced in ES6 they were not there in ES5, so I checked how this was being handled and I got to know that the transpiler converts let and const to var.

What?? converting let and const to var? Then what about all the concepts of hoisting and blocked scope, well don't worry I was really confused too, well after searching a bit more, I tried this website which lets you write ES6 code and you can see ES5 equivalent of it.

So I went there and I tried this snippet -

if(true){
  let a = 1
  console.log(a)
}
console.log(a)

well, the output was interesting -

if (true) {
  var _a = 1;
  console.log(_a);
}
console.log(a);

It created a behavior like that of let even with var, it smartly added an underscore inside the if block's a variable so that it doesn't exist outside of the if block.

Play around with this, try different things, and see what happens, let me know in the comments what you tried.

PolyFill

Ok so now we know that how transpilation works cool cool cool, but transpilation basically translates the code right? What about the features that don't even exist in the previous version? Like if we take an example of ES6 to ES5 conversion, and talk about the includes function, it didn't even existed in ES5. So how will we translate something that does not even exist there?

You can try using that ES6 to ES5 converter and it will show the same code that you will try.

Thats where poly filling comes in, the official definition of poly filling according to MDN is -

A polyfill is a piece of code (usually JavaScript on the Web) used to provide modern functionality on older browsers that do not natively support it.

Now how do you do polyfilling? Is it a software? Well, I was reading [Remy Sharp's] blog(https://remysharp.com/2010/10/08/what-is-a-polyfill) (Person who introduced the term), and some implementation of poly filling, I understood that poly filling is not some tool, its more of a concept that you apply in your code to increase compatibility of your application.

Here is an analogy to help you understand the relationship between traditional transpilation and Poyfilling -

Imagine that you got your hands on a time machine and you time travelled to ancient times, excited to talk to the people there, you forgot that the language is different and they won't understand what you are saying, so what will you do?

You will transpile your language to their language so that they can understand it. Now you want to talk more with these people and want to allow even communication from a distance, you thought of using the phone, but wait that did not exist at that time, how will you even translate a thing that did not even exist at that time?

Well that's where poly filling can help you to bridge the gap, you collected two cups and joined them using a long string, and now you have kind of a phone for long-distance communication.

Now I am still a little confused as if this is the right way to explain it. I mean let and const also don't exist in ES5 and it was translated, right?

What I think is transpilers handle these by finding equivalent constructs, while polyfills are employed when introducing entirely new concepts. I mean the concept of the variable was still there in ES5 right, but a function that can tell whether an element exists or not in an array, that never even existed there.

Let me know in the comments if you have a better explanation to address this.

How Transpilation happen in our Projects?

Well, before reading about this topic, I didn't even know that this was happening, so I possibly can't configure Babel in my React project or in any other framework or library.

So in popular frameworks and libraries like React, angular, etc., this is already pre-configured, like if I take the example of React, normally devs create a React environment through two ways -

1. CRA (Create react app)
2. Using Vite

Now CRA under the hood uses Webpack which is a Bundler. And another way I told you i.e 'Using Vite' well Vite is a bundler as well. Now we won't go in much depth about bundlers in this blog, but I'll give you an overview as it is necessary to understand, So bundlers like Webpack or Vite are tools that are used to merge two more JS files into one file to form a bundled file.

But this is not all that these bundlers do, they also take care of transpiling for us, so when you run npm run build in your terminal it creates an optimized bundled and transpiled code for you which you deploy. So you never get to know what happened.

The need for transpilation is more in these frameworks and libraries more than that of vanilla JS projects because Vanilla JS projects have the only problem that 'what if someone ran this in ES5 browser' but that's not the case anymore, its really rare that there is any browser which does not supports ES6.

But a library or framework like react which is written in JSX it needs to be transpiled every time because the browser has no clue about JSX.

Now there was one last thing that I was confused about, My confusion was that

'ok I ran npm run build and my code got transpiled and all good now browser will have no problem understanding my JSX.'
But what about during development?

I had questions like -

1. Is it getting re-transpiled on the go?
2. If I remove one semicolon from my code the whole code gets retranspiled?

Well if removing one semicolon were to trigger a full re-transpilation then Developing a web app wouldn't be easy, because if you are maintaining a huge codebase where you are continuously making changes and you have to wait for like 30 seconds to 1 minute because you have this huge code base, your Developer Experience and your Productivity both would be very bad.

That's where bundlers like Webpack and Vite show the magic, so these bundlers have something called HMR (Hot Module Replacement)

It smartly re-transpile only the modules that actually changed and not the entire application. Once the modules (which were changed) are re-transpiled, the new code is injected into the running application on the fly.

How does it 'inject' code in my application on the fly? Well when our frontend server starts a web socket connection is created between our frontend server and browser, and when something changes, this information is sent through a browser using a web socket.

Yeah!! Sounds really interesting right? And unbelievable right? I mean I thought I at least knew my front end, I thought I knew everything happening in my front end and how it was happening. But here we are, a WebSocket connection is being executed through my front end and I didn't even know.

I wanted to test this, so I tried turning this HMR feature off through the web pack config and this was the result.

So to test this by selecting all of the text, changing one line, and saving it and this was the result.

HMR ON -

You see how the text changed and neither the page reloaded nor the text selection went away.

HMR OFF

Here you see the page reloaded, so there is no HMR and the whole transpilation is being done again from scratch.

How can you try this?

For testing this create a Create react app, and then run this command npm run eject, after this you will be able to see the config file
which contains the configuration of webpack, in the webpack.config.js file change this variable shouldUseReactRefresh to false, restart the server, and just try changing something.

Let me know in the comments what happened.

We will talk about the working of HMR and the frontend server in another blog, it's a big topic, and need its own blog for that.

So there it is, we understood the fancy and technical words like Webpack, Babel, and Transpilation and learned some more things on the way, we talked about how transpilation works, how a newbie developer like me or someone else doesn't even need to know this stuff to run a project. And much more.

Thanks for reading this far😁.