How We Migrated from Javascript and Flow to TypeScript at Osome

This fairy tale started a long time ago. Often, you make a bet on the wrong horse in life, but, unfortunately, you cannot do anything about it. You're always working with limited information (especially about the future), when making decisions, so essentially you are betting. This is life: sometimes it happens in your personal life and sometimes in your work life. But the only thing that really matters is how you learn from your bets.

One such decision was betting on Flow. If you aren’t aware of what Flow is, it is Facebook's JavaScript language superset that brings types and type system to JavaScript. The idea is pretty straightforward: write your code with a kind of syntactic sugar Flow that takes your files and builds an extended abstract syntax tree (AST), analyze it, and if everything works, transform it into regular JavaScript.

Although, we started developing our apps in 2017/2018, when TypeScript tooling, support, and documentation were not as strong as it is now, and there was no clear winner between Flow and TypeScript. The developers who started it, were more familiar with Flow. The turning point was around 2019, when TypeScript started developing much faster, and Flow was essentially abandoned and we too decided that we should migrate to TypeScript at some point. Flow users will be faced with the following problems:

• Undocumented features, that you can only know by diving into Facebook's codebase or Flow source code
• Slow compilation process, that blows out your computer
• Substantially weak support from the maintainers, regarding your issues and problems
• Abandoned userland typing's for npm packages

This list can go on and on but within Y amount of time, we got the idea that we should transition into another superset of JavaScript. Well, if you ask me why, there are a few reasons, which can be divided into two subsections:

Company-specific reasons:

• We already utilized TypeScript on the backend and have the required expertise and experience.
• We already had contract-first backend programming. Where SDKs for different clients are automatically generated based on specification.
• A significant part of our frontend is CRUD-like interfaces, based on backend API. Therefore, we want to use SDK for our frontends with strong typings.

Common sense-based reasons:

• Great documentation, many tutorials, books, videos, and other stuff.
• Weak type system. It is debatable because it's not like ML typesystem (Reason, Haskell, Ocaml), so it easily provides a clear understanding to people from any background.
• Great adoption by the community, most of the packages you can imagine have great TypeScript support.

How has it started?

As far as I know, migration was started by adding tsconfig.json with:

{
  // ...
  "compilerOptions": {
    "allowJs": true
  }
}

This TypeScript flag allowed us to start adding ts and tsx files to our system.

Following this, we began to enable ts rules one after the one (the options below are just for example — choose those which fits to your codebase)

But we already had Flow, so let's fix it and add to .flowconfig

module.name_mapper.extension='ts' -> 'empty/object'
module.name_mapper.extension='tsx' -> 'empty/object'

Now, you are ready to start the migration. Let's go! Wait, it's that easy?

Where did we end up?

Due to a variety of reasons, we ended up hanging in limbo. We had plenty of files in Flow and in TypeScript. It took us several months to understand that it was not okay.

One of the main problems here is that we didn't have a plan as to which files we should start with, how much time and capacity it should take, and a combination of other organizational and engineering-based issues.

Another day, another try

After accepting the fact that the health of our codebase is not okay, we ultimately settled on trying again.

This is one more time when graphs save the day.  Take a look at this picture:

Any codebase is no more than a graph. So, the idea is simple: start your migration from the leaves of the graph and move towards the top of the graph. In our case, it was styled components and redux layer logic.

Imagine you have sum function, which does not have any dependencies:

function sum(val1, val2) {
   return val1 + val2;
} 

After converting to TypeScript, you get:

function sum(val1: number, val2: number) {
   return val1 + val2;
} 

For example, if you have a simple component written in JavaScript when you convert that specific component, you immediately receive the benefits from TypeScript.

// imagine we continue the migration and convert the file form jsx to tsx
import React from 'react'

// PROFIT! Here's the error 
const MySimpleAdder = ({val1}) => <div>{sum("hey", "buddy")}</div>

// => Argument of type 'string' is not assignable to parameter of type 'number'.

So, let's create another picture but for your current project.

The first step is to install dependency-cruiser.

Use CLI

depcruise --init

Or add .dependency-cruiser.js config manually. I give you an example that I used for a high level overview of such migration.

module.exports = {
  forbidden: [],
  options: {
    doNotFollow: {
      path: 'node_modules',
      dependencyTypes: ['npm', 'npm-dev', 'npm-optional', 'npm-peer', 'npm-bundled', 'npm-no-pkg'],
    },
    exclude: {
      path: '^(coverage|src/img|src/scss|node_modules)',
      dynamic: true,
    },
    includeOnly: '^(src|bin)',
    tsPreCompilationDeps: true,
    tsConfig: {
      fileName: 'tsconfig.json',
    },
    webpackConfig: {
      fileName: 'webpack.config.js',
    },
    enhancedResolveOptions: {
      exportsFields: ['exports'],
      conditionNames: ['import', 'require', 'node', 'default'],
    },
    reporterOptions: {
      dot: {
        collapsePattern: 'node_modules/[^/]+',
        theme: {
          graph: {
            rankdir: 'TD',
          },
          modules: [
            ...modules,
          ],
        },
      },
      archi: {
        collapsePattern:
          '^(src/library|src/styles|src/env|src/helper|src/api|src/[^/]+/[^/]+)',
      },
    },
  },
};

Play with it — the tool has a lot of prepared presets and great documentation.

Tools

Flow to ts — helps to produce raw migration from Flow to TS files and reduces monkey job.
TS migrate — helps convert JS files to Typescript files. A great introduction can be found here.

Analyze and measure

This was not my idea but we came up with the idea of tracking the progress of our migration. In such situations, gamification works like a charm, as you can provide transparency to other teams and stakeholders, and it helps you in measuring the success of the migration.

Let's dive into technical aspects of this topic, we've just used manually updated excel table and bash script that calculates JavaScript/Typescript files count on every push to the main branch and posts the results to the Slack channel. Here's a basic sample, but you can configure it for your needs:

#!/bin/bash

REMOVED_JS_FILES=$(git diff --diff-filter=D --name-only HEAD^..HEAD -- '*.js' '*.jsx')
REMOVED_JS_FILES_COUNT=$(git diff --diff-filter=D --name-only HEAD^..HEAD -- '*.js' '*.jsx'| wc -l)

echo $REMOVED_JS_FILES
echo "${REMOVED_JS_FILES_COUNT//[[:blank:]]/}"

if [ "${REMOVED_JS_FILES_COUNT//[[:blank:]]/}" == "0" ]; then
echo "No js files removed"
exit 0;
fi

JS_FILES_WITHOUT_TESTS=$(find ./src  -name "*.js*" ! -wholename "*__tests__*" -print | wc -l)
JS_FILES_TESTS_ONLY=$(find ./src  -name "*.js*" -wholename "*__tests__*" -print | wc -l)
TOTAL_JS_FILES=$(find ./src  -name "*.js*" ! -wholename "*.snap" -print | wc -l)

JS_SUMMARY="Total js: ${TOTAL_JS_FILES//[[:blank:]]/}. ${JS_FILES_WITHOUT_TESTS//[[:blank:]]/} JS(X) files left (+${JS_FILES_TESTS_ONLY//[[:blank:]]/} tests files)"

PLOT_LINK="<excellink>"

echo $JS_SUMMARY


AUTHOR=$(git log -1 --pretty=format:'%an')
MESSAGE=":rocket: ULTIMATE KUDOS to :heart:$AUTHOR:heart: for removing Legacy JS Files from AGENT Repo: :clap::clap::clap:\n\`\`\`$REMOVED_JS_FILES\`\`\`"

echo $MESSAGE
echo $AUTHOR

SLACK_WEBHOOK_URL="YOUR_WEBHOOK_URL"
SLACK_CHANNEL="YOUR_SLACK_CHANNEL"
curl -X POST -H 'Content-type: application/json' --data "{\"text\":\"$MESSAGE\n$JS_SUMMARY\n\n$PLOT_LINK\",\"channel\":\"$SLACK_CHANNEL\"}" $SLACK_WEBHOOK_URL

Stop the bleeding of the codebase

Migration is a process; you can't do it in one day. People are fantastic, but at the same time, they have habits, they get tired, and so forth. That's why when you start a migration, people have inertia to do things the old way. So, you need to find a way to prevent the bleeding of your codebase when engineers continue to write JavaScript instead of Typescript. You can do so using automation tools. The first option is to write a script that checks that no new JavaScript files are being added to your codebase. The second one is to use the same idea as test coverage and use it for types.

There is a helpful tool that you can use for your needs.

Set up a threshold, add an npm script and you are ready to start:

"type-coverage": "typescript-coverage-report -s --threshold=88"

Results

Today, there is no Flow or JS code in our codebase, and we are pleased with our bet to use TypeScript, successfully migrating more than 200k files of JavaScript to TypeScript.

---

Feel free to ask questions, express any opinions or concerns, or discuss your point of view. Share, subscribe, and make code, not war. ❤️

If you'll find an error, I'll be glad to fix it or learn from you — just let me know.

If you want to work with me you can apply here, DM me on Twitter or LinkedIn.

Comments

[Rajesh Royal]

I made the same design of moving all the projects to TypeScript when I joined, and I'm glad I made that decision. Although some of the developers still has a tendency to use any which makes TS no use 🙁

[Andrey Frolov]

And whats the problem with a small amount of any's in project?

[Rajesh Royal]

We had big problem. cannot desctructure X from undefined

[Andrey Frolov]

But the typescript type system is not Ocaml, Haskell, or Rust like; it doesn't give you any 100% that you don't get any runtime errors or null pointer errors :d

Yes, it prevents a huge surface of errors, but not as much as it looks in theory.

[Rajesh Royal]

agreed 💯

[Andrey Frolov]

Good point, but It's not so true, the development is not about black and white and good or bad, you always have semitones - tradeoffs. For us, it was an appropriate decision to make a sharp migration and start to check type coverage to reduce the amount of any's iteratively

[John Peters]

Sometimes our legacy prejudices blind us. For Javascripters, they rejected TypeScript from the start. That's why newfound issues were solved using various Javascript libraries none of which used the same dependencies or coding patterns. Alas NPM became a virtual cesspool as a result.

TypeScript has its roots in over 35 years of other languages such as C#, Java and even Pascal. Many Javascripters and large companies see the value now. Demand is only going to grow. Being a part of that need is a great place to be right now.

[Andrey Frolov]

Thank you for your comment. But what do you mean by demand right here?

[John Peters]

TypeScript is becoming very popular in large companies.

[Andrey Frolov]

de facto standard :d

[Denis Romanenko]

Thanks God it was not Dart. Canonical still doesn’t have strength to admit that they bet on the wrong horse…

[insidewhy]

Using Dart on a flutter project at work. So much disappointment, so much boilerplate, so much having to give up on doing things because the type system is too basic.

[Andrey Frolov]

Interesting point. What do you mean by too basic?

[Andrey Frolov]

Nowadays most people die of a sort of creeping common sense, and discover when it is too late that the only things one never regrets are one's mistakes :d

[KonNikiforov]

Thx, get some insights for my current project!

[Andrey Frolov]

appreciate it

[artemekaterinenko]

Great stuff! This will save us a lot of time for the future migration

[Andrey Frolov]

Thank you mate

[Sergey Korovin]

Great article, found some interesting ideas for me

[Andrey Frolov]

My pleasure