DEV Community: Louis

18 Reasons to Use TypeScript Since Yesterday.

Louis — Wed, 04 Jan 2023 23:12:54 +0000

If you're anything like me, you use Typescript because you need to. Your first startup in 2013 had a gigantic Angular.js codebase written in pure JS, and you were a scrappy hacker who thought he could change the world by plowing through the code and single-handedly maintaining it all.

After a huge QoL refactor, you learned that the time it took was not worth the ROI and that you needed a tool. You wanted something that could solve the problem systematically but is still lenient enough for you to move fast while not being some quick annotation with arcane tooling that is not smart enough to rename your function argument.

After trying out Flow, Haxe, ReScript, Clojure... you found TypeScript. The light clicked in your head💡

In this article, I am going to hype the $$$$ out of TypeScript and why you, yes, you, MUST use it. Having used it since 2015, I have seen it all - slow as $$$$ script runtime, messy type collusion, enum not working as expected, and did someone mention emitDecoratorMetadata?

But hey, eight years in the trench, why didn't I just switch to Rust, C#, Zig, Dart, or some other "better" language?

Read on.

👉 Table Of Contents (TOC).

What is Typescript?
Should I use It?
Powers of Typescript.

Ease of Learning
Flexible Type Checking System
Concise and Readable Syntax
Intuitive Syntax
Modern Tooling
High Productivity
Realistic Promises
Flexibility
Large Community
Clean Code Made Easy
A JS Superset
A Compiled Language
Reduced Tooling
High Performance
No Need to Write Unit Tests
Types Are Extremely Useful
Lower Tech Debt
To Boldy Refactor

What is Typescript?

It is JavaScript, with optional typing, that you can refactor. The code below is both JS and TS:

const derp = "world"

console.log(`Hello, ${derp}`)

The nice thing about TS is that I can press F2 on that derp variable and rename it, and it will work even if I did it in an external module.

Why is that? All thanks to the TypeScript AST produced by TS language server running under the hood inside VSCode. The collaboration between the language and the editor makes it do things that are impossible otherwise.

Those who have yet to experience the joy of TypeScript, simply have yet to experience the horror of "JavaScript with friends."

Should I use TypeScript?

YES.

Even for a tiny little module with 1 line of code?

Why YES:

export const life = 42

Or you can suffer with babel tooling, custom webpack configs, three separate rollup build configs targeting iife, cjs, esm, and heck, maybe WASM (assuming life is an uint8).

If not for you, think of the consumer of your module - is life a string or a number? The people need to know!

1. Ease of Learning.

🔝 Go To TOC.

"strict": "false"

Disable strict mode, and you will be writing JavaScript. When the static compile-time warnings bother you to a certain threshold (and they should bother you), start reading into the detail as to why the warnings are as such and fix them!

Did you know? VSCode codebase itself was not strict until Feb, 2020 [0]

2. Flexible Type Checking System.

🔝 Go To TOC.

"strict": "false"

Just disable strict mode, and you will wish it was slightly stricter. But this:

type ApiRoute = "/api/${string}"

Is an example of a flexible type system, that can prevent silly mistakes across your entire codebase.

The BubbleSort implemented purely using the type system [1], is also a good example of how flexible and powerful TypeScript is. Those who use it to claim TypeScript is "strict" missed the whole point of the demonstration.

3. Concise and Readable Syntax.

🔝 Go To TOC.

TypeScript infers type for you:

let foo = "bar" // foo is a string

But you can make it do even more:

let foo = "bar" as const // foo is strictly "bar"
foo = "lol" // -> error

TypeScript self-documents its arguments. Thus you only need to document the high-level idea of your function:

/**
 * Add two numbers
 */
function add(a = 0, b = 0){
    return a + b
}

add(1, 2) // 3
add(1, "0") // compile err

Utilize it wisely, and you will have a happy little codebase.

4. Intuitive Syntax.

🔝 Go To TOC.

TypeScript type syntax shares many resemblances to that of C# and C++.

To the experienced, it's quite intuitive (which explains its high adoption rate).

To those who are new to programming in general, the learning curve to some of the advanced syntax might be as high as learning C++ itself.

This explains why Python triumphs when it comes to teaching programming - it's more intuitive to most people because there are no braces. Refactoring a large python codebase, on the other hand, is a massive chore if the tab alignment is inconsistent between modules. Jupyter notebook had made it much better over the years, tho!

Just disable strict mode, you newb! - jokingly, I said.

5. Modern Tooling.

🔝 Go To TOC.

TypeScript is a compiled language. Thus, the same code can now be compiled to work on various runtimes, architectures, and systems. The drawback is, you will need to compile it OR use a runtime that runs TypeScript natively.

If TypeScript does not support a certain runtime, or if the compiler is unavailable in a certain environment, it's almost trivial to add them - use esbuild [2].

The power of TypeScript is not about the language itself, but the result of how it is interpreted. The TypeScript AST is integrated into the TypeScript language server within the editor, which enables features such as autocompletion, module import suggestions, type inference, and cross-module refactoring.

These modern tooling features might not benefit small projects as much as it does big ones. However, big project starts small, and it's better to start with the right foundation and footing, than retrospectively add them later on.

6. High Productivity.

🔝 Go To TOC.

Just disable strict mode, and enjoy all the speed and zero friction:

Type coercion/inference
Cross-module variable renaming/refactoring
Import suggestion
API typing and type-safe API modules
Scoped variable renaming (yes, try it with plain JS)
Any other TypeScript features

The key productivity gained by TypeScript in an enterprise team environment is the ability to use and explore module APIs without even having to check the documentation. More so when exploring 3rd party vendor APIs.

7. Realistic Promises.

🔝 Go To TOC.

Straight from the TypeScript front page:

TypeScript is JavaScript with syntax for types.
TypeScript is a strongly typed programming language that builds on JavaScript, giving you better tooling at any scale.

It gave me exactly what it promised:

A typed language (optionally strong, just disable strict mode)
Better tooling via VSCode integration, as well as tools created using and for TypeScript itself - ESBuild, PNPM, TurboRepo, etc...

8. Flexibility.

🔝 Go To TOC.

"strict": "false"

Just disable strict mode, and you are practically writing JS, but with a .ts file extension.

9. Large Community.

🔝 Go To TOC.

Do you have a TypeScript question and need to speak to a community member? The TypeScript Discord server is very active: https://discord.gg/typescript

So is their Stack Overflow: https://stackoverflow.com/questions/tagged/typescript

This is the benefit of speaking a common language that's slightly more powerful (dynamic) than the base language. It's like the original British English vs. the Americanized English. The latter sees massive adoption due to its flexibility in accommodating different culture, meanwhile, the community that uses it continue to grow due to the sheer size of the continent that spawned the dialect.

10. Clean Code Made Easy

🔝 Go To TOC.

TypeScript makes it easier for the editor to understand your code - which is required for better tooling. This, in turn, promotes cleaner code. Experienced developers leverage these properties to organize their code to be maintainable and easy to refactor - for the editor and their team.

I often see inexperienced dev struggles with organizing their code because, more often than not, they tried to be "clever" with it. This happens in almost any language - C++, Python, Haskell, Scala, you name it. The elusive and novel ternary or pipe operator they recently learned often leads them to utilize it immediately - and for a good reason. That's how one learns the language syntax. This often happens because there is no clear guidance or clear-cut benefit perceived - likely, they did not use the tooling. I would argue it is much easier to write complex code that's impossible to debug in JavaScript (golf-style), than TypeScript.

11. A JS Superset.

🔝 Go To TOC.

TypeScript IS a JavaScript superset - just disable strict mode and ignore any compiler warning!

  const star = {}
  star.trek = 47

The above code "compiles" to JS, and is also valid JS. The "superset" property of TS is that it added the TS compiler warning that tells you exactly what is wrong - the original object does not have the property "trek." The warning is a "feature" of the superset property - plain JS would not be aware of such an idea.

Side note, this works:

  const star = {}
  star["trek"] = 47

12. A Compiled Language.

🔝 Go To TOC.

Being a compiled language, you can leverage the TypeScript AST to do some pretty cool stuff. Namely, you can use the latest ES syntax and let the compiler handle the compatibility for you, as well as optimization. Even better, it can target the output to node, browser, worker runtime, etc...

On the other hand, you can convert the typing from other languages into TypeScript AST - from Rust to TypeScript with TypeShare for example [3]. This allows TypeScript developer to consume Rust native modules with API documentation via the Rust typing itself.

13. Reduced Tooling.

🔝 Go To TOC.

With the advent of parcel [4], esbuild [1], and tsup [5], you can say goodbye to extra tooling. Use one of these bundlers, and you will have TypeScript transpiling as fast as javascript being optimized by terser. This is because these bundlers skipped the type-checking steps and went straight to transpilation.

This allows the editor integration to shine, and separate the linting steps from your building steps if you have a CI/CD pipeline.

14. High Performance.

🔝 Go To TOC.

TypeScript produces much more performant code because the benefit of using it - editor integration - encourages developers to write better, cleaner code, which is easier to optimize.

15. No Need to Write Unit Tests.

🔝 Go To TOC.

The compiler and type system catches most of the silly mistakes possible that need to be tested with unit-test. This allows unit tests to focus on core libraries, and the team can focus on E2E test - the more important one!

If you are a junior developer working under a manager who's adamant about unit tests and the 95% code coverage threshold, my condolence. You might try to convince them to disable strict mode, write your test with a TS transformer that disables strict mode, or simply write the test and add // @ts-ignore at the top of the file - reap the benefit, ignore all the warning!

16. Types Are Extremely Useful.

🔝 Go To TOC.

When working with other people, external APIs, and overall, any distributed system that requires you to study payloads of API that you did not own - having decent (even incomplete) types will save you $$$$ and $$$$.

Try to work with the Stripe API without the typing, and deploy it to prod.

I $$$$ dare you.

17. Lower Tech Debt.

🔝 Go To TOC.

Switching from TypeScript to Rust is a rewrite, that is not tech debt.

Using plain JavaScript, with zero editor integration, resulting in inconsistent code that's impossible to refactor and maintain, is a tech debt.

Arguably, you can incur tech debt by using TypeScript if you are using it without trying to utilize the benefit of the editor and tooling integration the ecosystem provides, but why would you?

It's like using C++ without a linker. Now you must convert C++ to Assembly instead of writing a linker - is that a tech debt or the lack of tooling utilization? I'd argue the latter.

The same can be applied to any language and ecosystem - try to use Rust without Crate, Ruby without Gem. A language represents not just its syntax, but its entire tooling ecosystem.

Typescript's encouragement of clean-code erases tech debt.

18. To Boldly Refactor.

🔝 Go To TOC.

Finally, the key power of TypeScript, compared to plain JS or most web-based languages: is the ability to refactor a large codebase, with thousands of modules, with ease. This enables the codebase to evolve from small to medium, to large, with progression and iteration instead of a large wholesale rewrite.

This would not be possible without the tight integration between TypeScript and the VSCode editor. This killer combination enables module moving, file renaming, and import resolution refactoring with a click of a button.

DID YOU KNOW? There are plenty of similar code editor enhancement solutions in other compiled language. A popular one within the C++ professional community is the Tomato kit for Visual Studio - they have provided code refactoring, renaming, etc... for C++ since 2010s [6]

Conclusion

🔝 Go To TOC.

USE TYPESCRIPT Y'ALL!

References.

🔝 Go To TOC.

[0] https://code.visualstudio.com/updates/v1_43#_strict-true
[1] https://www.youtube.com/watch?v=Uf1D0GoEzlY
[2] https://esbuild.github.io/
[3] https://github.com/1Password/typeshare
[4] https://parceljs.org/
[5] https://tsup.egoist.dev/
[6] https://www.wholetomato.com/

Note about oh-my-posh, pwsh, wt + OSC99

Louis — Thu, 01 Sep 2022 15:38:49 +0000

When using a fancy theme in oh-my-posh, and we want to ensure the OSC99 gets registered to recall the previous PWD, make sure to fork the theme and add this to it:

   "pwd": "osc99"

Without this, osc99 will not be emitted, thus preventing Windows Terminal from restoring your session.

NOTE: This was corrected thanks to @jandedobbeleer (see comment section). Previously, the key and value was "osc99": true

HOWTO: Injecting UI into Website with Browser Extension

Louis — Fri, 22 Jul 2022 16:17:00 +0000

TL,DR:

pnpm create plasmo inject-ui
cd inject-ui
touch content.tsx

content.tsx

const Overlay = () => (<h1>HELLO WORLD</h1>)
export default Overlay

pnpm dev

Load the extension in Chrome, refresh and you should see HELLO WORLD injected into the top of the webpage.

Explanations

Browser extensions can inject DOM elements into a website using a content script. However, the injected element might inherit the website's style without proper encapsulation. This is often undesirable for any site-agnostic browser extension.

There are 3 solutions to the problem above:

A. Render the element with a style tag that does css unset.

A is often seen employed by ad. It works well for self-contained, simple components that render images. However, it is very hard to leverage a design system that has inherited styling, since the unset cascades down. Unless you have a dedicated style injection scheme, it is very difficult to use and the extra style tag makes the code not very clean.

B. Render the element inside an iframe.

B isolates the element well and is the go-to technique of many old browser extensions. However, the iframe container can be hard to work with when trying to blend the injected component seamlessly into the website. Iframe's elements are nested inside an inner document within its frame, with its own head and body. This makes it tricky to style and control elements inside the iframe, as well as align them with the elements of the website.

C. Render the element inside a Shadow DOM.

C is the best technique for web extension. Introduced as the key part of the web component API, the idea is that the shadow DOM is a separate DOM attached to the underlying main DOM. Nothing inside it can affect the main world. The Plasmo framework uses this technique behind the scene to enable seamless injection of React component into any webpage. We call this feature "content script UI."

HOWTO: Consuming CJS in a Typescript module

Louis — Wed, 26 Jan 2022 12:36:00 +0000

At Plasmo we leverage Typescript for our web projects.
CJS project like humanparser would have example like this:

const human = require('humanparser');
const fullName = 'Mr. William R. Hearst, III';
const attrs = human.parseName(fullName);

console.log(attrs);

The require statement is how we import the humanparser module into a CJS codebase. This statement can be translated into TypeScript/ESM 2 ways:

import * as humanParser from 'humanparser'

import humanParser from 'humanparser'

Which is it?

The answer lies in the source code of the humanparser module itself. This is an artifact from the evolution of ES, from ES3-> ES5 -> ES6 and beyond (now ESM). There were a couple of ways a module could be exported.

The first, commonly used during the transition between ES3 -> ES5 (early nodejs days), was to assign an entry function or object into module.exports global:

module.exports = stuff

In this example, the module.exports global was used to bridge into the require statement. Because module.exports represent "everything" that was exported from this module, we must use the import all statement:

import * as humanParser from 'humanparser'

The * represents the module.exports object.

Another way, which occurred during the transition between ES5->ES6, is to export a default property as the entry for your module:

module.exports = {
    default : stuff
}

This assigned stuff into the defaults property exported by the module. The reason why this was done is because in ES6, when doing:

import humanParser from 'humanparser'

The above code is actually importing the default props exported by a module. (The export statement in ES6 would be: export default stuff). The import statement above is doing something like this in equivalent CJS code:

const humanParser = require('humanparser').default

Now, back to humanparser's source code, their export statement looks like this:

const parser = module.exports = {};
parser.parseName = function(){}
parser.getFullestName = (str) => {}
parser.parseAddress = (str) => {}

Because there's no default prop being exported, we can import the parser two ways, either importing the whole object:

import * as parser from "humanparser"

OR, importing the props that was exported separately:

import {parseName, parseAddress} from "humanparser"

Personally, I like to import the named export, it helps with code intelligent, and I don't have to deal with the import all module namespace issue.

Back Story

Back in the olden day, pre-2015 to be exact, where angular.js were still trendy, react was still the new kid in the block, and people were comparing Corodva phonegap to react native, there was a transition from ES3/ES5 (commonJS or CJS) to ES6 (ES2015, or ESM, or MJS - modulejs). I suspect whoever named the extension was a huge fan of the late pop king.

Cautions

If you are looking at new projects, be cautious of pure ESM module. They will only allow you to import them with:

import mod from "module-name"

The tricky part is that, your project must also be a module - i.e, it cannot be compiled into common js (which converted all import statement into cjs require call). It must be an MJS/ecmascript module file (with the .mjs extension) OR that your package.json have specified the module property. Otherwise, it would simply fail to compile, and you won't be able to import it using require because ESM code looks like this:

export default stuff

Instead of this:

module.exports = stuff

In the CJS example, module.exports are bridged into the require statement for importing in other modules. Meanwhile, in the ESM example, the stuff is being exported by the export statement , and thus can only be imported via the import statement.

The dilemma here is that if you're using TypeScript and was hoping that it would play well with ESM only module, you are in for some sour candy - it does not. Typescript compiler doesn't yet care if a module it imported is ESM or not - it converts all to CJS import unless you configured it properly. You would also need to tell nodejs to use an experimental flag when running your compiled code: https://nodejs.org/api/esm.html#customizing-esm-specifier-resolution-algorithm

P.s: this post is a brain-dump. If it's useful, you're welcome. If not, here's the MIT license for this otherwise unreadable post:

Copyright 2022 L❤☮🤚

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Notes on default ES behavior

Louis — Tue, 07 Dec 2021 06:48:26 +0000

const n = null

console.log(n?.something) // undefined

const o = {}

console.log(o?.something) // undefined

function test(d = 0) {
    console.log({ d })
}

test(o?.something) // { d : 0 }
test(n?.something) // { d : 0 }

Undocumented EAS flag in Expo SDK 43

Louis — Mon, 22 Nov 2021 03:47:54 +0000

I was working on deploying ponder's native app to TestFlight using Expo EAS when I found some interesting undocumented EAS feature flags that I think folks should know. These are especially useful if you have a monorepo.

First, I just wanted to say thank you to the Expo team for making native deployment a breeze. And the fact that their source code is open really help resource-constrained team (like mine) to solve our own problem and get stuffs done without having to wait for their next release.

When using eas build, the eas-cli first create an artifact of the project's source code, before pushing to the cloud for remote building. The default behavior is to use git as a source of truth to generate the latest source code artifact.

However, if the monorepo contains server-side module or any code unrelated to the expo project, it would also be included in the uploaded artifact. This results in an additional security risk - i.e if expo's artifact storage were to be compromised, our internal business logic will be exposed.

At the moment, the default git workflow does not provide a way to ignore file during the artifact creation process (since it's just invoking git internally). Thankfully, as of Expo SDK 43, a new flag EAS_NO_VCS=1 was introduced to opt-out of using git or any VCS to create the artifact using shallow file copy. With this flag, EAS then read from an .easignore file to determine which file it should not copy into the cloud build artifact. The syntax for the .easignore file is the same as a .gitignore file:

# add any modules in your monorepo that is unrelated to EAS
service-a/
server-b/
socket-c/

However, there's one issue: monorepo. In git workflow, expo can simply rely on git to get the project structure. With the plain workflow, expo can now only see the current directory it is running in. Thankfully, the expo team had implemented a solution for this, albeit undocumented.

I found this solution while reading through eas-cli source code, looking for a way to patch it. Apparently, eas-cli also read another flag in the environment called EAS_PROJECT_ROOT: source. This variable is used by the EAS shallow copy method to create the complete project artifact.

To use this flag, first I move my .easignore file to the root of my monorepo. Then, I run EAS like this:

cross-env EAS_NO_VCS=1 EAS_PROJECT_ROOT=.. eas build

And voila - everything worked. Now instead of uploading a whooping 160MB artifact, our deployment upload a measly 2.6MB artifact of front-end specific code!

TypeScript Dynamic Module Import

Louis — Mon, 22 Nov 2021 00:43:00 +0000

Can you tell the difference between these two import statements?

import Button from "@kenaz/button";

const buttonModule = await import("@kenaz/button");
const Button = buttonModule.default;

Simple right? - The first imports the module statically at build-time, whereas the second imports the module dynamically at run-time.

With the first code snippet, most ES bundler will include the button component in our final bundle. Whereas with the second snippet, our button could be split into a separate chunk. This could be an advantage when your page gets super duper complex with 1000 states, but the button is only needed in 1 of those states.

Now, can you tell the difference between these twos?

async function loadButton() {
  const buttonModule = await import("@kenaz/button");
  return buttonModule.default;
}

async function loadComponent(name = "@kenaz/button") {
  const module = await import(name);
  return module.default;
}

Disregard the context and the use case supplied via the naming of those functions, these two dynamic import statements do not seem to differ that much. At least, that's what I thought when I was using these two interchangeably while developing locally with a dev server that was transpiling these code on-the-fly.

However, the difference came to light when I created a production bundle while trying to import a local module using the 2nd snippet:

const Button = await loadComponent("./button")

It threw this error:

Cannot find module './button'
Require stack:

/app/dist/index.js Error: Cannot find module './button'

I filed an issue for the ncc bundler regarding a similar behavior

It turned out that when passing a local path to import, the code becomes unanalyzable by the TypeScript compiler. Thus, the compiler does not bundle the local import code, leaving us with the error above.

The more I think of this, the more I realized this behavior is reasonable. I.e, if the module name is arbitrary, how could the compiler know which module would be imported? Should it just compile every possible combination of file in the project then? That's probably not a good solution...

My conclusion: for local module (owned by my project such as "./button"), statically analyzable dynamic import is required for most bundler to find and compile those module correctly. On the other hand, if the module is in a separate module (a dependency such as @org/some-module), the node runtime can crawl the node_packages directory for them.

Quick guide to AudioWorklet 🔉

Louis — Fri, 05 Nov 2021 19:17:09 +0000

Ponder is a voice-first application with audio recording and processing as its core features. Unlike most audio Web Apps where the processing is done after the recording is finished, Ponder processes the audio in real-time to provide advanced business logic and features such as transcription. There are two requirements:

Storing audio with full integrity and cross-device playback
Processing the audio while recording in real-time

This article is a guide to how Ponder accomplishes these requirements using AudioWorklet.

🐞 The problem with ScriptProcessor

The WebAudio ScriptProcessor API was deprecated in favor of the AudioWorklet in 2014. However, the AudioWorklet API was unavailable on iOS Safari until the recent 15.0 update.

Over six months of beta testing our product, we received feedback from our customers with concerns about the recorded audio quality when using Safari on iOS.

We investigated two solutions to mitigate the issue:

Splitting the audio stream using two separate ScriptProcessor nodes:
- Node A with a buffer rate of 8192 for smoother audio playback
- Node B with a buffer rate of 4096 for frequent transcription requests
Using a WebWorker to process the audio buffer in a background thread, using GoogleChromeLabs/comlink and GoogleChromeLabs/worker-plugin

The audio stream splitting method was a neat technique that helped with optimizing our audio processing pipeline, and the WebWorker processor setup, with full TypeScript interop through comlink, was a breeze to work with.

The resulting code was concise and pretty, but it did not solve the audio problem. If anything, these improvements made our system even more fragile. On Android, the audio worked well. On iOS however, the audio suffered issues from lagging to stuttering, to missing a large chunk of recorded audio. We observed an improvement to the smoothness of our UI, however our priority is the quality and integrity of the audio.

From a first principle perspective, the audio buffer that we were getting data from is still being generated by the ScriptProcessor API's callback. This led us to conclude that it would likely remain until we replaced the ScriptProcessor API.

🐴 ScriptProcessor implementation

We initially used the approach described in this google documentation for our old ScriptProcessor based audio recorder.

Below is the most relevant code snippet inspired by that tutorial:

🦄 AudioWorklet implementation

In this section, we shall implement an in-place replacement for the above ScriptProcessor code, using the AudioWorkletProcessor API. The two key code paths were marked A and B:

A: Replace ScriptProcessor with an implementation using AudioWorkletProcessor
B: Get the data from AudioWorkletProcessor and consume it

A: Replacing ScriptProcessor

First, let's create a recorder.worklet.js file for our AudioWorkletProcessor:

This file should be served by your static web server. In a nextjs application, you can simply place it under the public directory. Then, in your web app, register the Worklet processor as follows:

B: Reimplementing ScriptProcessor

Currently, the AudioWorkletProcessor emit audio data blocks of 128 frames long - i.e a Float32Array of 128 data sample doc📄. To emulate the behavior of ScriptProcessor, we will need to buffer the sample data produced by AudioWorkletProcessor.

While researching this setup, I stumbled upon a helpful gist by flpvsk. The buffer implementation below borrowed many ideas from that gist.

Buffering is a technique in data processing where instead of consuming the input data right away, we store the data in a temporary location and wait until a certain threshold is met to start reading the data. The key use case is when the input is a stream of data, and the output is a decoded representation of the streamed data, such as audio and video.

Why do we need buffering? - Let’s take a look at a communication example: Alice is sending Bob a sequence of letters: H, E, L, L, O. If Bob’s short term memory was not functioning, he might have forgotten about the letter H the moment he received the letter E, and so on and so forth. Thus, the lack of buffering prevented Bob from forming a coherent word from Alice, leading to miscommunication. If his memory is normal, he would buffer the letters until his brain recognizes that a group of letters matches a word in his vocabulary.

The idea is similar when processing microphone input for human speech. The basic groundwork to set up a buffer requires 3 fields:

bufferSize: A variable to track the size of the buffer
_byteWritten: A pointer to track the location of the last byte written (i.e, an index)
_buffer: The buffer itself

To capture the Float32Array of data samples produced by AudioWorkletProcessor mentioned earlier, we implement its process method and store the data as follows:

The append method above allows us to fill the data buffer with microphone samples. Next, we will need to check if the buffer is full to emit the data back to our main thread for upstream processing. We will also want to reset the _bytesWritten pointer. Let's call this operation flush.

To make it easier to work with our buffer and implement flush, we can add a couple of helper methods:

initBuffer: Reset our _byteWritten pointer to its initial position
isBufferEmpty: Check if our pointer is at its initial position
isBufferFull: Check if our pointer is of the same size as our pre-allocated buffer

With our helpers, we can implement flush method as follows:

Finally, we can invoke this flush method at the beginning of our append like so:

With that, our re-implementation of the ScriptProcessor node with a 4096 buffer size and a single input/output channel is now complete. The full code is published here.

Lastly, you can retrieve the data and consume it the same way you would with ScriptProcessor in your web app:

NOTE: GCP's speech to text service requires an Int16Array buffer. An example utility to convert the Float32Array is shown below:

Have any questions or feedback? Feel free to leave a comment here or on the gist itself!

🧪 Why did we choose a buffer size of 4096?

Recall our buffering example above: the buffer size in that example is a single alphabet letter. For speech audio communication however, there is no direct equivalent. Thus, we need to experiment with the delay in real-time transcription responses coming from Google Speech to Text service.

A smaller buffer size results in more frequent invocation of the flush method, and vice versa. Flush effectively sends data to the speech service.

For our experimentation, we track 2 timestamps:

Time when flush was first called
Time when our client received the first transcribed word

By subtracting t1 from t2, we can obtain the total delay:

AudioWorklet emits a minimum of 128 bytes per tick. Following ScriptProcessor's implementation, our buffer size should be a power of 2 between 128 and 16384. For each size, we repeat the experimentation 10 times. The results are in the table below:

To analyze the data, we can visualize it with a box plot:

Based on the plot, 4096 was chosen since it has the lowest delay, while also having a reasonable buffer size to reduce processing frequency.

After switching to our new AudioWorklet implementation, Ponder has stopped seeing audio quality issues such as lagging, stuttering, or missing audio. Furthermore, the transcription speed is noticeably faster.

🙏 Thank you to our beta testers!

This post is dedicated to the constructive feedback that motivated our engineering team to keep on improving Ponder's asynchronous coaching experience. The architecture improvement detailed in this post is now on our staging environment and is being tested by our internal team. We will release it into production under Ponder v1.5.0. Stay tuned!

typescript and create-react-app .env

Louis — Tue, 06 Aug 2019 14:50:43 +0000

You can create a type definition file that extends the NodeJS namespace to enable auto-completion for your env configuration. This is quite useful if you plan to reduce the amount of time spent looking up your .env file:

Check bottom for the solution. For now, let’s wander back a bit. I first thought about this problem when I found that process.env.NODE_ENV is typed and has code completion, while the other does not:

This piece of information had been in my mind since the first time I saw it around 2017. However, I never really looked into it too deeply as back then I had no need to leverage client-side environment variable.

At Plasmo, we decided to leverage client-side environment to swap between staging and production endpoints. Initially with 2 variables it was manageable. But when that number grew to 13, it became a bit annoying to lookup the .env file, as the variable names are increasingly getting longer. This annoyance can be tolerated for a while, as we were focusing on our product. However, in the back of my mind, there was a strong drive to squash this “bug.”

Technically this is not a bug, that is why there were "”.

A big win of using TypeScript is that refactoring is a breeze, and that is the key to move fast — starting with one file, grow it to 300 loc, then split it into several files, rinse repeat. This applies to our type definition as well, as using a single .d (definition) file to describe both our api typing as well as our theme typing becomes convoluted. So we decided to finally split our typing into these neat files:

Finally, I can bid our good ol’ react-app-env.d.ts farewell. But much to my surprise, when resuming CRA (create-react-app) server afterward, that file was resurrected. After going through CRA's internal packages for a bit, I found that react-scripts (the main program powering CRA) was programmed to generate this file.

This is when my brain started to notice the name of this file: react-app-env. Every magic string has meaning, said the basic rule of edb (on Windows you can use x64dbg). To generate a file with such specific name means there was some significant use of it, probably within the organization of ${process.env.REACT_APP_AUTHOR}.

Digging deeper with duckduckgo.com with the search term “react-app-env.d.ts declare global,” the third result (note: not 1st or 2nd, but 3rd) gave some great insight, in particular, in this comment.

The github issue's author was trying to tackle the “bug” at hand, and the code snippet was extremely insightful:

declare namespace NodeJS {
  interface ProcessEnv {
    NODE_ENV: 'development' | 'production' | 'test'
    PUBLIC_URL: string
  }
}

In brief, the typescripts support of CRA consume an extended definition under the NodeJS namespace. To extend it for our environment, simply put the snippet above in your react-app-env.d.ts

BONUS

You now know how to extends your environment typing for some good old lazy process.env auto-completion. But now you need to deal with global API placed under Window (like Stripe.js). Below will save you a trip to the duck (This is a simplified version of our react-app-env.d.ts file:

/// <reference types="react-scripts" />
declare namespace NodeJS {
interface ProcessEnv {
    NODE_ENV: 'development' | 'production' | 'test'
    PUBLIC_URL: string
    REACT_APP_HASH: string
    REACT_APP_API_URI: string
    REACT_APP_WS_URI: string
    }
}
interface Window {
    Stripe: any
}