DEV Community: JiaLiPassion

Build Typescript Project with Bazel Chapter 2: File Structure

JiaLiPassion — Mon, 30 Mar 2020 18:57:51 +0000

Build Typescript Project with Bazel Chapter 2: File Structure

In the last chapter, we introduced the basic concept of Bazel. In this blog, I would like to talk about the file structure of Bazel.

Concept and Terminology

Before we introduce the file structure, we need to understand several key concepts and terminology in Bazel.

Workspace
Package
Target
Rule

These concepts, and terminology, are composed to Build File, which Bazel will analyze, and execute.

The basic relationship among these concepts looks like this , we will discuss the details one by one.

Workspace

A "workspace" refers to the directories, which contain

The source files of the project.
Symbolic links contain the build output.

And the Bazel definition is in a file named WORKSPACE, or WORKSPACE.bazel at the root of the project directory. NOTE, one project can only have one WORKSPACE definition file.

Here is an example of the WORKSPACE file.

workspace(
    name = "com_thisdot_bazel_demo",
)

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

# Fetch rules_nodejs so we can install our npm dependencies
http_archive(
    name = "build_bazel_rules_nodejs",
    sha256 = "ad4be2c6f40f5af70c7edf294955f9d9a0222c8e2756109731b25f79ea2ccea0",
    urls = ["https://github.com/bazelbuild/rules_nodejs/releases/download/0.38.3/rules_nodejs-0.38.3.tar.gz"],
)

load("@build_bazel_rules_nodejs//:defs.bzl", "node_repositories", "yarn_install")

node_repositories()

yarn_install(
    name = "npm",
    package_json = "//:package.json",
    yarn_lock = "//:yarn.lock",
)

# Install all Bazel dependencies of the @npm npm packages
load("@npm//:install_bazel_dependencies.bzl", "install_bazel_dependencies")

install_bazel_dependencies()

# Setup the rules_typescript toolchain
load("@npm_bazel_typescript//:index.bzl", "ts_setup_workspace")

ts_setup_workspace()

In a WORKSPACE file, we should

Define the name of the workspace. The name should be unique globally, or at least unique in your organization. You could use the reverse dns name, such as com_thisdot_bazel_demo, or the name of the project on GitHub.
Install environment related packages, such as yarn/npm/bazel.
Setup toolchains needed to build/test the project, such as typescript/karma.

Once WORKSPACE is ready, application developers don't really need to touch this file.

Package

The primary unit of code organization (something like module) in a repository
Collection of related files and a specification of the dependencies among them
Directory containing a file named BUILD or BUILD.bazel, residing beneath the top-level directory in the workspace
A package includes all files in its directory, plus all subdirectories beneath it, except those which, themselves, contain a BUILD file

It is important to know how to split a project into package. It should be easy for the users to develop/test/share the unit of a package. If the unit is too big, the package has to be rebuilt on every package file change. If the unit is too small, it will be very hard to maintain and share. So, this is not an issue of Bazel. It is a general problem of project management.

In Bazel, every package will have a BUILD.bazel file, containing all of the build/test/bundle target definitions.

For example, here is a of the Angular structure. Every directory under packages directory is a package of code organization, and also the build organization of Bazel.

Let's take a look at the file structure of gulpjs in Angular, so we can have a better understanding about the difference between Bazel and gulpjs.

gulp.task('build-animations', () => {});;
gulp.task('build-core', () => {});
gulp.task('build-core-schematics', () => {});

In most cases,

a gulpjs file doesn't have 1:1 relationship to the package directory.
a gulpjs file can reference any files inside the project.

But for Bazel,

Each package should have their own BUILD.bazel file.
The BUILD.bazel can only reference the file inside the current package, and if the current package depends on other packages, we need to reference the Bazel build target from the other packages instead of the files directly.

Here is a Bazel Package directory structure in Angular repo.

Build File

Before we talk about target, let's take a look at the content of a BUILD.bazel file.

package(default_visibility = ["//visibility:private"])

load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "lib",
    srcs = [":lib.ts"],
    visibility = ["//visibility:public"],
)

The language of the BUILD.bazel file is Starlark.

Starlark is a subset of Python.
It is a very feature-limited language. A ton of Python features, such as class, import, while, yield, lambda, is, raise, are not supported.
Recursion is not allowed.
Most of Python's builtin methods are not supported.

So Starlark is a very very simple language, and only supports very limited Python syntax.

Target

The BUILD.bazel file contains build targets. Those targets are the definitions of the build, test, and bundle work we want to achieve.

The build target can represent:

Files
Rules

The target can also depend on other targets

Circular dependencies are not allowed
Two targets, generating the same output, will cause a problem
Target dependency must be declared explicitly.

Let's see the previous sample,

package(default_visibility = ["//visibility:private"])

load("@npm_bazel_typescript//:index.bzl", "ts_library")

ts_library(
    name = "lib",
    srcs = [":lib.ts"],
    visibility = ["//visibility:public"],
)

Here, ts_library is a rule imported from @npm_bazel_typescript workspace, and ts_library(name = "lib") is a target. The name is lib, and this target defines the metadata for compiling the lib.ts with ts_library rule.

Label

Every target has a unique name called label. For example, if the BUILD.bazel file above is under /lib directory, then the label of the target is

@com_thisdot_bazel_demo//lib:lib

The label is composed of several parts.

the name of the workspace: @com_thisdot_bazel_demo.
the name of the package: lib.
the name of the target: lib.

So, the composition is <workspace name>//<package name>:<target name>.

Most of the times, the name of the workspace can be omitted, so the label above can also be expressed as //lib:lib.

Additionally, if the name of the target is the same as the package's name, the name of the target can also be omitted. Therefore, the label above can also be expressed as //lib.

NOTE: The label for the target needs to be unique in the workspace.

Visibility

We can also define the visibility to define whether the rule inside this package can be used by other packages.

package(default_visibility = ["//visibility:private"])

The visibility can be:

private: the rules can be only used inside the current package.
public: the rules can be used everywhere.
//some_package:package_scope: the rules can only be used in the specified scope under //some_package. The package_scope can be: __pkg__/__subpackages__/package group.

And if the rules in one package can be accessed from the other package, we can use load to import them. For example:

load("@npm_bazel_typescript//:index.bzl", "ts_library")

Here, we import the ts_library rule from the Bazel typescript package.

Target

Target can be Files or Rule.
Target has input and output. The input and output are known at build time.
Target will only be rebuilt when input changes.

Let's take a look at Rule first.

Rule

The rule is just like a function or macro. It can accept named parameters as options. Just like in the previous post, calling a rule will not execute an action. It is just metadata. Bazel will decide what to do.

ts_library(
    name = "lib",
    srcs = [":lib.ts"],
    visibility = ["//visibility:public"],
)

So here, we use the ts_library rule to define a target, and the name is lib. The srcs is lib.ts in the same directory. The visibility is public, so this target can be accessed from the other packages.

Rule Naming

It is very important to follow the naming convention when you want to create your own rule.

*_binary: executable programs in a given language (nodejs_binary)
*_test: special _binary rule for testing
*_library: compiled module for a given language (ts_library)

Rule common attributes

Several common attributes exist in almost all rules.For example:

ts_library(
    name = "lib",
    srcs = [":index.ts"],
    tags = ["build-target"],
    visibility = ["//visibility:public"],
    deps = [
        ":date",
        ":user",
    ],
)

name: unique name within this package
srcs: inputs of the target, typically files
deps: compile-time dependencies
data: runtime dependencies
testonly: target which should be executed only when running Bazel test
visibility: specifies who can make a dependency on the given target

Let's see another example:

http_server(
   name = "prodserver",
   data = [
       "index.html",
       ":bundle",
       "styles.css",
   ],
)

Here, we use the data attribute. The data will only be used at runtime.It will not be analyzed by Bazel at build time.

So, in this blog, we introduced the basic Bazel structure concepts. In the next blog, we will introduce how to query Bazel targets.

This Dot Labs is a modern web consultancy focused on helping companies realize their digital transformation efforts. For expert architectural guidance, training, or consulting in React, Angular, Vue, Web Components, GraphQL, Node, Bazel, or Polymer, visit [thisdotlabs.com]((https://www.thisdotlabs.com).

This Dot Media is focused on creating an inclusive and educational web for all. We keep you up to date with advancements in the modern web through events, podcasts, and free content. To learn, visit thisdot.co.

Event Coalescing to improve performance

JiaLiPassion — Thu, 16 Jan 2020 01:18:04 +0000

Event Bubbling の性能の問題

下記のAngularアプリケーションでのHTMLテンプレートコードを見ましょう。

<div (click)="doSomething()">
  <button (click)="doAnotherThing()">Button</button>
</div>

Event Bubbling のため、DivのなかのButtonをクリックしたら、両方のEvent ListenerがTriggerされます。この２つのEvent Listenerが全部Change Detectionを実行します。実際はこのようなケースでChange Detectionを一回だけ実行したいです。特にこのようなケースがAngular MaterialとかUI ライブラリで結構普通のケースですので、性能改善したいです。

実行順番

まず上記のケースでEvent ListenerとChange Detectionの実行順番を見ましょう。

export class AppComponent {
  doSomething() { // event listener for parent div
    console.log('event listener for parent div');
  }
  doAnotherThing() { // event listener for inner button
    console.log('event listener for inner button');
  }
}

そして、Zone.jsがEvent ListenerをMonkey patchされましたので、下記のような感じのコードになりました。

function eventHandler(...) {
  try {
    realHandler(...); // doSomething あるいはdoAnotherThingのDelegate
  } finally {
    applicationRef.tick();
  }
}

そしたら、Buttonをクリックするとき、実行の順番が下記のようになりました。

event listener for inner button
applicationRef.tick
event listener for parent div
applicationRef.tick

実際ほしいのは

event listener for inner button
event listener for parent div
applicationRef.tick

です。

解決方法

Change Detectionを同期ではなく、非同期で実行することです。

isChangeDetectionScheduled = false;
function eventHandler(...) {
  try {
    realHandler(...); // doSomething あるいはdoAnotherThingのDelegate
  } finally {
    if (isChangeDetectionScheduled) {
      return;
    }
    isChangeDetectionScheduled = true;
    requestAnimationFrame(() => {
      isChangeDetectionScheduled = false;
      applicationRef.tick();
    }); 
  }
}

のような感じのコードでChange DetectionをrequestAnimationFrameのSchedulerで実行させ、そしてもしScheduleされたTaskがあったら、スキップして、なかったら、Scheduleするということです。

設定方法

bootstrapModuleのとき、かきのOptionを設定することができます。

platformBrowserDynamic().bootstrapModule(AppModule, {ngZoneEventCoalescing: true})

副作用

このオプションをTrueにしたら、もともと同期のChange Detectionが非同期になりました、普通のアプリケーションには影響がないですが、Google 内部でのEdge Caseで同期のChange Detectionが求めるテストケースがあるらしくて、それ以外が特に既存のアプリケーションには影響ないはずです。
この機能がすでに最新バージョンで使えますので、なにか問題が発見されたら、ぜひAngular RepoにIssueを提出ください。

以上です。

どうもありがとうございました！

Build Typescript Project with Bazel Chapter 1: Bazel introduction

JiaLiPassion — Wed, 08 Jan 2020 15:47:55 +0000

Bazel is a fast, scalable, incremental, and universal (for any languages/frameworks) build tool, and is especially useful for big mono repo projects.

I would like to write a series of blogs to introduce the concept of how to build a typescript project with Bazel.

Chapter 1: Bazel Introduction
Chapter 2: Bazel file structure and Bazel Query
Chapter 3: Build/Develop/Test a typescript project

In chapter 1, I would like introduce basic Bazel concepts, and define some of the benefits we can expect from using Bazel.

What is Bazel?
Bazel: Correctness
Bazel: Fast
Bazel: Universal
Bazel: Industrial grade

What is Bazel?

As you may know, we already have a lot of build tools. They include:

CI tools: Jenkins/CircleCI
Compile tools: tsc/sass
Bundle tools: webpack/rollup
Coordinate tools: make/grunt/gulp

So what is Bazel? Does it simply replace Jenkins or Webpack?
@AlexEagle helped us answer this question at ngconf 2019, but here is a great picture that will explain a little as well.

So Bazel is a build tool, used to coordinate other tools (compile/bundle tools), and will use all the existing tools (such as tsc/webpack/rollup) to do the underlying work.

Another graph, also from @AlexEagle, will show this relationship more clearly.

Ok, so Bazel is at the same position as Gulp, why not continue to use Gulp?

To answer this question, let's think about what the goal of the build tool is.

Essential:
- Correct - don't need to worry about environmental pollution.
- Fast
  - Incremental
  - Parallelism
- Predictable - Same input will guarantee the same output.
- Reusable - Build logic can be easily composed and reused.
Nice to have:
- Universal - support multiple languages and frameworks.

Correctness

This is the most important requirement. We all want our build systems to be stable, and we don't want them to generate unexpected results. Therefore, we want every build to be executed in an isolated environment. Otherwise, we will run into problems if, for example, we forget to delete some temp files, forget to reset environment variables, or if the build only works under certain conditions.

Sandboxing: Bazel supports sandboxing to isolate the environment. When we do a Bazel build, it will create an isolated working folder, and Bazel will run the build process under this folder. This is what we would call "sandboxing". Bazel will then restrict the access to the files outside of this folder. Also, Bazel makes sure that elements of the build tool, such as the compiler, only know their own input files, so the output will only depend on input.
The rule can only access an input file. Unlike a Gulp task, a Bazel rule can only access the files declared as input (we will talk about the target/rule in detail later).

Here is an example of a Gulp task:

gulp.task('compile', ['depTask'], () => {
  // do compile
  gulp.src(["a.ts"])
      .pipe(tsc(...));
});

So inside of a Gulp task, there is just a normal function. There is no concept of Input, and dependencies only tells gulp to run tasks in a specified order, so the task can access any files, and use any environmental variables with no restrictions. Gulp will have no idea which files are used in this task, so if some logic depends on the unintended file access/environment reference, it is impossible for Gulp to guarantee that the task will always generate the same results.

Let's see a Bazel target.

ts_library(
    name = "compile",
    srcs = ["a.ts"],
)

We will talk about the Bazel target/rule in more detail in the next chapter. Here, we will declare a Bazel target with a ts_library rule. Unlike with a Gulp task, here we have a strict input which is srcs = ["a.ts"], so when Bazel decides to execute this target, the typescript compiler can only access the file a.ts inside of the sandbox, and nowhere else. Therefore, there is no way that the Bazel target will produce wrong results because of the unpredictable environment or input.

Fast

Bazel is incremental because Bazel is declarative and predictable.

Bazel is Declarative

Let's use Gulp to compile those two files, in order to demonstrate that Gulp is imperative and Bazel is declarative. Let's see an example with Gulp.

// gulpfile.js
gulp.task('compile', () => {
  gulp.src(['user.ts', 'print.ts'])
      .pipe(tsc(...))
      .pipe(gulp.desc('./out'));
});

gulp.task('test', ['compile'], () => {
   // run test depends on compile task
});

When we run gulp test for the first time, both the compile, and the test tasks will be executed. And then, even if we don't change any files, those two tasks will still be executed if we run gulp test again. Gulp is imperative, so we just have to tell it to do those two commands, and Gulp will do what we asked. Specifically, it checks the dependency, and guarantees the execution order. That's all.

Let's see how Bazel works. Here, we have two typescript files: user.ts, and print.ts. print.ts uses user.ts.

// user.ts
export class User {
  constructor(public name: string) {}

  toString() {
    return `user: ${this.name}`;
  }
}

// print.ts
import {User} from './user';

function printUser(user: User) {
  console.log(`the user is ${user.name}`);
}

printUser(new User('testUser'));

To demonstrate that Bazel is declarative, let's use two Bazel build targets.

# src/BUILD.bazel
ts_library(
    name = "user",
    srcs = ["user.ts"],
)

ts_library(
    name = "print",
    srcs = ["print.ts"],
    deps = [":user"]
)

So we declare two Bazel targets, user, and print. The print target depends on the user target. All those targets are using the ts_library rule. It contains the metadata to tell Bazel how to compile the typescript files. And again, all this information is just a definition. It's not about commands, so when you use Bazel to build those targets, it is up to Bazel to decide whether to execute those rules or not.

Let's see the result first.

When we run bazel build //src:print, both the user and print targets will be compiled, which makes sense. When we run bazel build //src:print again, you will find Bazel will not run any targets because nothing changed, and Bazel knows it. As a result, Bazel decides not to run any targets.

Let's change something in user.ts, and see what happens.

// updated user.ts
export class User {
  constructor(public name: string) {}

  toString() {
    return `updated toString of user: ${this.name}`;
  }
}

After we run bazel build //src:print again, we may expect that both user and print will be compiled once more because user.ts has been changed, and print.ts references user.ts, and the print target depends on the user target. But the result is that only the user target has been compiled, and the print target has not. Why?

This is because changes in user.ts don't impact print.ts, and Bazel understands this.

Let's check out the following graph, which describes the input/output of the target.

So for user target, the input is user.ts, and we have two outputs. One is user.js, and the other is user.d.ts. The latter of the two is the typescript declaration file. So let's see the relationship between the user, and print target.

Here, we can see that the print target depends on the user target, and that it uses one of the user target's outputs, user.d.ts, as it's own input. So, because we only updated toString of user.ts, and the user.d.ts was not changed at all, Bazel analyses the dependency graph. As a result, it knows that only user target needs to be built. Further, it also knows that the print target doesn't need to be built because the inputs of the print target, which are user.d.ts and print.ts, have not changed. Because of this, Bazel decides not to build print target.

It is very important to remember that Bazel is declarative and not imperative.

Bazel analyses the input/output of all build targets to determine which targets need to be executed.

(We can generate the dependency graph with bazel query, and we will talk about it in the next chapter.)

So Bazel can do incremental builds based on the analysis of the dependency graph, and only build the really impacted targets.

Bazel is Predictable (Bazel's Rules are Pure Functions)

Also, all Bazel rules are pure functions, so the same input will always result in the same output, hence Bazel is predictable. We can use the input as a key to cache the result of each target, and save the cache locally or remotely.

For example, developer 1 builds some targets, and pushes the result to remote cache. The other developer can then directly use this cache without building those targets in their own environment.

So these amazing features make Bazel super fast.

Universal

Bazel is a coordinate tool. It doesn't rely on any specified languages/frameworks, and it can build within almost all languages/frameworks from server to client, and from desktop to mobile.

It is difficult and costly to employ a specialist team to handle builds with several build tools/frameworks when working on a full-stack project. In one of my previous projects, we used Maven to build a Java backend, used Webpack to build its frontend, and used XCode and Gradle to build iOS and Android clients. Consequently, we needed a special build team consisting of people that knew all of those build tools, which makes it very difficult to do an incremental build, cache the results, or share the build script with other projects.

Bazel is also a perfect tool for mono repo, and full-stack projects that include multiple languages/frameworks.

Industrial Grade

Bazel is not an experimental project. It is used in almost all projects inside Google. When I started contributing to Angular, I did not use Bazel. Because of this, the time that Angular CI was taking was about 1 hour. Once Bazel was introduced to Angular CI, the time reduced to about 15 minutes, and the build process became much more stable, and less flaky than before, even with double the amount of test cases. It is amazing! I believe that Bazel will be the "must have" tool for many big projects.

I really like Bazel, and in the next blog post, I would like to introduce Bazel's file structure with bazel query.

Thanks for reading, and any feedback is appreciated.

Enjoy this article? Head on over to This Dot Labs and check us out! We are a tech consultancy that does all things javascript and front end. We specialize in open source software like, Angular, React and Vue.