Switching from angular2-template-loader to @ngtools/webpack

I don't usually write "utility" posts that describe a way to fix a very specific problem but I ran into something that doesn't seem to have a good resolution path online.

A huge part of this article is all of the debugging/troubleshooting I've done to get everything to work. The entire project took me more than a week of focused work as well as bringing in another coder for pairing for several hours almost every day. I've had to switch from working on my laptop (which was getting overheated and stuttering at the constant recompiles) to my desktop which is currently heating my office way past comfortable temperatures. This was an excruciating process but...I did it!

The Problem

If you're using Angular (or "Angular2+"), you might be using the angular2-template-loader which allows you to do some neat things such as require all of your Component templates and SASS files which you can then run through other loaders in Webpack.

You'll send up with something like this for your components:

@Component({
  template: require('./button.component.html'),
  styles: [require('./button.component.scss')]
})
export default ButtonComponent {}

Whoa, check that out, we can use SCSS in our Angular components. That's kind of the power of it. Angular2-template-loader will then load up the html/scss (or rather its post-processed versions) and inline them within the template itself.

The problem is that this effectively disallows AoT or Ahead-of-time compilation. So while the angular2-template-loader is very popular, often used in tutorials, and very easy to setup, it also creates a problematic solution for the AoT compiler.

The AoT compiler

AoT stands for "ahead of time". The AOT compiler will look at the templates referenced in a component, parse them, and create the JavaScript logic to do what the templates ask it to do.

The best way for me to describe the AOT compiler is that instead of parsing through the HTML, figuring out where there are repeats, what components are referenced where, etc. at application boot time (when the browser loads the application), it happens during build time.

In our case at work, this process seems to take several seconds which is absolutely ridiculous. Plus, if you compile on application load, you have to include the compiler in your bundle. :(

But...angular2-template-loader doesn't do AOT (it doesn't claim it does!) and AOT can't happen with it.

The webpack loader alternative

The immediate alternative I've found is the @ngtools/webpack package which not only does AOT but also serves as a TypeScript loader! It does some other things as well but I want to focus on this first.

First, we gotta replace the old angular2-template-loader and whatever typescript loader you're using, this should look kind of like this at the end (and your TypeScript loader and your angular2-template-loader should be gone):

// somewhere in your webpack config
rules: [
  {
    test: /(?:\.ngfactory\.js|\.ngstyle\.js|\.ts)$/,
    loaders: [
       {
       loader: '@ngtools/webpack',
       },
    ],

  }
]

You also need to your webpack plugins:

import { AngularCompilerPlugin } from '@ngtools/webpack';

// and somewhere in your webpack config
plugins: [
    new AngularCompilerPlugin({
      tsConfigPath: './tsconfig.json',
      entryModule: './web/modern/main.ts#AppModule'
    }),
]

Fantastic! The only problem? All those requires..

Fixing the html requires

The problem now is that we have tons of components that have that fancy require for templates and styles. What do we do now?

Here's why I'm writing this so I can share/document this info and come back to it. Phew. So, @ngtools/webpack allows us to do similar requires but without the require.

Basically, we have to change

@Component({
  selector: 'button-component',
  template: require('./button.component.html')
})

into:

@Component({
  selector: 'button-component',
  templateUrl: './button.component.html'
})

Note that we're still referencing a templateUrl, not a template. NgTools brings that template in and does AOT on it. So how do we do this change on a large scale? By using the Unix tool grep and then using plain ol' node to do the changes

Grepping for all files

Grep is a unix tool present in Linux, macOS, and other systems. You can also get it on Windows. Unfortunately, Windows's Select-String will not the do the job we need it do to today (though go read about how to use Powershell's Select-String like grep if you're into that).

With grep we can select all the files that need updating.

grep -rl 'Component({' /path/to/application

The flag -r will make sure grep looks recursively through files in your application with the string Component({ in it.

If you just run this by itself, you'll get a long list of files that you need to update; however, we can use node to do the replacement for us.

Change your files

Here's the thing, I tried sed, I really did. But it took so long to do one small operation that I figured I might as well write a node script for it.

First, we need a really complicated Regex that can do the correct substitution and replace require('./template.component.html') with just ./template.component.html:

/Component\(\{(.*\n)+.*template: require\(('.*\.html')\)(.*\n)+\}\)/

Oh s***, wtf is that? Well, I hate to say this but this is our ticket to freedom. What this regex does is:

1. look for the Component({ string
2. it matches "however many newlines filled with characters" (that's the (.*\n)+)
3. it finds the template string with the require. Note the extra parentheses
4. group matching allows us to identify just the HTML string via ('.*\.html').
5. and then we match for "however many newlines filled with characters"
6. and finally we match for the closing }).

Basically anything that matches this pattern:

Component({
  something: else,
  doesnotmatter: whatshere,
  template: require('./path/to/template.html'),
  moreKeywords: withData
})

It doesn't matter how many keys are in that object or what data, as long as it has a template that requires an HTML file, we'll match on it.

Let's write the node script. It'll need to read the path of a file as an argument, do the substitution, and write to that file with the changes. I'm going to skip the craziness of explaining this step by step so here's the final JavaScript file:

const path = require('path');
const fs = require('fs');

function run() {
  /*
   * Args which should look like ['node', 'path/to/this/script.js', '/path/tofile.ts']
   */
  const args = process.argv;
  const files = args.slice(2);

  files.forEach(file => {
    changeFile(file);
  });
}
function changeFile(relativeFilePath) {
  const filePath = path.resolve(__dirname, relativeFilePath);
  const fileString = fs.readFileSync(filePath).toString();
  const regex = /Component\(\{(.*\n)+.*template: require\(('.*\.html')\)(.*\n)+\}\)/;

  const match = fileString.match(regex);

  if (!match) {
    return;
  }

  const htmlPath = match[2];

  if (!htmlPath) {
    return;
  }

  const replacementLookup = `template: require(${htmlPath})`;
  const replacement = `templateUrl: ${htmlPath}`;
  const newFileString = fileString.replace(replacementLookup, replacement);

  fs.writeFileSync(filePath, newFileString);
}

run();

Basically, this script will:

1. read in script arguments to get the file paths
2. read the files one by one
3. use that fancy regex to find a match. match group 2 (3rd item in the array) will be our html url
4. do a replace on the file string where the original template value gets replace with the new one
5. save it!

Note This script is pretty handy. You can use it to update the styles as well if you run into that issue in the troubleshooting section. :)

Put it all together

Ok, so to put it altogether, we use one more unix tool, xargs. It'll pipe the result of our grep into our node script which will then perform our replacement.

Caution This is a destructive action. Meaning that it'll affect each file and run directly. I'm not responsible for whatever problems you might run into with these scripts. MAKE SURE that you're using something like git and have committed all of your code changes up until this point so you can do a quick git hard --reset should you run into problems and get back to a working version of your project. PLEASE do make sure of this.

grep -rl 'Component({' /path/to/application | xargs node replace.js

The script will get each file as a separate argument so the xargs actually calls node replace.js path/to/component1.ts path/to/component2.ts and so on.

After this, you should be done!

The results

I want to share some preliminary benchmarks that I've done:

1. initial load time in a dev environment dropped from 10 seconds to 3 seconds uncached
2. initial load time (cached) dropped from 8 seconds to 1.8 seconds in a dev environment
3. compiler time is way more resource intensive (my office is a sauna)

I cannot wait to try this out in a production environment.

Template problems?

The compiler will call you out on your issues. I've mentioned it in Potential issues but it bears saying separately first: angular-cli will compile your templates. That means that it'll check for variables, it'll check for bindings, it'll check for everything. And it will let you know if you messed up. Common issues that I've had to fix:

1. referencing variables in the template that don't exist in the component
2. calling a function with the wrong number of arguments
3. passing in a variable to a component which does not have an input to receive that data

Potential issues

There were some issues I ran into during this conversion and I wanted to share how I was able to resolve them. Most of these have opened and/or closed issues on the angular-cli repo. :/ If you look up the errors directly, you can find those and follow the conversation. I wanted to provide you with how exactly I solved the problem and what other solutions were suggested.

IMPORTANT NOTES:

1. MAKE SURE you've saved your work and committed it in Git -- and NOT to your master branch. SERIOUSLY. I had to keep referring back to the master branch and the clean codebase
2. MAKE SURE you don't try to change things ahead of time. Look at the potential issues that YOU experience
3. It's ok if you have to undo some of the solutions
4. I can't fix this for you

ERROR in : Cannot determine the module for class AppRootComponent in /path/to/component! Add AppRootComponent to the NgModule to fix it.

I saw this come up in a few different places in issues and elsewhere. There are four solutions I've found to this issue:

Possible solution 1

You probably have bad import paths that webpack/node doesn't care about but the compiler does. Most of the time, this is due to capitalization. Make sure if you're importing a file that has capitalization in it (like AppRootComponent.ts) that you're capitalizing correctly in your import path. You can actually do an import like ./path/to/approotcomponent, and node/webpack won't complain.

Possible solution 2

The other possibility is that you have components that either aren't part of a module or are simply unimported but still in the working directory. Check for either of those situations and either put those components into modules or remove them.

Possible solution 3

Lastly, and this was my situation, you're using components in your main module. Urgh, I hate this problem because I think it shouldn't be causing these issues. Basically, if you have an AppRootComponent and you're using it to bootstrap Angular, put it in another module first, import that module, and then bootstrap.

Possible solution 4

There's another hidden error. As I was working through making everything work and through all of the other problems, I found out that I was still able to bootstrap AppRootComponent at the end of my journey, as long as it was part of my entry module (AppModule). So...I ended up reverting the solutions above once that happened.

ERROR in path/to/component/button.component.ts.ButtonComponentComponent.html(1,21): : Expected 0 arguments, but got 1.

This usually means that you have a typing issue in your template itself. There are several situation this shows up in:

1. you're calling a method or using a variable that was marked as private in your component
2. you have a typing issue. Eg. you're passing the wrong number of arguments into a method (this was my situation!)

TypeError: Cannot read property '_ngToolsWebpackPluginInstance' of undefined

This error occurred to me when using HappyPack with @ngtools/webpack. They're not compatible so just use @ngtools/webpack directly rather than with HappyPack

Module not found: Error: Can't resolve './main.ts.ngfactory' in '/path/to/application/directory'

Another strange problem that I don't understand. Note that main.ts is my entry file so that name may be different for you (such as app.ts). There are three solutions that I've found to this:

1. adjust your rootDir parameter in tsconfig check out the relevant issue
2. install enhanced-resolve via npm install enhanced-resolve@3.3.0 here's the relevant issue
3. you can add .ngfactory into the extensions that webpack needs to resolve in extensions.resolve in your webpack config

If none of this works, go ahead and adjust the @ngtools/webpack config by not specifying the entryModule and specifying an absolute path to the mainPath.

{
  mainPath: path.resolve(__dirname, '/path/to/entry/file')
}

This disables lazy route loading (automatic code splitting by route) which is a bummer but it was necessary in my case and the code splitting wasn't an issue for me anyways because I use an Angular/AngularJS hybrid app where the router rests on the AngularJS side.

Phew, let's keep going!

Potential issues in more complicated build chains

I wanted to split this section off because it's more likely that it's my own specific application architecture that causes these issues rather than something ngtools/webpack is responsible for.

TypeError: library_1.default is undefined

This issue basically means that a module is not available. It was an issue because the TypeScript compiler was compiling JavaScript files. Why is that a problem?

While Babel has no issue with this:

import _ from 'lodash';

TypeScript requires this syntax:

import * as _ from 'lodash';

What that meant for me was to switch to the TypeScript compiler entirely and adjust all of the import paths -- I ripped out our Babel compiler and changed the @ngtools/webpack loader to match against all .js files.

Here's the look up regex if you want to adjust the replace.js file I mentioned above: /import ([a-z0-9A-Z_]+) from/g;. I could write a whole article about the process so if you do get here, just comment below! And I'll give you the step-by-step process.

But, make sure you don't change the import paths for your own modules!

Host should not return a redirect source file from getSourceFile

There's an open issue on Github for this error and it suggests patching the compiler... yeah. The patch does work (I've tried it) and it basically involves:

1. looking up ./node_modules/@angular/compiler-cli/src/transformers/program.js
2. finding a line that starts with if (this.hostAdapter.isSourceFile(sf.fileName)) {
3. and putting if (sf['redirectInfo']) { sf = sf['redirectInfo'].redirectTarget; } right before

That essentially replaces a source file redirect with the actual file it redirected to. 🤷‍♀️ doesn't mean much to me; however, if you don't want to manually patch stuff in node_modules (even with the useful postinstall script included in the Github issue), you can do what I did and instead of step 3 where you set the redirect, you can log out the problematic files!

if (sf['redirectInfo']) {
  console.log('SF: ', sf.fileName);
}

And then tackle the problems that show up. In my case, the library @turf/turf was having some issues and namely, each module in @turf/turf was importing @turf/helpers and that was causing issues. I was able to solve those problems by:

1. destructuring any imports from the problematic library
2. using/installing submodule of the library directly

And I'm gonna also throw this in there: look for alternatives if it's not a huge cost to you.

SASS/SCSS not loading?

I encountered this issue as well. There's a closed ticket for it with not much information; however, if you're wanting to use SASS (and haven't before!), check out the thread for information on the SASS loader.

However, if you've used it before and things aren't working now, you just need to change how you load your styles:

// from
@Component({
    styles: [require('./path/to/component.scss')]
})

// to
@Component({
    styleUrls: ['./path/to/component.scss'] 
})

Additional Resources

There were a few resources that helped me along the way:

• Upgrading a project without cli to Angular 6

But Why?

Why am I sharing this long-ass article that has no clear path to navigate through it? So you can understand my journey. How I've struggled, what I've seen, and where I had to look for answers.

I'm a new person today. I look in the mirror, and I don't see the Antonin that I saw two weeks ago when I was preparing to make this loader change. I'm not the same developer.

These were trying times. My sanity has been tested. My perseverance has been tested. There were many nights when I wanted to shout at webpack, at Angular, at Angular-CLI. To blame someone for this but I couldn't. Code just...works this way sometimes. And sometimes, issues aren't reproducible and so they can't be addressed by the open source maintainers. And yet, it worked out. It worked.

Check out the screenshot at the top of the article. Look at how many changes I've had to make! It was...a lot of work.

Despite all of that...I'm glad that ngtools/webpack exists and that angular2-template-loader existed when I first started transitioning to Angular. Without the ngtools, I wouldn't have been able to cut down on our load time enough for us to move onto the next stage of our application development: rewriting everything in React.