DEV Community: Michael D. Callaghan

Upgrading Angular from 8 to 10 (Updated to 11)

Michael D. Callaghan — Thu, 21 Jan 2021 00:00:00 +0000

Angular is on a six-month release cadence, which means you need to stay on top of them in your own projects. The last thing you want to do is wake up one day and find that the latest version was just released and you are stuck on a version from two and a half years ago. Fortunately, the Angular team has made it very easy to upgrade.

Official Upgrade Guidance

Did you know that Angular offers official upgrade guidance? If you have not seen the website, take a look here at the Angular Upgrade Guide.

My Project

The project I am upgrading is the demo application presented in my book and course on Ionic and Angular development. You can find both at Gumroad.

The application itself is an Ionic app, written to run equally well on both desktop and mobile devices.

At the time I began, it was an Angular 8 app that had been automatically generated by the Ionic CLI.

You can see the app as it was just before I started the upgrade at its public GitHub repo.

I do not want it to fall too far behind, so I decided I should upgrade it to Angular 10 and make sure it works.

Angular 8.x to 8.y

The first thing I did was make sure the project was on the latest version of Angular 8. I did this by first making sure I had a clean repo (I did) and then entering the following command (and its output):

npx ng update @angular/cli@8 @angular/core@8

The installed Angular CLI version is older than the latest stable version.
Installing a temporary version to perform the update.
Installing packages for tooling via npm.
Installed packages for tooling via npm.
Using package manager: 'npm'
Collecting installed dependencies...
Found 35 dependencies.
Fetching dependency metadata from registry...
Package '@angular/core' is already up to date.
    Updating package.json with dependency @angular-devkit/build-angular @ "0.803.29" (was "0.803.25")...
    Updating package.json with dependency @angular/cli @ "8.3.29" (was "8.3.25")...
UPDATE package.json (1619 bytes)
✔ Packages installed successfully.

I specifically asked the CLI to update to @angular/cli and @angular/core version 8, which will update to the most recent minor/revision available to that major version.

I used npx to use the local Angular CLI version in the project. As you can see from the warning that it had its own idea and installed a temporary version to perform the update.

Apparently I was pretty close to current on Angular 8. It did not do much.

Next, I committed the code. Angular will not update on an unclean working repo.

git commit -am "Updated to latest Angular 8"
[master e9e9c71] Updated to latest Angular 8
 2 files changed, 3429 insertions(+), 2214 deletions(-)

You can compare what changed by reviewing the diff on GitHub. Again, not much happened.

Angular 8 to Angular 9

Next, I upgraded the project to Angular 9, with this command (it should look familiar).

npx ng update @angular/cli@9 @angular/core@9

This command did a lot more than the last one because now we are looking at an update of a major version.

Here is the output:

The installed Angular CLI version is older than the latest stable version.
Installing a temporary version to perform the update.
Installing packages for tooling via npm.
Installed packages for tooling via npm.
Using package manager: 'npm'
Collecting installed dependencies...
Found 35 dependencies.
Fetching dependency metadata from registry...
    Updating package.json with dependency @angular-devkit/build-angular @ "0.901.12" (was "0.803.29")...
    Updating package.json with dependency @angular/cli @ "9.1.12" (was "8.3.29")...
    Updating package.json with dependency @angular/compiler @ "9.1.12" (was "8.2.14")...
    Updating package.json with dependency @angular/compiler-cli @ "9.1.12" (was "8.2.14")...
    Updating package.json with dependency @angular/language-service @ "9.1.12" (was "8.2.14")...
    Updating package.json with dependency typescript @ "3.8.3" (was "3.4.5")...
    Updating package.json with dependency @angular/common @ "9.1.12" (was "8.2.14")...
    Updating package.json with dependency @angular/core @ "9.1.12" (was "8.2.14")...
    Updating package.json with dependency @angular/forms @ "9.1.12" (was "8.2.14")...
    Updating package.json with dependency @angular/platform-browser @ "9.1.12" (was "8.2.14")...
    Updating package.json with dependency @angular/platform-browser-dynamic @ "9.1.12" (was "8.2.14")...
    Updating package.json with dependency @angular/router @ "9.1.12" (was "8.2.14")...
    Updating package.json with dependency zone.js @ "0.10.3" (was "0.9.1")...
UPDATE package.json (1620 bytes)
✔ Packages installed successfully.
**Executing migrations of package '@angular/cli'**

❯ Update an Angular CLI project to version 9.
UPDATE angular.json (5386 bytes)
  Migration completed.

**Executing migrations of package '@angular/core'**

❯ Static flag migration.
  Removes the `static` flag from dynamic queries.
  As of Angular 9, the "static" flag defaults to false and is no longer required for your view and content queries.
  Read more about this here: https://v9.angular.io/guide/migration-dynamic-flag
  Migration completed.

❯ Missing @Injectable and incomplete provider definition migration.
  In Angular 9, enforcement of @Injectable decorators for DI is a bit stricter and incomplete provider definitions behave differently.
  Read more about this here: https://v9.angular.io/guide/migration-injectable
  Migration completed.

❯ ModuleWithProviders migration.
  In Angular 9, the ModuleWithProviders type without a generic has been deprecated.
  This migration adds the generic where it is missing.
  Read more about this here: https://v9.angular.io/guide/migration-module-with-providers
  Migration completed.

❯ Renderer to Renderer2 migration.
  As of Angular 9, the Renderer class is no longer available.
  Renderer2 should be used instead.
  Read more about this here: https://v9.angular.io/guide/migration-renderer
  Migration completed.

❯ Undecorated classes with decorated fields migration.
  As of Angular 9, it is no longer supported to have Angular field decorators on a class that does not have an Angular decorator.
  Read more about this here: https://v9.angular.io/guide/migration-undecorated-classes
  Migration completed.

❯ Undecorated classes with DI migration.
  As of Angular 9, it is no longer supported to use Angular DI on a class that does not have an Angular decorator.
  Read more about this here: https://v9.angular.io/guide/migration-undecorated-classes
  Migration completed.

Your project has been updated to Angular version 9!
For more info, please see: https://v9.angular.io/guide/updating-to-version-9

There were some warnings, which I will take care of eventually. For now, they did not affect the application. I then committed the changes.

git commit -am"Upgraded to Angular 9"
[master ba20fd7] Upgraded to Angular 9
 3 files changed, 3695 insertions(+), 2830 deletions(-)

The GitHub diff is here.

The only thing of note that it changed was that it added this bit of JSON to the angular.json file:

{
  "type": "anyComponentStyle",
  "maximumWarning": "6kb"
}

Angular 9 to Angular 10

Finally, I let the CLI upgrade from Angular 9 to Angular 10.

npx ng update @angular/cli @angular/core              
The installed local Angular CLI version is older than the latest stable version.
Installing a temporary version to perform the update.
Installing packages for tooling via npm.
Installed packages for tooling via npm.
Using package manager: 'npm'
Collecting installed dependencies...
Found 35 dependencies.
Fetching dependency metadata from registry...
    Updating package.json with dependency @angular-devkit/build-angular @ "0.1001.7" (was "0.901.12")...
    Updating package.json with dependency @angular/cli @ "10.1.7" (was "9.1.12")...
    Updating package.json with dependency @angular/compiler @ "10.1.6" (was "9.1.12")...
    Updating package.json with dependency @angular/compiler-cli @ "10.1.6" (was "9.1.12")...
    Updating package.json with dependency @angular/language-service @ "10.1.6" (was "9.1.12")...
    Updating package.json with dependency typescript @ "4.0.3" (was "3.8.3")...
    Updating package.json with dependency @angular/common @ "10.1.6" (was "9.1.12")...
    Updating package.json with dependency @angular/core @ "10.1.6" (was "9.1.12")...
    Updating package.json with dependency @angular/forms @ "10.1.6" (was "9.1.12")...
    Updating package.json with dependency @angular/platform-browser @ "10.1.6" (was "9.1.12")...
    Updating package.json with dependency @angular/platform-browser-dynamic @ "10.1.6" (was "9.1.12")...
    Updating package.json with dependency @angular/router @ "10.1.6" (was "9.1.12")...
UPDATE package.json (1620 bytes)
✔ Packages installed successfully.
**Executing migrations of package '@angular/core'**

❯ Missing @Injectable and incomplete provider definition migration.
  As of Angular 9, enforcement of @Injectable decorators for DI is a bit stricter and incomplete provider definitions behave differently.
  Read more about this here: https://v9.angular.io/guide/migration-injectable
  Migration completed.

❯ ModuleWithProviders migration.
  As of Angular 10, the ModuleWithProviders type requires a generic.
  This migration adds the generic where it is missing.
  Read more about this here: https://v10.angular.io/guide/migration-module-with-providers
  Migration completed.

❯ Undecorated classes with Angular features migration.
  In version 10, classes that use Angular features and do not have an Angular decorator are no longer supported.
  Read more about this here: https://v10.angular.io/guide/migration-undecorated-classes
  Migration completed.

As before, I committed the code and checked the behavior of the application.

git commit -am"Upgraded to Angular 10"
[master a0114c1] Upgraded to Angular 10
 2 files changed, 1945 insertions(+), 897 deletions(-)

The GitHub diff is here. As you can see, it only updated packages. Though it showed many of the same warnings, it did not find anything in the code it needed to migrate.

As I reviewed those migration guides, I also did not find any issues to update. I cannot confirm this, but I believe that the Ionic team keeps on top of these things and uses the latest Angular guidance in their application generators.

Upgrading to Angular 11

A few months have gone by since I originally wrote this article. Angular 11.1 was just released, so I thought it would be a good idea to update both the article and the application.

Angular 10.2

First, following the same pattern as before, I wanted to ensure I was on the latest version of Angular 10. I did that quickly and without error.

npx ng update @angular/cli@10 @angular/core@10
The installed local Angular CLI version is older than the latest stable version.
Installing a temporary version to perform the update.
Installing packages for tooling via npm.
Installed packages for tooling via npm.
Using package manager: 'npm'
Collecting installed dependencies...
Found 35 dependencies.
Fetching dependency metadata from registry...
    Updating package.json with dependency @angular-devkit/build-angular @ "0.1002.1" (was "0.1002.0")...
    Updating package.json with dependency @angular/cli @ "10.2.1" (was "10.2.0")...
    Updating package.json with dependency @angular/compiler @ "10.2.4" (was "10.2.3")...
    Updating package.json with dependency @angular/compiler-cli @ "10.2.4" (was "10.2.3")...
    Updating package.json with dependency @angular/language-service @ "10.2.4" (was "10.2.3")...
    Updating package.json with dependency @angular/common @ "10.2.4" (was "10.2.3")...
    Updating package.json with dependency @angular/core @ "10.2.4" (was "10.2.3")...
    Updating package.json with dependency @angular/forms @ "10.2.4" (was "10.2.3")...
    Updating package.json with dependency @angular/platform-browser @ "10.2.4" (was "10.2.3")...
    Updating package.json with dependency @angular/platform-browser-dynamic @ "10.2.4" (was "10.2.3")...
    Updating package.json with dependency @angular/router @ "10.2.4" (was "10.2.3")...
  UPDATE package.json (1620 bytes)
✔ Packages installed successfully.

This made no code changes, which I did not really expect.

TSLint and Codelyzer

This is where I finally ran into some minor trouble. My initial migration to Angular 11 failed due to some incompatible peer dependencies with Codelyzer.

Angular 11 finally does away with TSLint and Codelyzer for its linting tools, so I had to remove them before proceeding.

npm uninstall -D codelyzer tslint
git commit -am"Removed tslint and codelyzer"

This went smoothly, and simply removed those two entries from my package.json file.

At that point, the ng lint command not longer worked. Executing it resulted in this helpful error message:

TSLint's support is discontinued and we're deprecating its support in Angular CLI.
To opt-in using the community driven ESLint builder, see: https://github.com/angular-eslint/angular-eslint#migrating-from-codelyzer-and-tslint.

I will leave that for another time, but it seems as though it will be reasonably straightforward.

Angular 11

Now it was time to upgrade from Angular 10.2 to Angular 11.1. The command and its output are shown below:

npx ng update @angular/cli@11 @angular/core@11
The installed local Angular CLI version is older than the latest stable version.
Installing a temporary version to perform the update.
Installing packages for tooling via npm.
Installed packages for tooling via npm.
Using package manager: 'npm'
Collecting installed dependencies...
Found 33 dependencies.
Fetching dependency metadata from registry...
    Updating package.json with dependency @angular-devkit/build-angular @ "0.1101.0" (was "0.1002.1")...
    Updating package.json with dependency @angular/cli @ "11.1.0" (was "10.2.1")...
    Updating package.json with dependency @angular/compiler @ "11.1.0" (was "10.2.4")...
    Updating package.json with dependency @angular/compiler-cli @ "11.1.0" (was "10.2.4")...
    Updating package.json with dependency @angular/language-service @ "11.1.0" (was "10.2.4")...
    Updating package.json with dependency karma @ "5.2.3" (was "4.1.0")...
    Updating package.json with dependency protractor @ "7.0.0" (was "5.4.3")...
    Updating package.json with dependency @angular/common @ "11.1.0" (was "10.2.4")...
    Updating package.json with dependency @angular/core @ "11.1.0" (was "10.2.4")...
    Updating package.json with dependency @angular/forms @ "11.1.0" (was "10.2.4")...
    Updating package.json with dependency @angular/platform-browser @ "11.1.0" (was "10.2.4")...
    Updating package.json with dependency @angular/platform-browser-dynamic @ "11.1.0" (was "10.2.4")...
    Updating package.json with dependency @angular/router @ "11.1.0" (was "10.2.4")...
  UPDATE package.json (1568 bytes)
✔ Packages installed successfully.
** Executing migrations of package '@angular/core' **

❯ In Angular version 11, the type of `AbstractControl.parent` can be `null` to reflect the runtime value more accurately.
  This migration automatically adds non-null assertions to existing accesses of the `parent` property on types like `FormControl`, `FormArray` and `FormGroup`.
  Migration completed.

❯ ViewEncapsulation.Native has been removed as of Angular version 11.
  This migration replaces any usages with ViewEncapsulation.ShadowDom.
  Migration completed.

❯ NavigationExtras omissions migration.
  In version 11, some unsupported properties were omitted from the `extras` parameter of the `Router.navigateByUrl` and `Router.createUrlTree` methods.
  Migration completed.

❯ Updates the `initialNavigation` property for `RouterModule.forRoot`.
  Migration completed.

❯ NavigationExtras.preserveQueryParams has been removed as of Angular version 11.
   This migration replaces any usages with the appropriate assignment of the queryParamsHandling key.
  Migration completed.

❯ The default value for `relativeLinkResolution` is changing from 'legacy' to 'corrected'.
This migration updates `RouterModule` configurations that use the default value to 
now specifically use 'legacy' to prevent breakages when updating.
  UPDATE src/app/app-routing.module.ts (782 bytes)
  Migration completed.

❯ `async` to `waitForAsync` migration.
  The `async` testing function has been renamed to `waitForAsync` to avoid confusion with the native `async` keyword.
  UPDATE src/app/app.component.spec.ts (2562 bytes)
  UPDATE src/app/home/home.page.spec.ts (647 bytes)
  UPDATE src/app/roster/roster.page.spec.ts (661 bytes)
  UPDATE src/app/student-info/student-info.page.spec.ts (697 bytes)
  Migration completed.

❯ Removes `canActivate` from a `Route` config when `redirectTo` is also present.
  Migration completed.

As you can see, there are some things going on here other than simply updating some libraries.

The async testing function has been renamed to waitForAsync to avoid confusion with the native async keyword. The migration tool converted them automatically.
In my app's routing module (app-routing.module.ts), the migration added relativeLinkResolution: 'legacy' to the config:ExtraOptions parameter in the RouterModule.forRoot call.

As with the rest of the migrations, this one made no changes to the app's functionality, so I went ahead and committed the code.

Summary

Will your upgrade go as smoothly? I cannot guarantee it, of course. Honestly, the application is modest enough that I did not expect any problems.

As a convenient summary, here the commands I used (without the command output being shown).

npx ng update @angular/cli@8 @angular/core@8
git commit -am "Updated to latest Angular 8"
npx ng update @angular/cli@9 @angular/core@9
npx ng generate component Loading --spec=false --dry-run\n
git commit -am"Upgraded to Angular 9"
npx ng update @angular/cli @angular/core
git commit -am"Upgraded to Angular 10"
npx ng update @angular/cli@10 @angular/core@10
git commit -am"Upgraded to Angular 10.2.4"
npm uninstall -D codelyzer tslint
npx ng update @angular/cli@11 @angular/core@11
git commit -am"Upgraded to Angular 11.1"

As I said, your experience may not be the same as mine. You may not even need npx. I hope this article helped you, even a little, and I wish you every success in your own endeavors.

Discount Code

If you are interested in either the book or the course on building web and mobile Apps with Angular and Ionic, here is a code good for 25% off the regular price.

Book: https://gum.co/NlVUr/blog25
Course: https://gum.co/FyZHi/blog25

Diagnosing Random Angular Test Failures

Michael D. Callaghan — Thu, 14 Jan 2021 00:00:00 +0000

Have you ever had an intermittent or random failure in your unit tests? I did, and I was pulling my hair out trying to figure out the problem. Below I will describe how I finally managed to find the offending tests and solve the problem.

Background

I am using Angular 10 in this specific project, writing unit tests with Jasmine, and running them with Karma.

My karma-jasmine configuration when I began was completely empty, meaning I was using the defaults. Not understanding those defaults caused much of my problem. Out of the box, your unit tests are run in a random order. I was vaguely aware of this, but it had completely slipped my mind at the time. If you take nothing else away from my story, remember this: Whenever you experience intermittent or random test failures, you can almost be sure that one test is causing another to break. The trouble is figuring out which combination.

After the First Failure

As soon as I saw my first failure, I naively assumed it was the most recent test I added. Naturally, I removed it. At that point, the failure went away. I spent the next half hour trying to rewrite the "offending" test to figure out how it caused a failure in an unrelated test. The fact that the new test and the failing test were unrelated should have been my first clue that something else was at work here.

Start Disabling Tests

As I write this, my project has 153 unit tests. The failure was occurring in an afterAll function, which I did not even have. The error referred to a specific component, but not a specific test. I could not even determine which test to skip. Instead, I decided to start running a subset of tests by selectively disabling some of the other tests and test suites.

Disable Random Test Ordering

By this point, I was reasonably sure the problem was one test causing a failure in another one. The trick now was to figure out which one. On a whim, I decided to turn off random ordering in the Jasmine configuration. As I said, I am running my tests with the Karma test runner. So, inside the karma.conf.js file, I simply needed to add some Jasmine-specific configuration to the client section.

Before, that section looked like this:

client: {
  clearContext: false // leave Jasmine Spec Runner output visible in browser
},

To disable random test order, I added this:

client: {
  clearContext: false, // leave Jasmine Spec Runner output visible in browser
  jasmine: {
    random: false
  }
},

This would guarantee that my tests would run "in order," whatever that means. It occurred to me that this probably would not reveal any new insights. I was right. All tests passed. The good news is that the test run was now repeatable. Every test run passed now.

Did it solve my problem? Nope, but at least I felt I was making progress.

You Can Specify the Random Seed!

The next thing I did was to turn random ordering back on, but this time provide my own random seed. If you are not familiar with a seed, it is a number used to initialize (or "seed") the random number generator. The benefit of this approach is that using the same value to seed the random number generator will provide the identical sequence of random values on subsequent runs. Thus, once I found a seed that caused my test failure to show itself, I could continue using that seed during my investigation.

I started with the seed 1234. Because I had no idea what value might cause the problem, it really did not matter. On the first run with that value, all the tests passed, so that was no help. I continued changing the seed value until my test failed. Fortunately, it only took me a couple of attempts. I ended up with a configuration like this:

client: {
  clearContext: false, // leave Jasmine Spec Runner output visible in browser
  jasmine: {
    random: true,
    seed: '12345'
  }
},

Start Isolating Tests

Now that I could reliably reproduce the problem, it was time to start isolating the failed test from the working ones.

On my first attempt, I simply ran only the failing test in that test suite, by changing describe to fdescribe. This block had only one it block, so I was only running a single test from this suite. The rest of the test suites continued to run as normal. I was not yet ready to turn them off.

As I expected would happen, the test passed. When run in isolation from the other tests in the test suite, it worked just fine.

As a sanity check, I disabled every other test in the project, and as expected, the remaining test passed.

Next, I began turning on one test at a time. I considered turning on half, and then the other half. However, this particular suite only had ten tests in it, so I simply started at the top and turned them on one by one. As luck would have it, the very first test I enabled caused the problematic test to fail.

Now the real investigation began. What code did the first test run that caused the second test to fail?

Give Asynchronous Code a Closer Look

As I looked closer at the failing test, I noticed that it was testing a failure scenario. A passing test actually meant that an error occurred inside my component. This test used the async/await pattern to wait on the asynchronous service call it had to make.

I had seen this problem before, and almost certainly knew what had caused it. The function in many of my components that made asynchronous service calls were not returning the promise to the caller. Consider this block of code (not my actual code):

// Component function
getSomeData() {
  // Service function
  return this.service.getData()
    .then( data => { ... }
    .catch( error => { ... })
}

In this code, the promise returned by service.getData() is being returned by the component's getSomeData() function to its caller. If you forget to do that, the testing code will have nothing to await. My component had many such functions, and I had been meticulously going through them all to make sure they returned the service call's promise. I immediately checked the particular function being exercised by the failing test. To my surprise, it was fine.

That, however, led me to another realization. The test itself was bad. It looked like this:

it('should set an error if it throws', async() => {
  // service here is a test double, a mock of the real service
  service.getData.rejects({ message: 'test-error' });
  fixture.detectChanges();
  await component.getSomeData();
  expect(component.error).toBeTruthy();
}));

The testing code was using async/await, but had no try/catch! I decided that I would rewrite the test without async/await, instead using catch. The modified test looked something like this:

it('should set an error if it throws', ()=> {
  // service here is a test double, a mock of the real service
  service.getData.rejects({ message: 'test-error' });
  fixture.detectChanges();
  component.getSomeData()
    .catch(err => {
      expect(component.error).toBeTruthy();
    });
});

At last, I was pretty sure I had found the problem. I eagerly enabled all units tests and reran the tests with the same seed I had been using. The problematic test failed again, with the exact same error!

New Test Suite

At this point, I decided to start with a clean slate. I disabled all tests in that component's test suite. Then I used the Angular CLI to create a brand new component, complete with its unit test boilerplate.

Once that was done, I painstakingly began copying tests one at a time from the old test suite into the new one. I started simply, copying just enough code to initialize the component. That test failed due to missing service doubles. Next, I copied those test doubles (mostly service mocks) into the new suite.

With the test doubles in place, my component creation test passed. One down, nine to go.

The next test up was the failing one. It did something only other test in the suite does. It changed the behavior of one of my mock services. Could that be the piece I had been missing?

Suspect Your Test Doubles

I deleted my test doubles and reentered them individually. While doing this, I noticed that two of my mock services were created as object literals directly in the test suite. This was odd, as I also had a mock service defined as a class in another file. I decided I should try to use that instead.

The code used to look something like this:

const service = {
  getData: () => Promise.resolve({data});
}

// This was provided to the component's testing module like this:
providers: [
  { provide: DataService, useValue: service }
]

I moved the getData call into my Mock Service and changed the provider to look like this:

let service: DataService;

// This was provided to the component's testing module like this:
providers: [
  { provide: DataService, useClass: DataService }
]

I had left in the variable declaration so that I could redefine its behavior in the error path. But now I had a compiler error. This line in my failing test was no longer valid.

  service.getData.rejects({ message: 'test-error' });

That is because service was now strongly-typed to be a DataService, and getData returned a promise. It had no function named rejects. The solution was to replace the getData function completely, but just for this test.

Recall that I said this test was inside its own describe block, so I added a beforeEach to it, where I could replace the behavior of the getData function to reject instead of resolve the promise. Now my test looked something like this:

describe('getSomeData (error path)', () => {
  // Redefine service.getData here, outside the test itself.
  beforeEach(() => {
    service.getData = sinon.stub().rejects({ message: 'test-error' });
  });

it('should set an error if it throws', ()=> {
  // service here is a test double, a mock of the real service
  fixture.detectChanges();
  component.getSomeData()
    .catch(err => {
      expect(component.error).toBeTruthy();
    });
  });
});

Here I am using sinon to create the stub, but you can use a similar strategy with Jasmine's own functions.

The beforeEach on the overall test suite itself creates a brand new instance of the mock service before each test. Then, in this test, its getData function is replaced with a new function that returns a rejected promise instead.

I was feeling pretty good about where this was heading. I enabled all the tests and reran them. All passed. Finally, I removed the seed value from the Jasmine configuration and let the tests run in a random order. The test has been passing ever since.

Summary

I have been careful not to show any of the actual test code that caused my problem, but instead just enough code to get my point across. My goal was to discuss how to isolate the problems in your tests, rather than discuss testing strategy. To that end, I hope I have succeeded.

In short:

Understand that tests run in no particular order, and that is ideally what you want.
Turn off random execution to help find the culprit.
If sequential execution does not help, try various random "seed" values until the tests are failing consistently.
Isolate the failing test or test suite by using fdescribe, fit, xdescribe, and xit to turn tests on or off selectively.
Pay close attention to asynchronous code and test doubles.
Ensure your test doubles are typed correctly, especially your mock services.

I hope you found this summary to be worthwhile and useful. If you have any tricks or tips of your own when it comes to diagnosing unit tests, please feel free to share them with me.

Angular Advocate

If you are interested in more content like this, please consider my recently-released book, Angular Advocate: How to Awaken the Champion Within and Become the Go-to Expert at Work, available in a Kindle Edition at Amazon or DRM-free on Gumroad.

What Did Prettier Do to My HTML?

Michael D. Callaghan — Wed, 06 Jan 2021 00:00:00 +0000

I have resisted installing and using Prettier for a long time now, mostly because I was happy enough with the job VS Code does formatting my code. Then over the Christmas break, my son convinced me to install it. After I did, he said, “Now open an HTML file and I’ll show you something you aren’t going to like.” Sure, now he tells me! That struck me as ominous, but I decided to go along for the ride.

We opened an HTML file and immediately reformatted the file using Prettier’s default settings. Near the top of the file was a block of HTML rendering an unordered list, representing a navigation menu.

<ul>
  <li>
    <a routerLink="/" (click)="closeSidebarPanel()">Home</a>
  </li>
  <li>
    <a routerLink="/reports/" (click)="closeSidebarPanel()">Reports</a>
  </li>
  <li>
    <a routerLink="/annual-sales/" (click)="closeSidebarPanel()">Annual Sales</a>
  </li>
  <li>
    <a routerLink="/product-maintenance/" (click)="closeSidebarPanel()">Product Maintenance</a>
  </li>
</ul>

The last two lines with <a> tags are longer than 80 characters, which is Prettier’s default maximum line length. During a reformat to wrap those lines, Prettier needs to make a decision on how to do so without semantically affecting the rendered output. The decision most formatters would make might end up with a block of code like this.

<ul>
  <li>
    <a routerLink="/" (click)="closeSidebarPanel()">Home</a>
  </li>
  <li>
    <a routerLink="/reports/" (click)="closeSidebarPanel()">Reports</a>
  </li>
  <li>
    <a routerLink="/annual-sales/" (click)="closeSidebarPanel()">
    Annual Sales</a>
  </li>
  <li>
    <a routerLink="/product-maintenance/" (click)="closeSidebarPanel()">
    Product Maintenance</a>
  </li>
</ul>

The problem with this is that the HTML <a> tag is an inline-display element, meaning it flows along with the text that’s around it. A code formatter has no way to know whether the whitespace it just added to the beginning of the line is significant. It is entirely possible, even likely, that it just inadvertently added an extra space to the menu label.

What does Prettier do instead? It tries to make an intelligent decision on how to wrap and reformat those lines based on an item’s CSS display rules. If the element is a block-level element, then adding space around it will not affect its layout at all. On the other hand, if the element is an inline-display element, it will take care not to add any additional whitespace. What it does instead is rather clever. This is the final output.

<ul>
  <li>
    <a routerLink="/" (click)="closeSidebarPanel()">Home</a>
  </li>
  <li>
    <a routerLink="/reports/" (click)="closeSidebarPanel()">Reports</a>
  </li>
  <li>
    <a routerLink="/annual-sales/" (click)="closeSidebarPanel()"
      >Annual Sales</a
    >
  </li>
  <li>
    <a routerLink="/product-maintenance/" (click)="closeSidebarPanel()"
      >Product Maintenance</a
    >
  </li>
</ul>

The whitespace inside the tag itself, around its attributes, is never significant. Thus, it wraps the line between the final attribute and the angle bracket that closes the tag. That angle bracket is placed on the next line (remember, that whitespace is irrelevant), followed immediately by the text of the link itself. The link is then closed, again without any whitespace added to the text.

What about that closing angle bracket?

Why does it do this, though?

    <a routerLink="/product-maintenance/" (click)="closeSidebarPanel()"
      >Product Maintenance</a
    >

And not this, which might be more intuitive?

    <a routerLink="/product-maintenance/" (click)="closeSidebarPanel()"
      >Product Maintenance</a>

The reason, as I understand it, is that Prettier is intentionally placing the closing angle bracket on its own line, vertically lining it up with the tag’s opening angle bracket. In this way, it visually appears as a semantic block, exactly the way curly-brace languages line up their braces. Viewed from that perspective, I find it to be both elegant and clever.

My son was right. I did not like it when I first saw it, but with understanding comes appreciation and acceptance.

HTML Attribute Organization

Another change I did not expect is the way Prettier organizes HTML attributes when the tag has a lot of them. It formats them with one attribute per line, and closes the tag with the /> on the final line, as it did with the <a> tag above. I found a couple things to like about this.

It is really easy to find, edit, add, and remove attributes. This is particularly noticeable if you use a source control system such as Git (you do use source control, right???).

Consider the following HTML element definition:

<input #emailAddress type="email" class="email-input" placeholder="Email Address" required id="email-address" title="Enter your email address" name="email-Address" [(ngModel)]="emailAddress">

That is quite the list of attributes, but it is by no means unusual. At first glance, is this element required? Does it have an id attribute? Finding the answers to those question is not difficult, but it does take a little effort to scan the attributes when they are organized that way.

This is how Prettier reformats that element.

<input
  #emailAddress
  type="email"
  class="email-input"
  placeholder="Email Address"
  required
  id="email-address"
  title="Enter your email address"
  name="email-Address"
  [(ngModel)]="emailAddress"
/>

Notice that it did not change the order of the attributes. It honored my decision of where each belongs. Some formatters insist on alphabetizing them. That may even be a Prettier option, but as I said above, I am only working with its defaults for now. Maybe I will investigate and tweak it later, but one of its advantages is that you can use it “right out of the box,” as it were.

The final piece of brilliance with this formatting decision concerns source control, which I mentioned earlier. In the first element definition above, if I were to add a new attribute (say, disabled), the entire line would show as “changed” in the source control’s “diff” screen. Granted, finding such a small change would be reasonably simple and require very little mental effort.

Now imagine many such changes. Each one by itself is simple, but adding dozens or hundreds of these changes can start to feel like a burden.

Prettier’s decision of displaying one-attribute-per line means that each change affects exactly one line. Changes are easy to spot; they clearly stand out from the surrounding code, allowing you to see and understand what changed quickly and easily.

This advantage carries through to adding attributes right before the closing /> characters. Because a new attribute is on its own line, it will display as a single change in source control.

To me, this is a huge win.

Bottom Line

I will keep using Prettier, at least for my own projects. Next is getting my team at work to share in my appreciation.

The longer explanation of the whitespace decision can be found on the Prettier blog.

Upgrading an AngularJS Project to Angular

Michael D. Callaghan — Wed, 30 Dec 2020 00:00:00 +0000

For the past few months I have been involved with migrating an AngularJS 1.4 app to a more modern version of Angular. Below I will describe some of the processes, techniques, and issues I have encountered to make the migration successful.

Preparation

Before starting the migration, there are a few things that will make it easier, or harder, depending on your project’s configuration.

TypeScript

I was fortunate in that the entire AngularJS project I was migrating was already written in TypeScript. Each AngularJS controller was already a single class. If that had not been the case, I would still consider the controller itself to be a component in the new project.

Routing

My project used the Angular UI router. In your new project, I recommend using the default app routing module. Make a note of each state or route in your app. Copy them down with their relative URLs so that you don’t forget any.

ControllerAs or $scope pattern

Your AngularJS project is probably built with one of two patterns: You either use $scope to reference variables on your controller, or you created a “view-model” object and referred to it by name in your template. With an Angular component, you will use neither. When you migrate your HTML templates, you will remove all instances of the view-model object. If you used $scope, you probably won’t have to do anything to bind your variables.

UI Directives

In one of my projects, all our UI directives were already written as components. In another, the one I am currently migrating, they are not. The good news is that UI components and directives migrate just as easily as pages. To Angular, they are all simply components.

Bower Packages

Bower has all but been abandoned. Check your bower.json file and bower_components folder for any libraries you think you may need to keep. By now, most every bower package your old AngularJS project uses can be found in npm. Don’t bother trying to find new versions until you know you need them. What I found is that I could ignore them mostly, finding them in npm as necessary.

Start a new project

Starting with the latest Angular CLI, the first thing I did was create a brand new project with ng new. That provides a simple skeleton app, scaffolded with a single page and pre-configured routing.

Migrate one page at a time

To migrate a page, I used the Angular CLI to create a page in the new project, using the same name as the old page. For example,

ng generate component ProductDetail

Remember: All pages in Angular are components.

By default, this command creates four new files in a folder called product-detail:

product-detail.component.html
product-detail.component.ts
product-detail.component.scss
product-detail.component.spec.ts

It will also modify app.module.ts to reference your newly-created component.

What about lazy-loaded page modules?

The project I migrated does not use lazy-loaded pages, but I recommend that you do so if you can. If I have time, I may convert this project to do that, but it’s definitely out of scope for now.

Copy the controller code into the component

The first thing I do with any page is copy the controller code from the old page into the new page’s component class. An empty component class looks something like this:

import { Component, OnInit } from "@angular/core";

@Component({
  selector: "app-product-detail",
  templateUrl: "./product-detail.component.html",
  styleUrls: ["./product-detail.component.scss"],
})
export class ProductDetailComponent implements OnInit {
  constructor() {}

  ngOnInit(): void {}
}

Variables

First, find all all your variables in the old controller and copy them to top of the class, just above the constructor.

Constructor and ngInject

My controllers were already written in TypeScript, so I started with a class that had a constructor. Everything being “injected” into my controller are listed as parameters to the class constructor. Those got copied and pasted into the TypeScript component class without modification.

If you are not using TypeScript in your AngularJS controller, you will still be able to find the list of items being injected into your controller by the array of dependencies being passed to your controller’s function parameters. Simply copy those to your new component’s constructor.

The trick at this point is to provide the appropriate import statement and types for each one. You may be able to put that off for a while, but eventually your component will need to know what those dependencies are. Each one will need to be matched to a type and an import at the top of the class file.

In one case, my existing page listed eight such dependencies. Each one was either a bower package, an Angular service, or an internal service. Fortunately, I was able to find an npm equivalent for each bower package. The Angular services were not quite as simple, and I will touch on them later.

The others were services internal to the project. For those, I needed to migrate or replace them. Because those are project-specific, I can only offer very generic advice.

Try to use your internal services as-is, if you can. If you cannot, you may need to mock them temporarily to get your page to render.

You will probably have more than one call to AngularJS’s $http service, which you will need to upgrade to Angular’s HttpClient. That is not terribly difficult, but you may want to mock those services so that you can focus on one task at a time.

Another quirk is that all my older project’s constructor parameters were tagged as public, which is not desirable in my opinion. When I copied them to the new component, I changed them to private.

Functions

Next, copy every function from your controller into the blank space between ngOnInit() and the final closing brace.

If your controller contains any initialization logic, you may want to start with putting that into ngOnInit(). Try not to put too much logic into your constructor.

If you are fortunate enough to be copying TypeScript code, you won’t need to do much more. But if you are copying JavaScript, you will need to remove the word function from each one of them. If your project has anonymous functions tied to controller variables, you may have a little extra work to do.

For example, my controller functions all looked like this:

function doSomething() {
  ...
}

In this case, all I had to do was remove the word function and the rest of it could stay the same (for now).

However, your controller functions might look like this:

var doSomething = function() {
  ...
}

In either case, my recommendation is that make all your functions look like this:

doSomething() {
  ...
}

Dealing with this

Once my functions were fixed, the next thing I discovered was that many of them had the following first line:

var self = this;

The rest of the function referred to the variable self to read or write to variables on controller object itself. I won’t go into the joys of this in JavaScript. You can find plenty of angry rants elsewhere for that. The good news is that this sort of thing simply isn’t necessary with TypeScript classes, which is what your new component is.

So, to deal with that, the first thing I did was remove every line like the above. Then I converted all instances of self. to this. throughout the entire component. Yes, I used a blind find/replace, but so far it has never been an issue.

I also found some places in my code that calls into other functions as a callback to another function, like this:

this.doCoolThingWithCallback(input, this.thingCallback.bind(this));

The reason for the bind is so that this is a reference to the caller inside thingCallback. With TypeScript, simply replace that with an anonymous arrow function, which solves the “this” problem.

this.doCoolThingWithCallback(input, () => this.thingCallback());

Calling controller functions

If you are following along, you may have some function calls flagged as errors because the functions are defined in your component. Simply prepend them with this. and you should be fine.

Anonymous functions

The next thing I do is replace every anonymous function with an arrow function.

service.getData()
  .then(function(data) {
    ...
  });

In the TypeScript component, that simply becomes this:

service.getData()
  .then((data) => {
    ...
  });

Promises

Many of my controllers use promises, and do so through the Bluebird library. TypeScript has built-in support for native Promises, so I have been able to remove Bluebird entirely.

What about async and await?

If you are migrating older AngularJS to Angular with lots of promises, you might be tempted to convert them all to async and await. I did that at first. I strongly recommend you resist that temptation. At this point, your goal is not to refactor, but to migrate. You want to change as little code as possible. If you have promises that are working with .then(), keep them.

Navigation parameters

My controller used $stateParams, which map nicely to Angular’s ActivatedRoute. During the component’s ngOnInit() function, I grab the snapshot from the ActivatedRoute and set the state parameters my component is already expecting.

For example, one component was looking for the following, injected into its original controller:

  $stateParams: { storeId: string, subsetId: string };

I moved that definition out of the constructor and into the component itself as a variable. Then, I modified ngOnInit to look like this:

  ngOnInit() {
    const snapshot = this.activatedRoute.snapshot;
    this.stateParams.storeId = snapshot.params.storeId;
    this.stateParams.subsetId = snapshot.params.subsetId;
  }

As you can also see, I also removed the $ from the variable name, which I did safely using the variable refactor tooling in my code editor.

Service refactorings

As I mentioned above, my page had some external dependencies injected into it. Those needed to be addressed. I still had some older AngularJS constructs being injected that I needed to fix.

$q

The first was that $q: ng.IQService was referenced. For that, I can simply remove it entirely and change anywhere it’s being used into a native TypeScript promise. For example, I had this use of $q:

this.$q.all(promises).then(
  (data) => {
    this.getAllProductsSuccess(data);
  },
  (data) => {
    this.getAllProductsFailure(data);
  }
);

I replaced it with this instead:

Promise.all(promises).then(
  (data) => {
    this.getAllProductsSuccess(data);
  },
  (data) => {
    this.getAllProductsFailure(data);
  }
);

In case it is not clear, the variable promises is defined as Promise<any>[]. I will eventually do something about the <any>, but for now it should be fine.

$location

The old AngularJS LocationService is used in my controller, but I’m not entirely sure why. In the case of the page I’m currently migrating, it was better to use the router, which is what I did instead. I found this function in the old code:

  navigateToListing()
    this.$location.path('/listing');
  }

That makes no sense to me, as I would prefer to use the router. So, I changed the constructor to get a reference to the Angular Router object with private router: Router, and changed the function to look like this instead.

  navigateToListing()
    this.router.navigateByUrl('/listing');
  }

If the only call to this function was from a button in the HTML, I could also use a [routerLink] attribute instead and remove the function call entirely, like this:

<button [routerLink]="/listing">Return to Listings</button>

FormController

If the page you are migrating has a FormController, as did mine, you may have a little more work to do. I had never used this before, but this page has a pretty complex form, so I think I understand why they used it initially.

The definition at the official AngularJS docs says:

FormController keeps track of all its controls and nested forms as well as the state of them, such as being valid/invalid or dirty/pristine.

That sounds like Angular’s Reactive Forms, so I immediately wondered whether I could replace the form with that. Peeking inside my template, I found many uses of ng-model, which performs two-way data binding from the form to the controller. That patterns sounds like Angular’s Template-driven forms, so it required additional investigation.

The service was injected into my controller as form: IFormController. So, the first thing I wanted to do is find out how much it is used, which I did at the command line with grep.

 grep 'this.form' src/app/features/product-subset-detail/product-subset-detail.component.ts 
    this.form.$setPristine();
    this.form.$setPristine();

Well, that doesn’t seem too bad. For now, I decided simply to delete the reference and comment those two lines. Had it been more involved, I would have looked into refactoring the form into a ReactiveForm. But, as I said earlier, you want to avoid heavy refactoring until after you get the page migrating and functioning at least at a basic level.

Interfaces

If your AngularJS project is written with JavaScript, you won’t have any interfaces. My project was in TypeScript, and had interfaces defined all over the place. During the migration process, I created a new folder just for them and copied each interface into its own file. This was absolutely unnecessary, but it cleaned up the code just a bit and made me happy.

The template

With the controller migrated to a new component, it was time to turn my attention to the HTML template. My component code was free of compiler errors. Whether or not it works will still depend on whether or not I missed anything.

ViewModel or $scope

If your controller uses $scope, your data bindings are probably already correctly mapped to your view. If your controller uses a viewmodel pattern, as mine all do, you need to get rid of that reference everywhere it exists in your template.

For example, my controllers all used a viewmodel object named for the page (rather than simply vm as I’ve seen many developers use). My data bindings all look like this:

<span>Product Name: </span>`

The quick solution was to use Find/Replace to remove all occurrences of ProductDetailVm. (don’t forget the dot). After that, the above data binding looks like this.

<span>Product Name: </span>`

Assuming I didn’t miss anything, the component should already have a property named productName.

Custom directives

At the very top of my first HTML template I found two separate custom directives. Those will obviously need to be dealt with at some point, but for now I chose to skip them.

Angular directives

Angular directives are much simpler to convert, so I decided to start there. Most of these can be handled with a simple find/replace operation:

AngularJS	Angular	Notes
ui-sref	[routerLink]
ng-if	*ngIf
ng-show	*ngIf	It might make more sense to use `*ngClass{hidden: condition}`)
ng-hide	*ngIf	It might make more sense to use `*ngClass{hidden: condition}`)
ng-repeat	*ngFor	Requires additional syntax changes, see below.
ng-model	[(ngModel)]
ng-class	ngClass
ng-click	(click)
ng-change	(change)
ng-disabled	[disabled]
ng-pattern	pattern
ng-maxlength	maxlength

Granted, all of these will need to be revisited at some point to ensure that they do the right thing. There are a few extra steps to be taken once the attributes themselves were changed.

ng-repeat and *ngFor

I had a data table, where each table row <tr> is repeated using ng-repeat. This construct needed to be migrated to use *ngFor with its modern syntax. It isn’t hard, but it’s also not a simple Find/Replace as many of these have been.

Before:

  <tr ng-repeat="item in displayedCollection">

After:

  <tr *ngFor="let item of displayedCollection">

As you can see, I replaced ng-repeat with *ngFor and fixed the looping expression.

ngIf “gotchas”

Remember that *ngIf literally adds or removes elements from your page’s DOM. This is important if you ever try to get a reference to an element from your component.

For example, I found code in my old controller that manipulated an HTML element directly. It called document.getElementById to retrieve a reference to that element. I prefer to use Angular’s @ViewChild decorator, as I find it to be a little cleaner. The “gotcha” is that if the element being referenced by @ViewChild happens to be hidden inside another element that has an *ngIf, it may not exist when you want to use it.

For this reason, I prefer to keep my use of *ngIf limited to very small elements, or not use it at all. Instead, I prefer to show/hide elements with a CSS hidden class, which is simply defined as .hidden { display:none; } in my app’s global style sheet. I find for most use cases, this works as well, if not better, than *ngIf.

Custom components

My form contained a custom DatePicker component that does not work with Angular. Fortunately I was able to find a replacement that did, which did not require too much additional customization.

I recommend that as you convert your pages, try to determine ahead of time whether or not you will be using a third-party component library (such as Ionic, Angular Material, Bootstrap, etc.). It might be easier if you take inventory of all the custom components in your existing application, and then decide how to replace them in the migrated app.

Bootstrap Grid???

Speaking of Bootstrap, my AngularJS app makes heavy use of Bootstrap’s grid system. At first I thought I would simply remove and it replace it with something else. The trouble was I did not quite know what that “something else” would be. I briefly considered using Ionic’s <ion-grid>, as it is quite straightforward to use Ionic components selectively. The unused portions get “tree-shaken” out at build time.

I also considered downloading a customized version of Bootstrap 3.4, including only the grid system.

Then I stumbled on a blog post by Chris Wachtman on replicating the Bootstrap grid system with CSS Grid. The code looks pretty clean, so I’m going to give it a try.

Unit Testing

I still need to migrate all my unit tests. You do have unit tests in your app, right? I certainly do, and many of them will need some love. One anti-pattern I uncovered during this migration is that many of the functions that make http calls don’t return the promise returned from the service.

For example, consider this function:

  getProduct(): void {
    this.loading = true;
    this.myService.getProduct(
      this.productId
    ).toPromise()
      .then(
        (data) => {
          this.getProductSuccess(data);
        },
        (data) => {
          this.getProductFailure(data);
        }
      );
  }

The first thing you may notice is the call to .toPromise(). The call to myService.getProduct() returns an observable. When I migrated all my services, I decided to embrace RxJS as much as possible. However, for the purposes of migrating the individual pages, it was simpler to leave the promise handlers in place, at least for now. This function is essentially identical to the original from the AngularJS project, with the simple addition of the call to .toPromise().

Every one of my service calls follows this same pattern.

Call the service
Handle the success
Or handle the failure

Have you spotted the problem yet? There are two remaining.

The first is that there is no finally() call to reset this.loading, which controls a visible loading indicator. That is handled in the both the success and failure functions. That’s minor, however, to the glaring problem preventing me from testing these functions property.

The promise returned from getProduct(...).toPromise() is never returned! This makes testing the function extremely difficult. Fortunately, simply adding a return statement to the front of it fixes it, and has no negative side-effects.

This is the current implementation:

  getProduct(): void {
    this.loading = true;

    return this.myService.getProduct(
      this.productId
    ).toPromise()
      .then(
        (data) => {
          this.getProductSuccess(data);
        },
        (data) => {
          this.getProductFailure(data);
        }
      ).finally(() => {
        this.loading = false;
      });
  }

I still need to revisit and possibly rewrite every existing unit test, but that will be a topic for another time.

What Next?

At this point, everything was migrated and it was time to fire it up to see how it looks. As you might guess, it still isn’t perfect. However, it seems to be mostly functional.

If you find any hints or tricks that work for you, which I did not address here, please let me know.

I hope that your migration goes well.

Angular Advocate

Did you like this article? If so, please consider reading Angular Advocate, my book on how you can become the go-to Angular expert at your company.

New Book Announcement - Angular Advocate

Michael D. Callaghan — Mon, 30 Nov 2020 15:30:58 +0000

How I became the go-to Angular Advocate at work – and how you can, too!

In late spring 2018, my manager approached me with some interesting news. He explained that our executive director had complained that although Angular was prevalent throughout his organization, his own developers were not very experienced with it.

What could be done about that? They discussed various options, from video course subscriptions, to books, to hiring an outside training firm to provide the necessary training.

At that point, my manager spoke up. “What if we ask Mike Callaghan to do it?” He then proceeded to recommend me in an executive meeting to train a group of our software developers how to use Angular.

Before I knew it, I was the Angular Advocate.

My Angular Journey

I’ve been using Angular since AngularJS 1.2 or so. My first ever Angular app was a mobile swimming coach, implemented with the Ionic Framework and AngularJS.

Today I build web applications with Angular almost exclusively, though I admit to having dabbled briefly with React, even so far as to writing a book on using React with Ionic.

When I start a new application, either personally or professionally, I tend to start with Angular.

What’s in this book?

This book consists of a series of essays I have written for various technical audiences over the past year or so. They are either directly or indirectly related to Angular development.

However, this is not a book for web development beginners. There are plenty of those. Instead, this book is designed to appeal to experienced web developers with some exposure or experience with Angular.

The content you will find is, admittedly, opinionated, as is the entire Angular ecosystem.

I describe solutions that worked for me and my team. There are probably other approaches and solutions that would also work, but I will not dwell on them.

The content of this book represents my experience, and I hope you will be able to learn something from it.

Topics Covered

Here are some of the topics that are covered in the book. Notice it is not all simply Angular. There are other useful subjects that the Angular Advocate needs to understand.

How to introduce coworkers to TypeScript.
How to provide coworkers a Gentle introduction to Angular.
How to create a simple “no-code” Angular component.
How to display “offline” status in an Angular app.
How to support [(ngModel)] in a custom Angular component.
When to use route resolvers, and when not to use them.
How to keep up with Angular releases. Fortunately, @Angular makes it easy to do.
Understanding service APIs from both the producer and consumer point of view.
Keeping your deployment options open.
How to spin up a test API quickly and easily with Firebase.
The benefits of Progressive Web Apps (PWAs) and when to use them.
How to embrace RxJS and become truly effective.
How to use Test-driven-development (TDD) effectively with Angular.
Knowing what code to test (and what code not to test).
Knowing what all those weird symbols on the command line are all about.
How to revert your (or your team’s) mistaken git commits.
How to provide source code without the overhead of the git repository.
Understanding how people learn, before you need to teach.
Knowing the basics of how to teach is critical on the path.
How to get people to pay more attention when you speak.
Why you should pay attention to spelling and grammar.
Understanding the basics of agile planning.
Where to get Creative-Commons licensed images for almost any purpose.

How to Reserve Your Copy

If these things sound interesting, and I hope they do, please consider placing an order today at AngularAdvocate.com.

I am also releasing the first three chapters, free of charge. You can download the free PDF today.

A Bizzare eBook Giveaway

Michael D. Callaghan — Thu, 05 Nov 2020 00:00:00 +0000

Anyone who buys any one of my eBooks on Gumroad during November will automatically be entered into a drawing to win a Nook Color from Barnes & Noble preloaded with a copy of every title I have released in 2020.

This is the most bizarre giveaway I could think of

Here is how it will work. During the month of November, all you need to do is buy any of my eBooks at Gumroad. At the end of the month, I will select one person at random and ship them a Nook Color tablet with a copy of each all three books I have released in 2020.

There will also be some “runner up” prizes (see below).

The Books

Here are the books in question, along with a link to purchase each of them.

Don’t Say That at Work: Lessons Learned from a Lifetime of Mistakes

Have you ever been told that your communication style is “too confrontational?” Do you have problems persuading your coworkers and managers without sounding arrogant or condescending? Have you said something and immediately regretted it? Have you ever said something you can’t take back? Have you ever wondered later, ‘what was I thinking???’

We have all been there. We all make mistakes. We all suffer from a lapse in good judgment from time to time. Sometimes these mistakes are a “one off” but when made repeatedly can lead to a domino effect of problems. When that happens, it can be hard to recover.

Order Here: Don’t Say That at Work: Lessons Learned from a Lifetime of Mistakes

Developing Mobile Apps with Ionic and Angular

Learn to Build Mobile Apps! The Ionic Framework makes it easy

The Ionic Framework supports a variety of mobile platforms. Throughout this book, I will cover the important aspects of development with Ionic and Angular, going from the initial idea all the way to the Apple App and Google Play Stores.

Order Here: Developing Mobile Apps with Ionic and Angular

Developing Mobile Apps with Ionic and React

Learn to Build Mobile Apps! The Ionic Framework makes it easy

The Ionic Framework supports a variety of mobile platforms. Throughout this book, I will cover the important aspects of development with Ionic and React, going from the initial idea all the way to the Apple App and Google Play Stores.

Order Here: Developing Mobile Apps with Ionic and React

Bonus - Angular Advocate (Unreleased)

As an added bonus, the winner will also receive an advanced preview copy of my upcoming book, Angular Advocate: How to Awaken the Champion Within and Become the Go-to Expert at Work (Working Draft Title).

A Nook Color?

Why am I giving away the books on a Nook Color? Admittedly it’s a novelty. I was recently going through my old electronics and found my Nook Color from about 10 years ago. I charged it and turned it on. It still works! So, rather than throw it away or recycle it, I thought it would be fun to load my eBook titles on it and give it away.

No seriously? A Nook Color?

Sure, why not? Let’s face it, you aren’t really winning the Nook, but the titles loaded on it. I just thought it would be amusing to put the books on the device and award them that way. If you win and really don’t want the Nook, just let me know. I’ll understand, honest.

Prizes

1 Grand Prize: The Nook Color with all four of my books (including the unreleased one).
5 1st Prizes: All three of my current books.
10 2nd Prizes: Any one of my current books of your choice.

What’s the fine print?

I will only ship the Nook to an address in the United States. If the Grand Prize winner is from outside the US, the winner will automatically receive one of my 1st prizes, and then I will select an additional winner.

If you want to enter the giveaway without purchasing anything at all, that’s fine, too. Simply follow me on twitter (@WalkingRiver) and tweet a link to this post, including the hashtag #BizarreGiveaway. I’ll make sure you’re entered.

A Simple Angular Component

Michael D. Callaghan — Tue, 03 Nov 2020 00:00:00 +0000

Note: this is an excerpt from my upcoming book, Angular Advocate.

Angular components do not need to be complicated. In fact, with a little HTML and CSS, it is reasonably straightforward to build a component you can reuse in all your projects. I will detail the creation of a “Loading” indicator.

On one of my projects I needed to display a small screen that simply tells the user that data is being loaded from a remote service. Angular makes this almost too easy.

Create the Component

To create the component, I used the Angular CLI.

npx ng generate component Loading --spec=false --dry-run

The output is essentially the same with either.

CREATE src/app/loading/loading.component.scss (0 bytes)
CREATE src/app/loading/loading.component.html (26 bytes)
CREATE src/app/loading/loading.component.ts (272 bytes)

This command asks the Angular CLI to generate anew component named “Loading”, not to bother generating a test file (I will explain why not shortly) and then simply show me what the command will do (–dry-run).

I almost always do a dry run before having the CLI generate anything for me. That way, I can what files it will create and modify and where it will put them. On some projects I like to organize components differently than the default. Seeing the file paths before creation gives me a chance to correct them, simply by pre-pending the path to the name of the component.

In this case, I am comfortable with the component living in its own folder under app, so I can rerun the command without the --dry-run flag.

npx ng generate component Loading --spec=false          

CREATE src/app/loading/loading.component.scss (0 bytes)
CREATE src/app/loading/loading.component.html (26 bytes)
CREATE src/app/loading/loading.component.ts (272 bytes)

A note about the npx prefix: I need to add this to the command because my ng is not installed globally. Using npx causes the Angular CLI installed in my project’s node_modules folder to be used.

Component Code

This is the simplest part because there really is no logic to speak of. I am simply creating a visual component with no other behavior.

Inside the file loading.component.ts, the generated code looks like this:

import { Component, OnInit } from '@angular/core';

@Component({
  selector: 'app-loading',
  templateUrl: './loading.component.html',
  styleUrls: ['./loading.component.scss'],
})
export class LoadingComponent implements OnInit {

  constructor() { }

  ngOnInit() {}
}

As I said, there is not much here. After the two imports is the @Component decorator, which defines how the component will be implemented. selector defines the custom component’s HTML tag. This is how the component will be placed on a page.

<app-loading></app-loading>

The next two lines tell the Angular compiler (and us) where to find the markup and styles for the component, respectively.

Next is the class body itself, consisting of two empty functions. I need neither of those, so will delete them entirely, replacing them with two variables.

import { Component, Input } from '@angular/core';

@Component({
  selector: 'app-loading',
  templateUrl: './loading.component.html',
  styleUrls: ['./loading.component.css'],
})
export class LoadingComponent {
  @Input() label = '';
  @Input() shown = false;
}

The @Input decorators tell Angular to expose those two variables as attributes on the custom HTML tag.

label will be bound to some text in the HTML so that I can tell the user exactly what is loading. If you do not need that, you could eliminate it entirely.

shown allows the host to show or hide the component as necessary. Something like this:

<app-loading label="Loading data now..." [shown]="isLoading">

With this example markup, I have hard-coded the loading message, but have bound the shown attribute to a variable on the host component. Whenever isLoading is true, the loading component will be visible; otherwise it will be hidden. That is all the host needs to be concerned with. How the visibility is implemented inside the loading component is irrelevant to the host.

Markup

Now let us take a look at the markup. This, too, is pretty simple, almost trivial.

<div class="wrapper" [ngClass]="{hidden: !shown}">
  <img src="/assets/img/loading.gif">
  <h1>Please Wait While We Complete Your Request</h1>
  <p></p>
</div>

The component consists of a single <div> with a class called wrapper. We will see more of that in the next section on styling. Inside this <div> are three more elements:

An <img> tag pointing at an animated gif. This is one I am not really happy with. I would prefer to isolate the image so that using this component is a simple matter of dropping it into another project. However, this is not about component reuse. If it were, I would probably encode the image as a BASE-64 string and include it directly in the <img> tag itself.
A title represented by an <h1> tag containing a hard-coded message to the user.
The final piece of content is a <p> tag with its text bound to the @Input() label field on the component. Whatever the host component passes as the label attribute will be displayed here.

Styling

The real magic happens in the component’s stylesheet. I will show the entire thing, followed by an explanation of the relevant sections. The stylesheet is SCSS, but it really does not need to be. The code uses no specific SCSS features, and should probably be renamed with the .css extension. I will leave that as an exercise for the reader.

h1 {
  line-height: 30px;
  font-size: 24px;
}

img {
  width: 74px;
  height: 74px;
  display: inline-block;
}

.hidden {
  display: none;
}

.wrapper {
  text-align: center;
  position: absolute;
  z-index: 9000;
  width: 480px;
  height: 326px;
  top: 100px;
  left: 50%;
  margin-left: -215px;
  background-color: #ffffff;
  outline: 9999px solid rgba(217, 217, 217, 0.95); 
  font-weight: 400;
  line-height: 18px;
  padding: 60px 20px 20px 20px;
}

h1

The first rule is for the <h1> tag, and it is pretty straightforward. It simply sets the font-size to 30px, and the line-height to a slightly lower value. These values do not materially change the component. They are purely aesthetic and you could change them to reflect your own personal style. One thing of note is that the Loading Component will inherit its host’s font selection, whatever that may be.

img

The image tag, as I mentioned above, is hard-coded to a specific animated gif. The style sheet sets its size to be a 74px square, and sets it to display as an inline-block. Without that rule, CSS would not honor the width and height.

.hidden

The component’s visibility is driven by this class. The wrapping <div> either does or does not have this class set, based on the value of the shown attribute.

Why did I not put the hidden class on the host and let the host handle it directly? The reason I wanted to use shown is so that I could change the visibility implementation at will, without changing any of the host code.

For example, I could add some CSS animation or implement some other complex code, all without the host components even knowing about it. They would continue to set [shown] as they do now.

.wrapper

This is the big one, so I will show the code again for convenience, explaining it as I go.

.wrapper {
  text-align: center;
  position: absolute;
  z-index: 9000;

These first lines are just a bit of setup. They indicate that everything inside the wrapper will be centered, text and images both. The component will be positioned at an absolute position on the screen. The z-index of 9000 is a relative position of depth. Elements with larger numbers appear “on top of” or “in front of” elements with a z-index value that is smaller. Setting the loading component’s z-index to 9000 gives it a decent likelihood that no other elements will appear in front of it. Should you find that is not the case, set a higher value. Browsers do not seem to have a standard “maximum” value, but most modern browsers should allow values up to 2³¹ - 1.

  width: 480px;
  height: 326px;
  top: 100px;
  left: 50%;
  margin-left: -215px;

This next block helps to position the loading component. It sets a fixed width and height, and positions the top of the component at 100px from the top of the screen. Then it does something a bit clever. The component’s left side is set at 50% of the host’s width. Then it sets a negative margin of half the component’s width. This effectively causes the entire component to be perfectly centered horizontally inside the host.

  background-color: #ffffff;
  outline: 9999px solid rgba(217, 217, 217, 0.95); 
  font-weight: 400;
  line-height: 18px;
  padding: 60px 20px 20px 20px;
}

Finally, you can see some various rules that dictate how the component looks. It has a background color of white, indicated by the value #ffffff.

The clever bit I find is the next line: outline. The component’s outline is being defined as a 95% opaque (i.e., 5% transparent) solid gray line 9999px wide. This ends up covering the entire host component with the outline, preventing it from being selectable.

The last three lines set the text font-weight to 400 (normal), a default line-height of 18px, and some internal padding to provide whitespace.

And that is the entire component!

Use

I hinted at its use above, but there are three things you would need to use it in your own project.

Include the source files.
Ensure that the component is declared and exported in whatever Angular module you intend to use it.
Supply the HTML markup to call it, which looks like this.

<app-loading [label]="loadingText" [shown]="isLoading"></app-loading>

In this sample code, I am using Angular’s attribute binding syntax to bind the label and shown attributes to the host component’s loadingText and isLoading variables, respectfully. Changes to these variables on the host component will cause Angular to re-render the loading component as necessary.

The Result

When it is all assembled and working on an actual web application, this is what it might look like:

Summary

Angular components do not need to be complicated. In fact, sometimes they do not even need any imperative code. In this article I have created a simple Loading Component that can be easily reused anywhere in my application.

Further, with just a little more effort, I could build a completely standalone component that I could drop into any project I wish.

What do you think? How could this component be improved? Let me know your thoughts.

Test-Driven Development in Angular

Michael D. Callaghan — Mon, 26 Oct 2020 00:00:00 +0000

I tried something "new" this afternoon. I built an Angular service in a true TDD fashion. I wrote the tests first, discovering the service interface along the way. This is how it went. I invite you to follow along.

Background

I am not a fan of writing unit tests for Angular apps. The tooling I am using (Jasmine and Karma) feel like afterthoughts. They work and they have gotten much better over the past few years, but they still seem like they were written to bolt onto Angular, rather than being built as part of the ecosystem.

Then I started thinking that maybe the problem is with me. Maybe I despise writing tests because I have not truly adopted test-driven-development in my Angular apps. I used to use TDD all the time with .NET and C#.

So today I decided to go back to that philosophy and build a modest service using strict TDD principles. This is how it went.

The Service

The service itself is simple enough. I want to build a means of setting and retrieving two different unique IDs my app can use when making service calls. The first is a "conversation ID" that will be set as an HTTP header for all network calls for a specific user for a given session. It will not change until the application user manually refreshes the screen, closes the browser, or logs out and back in.

The second is the "correlation ID." This will also get sent with each HTTP call, but it changes with every request.

Not only will these IDs be set as custom HTTP headers on all web requests, they will be logged with all such requests and responses. They can then be used to correlate several layers of service requests and responses back to the user and high-level function that initiated them.

The name of my service is simply correlation. I created it with this Angular CLI command:

npx ng g service services/correlation/Correlation

CREATE src/app/services/correlation/correlation.service.spec.ts (382 bytes)
CREATE src/app/services/correlation/correlation.service.ts (140 bytes)

This creates two files in their own folder at ./src/app/services/correlation. I got a nearly-empty service file and a test (spec) file with one test.

As I usually do, pre-pending npx causes the system to use the locally-installed Angular CLI.

The Generated Test

I want to start by reviewing the test code that was generated by the Angular CLI. I do not mean for this to be a comprehensive introduction to testing, but I will explain the basics. It should be enough for you to follow along and also modify your own tests.

import { TestBed } from '@angular/core/testing';
import { CorrelationService } from './correlation.service';

describe('CorrelationService', () => {
  let service: CorrelationService;

  beforeEach(() => {
    TestBed.configureTestingModule({});
    service = TestBed.inject(CorrelationService);
  });

  it('should be created', () => {
    expect(service).toBeTruthy();
  });
});

The first import line brings in the Angular testing class called TestBed. This class contains most of the basic testing framework.

The second pulls in the service to be tested, also known as the "System Under Test," or SUT.

describe

With most JavaScript testing frameworks, tests are organized into one or more describe functions. These can be nested, as you will see shortly.

The describe function is called at least two parameters.

The test label. In this case, the name of the service to be tested.
The function that contains the tests themselves. Here it is an arrow function.

This function contains a single variable representing the service, but nothing is assigned to it yet.

beforeEach

Directly inside this function is another function call, beforeEach, which itself contains another arrow function. This function is called by the testing framework before every unit test.

This one calls the TestBed.configureTestingModule({}), and you can see that it is being passed an empty object as its only argument. This is the options, and can accept just about everything a normal Angular module can. Most tests use this to configure Angular's dependency injection system to inject test doubles required by the SUT. My service has no dependencies, so there is nothing to configure.

Other Functions

Not shown are some other functions that can contain setup/tear-down instructions:

beforeAll: called once before any tests are run.
afterAll: called once after all tests have been run.
afterEach: called after each unit test function.

it

This function defines a single unit test. You can create as many it functions as you want inside your describe. The generated test comes with a single it function. Its signature matches that of describe, in that it takes a label and a function defining the test.

When combined with its enclosing describe, the it functions should read like this:

[describe Label] [it Label]: Pass/Fail

Thus, when you read the one generated test, it should look like this:

CorrelationService should be created: Pass

Consider this phrasing when you create your own tests.

There is a lot more to Angular testing than this, but I wanted to make sure I explained what you would be seeing below before I begun.

The Tests and API

There are three primary things I need the service to do for me.

Give me the same conversation ID whenever I ask, unless one does not exist. In that case, it needs to give me a new one and return it.
Give me a fresh correlation ID every time I request one. I should never get the same ID twice.
Provide a way for me to force a fresh conversation ID.

These rules allowed me to come up with the following tests. Again, I am using Jasmine as my testing framework. I know a lot of people these days are using Jest, but the concepts should be the same regardless of what you use.

import { TestBed } from '@angular/core/testing';

import { CorrelationService } from './correlation.service';

describe('CorrelationService', () => {
  let service: CorrelationService;

  beforeEach(() => {
    TestBed.configureTestingModule({});
    service = TestBed.inject(CorrelationService);
  });

  it('should be created', () => {
    expect(service).toBeTruthy();
  });

  describe('resetConversationId', () => {
    it('should return different values on subsequent calls', () => {
      const firstId = service.resetConversationId();
      const secondId = service.resetConversationId();
      expect(firstId).not.toEqual(secondId);
    });

  });

  describe('getConversationId', () => {
    it('should return identical values on subsequent calls', () => {
      service.resetConversationId();
      const firstId = service.getConversationId();
      const secondId = service.getConversationId();
      expect(firstId).toEqual(secondId);
    });

  });

  describe('getCorrelationId', () => {
    it('should return different values on subsequent calls', () => {
      const firstId = service.getCorrelationId();
      const secondId = service.getCorrelationId();
      expect(firstId).not.toEqual(secondId);
    });
  });
});

Even if you are not intimately familiar with Angular testing in Jasmine, I think these tests are easily understood.

Naturally, though, none of these tests will run. In fact, they will not even compile. The functions on the service do not yet exist.

Auto-generated Service Code

Fortunately, VS Code will do the heavy lifting for me. All I have to do is put my edit cursor on one of the function names, click the yellow light-bulb (for Auto Fix), and choose Add all missing members.

The code it builds is not ideal and will still require some editing, but at this point the tests will compile.

import { Injectable } from '@angular/core';

@Injectable({
  providedIn: 'root'
})
export class CorrelationService {
  resetConversationId() {
    throw new Error('Method not implemented.');
  }
  getConversationId() {
    throw new Error('Method not implemented.');
  }
  getCorrelationId() {
    throw new Error('Method not implemented.');
  }

  constructor() { }
}

Make Them Run (and Fail)

Now I have code that compiles, implemented in such a way that all three tests will fail with an expected exception. The first thing I need to do is remove the exceptions. My class now looks like this.

export class CorrelationService {
  resetConversationId() {
  }
  getConversationId() {
  }
  getCorrelationId() {
  }

  constructor() { }
}

I am afraid one of those tests will now pass, but should not. Each function call in the test code evaluates to undefined. This causes the test should return identical values on subsequent calls to pass, because undefined equals undefined.

I will have to edit the tests. I have two choices. I can add three more tests to ensure that no function returns undefined or I can add a check for undefined in the test that is checking for equality.

Some purists believe that every test should have a single assertion/expectation. I tend to be more of a pragmatist. If you are testing one high level "thing," then it is fine to have multiple expectations in a single test.

The new test now looks like this, and fails as expected.

describe('getConversationId', () => {
  it('should return identical values on subsequent calls', () => {
    service.resetConversationId();
    const firstId = service.getConversationId();
    const secondId = service.getConversationId();
    expect(firstId).toBeDefined(); // New code
    expect(firstId).toEqual(secondId);
  });
});

Note I am only checking on the first result to be defined. If the first call is defined and the second is not, the second expectation will then fail. I will let you decide which approach makes sense for your project.

Make Them Pass

According to TDD principles, the next step is to write the least amount of code that will cause the tests to pass. In theory, I should not have to touch the tests again. In practice, I probably will. This is a path of discovery, which I am writing as I go. Thus, you are learning right along with me.

resetConversationId() {
  return 'mike';
}
getConversationId() {
  return 'mike';
}
getCorrelationId() {
  return 'mike';
}

Technically, this will make the middle test pass, but not the others. It is time to think about how the service is supposed to work.

UUID

The business rules call for some sort of semi-unique identifier string. I plan to use a GUID or some variant thereof.

After a few seconds (ok, a minute or so) of research, I found the UUID npm package{:target="_blank"}. I will it use to generate both my conversation and correlation IDs.

Once the package is installed in my project, the CorrelationService now looks like this.

import { Injectable } from '@angular/core';
import { v4 as uuidv4 } from 'uuid';

@Injectable({
  providedIn: 'root'
})
export class CorrelationService {
  resetConversationId() {
    return uuidv4();
  }
  getConversationId() {
    return uuidv4();
  }
  getCorrelationId() {
    return uuidv4();
  }

  constructor() { }
}

Now the tests pass or fail as expected.

Make It Right

This code looks pretty good, almost complete. There are two things I think are missing.

The first is obvious: Subsequent calls to getConversationId need to return the same value. This means I need a place to store the value. There is also the scenario of the ID's initial value. How do we handle that?

I will tackle the second scenario first by modifying getConversationId to return the stored value, and also by modifying resetConversationId to set the stored value. This will cause the tests to fail, but that is why we write them in the first place. Right?

My modified service looks like this:

export class CorrelationService {
  conversationId: string;

  resetConversationId() {
    this.conversationId = uuidv4();

    return this.conversationId;
  }

  getConversationId() {
    return this.conversationId;
  }

  getCorrelationId() {
    return uuidv4();
  }

  constructor() { }
}

All the tests pass, because I had the foresight to call resetConversationId in the test expecting equality. In reality, this was not a good idea. My motive was good, but I do not believe a user should be forced to call resetConversationId before calling getConversationId. That should be up to the code.

So, now I want to remove the call to resetConversationId from the test, which will cause that test to fail.

To enable that code to pass again, I need to modify the service to ensure there is a value before returning it.

getConversationId() {
  return this.conversationId || this.resetConversationId();
}

Now all my tests pass, the service does the modest job it is meant to do, and my test coverage looks good.

The Final Test

Here is the final set of tests.

import { TestBed } from '@angular/core/testing';

import { CorrelationService } from './correlation.service';

fdescribe('CorrelationService', () => {
  let service: CorrelationService;

  beforeEach(() => {
    TestBed.configureTestingModule({});
    service = TestBed.inject(CorrelationService);
  });

  it('should be created', () => {
    expect(service).toBeTruthy();
  });

  describe('resetConversationId', () => {
    it('should return different values on subsequent calls', () => {
      const firstId = service.resetConversationId();
      const secondId = service.resetConversationId();
      expect(firstId).not.toEqual(secondId);
    });
  });

  describe('getConversationId', () => {
    it('should return identical values on subsequent calls', () => {
      const firstId = service.getConversationId();
      const secondId = service.getConversationId();
      expect(firstId).toBeDefined();
      expect(firstId).toEqual(secondId);
    });
  });

  describe('getCorrelationId', () => {
    it('should return different values on subsequent calls', () => {
      const firstId = service.getCorrelationId();
      const secondId = service.getCorrelationId();
      expect(firstId).not.toEqual(secondId);
    });
  });
});

The Final Service

Here is the entire service.

import { Injectable } from '@angular/core';
import { v4 as uuidv4 } from 'uuid';

@Injectable({
  providedIn: 'root'
})
export class CorrelationService {
  conversationId: string;

  resetConversationId() {
    this.conversationId = uuidv4();

    return this.conversationId;
  }

  getConversationId() {
    return this.conversationId || this.resetConversationId();
  }

  getCorrelationId() {
    return uuidv4();
  }

  constructor() { }
}

I probably could also dispense with the empty constructor, but something in the back of my mind is preventing me from deleting it.

Refactoring the Service

After I finished writing this, it occurred to me that there is a better way to initialize the service than with the || in getConversationId. Why not use the constructor to do its job and construct the object and initialize its internal state?

Before

As you may recall (or just look up and see), the getConversationId function looks like this:

getConversationId() {
  return this.conversationId || this.resetConversationId();
}

If the value of this.conversationId is not defined, the conditional "or" will cause the function on the right side to be executed. That function's side-effect is to initialize the value. TypeScript conditional "short-circuiting" prevents it from being called if this.conversationId already contains a value.

In this case, it is simple enough to follow, but you may be able imagine that in more complex classes it may not be.

After

Instead, I will move the call to resetConversationId into the constructor, guaranteeing that this.conversationId will always have a value. Thus, I can delete the conditional check from the latter function.

constructor() {
  this.resetConversationId();
}

getConversationId() {
  return this.conversationId;
}

To me, this is much simpler code and captures the meaning more clearly than before. Anyone looking at this code will understand that the service pre-initializes its state immediately.

The tests still pass, as they should. This ostensibly is why we write unit tests in the first place, to ensure that changes to the implementation do not break functionality.

Conclusion

From start to finish, this experiment took me just over two hours to complete (2:30 - 4:45 PM). I spent another 15 minutes or so doing the above refactoring and writing about it.

The tests were easy to write because the service itself did not exist when I began. By describing the tests as I expected them to work, the service API practically wrote itself.

I am not convinced that a more complicated service or a UI component will be as easy to write in this manner, but over all I am pleased with the result.

I will probably continue to develop the project this way, and can honestly recommend that everyone should give it a try some time. You may end being pleasantly surprised.

To RxJS or Not to RxJS

Michael D. Callaghan — Mon, 12 Oct 2020 00:00:00 +0000

Note: this is an excerpt from my upcoming book, Angular Advocate.

I am currently working on a project to uplift some AngularJS code to Angular 10. On a recent code review there were some concerns that came up about the heavy use of RxJS. I will attempt to address those concerns in this post.

Of particular note were the following assertions:

RxJS is hard to learn/read.
RxJS is hard to test.

Hard to Learn

Admittedly RxJS has a steep learning curve. That said, it should only take an week or so to become accustomed to using it. Once you understand the basics, the more esoteric operators are mostly a matter of finding them. There is a great website that makes this easier, which is where I found the fromEvent and combineLatest operators I use below. https://rxjs.dev/operator-decision-tree

Hard to Test

Testing Angular is itself often an exercise in frustration. The key is knowing what to test and what not to test. I wrote a blog post on that very subject. Some people go overboard and try to achieve 100% code coverage. I have found that more than about 80-85% is usually counter-productive. Often, people will write brittle (or worse, useless) tests to try to hit some magical coverage number.

RxJS is no more difficult to test than Angular, assuming you write your code to be testable in the first place. This is not always the case, especially when you try to do too much with it at once. My strategy is to create observables that do one thing that I can explain in a comment. For example:

Convert input event into stream of search terms.
Make web service call on selection change.

These sorts of observables are reasonably straightforward to write and test. More importantly, they are also straightforward to read and to modify.

The Code

The code in question includes a custom dropdown with search capability. The idea is that you can bind a very long list of items and either select one or search for one by name. The component fires a custom DOM event, selected-changed, to indicate that the user has made a new selection.

The dropdown represents a list of “stores.” Whenever a store is selected, the app needs to make a web request to load products associated with that store.

Though it renders and behaves like an HTML <select> tag, it does not actually contain a <select> element. Fortunately, this turned out to be irrelevant.

The first challenge was to figure out what event was being thrown. The docs showed that I could use (selected-change)="storeChange($event), so I wired it up to see what the event looks like. It seemed to be exactly what I had hoped: a custom DOM event.

The next step was to create an Observable I could subscribe to. I created a component variable I could use to subscribe to the event using Angular’s @ViewChild decorator.

@ViewChild('storeList') storeListDropdown: ElementRef;

That led to this small bit of code:

this.storeChanged$ = fromEvent(this.storeListDropdown.nativeElement, 'selected-changed')
  .pipe(tap(x => {
    console.log(x);
  }));

Now I could verify whether or not I could truly subscribe to these custom events. A quick test showed that indeed I could. Hurdle one cleared.

Make the service call

Next, I needed to take the newly-selected value and use it to make a service call. For various reasons, I cannot show the actual call here, but I think I can show enough to get my point across.

I only care about selections

It is possible for the user to select nothing, at which point I do not want to make a service call. So I added a filter to the observable’s pipe to ensure I am only getting selections.

this.storeChanged$ = fromEvent<CustomEvent>(this.storeListDropdown.nativeElement, 'selected-changed')
  .pipe(
    filter(x => x.detail?.value?.length),
    tap(x => {
      console.log(x.detail.value[0].value);
    }));

With that I was able to confirm that it only fired when the user selects something. There is another use case to clear the results if the user selects nothing, but that is not currently important.

Make the web call

Next up was to make the web service call. I modified the tap function to set a component variable that holds the selection, then added a switchMap to call an Angular service that hides the actual HTTP call. Now it looks like this.

this.storeChanged$ = fromEvent<CustomEvent>(this.storeListDropdown.nativeElement, 'selected-changed')
  .pipe(
    filter(x => x.detail?.value?.length),
    tap(x => {
      this.selectedStore = x.detail.value[0].value;
    }),
    switchMap(x => this.getProducts(this.selectedStore))
  );

I could drop the tap and done both the variable assignment and the service call inside of the switchMap operator. I may eventually do that, but for now I like the way that each step is visually separated.

Search filter

A single “store” in this project could return anywhere from 0 to potentially hundreds or thousands of products. Each record is small, so for now, we are keeping the filtering of products entirely client-side. To support that, the UI has a search box. Entering anything in the search box should cause the records to be filtered to those records matching the value entered. This called for another observable, which I will show in its entirety.

this.searchFilterChanged$ = fromEvent<InputEvent>(this.searchBox.nativeElement, 'input')
  .pipe(
    // tslint:disable-next-line: no-magic-numbers
    debounceTime(300),
    map(_ => {
      this.searchFilter = this.searchBox.nativeElement.value;

      return this.searchFilter;
    }),
    distinctUntilChanged(),
    startWith('')
  );

As you can see, I start by creating an observable from the HTML input event. That fires an event for every change to the text box. This stream is then sent through another RxJS pipe to do the following:

The observable is “debounced” so that any value is delayed by 300ms. This prevents the events from coming in too fast as the user types.
The distinctUntilChanged operator ensures that only changes to the value are sent. In other words, if the user repeated presses the space bar and the backspace key in rapid succession, resulting in no change to the text box, my code will never see it. It is unlikely, but possible.
Inside of map, I assign the value of the <input> element to a component variable named searchFilter and return that value. This successfully converts the event into a string I can use as a search value later.
The final operator is startWith(''), which will be important later. This initializes the observable stream with an empty string value.

Pager component

My final component is another custom one that handles paging. As I said, the results of the web service can potentially contain thousands of records, so I want to provide a user a simple way of paging through those results. Like my other custom component, it also fires a CustomEvent called nav-selected. I again used fromEvent and set it up similarly to the first one.

this.pagerChanged$ = fromEvent<CustomEvent>(this.pager.nativeElement, 'nav-selected')
  .pipe(
    map(x => x.detail.currentPage),
    startWith(1)
  );

This is the simplest of the bunch. Inside its pipe, I map the details from the custom event into the page the user selected. Once again, I start the stream with a default value.

Combine them all

Why did I go through all of this instead of simply using angular event binding? This is why. Now that each event is its own individual stream of events, they can easily be combined using the RxJS operator combineLatest. This operator accepts an array of observables as its input and returns a single observable. As its name implies, this observable fires when any of its input observable values changes. Whenever that happens, the subscriber gets the latest value of each of its constituent observables.

The caveat for this operator is that it will not emit an event until each observable in its input array has fired at least once. This is why each of my observables above (except the first one) used startWith to set an initial value. The store dropdown did not need one because until the user selects a store, there is no reason to do anything.

How does this all work in practice? Every time the user selects a new store, changes the search filter, or selects a new page on the pager, my subscriber gets the most recent value of each of those three observables. Here is the code.

this.filteredProducts$ = combineLatest([
  this.storeChanged$,
  this.searchFilterChanged$,
  this.pagerChanged$
]).pipe(
    map(([products, search, page]) => this.filterProducts(products, search, page))
  );

Inside of the map operator, you can see that each value from the array of observables is passed into its arrow function. I pass those values into a pure function on my component called filterProducts. This function handles the filter and paging on the products array to return a new array of products.

Keep in mind that the return of combineLatest, which is assigned to this.filteredProducts$, is itself an observable. This is important because of what happens next.

Inside the template

Because of the composition of these three independent observables into a single observable, binding to its results is almost trivial. Inside my HTML template, I have the following markup inside an HTML <table> tag.

<tr *ngFor="let product of filteredProducts$ | async">
  ...
</tr>

By using Angular’s async pipe, I never need to subscribe or unsubscribe to the observable. Angular handles all that for me. Whenever any of the three source observables changes, the table will update to reflect those changes.

The benefit of this approach

I started down this path of using RxJS for user events because I wanted to isolate each event and hide the irrelevant details from its consumers. By using fromEvent I am able to tweak and modify each event to get exactly the behavior I am looking for, separate from the others.

With that, the final observable created with combineLatest requires no additional manipulation. I can pass the values to a pure function to get exactly the data to be displayed, and then bind that data with a simple *ngFor and async pipe.

Testing each observable stream independently from the others also becomes almost trivial.

What I really like most about this approach, though, is how extensible it is. When I first wrote this code, I only had the custom dropdown and the text input box for searching. It was all working properly when I decided to add the custom pager control. The result was that I was able to include the pager, set up and test its custom event, and then simply wire it up to the combineLatest operator by adding it to the input array. Then I added it as an input to my filterProducts function.

What Do Others Think?

I double-checked my philosophy with some people I consider experts, both inside and outside of team. Here are some of the responses I received (mostly paraphrased):

That code is awesome!
For me it’s a tradeoff between simplicity (Angular’s event bindings) and power (RxJS fromEvent). In the majority of cases I only need the simple event bindings. As soon as you try doing something special (debounce, filter, delay, etc.), use RxJS.
Observables are a heavy pill to swallow. If you get why you’d use it (sync/async feel the same, pure function pipelines of data, deterministic, easier to unit test), cool. Otherwise, hide in the Model layer so dev can play away from ‘em elsewhere.
I struggled for a long time to understand RxJs (still don’t get everything) but once I had a use case, it was a no-brainer to use the library.
Every time I have to touch rxjs code it’s a major PITA and source of bugs for me. I’m not intelligent enough to understand it, no matter how many rxjs tutorials I read.

Interestingly, some suggested that learning to use state management with NgRx might result in a lower learning curve than raw RxJS. My experience has been the opposite.

Ultimately, I recommend that you use the best tool for the job. If that means going a little outside of your comfort zone to learn RxJS, so be it.

Everything you see above took about three days start-to-finish to get right, including the unit tests I have not shown. When I began, I knew almost nothing about any of these RxJS operators I ended up using.

How I Got My First Programming Job

Michael D. Callaghan — Fri, 28 Aug 2020 00:00:00 +0000

I wanted to become a software developer from the time I wrote my first “Hello World” app in 9th grade. After graduating high school in 1985, I entered the University of Maryland’s Computer Science program. It did not go well. I completed only four semesters of college over the next five years. My dream of becoming a professional developer faded. In 1995, nearly ten years after high school, I finally landed my first paid programming job. Here is how I did it.

A lot happened to me in the decade between high school and my first programming job. The highlights, in order were:

Started and dropped out of college twice.
Helped manage two local pizza delivery chains, neither of which was a Dominos.
Sold high-end TVs and audio equipment at a local AV chain store.
Created and folded three companies: desktop publishing, wedding photography, and business equipment refurbishment.
Packed up and moved from Maryland to New Hampshire to “start over,” leaving behind my former life as a devout atheist.
Got a job at a New England-based Apple Macintosh dealer.
Met and married a wonderful woman and began a life together.

I Used Macs Before They Were Cool

Before I moved to New Hampshire, I had already arranged for a job. I started as a retail sales rep at Computer Town in Nashua on July 27, 1992. We sold the complete line of Apple Macintosh products. System 7 had been released just over a year prior. Though Macs were starting to become popular, Windows 3.1 still held the lion’s share of the PC market.

I quickly learned all I could about the computer business and our product lines. I leveraged my experience refurbishing business equipment into a thriving used Mac business in an unused room on the second floor of our store.

Due to being consistently one of the top sales reps in our location, I was selected as one of the few to represent the company at MacWorld Boston every year, and MacWorld San Francisco the one time we attended.

Once a Programmer…

I enjoyed my time selling computers, which helped me hone my skills working directly with people. In the back of my mind, however, there was still an emptiness. I never shed the dream of programming computers for a living. During my spare time, I learned all I could about how to program the Macintosh computers I sold.

A/UX

Did you know that Apple had a Unix-based operating system before OS X? It was called A/UX and was based on UNIX System V Release 2.2. It looked like System 6 but had a real Unix core under the covers. Naturally, the geek in me needed to run this OS. I bought a copy and installed it on the Mac Quadra on my desk at work.

Filemaker Pro

One of my more ambitious projects was when I created my own customer management application in Filemaker Pro, a powerful (at the time) database system. I used it to help me communicate with my repeat customers, track sales, and generally provide me with an excuse to do some level of programming.

Code Warrior

Who remembers Code Warrior? It was an integrated development environment (IDE) for building apps on the Macintosh. I bought a copy as soon as I could and started writing Pascal and C programs.

Usenet News (nh.jobs)

One day I was perusing job listings on nh.jobs on usenet using a text-based news-reader. Netscape Navigator was brand new and most internet tools were still text-only. I stumbled upon a job post for a Junior Programmer in Nashua. The body of the post mentioned C experience but indicated that most relevant technologies would be taught on the job. I was intrigued. Could this be my chance to become a paid software developer?

Junior Programmer?

Back in those days, most job posts of this nature were created by the company or even the person doing the hiring. Email spam was also not a huge issue, so most people simply included their email address right in the post. That was the case here. I quickly sent an email referencing the job post. I briefly described myself as a hobbyist programmer who had been self-taught since 1981. I ended with the question, “what do you consider a ‘junior programmer?’”

The hiring manager emailed me back almost immediately, describing what he meant by junior. As I recall, the company was looking for someone they could teach application-specific integrated circuit design using VHDL. The only qualification he really wanted was some programming ability. He asked me if I could implement a doubly-linked list with operations to add and remove elements. I accepted his challenge, wrote a short command line program in C, and sent the source code to him a few hours later.

Interview

He invited me to his office for an official job interview. At this point, I was still feeling a bit nervous. I was not sure I was ready to leave the known for the unknown. I discovered the biggest hurdle I would face during the interview.

As I mentioned, the company made integrated circuits using VHDL, a language I had never heard of. The manager gave me a brief overview and it looked pretty straightforward. Two things gave me pause:

The entire company was on Windows. I had never used Windows before in my life. I had been a die-hard “Mac guy” since college.
The salary was fully 20% lower than I had been making at Computer Town.

They offered me the job, but I told them I needed to discuss it with my wife. She and I decided that taking a pay cut at this point probably made sense, as it would lead to better opportunities in the future. We suspected there was not much room for growth in retail.

Our decision made, I emailed the manager that I would accept their offer. The next day I gave my two-weeks’ notice to my manager at Computer Town, and started the new job less than a month later.

It Did Not Last

As it turns out, the job only lasted a few months. I quickly discovered a blind spot for hardware programming. Although the syntax of VHDL was straightforward enough, programming ASICs was very confusing to me and unforgiving. Something about the process eluded me. Perhaps if I were to try it again, with 25+ years of experience, it might not seem so bad.

One day my manager and the owner invited me into a conference room for a brief meeting. The owner asked me how I thought things were going. He knew things were not going well, and I confirmed it. He made me an offer that seemed odd at the time. If I chose to resign immediately, the company would provide me one month’s pay, and cover the cost of my health insurance for the rest of the year. As it turns out, resigning prevented me from filing for unemployment benefits. It did not matter, as I found a new job just a few weeks later.

As we were wrapping up, the manager had a final piece of advice for me. He suggested that I might want to consider a different career and give up my dreams of being a professional software developer.

Lessons Learned

Though my first real programming job was not the tremendous success I had hoped, it nonetheless served a valuable purpose. It changed the course of my career and my life. And I learned some things that I have tried to keep in mind ever since.

Always keep learning. Had I not continued to learn and behave as a programmer, I would never have been able to land this job.
Sometimes you need to take a step backward. Taking the pay cut was briefly painful, but having a real programming job on my resume opened doors for future opportunities.
You may have to cut your losses. The job was not what I had hoped it would be, and I was not what they had hoped I would be. In the end, we were both better off cutting ties after only a few months.
Always be on the lookout for opportunities and be willing to jump when they appear. Shortly before I lost this job, I had run into one of my Mac customers at Computer Town. He told me to call him if I ever wanted to be a Mac developer. After I lost this job, I called him and let him know what had happened. He offered me a job on the spot, at a salary a little higher than I had been making selling Macs.
Take other’s advice with a “grain of salt.” Had I listened to my manager’s advice to consider a different career, I can only imagine where I would be today. I am sure that I would not be midway through my third decade as a software developer, and it can be assumed I would not have the Most Magical Job on Earth.

My Disastrous 3-Year Programming Hiatus

Michael D. Callaghan — Mon, 17 Aug 2020 00:00:00 +0000

Ten years into an otherwise successful software development career, I experienced significant burnout and walked away from a 6-figure salary. I quit my job, sold my house, moved to a remote part of New Hampshire, and did not work again in software development for almost three years.

In mid-2005 I threw a temper tantrum and quit my high paid software development job at HP. Sure, I blamed it on factors other than myself, but looking back it was all on me. Not only did I quit my job, I walked away from my entire career in software development.

Real Estate

I decided to switch gears entirely and chase real estate deals full-time. Why did I think I had a “snowball’s chance” in this endeavor. It was mostly due to some early successes some friends and I had over the prior year.

Two friends and I partnered on some real estate deals during the mini boom of 2004. We bought and resold some foreclosed properties in Southern New Hampshire. Later that year, I successfully managed two very lucrative “flips” on my own and saved up a decent little nest egg. In early 2005 I took some of my profits and bought myself a Mustang Convertible. I was convinced that this was a better way to make a living than working for a huge corporation like HP. So when the opportunity presented itself, I gave my notice and walked away from a promising career.

Time to Move

Once I left HP and decided I was leaving software development for good, I used some of my cash to buy a brand new home in Central New Hampshire, and then sold the home in Southern New Hampshire where I had been living.

Once the move was complete and the other house sold, it was time to rev up the real estate machine I had been slowly building. Over the next few months I found a couple of new people to help with the tasks. One was a local real estate appraiser who was good with tools. The other was an unemployed friend from church. Both men were willing to work at no charge until after the sale of any house we bought. That alleviated any immediate cashflow issues. I still had to pay for materials, but labor was free until closing.

Reduced Success

Unfortunately, unbeknownst to me at the time, the real estate market in New Hampshire was beginning to cool. Properties were just as easy to find and acquire but they were becoming harder to sell. Those that did sell quickly were at much lower profit margins. There were some I had to hold for months, often at burn rates of $1000 or more per month, not counting material costs. Deals profiting $20,000 or more were vanishing quickly.

My next two deals netted me less than $10,000 total. Given that real estate flips can take a month or more to complete, this was not enough money to pay my expenses. We started dipping into savings.

Rent for Cash Flow?

For my next purchase, the thought occurred to me that I could make more money keeping it for a few years and putting a tenant in it. Against the advice of everyone I knew, that is what I did. A few months went by just fine, and then the rent checks stopped. Attempts to reach the tenants were in vain.

After two months of being unable to reach them, they finally sent me a letter saying they had moved out and that I could keep their security deposit. I immediately got my Realtor friend over to the house for a damage assessment. She called with the good news that the house was still in good shape, but that the tenants had left the 2-car garage/barn full of their stuff.

We put the house on the market immediately, but I had to deal with the barn. New Hampshire law requires that landlords keep their tenants’ property safe and secure for a minimum of 30 days after they move out. You cannot simply throw it all away. It would take longer than a month to sell, so I was not too concerned.

After the month elapsed, we still did not have an offer on the house. Neither did the tenant come back to claim the contents of the barn. We decided to lower the asking price of the house and have a garage/barn sale. It was the first time we really took inventory of what had been left. We found a really nice snowblower and grill, along with lots of clothes and housewares. Most everything sold in one weekend. What did not we donated.

Eventually, the house did sell. After paying my crew, the real estate fees, etc., the profit was less than $4000 for about eight months of work. You would expect that I had just learned a valuable lesson, but the harder lessons were still to come.

Renovation Profits?

Perhaps I could renovate the properties for a slightly better profit than a quick flip?

A Realtor friend helped me find a house in a small town on the southwest corner of the state, bordering both Massachusetts and Vermont. The house was in serious need of work, but she was confident she could sell it once the current tenant moved out. I made the purchase sight-unseen. Over the next month, some friends and I made the 200-mile roundtrip drive to the house each weekday. We had to rebuild the kitchen and the downstairs bathroom; replace the roof over the dining room and repair water damage in the dining room ceiling; and repair fire damage in the garage. The entire house was in desperate need of new carpet, so it was all replaced.

Once the work was complete, it did sell quickly. By this time, however, the real estate market was in a noticeable downturn. After factoring in all of the time, effort, and cash needed to sell the house, I netted $1500 on the sale.

Disasters Begin

My track record since leaving HP was not turning into the success story I had envisioned. I had not yet lost money but was quickly burning through my savings. I was simply not earning enough money fast enough to pay my bills. And then I was hit with two complete disasters in rapid succession.

I found a tiny but clean manufactured home on a huge, gorgeous lot with mountain views. I agreed to purchase it from the landlord and arrange a sale to the tenants. Once I took possession of the property, it became quickly apparent that tenants had no ability to buy it. After a few months of promises, I politely told them they would need to leave, so I could get it sold. I was happy letting them stay while they were trying to buy it, but I was not looking to be a long-term landlord. They agreed to go without a legal battle.

A few weeks later I received a phone call from the Town. The house’s water use had doubled over the past month, which they thought was unusual. They sent an inspector to the house, who heard running water through the door. Not getting an answer when he knocked, he immediately shut off the water and called me.

When I got to the house, I discovered that the tenants had left without telling me. They had also shut off the power, and thus the heat. This being the dead of winter in New Hampshire, the temperature tends to drop well below freezing. I entered the house to a disaster. One of the water supply lines had frozen and burst, and began filling the house with water. By the time I arrived, the water was gone. The water stains, on the other hand, were approximately 18 inches above the floor. The floor itself was gone in many places, having simply collapsed under the weight of so much water. The entire house was rendered uninhabitable.

I ended up selling it at a major loss to a builder who wanted the lot.

They Stole My Kitchen

Around the same time, I bought a tiny fixer-upper on the outskirts of the state capital. It was easily the worst-looking house on its street. It was more than 100 years old and it showed. The yard was pleasant and would make a nice starter home for a young family. We got to work repairing one bathroom and replacing the other. We refinished the hardwood floors, added new kitchen appliances, and rebuilt the front porch.

Once the renovations were complete, I put it on the market at its estimated value of $225,000. By this time, the market had taken a definite turn for the worst. Nothing seemed to be moving. After a few false starts, I reluctantly agreed to rent the house for a year, hoping the market would turn and I could sell. Or perhaps hoping that the tenant would be able to buy it. I was fine either way.

You can probably guess what happened. The rent check bounced. This tenant and I were on pretty good terms, so I emailed her to ask her what was going on. I heard nothing for about a week. Finally, she replied with an apology. She could not afford to stay and had already moved out.

When I got to the house, I found that the locks had been changed. I summoned a locksmith and the local police department to help me enter my own property. What we discovered upon entering was nothing like I would have ever imagined. The kitchen was gone. Every appliance, cabinet, and counter top — gone, stolen. The rest of the house was destroyed. There were holes in the walls, obscene graffiti scrawled everywhere. The pristine hardwood floors were scuffed and scratched. The bathroom we had replaced was trashed. One of the bedroom walls had been used to hang a dartboard. You could see the outline where the board had been and the evidence of how badly they played.

I immediately asked the police officer whether I could file a criminal complaint, for the kitchen theft if not outright vandalism. He replied that because my tenant had done it, it was not a criminal matter and that I would have to file a civil complaint.

I was out of money and my insurance would not cover this type of damage. In desperation, I had my real estate agent put the house on the market as-is, as a “gut job” fixer. She managed to find a buyer willing to pay $80,000 for it. I accepted.

The only thing of value I got from that house was a tennis racket I bought at a yard sale before I bought the house. It is probably the most expensive tennis racket ever sold, and I still play with it to this day.

Recovery Begins

That house was my last real estate deal. Not only were my savings gone, but I was in the hole more than $200,000. I had no idea what I was going to do next. It was 2008 and I had been out of the software development world for three years.

Then I got a call from a friend from high school. We discussed everything that had occurred. It was at that point he asked me something that has stuck with me ever since. “Have you considered that this isn’t your core competency?” He went on to remind me that ever since we met I exhibited a love and talent for computer programming unmatched by anyone else he knew.

He offered me a part-time software development gig with his young firm, working remotely on a 9-1-1 telecom product. I gratefully accepted the contract, which turned into a full-time job a few months later. This led to future opportunities working on telephony projects for other companies, including one in which I was a minority shareholder. The experience I gained there enabled me to get a long-term contract with Dell in Round Rock, Texas, and ultimately my current job at Walt Disney World. These other roles and jobs may become future blog posts.

Lessons Learned

It took me three more years to dig out from under the disaster that was my brief real estate career. Those losses were losses of cash, home equity, and even some retirement savings. I have not speculated in real estate since.

Some of the takeaways from this experience:

Never risk more than you can stand to lose. I leveraged one property against another on more than one occasion. This led to a domino effect where a loss on one house led directly to a loss on the next, and so on.
You make your money when you buy. Never assume the real estate market is always going up. Though it is true that long time horizons can sometimes bail you out of a mistake, there are still some areas that have not yet recovered.
Rental property management is hard. Hire a professional property manager. Let them do their job.

Finally, the most valuable lesson I learned came from my friend: Stick to your core competencies. It is one thing to make the occasional “small bet” where your downside is limited. That is not what I did.

I let my pride and my emotions drive me away from what could have been a long and lucrative career with HP. I lost tens of thousands of dollars, destroyed my personal credit for years, and nearly threw away my career. Three years in the software industry is an eternity. I was fortunate to have been rescued by someone willing to take a chance on me after being away for so long. I will be ever grateful.

Burn out is real. Every software developer I have ever met has felt it at one time or another. Let my experience be a warning to you not to throw away a promising career on a long-shot.

Debugging AngularJS from VS Code with Docker Containers

Michael D. Callaghan — Mon, 10 Aug 2020 00:00:00 +0000

This post describes how I built a docker container that runs an express server, which serves an AngularJS app, written in TypeScript, with live reloading, remote Chrome debugging, with Source Maps served from a different directory, and VS Code breakpoints.

The updates to this application now enable any developer to spin up a development Docker container quickly. Once running, it is reasonably straightforward to edit and debug the application with Chrome and Visual Studio Code, with the code executing inside of the container.

Problem

Given an AngularJS web app served by a Node/Express app, provide a Docker solution to simplify both deployment and development.

Requirements

Ensure existing Dockerfile will work to deploy into corporate container host. Modify it if necessary.
Without affecting the production Dockerfile, provide a way for new developers to get their environment up and running quickly.
(Optional) Provide a way for live editing of the running web app inside of the container.
(Optional) Provide a way to connect Chrome Dev Tools to the built code.
(Optional) Provide a way for Dev Tools to access source maps which are located outside of the built assets folder.
(Optional) Connect VS Code Debugger from outside the container.

Constraints

Must start with the official Dockerfile provided, which contains the base OS requirements for the particular Node environment and Docker host.
The app is built with AngularJS 1.4 and may not be upgraded for this project.
The primary development language for the application is TypeScript.
The app is built with the Grunt CLI and a pre-tested grunt configuration, which may not be changed.
The built application artifacts are bundled, minified, and packaged into the ./dist folder.
Source maps are generated but stored alongside the original TS files under the ./src folder.
The application uses many deprecated npm and bower packages, and even node itself is outdated.

Base Dockerfile

The first thing I did was grab the official Dockerfile provided for hosting Node/Express applications for the appropriate version of Node. Unfortunately, this particular app requires an old, unsupported version of Node. That, however, is a problem for another day. The strategies I outline should work regardless of the Node version in use.

FROM node:0.10.38-slim

# Expose PORT
EXPOSE 8626

# Set Current Working Directory
WORKDIR /app

# Set production environment
ENV NODE_ENV=production

## Copy dist folder to application
COPY dist/package.json /app/

RUN npm install

COPY dist /app

The base image is prebuilt image internal to the company, but it is based on Debian “Buster.”

The hope was that at the very least, the application could be built and installed with the official Dockerfile with no, or only minor modifications.

Upon the initial docker build command, the process hung at the npm install phase. Something the application needs required a step that used python. This was something other teams here had encountered and had a known solution. I simply needed to add a line to the Dockerfile to install Python, which I did by adding this command immediately after setting the current working directory.

# Install Python and supporting build tools
RUN apt-get update && apt-get install -y \
        python \
    && rm -rf /var/lib/apt/lists/*
# rm -rf ^ removes cache, reducing image size

At that point, my npm install got farther, but still failed due to an npm package that was being referenced with git+ssh protocol. Our official Docker image intentionally omits git. Instead of installing git into the image, I spent a few hours investigating why that particular package was being included that way, and what it was ultimately doing.

As it turned out, the package in question was only used to help manage bower package versions, and was only executed prior to the bower install. As our official CICD system has its own way of managing that, it became clear that this particular npm package is only needed during local development. I moved the npm package to the devDependencies section in package.json, and provided some new scripts that would only be run outside the container:

"postinstall": "npm run bower",
"prebower": "node ./scripts/npm/preinstall.js",
"bower": "bower install",
"postbower": "node ./scripts/npm/postinstall.js",

To prevent the git+ssh package from being installed as part of the docker build, I modified the final npm install command in the Docker file to include the --production flag, thus bypassing the devDependencies entirely.

Now, as soon as the npm install command completes, npm will execute the three bower scripts in order. Our CICD system runs npm install outside of a container and should have no problem running these commands.

The final Dockerfile now looks like this.

FROM node:0.10.38-slim

# Expose PORT
EXPOSE 8626

# Set Current Working Directory
WORKDIR /app

# Install Python and supporting build tools
RUN apt-get update && apt-get install -y \
        python \
    && rm -rf /var/lib/apt/lists/*
# rm -rf ^ removes cache, reducing image size

# Set production environment
ENV NODE_ENV=production

## Copy dist folder to application
COPY dist/package.json /app/

RUN npm install --production

COPY dist /app

The --production flag to npm install skips installing anything listed under devDependencies, so my git+ssh issue never arises.

The only potential downside I see is with the three new bower scripts. The docker build calls npm install as one of its final steps, but the older version of npm it is running does not execute the postinstall command, so will not run the bower scripts. This works because of a required build step that has to occur outside of docker build, namely a grunt build that packages the entire application into the ./dist folder. This includes all of the bower dependencies, so a separate bower install is simply not needed. In future versions where Node is upgraded, it might be required to run npm install --production completely outside of the Dockerfile and simply copy the node_modules folder.

As it is, everything should work for now.

Local Development

The next thing I wanted to do was to have a local docker image that could be used by new developers. Ideally, I would use the Dockerfile I already had working. However, it was not that simple. The first thing I realized is that none of the grunt commands would work inside the container because only the built artifacts are copied into it. There is also no grunt-cli installed into the container.

As I mentioned above, we have a complete system of grunt commands. The commands are used to serve the app under development with live reloading, and also to build and package the final application for deployment.

After a few failed experiments with the existing Dockerfile, I decided the most prudent course of action would be create a parallel Dockerfile.dev that was mostly the same, but that I could modify to suit local development needs. This is the file I came up with, which I called Dockerfile.dev.

FROM node:0.10.38-slim

# Expose PORT
EXPOSE 8627

# Set Current Working Directory
WORKDIR /app

# Set production environment
ENV NODE_ENV=development

The two files are identical, except I completely leave out the python installation, the production version’s npm install and COPY commands at the end. Instead, I handle the source code, npm, bower, and grunt a little differently.

Specifically, I added a series of scripts to package.json to handle most of the complexity. This will enable a new developer to clone the repo, perform an npm install, and then run a few more npm scripts to configure the Docker environment.

Local Docker Build

First, I needed a simple way to build a docker image from the new Dockerfile.dev file.

"docker:build": "docker build -f Dockerfile.dev -t my-cool-app-ui:latest .",

Now, executing npm run docker:build will create a new image based on the development Dockerfile.

Local Docker Container Execution

Next, I wanted to provide a similar experience for managing the local docker container.

"serve": "grunt serve --no-browser-sync",
"docker:start": "docker run -d --rm --privileged --name my-cool-app-ui --env-file .env -p 8627:8627 -v $PWD:/app my-cool-app-ui npm run serve",
"docker:stop": "docker kill my-cool-app-ui",

Because I did not want to install any npm packages globally into the container, I added grunt-cli to the application’s devDependencies. That enabled me to provide a serve script in package.json that could launch grunt inside the container.

The next bit of magic is inside the docker:start script. It is a simple docker run command. The parameters chosen are important.

Detached/Daemon Mode

The -d parameter causes the container to be started in a detached state, meaning it is not interactive. It will start and run in the background.

Remove on Exit

The --rm parameter is for housekeeping purposes. When the container exits, it is removed immediately, freeing up any resources it was consuming.

Privileged Mode

The --privileged parameter provides elevated access to the container. This was recommended to me by a colleague, but I am not convinced it is necessary. I may remove it and see whether it still works.

Container Name

The --name parameter simply provides a friendly name for the container, rather than one randomly-generated by Docker.

Environment Variables

The --env-file .env parameter indicates that the container should be started with the environment variables specified in a local file .env. In our applications, the .env file contains environment-specific configuration and secrets that are needed at runtime, but are not stored with the application or in source control. These types of things could include database connection strings, authentication tokens, etc. Storing them in a local file that is not stored in source control keeps sensitive information out of publicly-visible files (like package.json or Dockerfile).

Port Mapping

When grunt serve starts the Node/Express app, it is configured to listen on TCP port 8627. For some reason, the production version of the app is configured to listen on port 8626. The parameter -p 8627:8627 maps port 8627 locally to the container’s port 8627. I will revisit this inconsistency at some point and clarify which port it should be.

Local Volume Mapping

The next parameter -v $PWD:/app maps the current directory to a directory inside the container called /app. Recall that the Dockerfile indicates that the working directory for the container is /app.

For me, this was the real breakthrough. Rather than fighting with grunt, npm, and bower inside the container, I simply map the entire project folder into the container’s /app folder.

Docker Image to Use

The parameter my-cool-app-ui is the name of the Docker image to execute. This is second time this identifier is used in the command, which is an unfortunate naming on my part. I inadvertently ended up naming the image and the container identically. I intend to fix that in the future.

Command to Launch

The final three parameters of the docker run command, npm run serve, represent the command that Docker should execute inside the container. In this case, I am executing the serve script inside of package.json.

Stopping the Container

I also provided another script to stop the container, which simply executes the command docker kill my-cool-app-ui. This stops the detached container named my-cool-app-ui.

Interactive Alternative

Finally, I created an alternative version of the docker run command that works interactively.

"docker:run": "docker run -it --rm --privileged --name my-cool-app-ui --env-file .env -p 8626:8627 -v $PWD/:/app my-cool-app-ui /bin/bash",

It is essentially the exact command as the last one with two exceptions:

It uses the -it parameter to create an interactive session instead of a detached session.
It executes /bin/bash to create a shell instead of simply running grunt serve.

If you want to launch the container interactively, simply execute npm run docker:run. To stop and remove the container, you just need to log out of it.

This enabled me to experiment with various grunt commands and view web logs while I was still trying to figure things out. I felt it was useful enough to keep.

Debugging the TypeScript

The next (optional) requirement was the ability to use the Chrome debugger to set breakpoints in the application’s TypeScript source code. The problem was that the TypeScript and generated source maps are stored in completely different locations that the files generated by the grunt serve command.

The solution is far simpler than the amount of research and experimentation that went into it. It ultimately came down to including three lines of code into the project’s tsconfig.json file.

"sourceMap": true,
"mapRoot": "/src/client/my-cool-app-ui",
"sourceRoot": "/src/client/my-cool-app-ui"

These three lines need to be included in the compilerOptions section of the file. The first line was already there, indicating that source maps should be created.

The mapRoop and sourceRoot lines took some time to get right. These values end up being prepended to the name of the sourcemap file, which is specified at the end of each generated .js file.

When the browser downloads a JavaScript file, it checks the end of the file for the location of any source map. It then makes a separate HTTP request for the file found there. This works transparently when the source map file is located right next to the JavaScript file. In my case, however, that was not true. I discovered this by watching all of the HTTP 404 errors for each attempted request.

My solution was to prepend a different path to each source map, which corresponded to the source folder. With the mapRoot set appropriately, instead of an HTTP request that looks something like this…

http://localhost:8627/my-cool-app-ui/feature/coolthing.js.map

We would get this:

http://localhost:8627/src/client/my-cool-app-ui/feature/coolthing.js.map

That still resulted in a 404, even though the folder was now correct.

The solution to this was simple, however. I modified the Express app, only in development mode, to serve static files from the src/ folder. Voila! The source code and source map requests suddenly worked.

At that point, the Chrome debugger was fully functional, including the ability to set break points directly inside the TypeScript files.

VS Code Debugging

The final requirement was to make all of the debugging features work from inside of VS Code, which already supports Chrome debugging through its Debugger for Chrome Extension.

Once this extension was installed, I only had to add the appropriate configuration inside of .vscode/launch.json. This is the configuration that ultimately worked, which I believe is the default.

{
  "name": "Launch Chrome",
  "request": "launch",
  "type": "pwa-chrome",
  "url": "http://localhost:3000/individual-entry-codes",
  "sourceMaps": true,
  "trace": false
}

The only thing I did was change the url to a page I wanted to load, and the trace property. I set its value to true during my experiments, which provides more verbose logging in VS Code’s Debug Console. I set it back to false for most ordinary uses.

Docker Compose

This document describes my adventures in getting the UI project up and running in Docker. However, there is also the API project to consider. In production, the two execute in separate Docker containers primarily so they can be scaled independently from one another. A load balancer sits in front of them to manage the traffic and to provide a single base URL, enabling them to share cookies. The load balancer acts as a proxy, routing calls to the API based on the URL route. The next step in my journey is to replicate that experience locally.

Dockerizing the API

As it turns out, executing the API project in Docker worked exactly the same was as I described above for the UI project, proving that the process is repeatable.

Once I had both the UI and API projects running separately inside of their own Docker containers, I needed a way to route traffic between them.

HA Proxy

Enter HA Proxy. This docker image was recommended as the one to use, though I have also used Nginx and it works just fine. The are both serving the same function in this case. I will detail HA Proxy, as that is the one I used for this project.

What is It?

From the explanation in the Docker registry,

HAProxy is a free, open source high availability solution, providing load balancing and proxying for TCP and HTTP-based applications by spreading requests across multiple servers.

That is exactly what I needed.

Configuration

The hardest part about any of these tasks is figuring out the appropriate configuration settings. As I said, someone else gave me this, so it only took some minor tweaking.

The Dockerfile itself was only two lines, and also comes straight from DockerHub:

FROM haproxy:1.7
COPY haproxy.cfg /usr/local/etc/haproxy/haproxy.cfg

Nothing exciting to see there. It simply grabs a specific (albeit older) version of HA Proxy and copies a config file.

This is that config file:

global
    daemon # Run in the background

defaults # These apply to everything and were provided to me
    log global
    mode http
    option httplog
    option dontlognull
    timeout connect 5000
    timeout client 50000
    timeout server 50000

frontend localnodes # Public-facing config
    bind *:3000 # Listen on port 3000
    mode http                             
    acl is_api_url path_beg -i /api/ # Check to see if path starts with /api/
    use_backend api_server if is_api_url # If so, forward to api_server
    default_backend static_server # Otherwise, default to static_server (UI)

backend static_server # Static (UI) Server Config
    mode http
    balance roundrobin
    option forwardfor # Forward traffic to...
    server web01 ui:8627 # A server called ui on port 8627

backend api_server # API Server Config
    mode http
    balance roundrobin
    option forwardfor # Forward traffic to...
    server web01 api:8181 # A server called api on port 8181

The only edits I made to this file were on the two server lines. They were set to localhost, which worked when I ran the two Docker containers locally. They exposed their respective ports. However, when I decided to try using Docker Compose, I intentionally configured its network to hide those ports.

Docker-Compose Configuration

Now I will show the docker-compose.yaml file I came up with. The sole consideration for me was to have Docker automatically build (if necessary) and launch all three containers for me.

version: '2'
services:
api: # This is the config for the API
    image: my-cool-app-api:latest # Name of the docker image to run
    build: # Or build if it isn't available
      context: ../my-cool-app-api/ # Look here for its dockerfile
      dockerfile: Dockerfile.dev # Using this file as the dockerfile
    env_file: 
      - ../my-cool-app-api/.env # Get environment vars from this file
    volumes:
      - ../my-cool-app-api/:/app # Map the entire folder to /app
    restart: unless-stopped # Restart the container unless I stop it
    command: npm run serve # Run this command at container startup
    expose:
      - 8181 # Export port 8181
    networks:
      - my-network # Expose it to this named network

ui: # This is the config for the UI 
  image: my-cool-app-ui:latest # Name of the docker image to run
    build: # Or build if it isn't available
      context: . # Look here for its dockerfile
      dockerfile: Dockerfile.dev # Using this file as the dockerfile
    env_file: 
      - .env # Get environment vars from this file
    volumes:
      - .:/app # Map the entire folder to /app
    restart: unless-stopped # Restart the container unless I stop it
    command: npm run serve # Run this command at container startup
    expose:
      - 8627 # Expose port 8627
    networks:
      - my-network # Expose it to this named network

haproxy: # This is the config for the prooxy
  depends_on: # It depends on the following services
    - api # api, as defined above
    - ui # ui, as defined above
  image: my-haproxy:latest # Name of the docker image to run
  build: # Or build if it isn't available
    context: . # Look here for its dockerfile
    dockerfile: Dockerfile-haproxy # Using this file as the dockerfile
  ports:
    - 3000:3000 # Make port 3000 available to the host
  volumes: # Map my custom config file at runtime
    - ./haproxy-compose.cfg:/usr/local/etc/haproxy/haproxy.cfg
  restart: unless-stopped # Restart the container unless I stop it
  networks:
    - my-network # Expose it to this named network

networks:                               
    my-network: # Create a default named network for containers

A few things are worth noting.

The names “api” and “ui” appearing in both the docker-compose.yaml file the haproxy.cfg file have to match. Docker will effectively perform runtime DNS resolution, and those names end becoming the hostnames inside the Docker-managed network.

The expose lines in docker-compose.yaml expose those ports to the Docker-managed network, but do not expose those ports publicly to the Docker host. The ports entry in the haproxy configuration does expose port 3000 to the host.

Tying it All Together

Now that I have a working docker-compose.yaml file, the only thing left is to start it all up with the command docker-compose up. I like the fact that all log files are sent to stdout and that I can watch the progress. Once everything is up and running, I can launch a browser to http://localhost:3000 to test my application. And yes, the debugging strategies detailed above for VS Code still work.

The only significant piece missing is the ability to debug my API layer, as those specifics are dependent on the backend technology. The strategy will be similar, whether the API is written in Node, C#, Java, PHP, etc. You need to ensure that the appropriate debugging ports are exposed to the host, and then connect your debugger through that port as usual.

Final Notes

I spent a lot of time reviewing the file diffs on the pull request I opened to implement all of these changes. In doing so, I discovered a few discrepancies, missing files, and even one unnecessary file. Typing this lengthy explanation helped me to catch and correct those oversights.

And that is everything. I managed to cover every requirement and then some. The result is better and more functional than I had hoped. My only remaining hope is that my sharing this experience will help someone else save some time. If you happen to find any tweaks that make it even better, please let me know.