DEV Community: Yonatan Korem

If you use Amplify and Git branches, you are going to have a bad time

Yonatan Korem — Tue, 07 Jun 2022 11:38:25 +0000

AWS Amplify is...

A combination of tools that aim to assist with development, deployment, and hosting of your serverless application.

AWS Amplify is three things:

It's a cloud service that includes consoles, monitoring tools, build and deploy configurations, and so on.
It's a CLI tool that allows you to create Amplify environments, modify them, add resources, and manually publish if you so choose.
It's a library for frontend development that simplifies using the resources in your environment like authentication, HTTP calls to your gateways, and more.

You don't have to use the CLI or the library, but you would probably want to since it saves a lot of headaches.

Deploying applications with Amplify

When you create an Amplify app (which contains different environments under it), you can connect it to a branch in your repository. That creates a Hosting Environment. Every hosting environment can be connected to a specific Backend Environment.
When connected to a branch, Amplify will be notified whenever a code was pushed to that branch. At that point, Amplify will rebuild the frontend code in the branch and deploy it.
If connected to a backend environment, Amplify will also redeploy the backend, based on the contents of the branch.

Amplify CLI tries to follow how Git works

After you created a backend environment dev and deployed it, Amplify considers it as the equivalent of Git remote. When you want to work on it, you must pull that environment to your local machine. You can then change it and push your changes back to remote.

When you pull an environment with Amplify CLI, it does two things:

Downloads the content of the environment. The code of the lambdas, the paths defined in the gateway, the templates of the resources, etc.
Creates/modifies files that say "what is the current environment in this machine". Similarly to Git, where you can checkout multiple branches and your local Git will know which branch you are currently using. For example, a file called aws-exports.js that contains a JSON with a lot of configurations. When your application loads, you need to pass the content of that JSON to the AWS Amplify library so it will know what endpoint to contact when your code performs a HTTP call.

When you push an environment, Amplify CLI will first check against the remote to see if there is any local modifications. If there are, it will compile a .zip with all the files required and upload it to the one currently configured.

That's a lot of information. I thought Amplify was supposed to make my life simpler...

Tools that make your life easier usually do it by abstracting a lot of knowledge and technical understanding. Your Nespresso machine doesn't even show you the coffee grinds. You just put this aluminum pocket into the hole, push the button, and viola!

Amplify definitely will make your life easier. But this will come with a price, and when something breaks, you must understand what it does in order to fix the problem or preemptively avoid the pitfalls.

The reason Git and Amplify don't play well together

Imagine the following scenario:
You have a main Git branch and you have your mainBE backend environment that currently has a lambda function foo.

To develop a new backend service, you branch out of main to a feature branch called feat.

> git checkout main
> git checkout -b feat

You also clone mainBE to a new
environment: featBE

> amplify env checkout mainBE
> amplify add env featBE
> amplify push --y

This action changes team-provider-info.json by introducing a new key to the object: featBE. The value currently (almost) equals that of mainBE. It will now look like this:

"mainBE": {
    "awscloudformation": {
      [...]
      "Region": "us-east-1",
      "AmplifyAppId": "someRandomId"
    },
    "categories": {
      "function": {
        "foo": {
          [...]
          "envVarName": "environment variable value"
        },
    }
},
"featBE": {
    "awscloudformation": {
      [...]
      "Region": "us-east-1",
      "AmplifyAppId": "someRandomId"
    },
    "categories": {
      "function": {
        "foo": {
          [...]
          "envVarName": "environment variable value"
        },
        "bar": {
          [...]
        },
    }
}

You then add the new lambda called bar and push it to your feature environment.

> amplify add function
> amplify push

Now the two entries are different. featBE has details of a new lambda, while mainBE does not.

You develop the new service, test it locally and deployed, and everything is done. You then merge your feature branch back into the main branch.
Problem is, any CI/CD process that tries to deploy the backend will fail. The content of team-provider-info.json under the mainBE key has no definition for the bar lambda.

To successfully deploy mainBE with the new service, you will need to pull featBE, switch to mainBE, and push to it.

> amplify env checkout featBE
> amplify pull --y // to overwrite the current content, assuming this is done on a different machine
> amplify env checkout mainBE
> amplify push

Amplify will see the new content and update mainBE.
But, like before, this will update team-provider-info.json, which means your backend environment doesn't match the code in your source control.

You'll need to push this new difference into your git main branch. This might sound somewhat reasonable, but if your setup uses the Git flow of having dev, testing, and production branches (and maybe more), you'll find that this cycle of having to merge branches, merge backends, then merge again will grow exponentially with every branch in your flow.

For example: let's assume we already performed all the above mentioned, but we also have staging and production environments.

First, we need to merge the new feature to staging:

> git checkout staging
> git merge main

At this point, stagingBE won't build. We need to fix this by taking the backend definition in mainBE and applying them to stagingBE

> amplify env checkout mainBE
> amplify pull --y
> amplify env checkout stagingBE
> amplify push

Now our local branch is ahead of main (and staging)

> git checkout main
> git commit -m "Updated Amplify setup of stagingBE"
> git push

main is updated, but now main and staging are not aligned

> git checkout staging
> git merge main

stagingBE should be ok now, but to make sure that the deployed backend matches 100% to the source code, we will redeploy

> amplify push --y

Now staging environment is ok, but we still have production to deal with

> git checkout production
> git merge staging
> amplify env checkout stagingBE
> amplify pull --y
> amplify env checkout productionBE
> amplify push --y
> git checkout main
> git commit -m "Update Amplify setup of productionBE"
> git push
> git checkout staging
> git merge main
> git checkout production
> git merge staging
> amplify push --y

This is the abbreviated version as most Amplify CLI commands require more information either manually entered or via headless mode.

This whole process can take at best ~15 minutes because Amplify takes time to pull and push. At worst, your CI/CD: tests, automation, deployments, etc will need to execute between after each git push in this process, which will take much much longer.

In the end, no matter what you do, you will end up in a situation where the deployed backend is not based on your source code, which is not a situation you want to be in, certainly not on a regular basis.

By the way, the CI/CD built into the Amplify console won't do all of this. It just takes whatever there is in your source code and tries to deploy.

Finale

All in all, Amplify is a pretty powerful tool but seems to support a common, basic use case. If you need anything more than this, you're probably better off using other tools.

What Test Driven Development is really about

Yonatan Korem — Fri, 27 May 2022 17:28:56 +0000

When I write a post, I tend to start with some definitions. What is this framework? What is the meaning of the term ?

This time, we are going to mix it up. Take a moment and think about the meaning of Test Driven Development or TDD for short? What does it mean to work this way, and why would you do it?

This is where you take a moment...

The answers I usually hear

"To work TDD, you write all your tests, and then implement the code to make them pass"

"TDD is about writing as many tests as you can to make sure you have 100% code coverage"

"TDD means you don't have a design"

"TDD is only used with low level unit tests"

The answers I hear are not wrong, but in my mind, are not accurate.
TDD is not about code coverage, though you'll probably have a high coverage with it.
TDD is not about having as many tests as you can, though the ones you'll end up with will probably cover a lot of edge cases.
TDD is not about having no design at all, just not having it up front. But you'll have one in the end.

I think that TDD is not even about tests.

TDD is about being like water.

You've probably encountered Bruce Lee's famous quote:

"Be like water, my friend"

By working with TDD, you allow yourself the freedom of not being trapped in a pre-determined design that might and might not be the right now. You shape the vessel that you need your software to fill, and fill it a little. Shape it some more, fill it some more. Your code adapts to an increasingly specific vessel.

TDD is about thinking like an artist

A sculpture doesn't carve the wood to the exact shape of the image they wish to create. They carve enough to get a rough form that looks right, and then carve some more. Slowly but inevitably they will get to the point where the carving is about the finest of details.

TDD is about knowing you are on the right path

When you work in the TDD flow, you are always adjusting, but rarely going backwards. Each test paves just a little more of the road. You rarely go backwards because you might take a wrong turn, but you will know you went backwards right away. Allowing you to fix it, go back, and try a different direction.

Enough philosophy for now

TDD is simple:

Write the simplest test that should break. You know what it is, because you know what the implementation is lacking.
then you make it work, without braking any other existing test. It doesn't have to be the best solution. Most of the time, the most straightforward approach is best.
Clean up your (tiny) mess. You can't create a giant mess in 5 minutes. It's impossible. Cleaning up a tiny mess will take a tiny bit of time.
Go back to step 1 until you can't think of any more tests.

A while ago, I watched an online lecture from uncle Bob. It wasn't even about TDD, but the topic found its way there. In that lecture, somewhat off-handedly, uncle Bob said:
"You avoid writing the tests that you know you'll need in the end. Instead, you write a test which proves that what you thought was wrong with your code, is real."

That sentence helped me understand why TDD was difficult for me to do at first. I would pick the tests that were the end goal and found myself not having that short iterations, where I could feel the code evolve. I would just write the tests that proved what I needed and then I would write the code I knew I wanted to write.
I was not like water.

Emergent Design

I once did a programming Kata that asked you to write an online API for a persistent stack. I decided to attempt it with TDD.
I avoided this process: I'll use a MongoDB for the persistency, and I'll do the backend with ExpressJS, and so on...
Instead, TDD meant that I first need the stack itself. Once that was made, I knew I could put that library anywhere and it was well tested. Then I built just the API and ran it locally. Now I know the API works. It returns garbage, but it works.
Put the stack library in the logic of the API, and I got a HTTP stack going.
The stack just used in-memory storage, so as long as the server was running, it worked.
I wrote a persistency interface that just wrote to a JSON file. That worked well, so I just put that layer into the stack, but it broke the tests because the tests did not mock it.
"Oh, right!" I realized that I should inject that layer into the stack so it just uses it like an array.

I didn't have to spend time designing up-front too much.
I didn't spend time on large refactoring sessions.

There are endless columns, write-ups, and articles about doing "enough design", and not solving problems you don't have. In its essence.
Same goes for tests, clean code, clean architecture, and so forth.

TDD is about discipline

It is a discipline you employ that helps you. It helps you create software that is clean, testable and tested, flexible, and long lasting.

Finding the limitations of AWS Amplify

Yonatan Korem — Thu, 27 Jan 2022 10:23:45 +0000

Preface

This post is based on the experience I had with Amplify@7.6.3 in November 2021.
When you read this, things might be different...

AWS Serverless infrastructure

AWS cloud is rich platform, full of servers, elastic searches, databases, and a lot more cloudy stuff. AWS also provides tools to build your application "serverless". That is, write your backend as functionalities and let AWS handle the deployment of it. For example, instead of creating a project for a micro-service and deploying it to a cloud server instance, you just write an AWS Lambda and tell AWS via configurations how it connects to other elements like APIs, databases, or other AWS Lambdas.
AWS takes your Lambda and does all that infrastructure work for you.

There are pros and cons for using a serverless infrastructure at large and AWS specifically. Those relates to your level of control, payment model, and so on. But for now, I'm going to leave all that aside and focus on two AWS tools for building a serverless application

SAM (Serverless Application Model) and Amplify

SAM is one tool AWS provides that allows you to define, via a .yaml file, your backend. What resources you need, how they connect to one another, and so forth.
Amplify is another tool provided by AWS. In fact, it is three tools that work together.

The first tool is the Amplify CLI. The CLI gives you the ability to define your backend, similarly to SAM (since it is sort of a friendlier interface over SAM), but in a command line. You just tell Amplify that you want to create a new Lambda, that it needs access to specific global environment variables, and connect it to your DynamoDB triggers (events that the lambda will be invoked when they occur). Few button clicks, and you're good to go.

Second is the Amplify console. It lets you monitor different deployments, called environments, attach it to your repo and have CI\CD that deploys your backend and frontend when you commit changes to it. It allows you to customize the build process, manage domains, control access and various other options.

The third, is the Amplify library. The library is used in your frontend. It can completely configure your frontend in a couple of lines of code, since Amplify CLI created files that the Amplify library uses to configure everything. You give the library the config file and off you go adding authentication and API calls to your frontend.

Amplify CLI and GIT

Amplify manages the project in the top level as an "Application". Under that, there could be multiple backend environments, and multiple frontends - each deployed from a connected repository branch. You push your changes into your github repo, under the "testing" branch and Amplify will redeploy the front end that is connected to that branch, and update the backend that is connected to that frontend.

The use of multiple environments tries to create a similar way of work as GIT. You can checkout backend environments. Switch between checked out environments. Push (deploy) and pull to your local machine.
You can even create new environments based on the current state of others (i.e. git checkout -b ...)
You perform all these actions via the Amplify CLI tool with commands like amplify env checkout ENV_NAME or amplify init.

"When this baby hits 88 mph, you're gonna see some serious shit"

We've been working on a web application that required fairly straightforward backend: a database, some HTTP methods, authentication, and that's about it. Then, the company had an idea of what direction the product will take and it involved doing a lot of new stuff. We wanted to take a couple of days to see if the idea is technologically viable. We knew that we are going to work fast, encounter a lot of unknowns, and push the boundaries of our knowledge. The hope was that in one week, we'll be able to show the company some serious shit.

The setup

We were a number of developers, split into three different areas of responsibilities.
One team focused on creating a custom authentication flow with AWS Cognito.
The second focused on mostly frontend exploration, with some need for storage and a way to access it.
The third focused on integration with a third party tool that required lambdas that would be triggered by AWS on certain events.

Where Amplify is powerful

Setting up a new Amplify application, hooking it up to our repo, creating a backend environment, deploying the front end, and connecting it to that backend was a snap.
Adding an API gateway with a specific path, along with a lambda and dynamoDB was a single command in CLI.
Hats off to AWS, because it was amazingly simple and easy.
It even managed the CI/CD fairly well. When we pushed changed to the repo, it would update the backend and deploy the frontend all on its own.

Where we struggled with Amplify

We had three teams, so naturally we had four branches: main and one for each team. The way Amplify works with your repo, and allows you to switch environments, is by having files that define the deployment, and those that define the current environment you are working on. For example, the actual code of your lambda is stored. The file aws-exports.js that contains information about the active environment is not stored.
Thing is, git and amplify don't actually work together. When you change your branch, the files that are not tracked by git remain the same. You are still on the old environment, but now Amplify sees that your local backend setup is different than what is in the environment.
We tried working the way AWS recommends but it didn't take long until the CLI seemed to go crazy.

I would try to change my environment and it would ask for authentication again, even though I just did. I would pick using access keys and nothing would happen.
If I picked using an AWS profile, it would error out with some error that did not explain itself well.

Because you can deploy your local changes to the cloud without going through the repo, it's easy to collide with others working on the same environment. Naturally, we'd want to have different environments, but when the CLI often went crazy we avoided it and tried to work safely.

Where Amplify plain does not work

The moment we started working on the custom authentication, we encountered a major boundary of Amplify.
AWS custom authentication requires connecting lambdas to several Cognito triggers. Amplify CLI just does not support it. We had to manually connect everything via the Cognito console.

We had an issue that blocked us for a whole day (which was about 20% of the total time allocated for us). We had a working API Gateway that connected to a lambda, which accessed a DynamoDB table. We needed another one. Created a new path in the API, created a new lambda and DynamoDB, and so on. Everything looked exactly like the existing infrastructure. Only the new stuff didn't work. We would get CORS error all the time. Tried to check the CORS settings, authentication settings, it all seemed ok. Tried to do the process again and again. Tried the create a new API Gateway. Nothing worked...

Only later one of the developers noticed that the API gateway resource did not update correctly. Only when we created a new environment and the API was reconfigured prior to being deployed, that the issue was finally fixed.

The last day

As these things go, somehow everything managed to cross the finish line at the very last minute. But we all felt that we succeeded in spite of Amplify.

Part of the team was more familiar with SAM, and by the end of it they said that SAM, while being more complex and leave a lot of the responsibility with the developers, is probably the better tool since it give you the power to configure everything.

Final thoughts

Amplify is a good tool when your requirements match its capabilities. You could setup a live web page with login capabilities, that stores and loads data via a RESTful API, in literally a few minutes. If you need anything more than that, Amplify is limited and you might find yourself struggling to manage some elements via the AWS console and some via Amplify.

Amplify CLI is a great tool in concept, but we felt it was too unstable and unpredictable. But if you only use a single environment, you won't encounter the most troubling aspects of it.

The Amplify library is pretty good! My only major issue with it is that you have to use Amplify CLI with it or it won't work.

In the end, it's the question of what would you rather have: a big toolbox that allows you to build and control whatever you want, or a pocket knife that is great when you need to cut something but you would have a bad time trying to hammer a nail with it.

"Single Responsibility" will make your code better without you even noticing

Yonatan Korem — Tue, 02 Nov 2021 11:12:07 +0000

This column is not about what is the "Single Responsibility Principle". There are a lot of other places you can read all about it. Robert Martin's blog is a good place to start.

It is also not about how strict you should be applying single responsibility.

This column is about how following this principle will make your code better. Your code will be shorter, simpler, maintainable, testable, and various other adjectives.

Exercise in imagination

Imagine a hamburger cookbook. It tells you what you need to make the patty, how to grill it, how to make and bake the buns, and how to put it all together. Now imagine that it is all on the same page.
The ingredients list is long and difficult to understand what goes in the bun and what in the burger.
The preparation guide tells you how to do everything combined. The steps jump between the patty and the bun - it's time saving because while the bun bakes, the next couple of steps is about the burger. But it is also comprised of dozens of steps.

SRP Makes your code easier to follow

Why do cookbooks separate the items? It is much better to do all at same time, plus it saves paper because there is less white spaces. They separate it to make each part easier to follow and understand. It also makes it harder to confuse the ingredients.

Your class/function/file becomes simpler to follow because it focuses on one subject. You can feel safe that your backend API controller only handles defining the API and validating the inputs. Nothing more and nothing less.

SRP makes your code shorter

With the same example of the cookbook: instead of a single recipe taking three pages, you get two parts - each taking two pages. Combined you might end up with more pages, but each component of the hamburger is shorter.

In the cookbook, though it has more pages, it is easy to find what you are looking for. It has a structure and logic to it.

True, you will end up with more files in your repository. But each one is shorter, and the entire thing is organized and structured so anything you need, you know where to look.

SRP will make it easier to debug problems

If you use the bad cookbook to make your burger and confuse the seasoning between the bun and patty, you could get a salty hamburger and tasteless bun, because a portion of the salt that was meant for the bun is in the patty.
If you use the normal cookbook, you might get a poorly seasoned hamburger, but it will not impact how the bun ended up.

Problems are easier to find because they are contained.
Your API responded with an error. It responded, so the controller got the message. You look at the body and all the fields have the correct type and length. Like before, the controller only defines the API and validates inputs.
Together that automatically means there is no reason to investigate the controller.

SRP makes your code easier to unit test

When your class has a single responsibility, it depends on other components for any other functionality it requires.
The class handles some logic, which means it does not handle creation of its dependencies. Hence, your class gets the dependencies from outside.

When your class does that, it becomes easier to inject fake dependencies - which is exactly what you need to unit test.

SRP helps naming things

From the function level, to your folder structure, having "things" be responsible for one thing makes it easier to name them.

A component that does one specific thing, can have an equally specific name, at the same abstraction level.
When the class encapsulates HTTP calls to get and set Contact from a DB, it becomes clear that it should be named ContactGateway
The folder that contains all the components and logic that handles the login page, would probably be named LoginModule
When your library handles thread pools using a SQL server, it would probably be named SqlThreadPool.

SRP helps following established design patterns

Now that your class does one thing, and is named to match, you'll find yourself following established design patterns. When you separate the modeling of a graph from the algorithm that traverses it, you end up with the Strategy class and the context it operates on.
The graph is comprised of individual nodes, each encapsulating its data and logic. Congratulations! You are using the Composite design pattern.

SRP makes your code readable

When you combine all of the above, you get code that is shorter, highly focused, with lower cyclomatic complexity. Plus, the code is easy to find.

These are all things that every developer will appreciate. Think of the times where you had to juggle multiple trails of thought to try and figure out why something does not work right.

SRP makes your project better

Because it helps generate code that is easier to maintain, test, refactor, and extend.
Maintainable code is critical for allowing your project to continue consistent and fast development.

In conclusion

SRP helps you make better hamburgers.

Designing a Robust HTTP Error Handling System in Angular

Yonatan Korem — Tue, 12 Oct 2021 06:25:02 +0000

A Story Told Over and Over

I've been a part of a number of different Angular front end projects in the past years. In every one of those projects, there came a point in time where we were asked to implement:

When a call to the backend fails due to authentication errors (401), route the user to the login page.

It didn't take long until another use case arrived:

When a call to the backend fails, alert the user of the failure with a generic "The requested operation failed".

Soon after, another use case:

When the user tries to get the information of an existing contact, and the operation fails, display a toast "Contact not found".

This would escalate further when the same operation failure would need a different error handling when done from different components.

If the user tries to edit a contact and the error is 404 (not found), then toast "Contact not found, would you like to create one?" along with a "Create" button.

I've seen, and been involved with the design decisions that attempted to handle these use cases. Here are a couple of case studies.

The Idealized

This approach tried to contextualize the error itself. An interceptor would wrap the error and give it a default message. The error would not be handled by the different layers. Instead each one could attach a new message. When a specific layer would want to "finish" handling the error, it would manually call an error handling service that would pick the appropriate message and display it.

In theory, this should have worked and support all the different use cases. It did, but it was complex. Too complex. Developers would not use it correctly, and defects would pop up. The gap between the theory described, and the practicality of it was tremendous.

Plus, it still required someone, somewhere, to handle the error. If no one does, all this mechanism does nothing.

The Simplistic

This approach went the complete opposite way. Have a simple mechanism: an interceptor would have a hard coded black list of errors it would always handle, like authentication issues that it would reroute. It also had a hard coded white list of URLs and error codes it would not handle at all.

This design was good, but it left large gaps. Any change to the URLs, any change to the possible error codes returned, would mean one of those hard coded lists would need to be manually updated. It also still didn't solve the issue with errors not being caught at all.

We Need to Get Back to Basics

If we look at the requirements fresh, we can see that the basic message of "The requested operation failed", is the message we would want to display if no one else handled the error. That means that we have to first let all components and services the opportunity to handle the error and only if none of them does, then we should display the default message.

Here lies the root of the problem with all the designs I've encountered: An interceptor is the first component that has the opportunity to handle the error, not the last one.

Introducing the ErrorHandler

Angular has a built in service called ErrorHandler. Any error that your app does not handle will reach this service. The Angular service just outputs the exception to the console. If you want to display a toast for specific unhandled errors, all you need to do is:

// my-error-handler.service.ts
class MyErrorHandler implements ErrorHandler {
  handleError(error) {
    // do something with the exception
  }
}

// app.module.ts
@NgModule({
  providers: [
    { provide: ErrorHandler, useClass: MyErrorHandler }
  ]
})
class AppModule {}

The only difficult part here is that ALL uncaught exceptions end up here, not just HTTP ones. Luckily, we can differentiate between them with this:

if (error instanceof HttpErrorResponse) {
 // Handle HTTP errors
}

This will cover our fallback use case so no error goes unhandled, but what about errors we want to always handle the same way?

Enter the HTTP Interceptor

While the ErrorHandler is our last line of defense, the interceptor is our first. That makes it ideal to handle the authentication errors which we would want to re-route back to a login page.

// my-interceptor.ts
class MyInterceptor implements HttpInterceptor {
   intercept(req: HttpRequest<any>,next: HttpHandler): Observable<HttpEvent<any>> {
  return next.handle(req).pipe(
    catchError(e => {
      // route if the error is an authentication error
    })
  );
}

// app.module.ts
@NgModule({
  providers: [
    { provide: ErrorHandler, useClass: MyErrorHandler },
    { provide: HTTP_INTERCEPTORS, useClass: MyInterceptor, multi: true },
  ]
})
class AppModule {}

For Everything in Between

We took care of the first two requirements. Let's handle this next:

When the user tries to get the information of an existing contact, and the operation fails, display a toast "Contact not found".

Our instinct might be to let the service that performed the HTTP request will handle it within the scope of the observable.

@Injectable()
export class MyService {
  constructor(private http: HttpClient) {}

  getEntity(entity: Entity): Observable<Entity> {
    return this.http.get(url).pipe(
      catchError(e => {
        // toast the appropriate message
      })
    );
  }
}

Which is OK until the last requirement appears:

If the user tries to edit a contact and the error is 404 (not found), then toast "Contact not found, would you like to create one?" along with a "Create" button.

We need a way for all the parties involved to have the chance to say "please toast this message" and only when everyone finished, then decide what to show.

RxJS handles all your needs

RxJS has two operators that we need to implement our solution:
CatchError and Finally.

CatchError is triggered when an error happens in the stream, and it must return another observable, or throw an error.

Finally is triggered when the stream completes, or when it errors.

The important part here is the order of them being called when the observable is constructed with multiple of both.

// serviceA
getFromBackend(): Observable<ReturnType> {
   return this.http.get(...).pipe(
      finally(() => // some service level cleanup)
   );
}

// serviceB
findElement(): Observable<ReturnType> {
   return this.serviceA.getFromBackend().pipe(
      catchError(e => {
         // log something
         throw e;
      }),
   );
}

// componentC
onButtonClick(): void {
   // set the button to disabled
   this.serviceB.findElement().pipe(
      catchError(e => of({})),
      tap(value => { 
         // do something with the value 
      }),
      finally(() => {
         // set the button back to enabled
      })
   ).subscribe();
}

When the backend returns an error, the order of calls will be:

catchError - serviceB
catchError - componentC
finally - serviceA
finally - componentC

This is exactly what we need - anyone that wants to do something with the error does it first. Then, in a "finally" operator we could trigger the toast service. (Thank you to @elirans for the idea)

We don't want any element that wants to toast, to rely on someone else triggering the toast service. Anyone that wants to toast will need both the catchError and finally operators. Trouble is, there will be a lot of repeated code, and the risk of missing some crucial part of the behavior is high.

Because of that, we're going to create our own pipe operator!

It's really not that scary

A pipe operator is just a function that takes a stream as an input and returns a stream as an output.
In reality, most pipe operators are factory methods that return a pipe operator. We'll do just that.

// toastOnError.ts
export function toastToConsoleOnError<T>(messageToToast: string): (source: Observable<T>) => Observable<T> {
   let errorToToast: { toast: string };
   return function(source: Observable<T>): Observable<T> {
      return source.pipe(
         catchError(e => {
            e.toast = messageToToast;
            errorToToast = e;
            throw e;
         }),
         finally(() => {
            if (errorToToast && errorToToast.toast) {
               console.log(errorToToast.toast);
               errorToToast.toast = null; // since we save the reference to the error object, any future access to this field will get a null value.
            }
         })
      );
   }
}

// serviceB
findContact(searchTerm: string): Observable<Contact> {
   return this.serviceA.getFromBackend(searchTerm).pipe(
      toastToConsoleOnError('Contact not found');
   );
}

// componentC (Component level toast)
onEditRequest(): void {
   this.serviceB.findContact(this.searchTerm).pipe(
      toastToConsoleOnError('Contact not found. Would you like to create one?')
   ).subscribe();
}

// componentD (Service level toast)
onQuickViewRequest(): void {
   this.serviceB.findContact().subscribe();
}

While the specific implementation above has its weaknesses (for example, if one element uses the operator, and another does not, you'll get two toasts), the core idea is the same and you can adjust the implementation for your needs: Maybe you need a way to mute the toasts, or maybe you want to toast if there is a condition met on the error.

With this new operator, if someone wants to toast they will, unless someone with more context also wants to toast, and we won't have two toasts popping up.

Our journey complete

We broke down the problem into three sections, and deal with each using a different mechanism:

Use a HTTP interceptor for handling errors that is always the same.
Use the Angular ErrorHandler as a failsafe to catch any error that is not handled elsewhere.
Create a pipe operator that uses catchError and finally to allow elements to store/overwrite the toast to display, and display it in the finally operator.

Why I fell out of love with inheritance

Yonatan Korem — Wed, 18 Aug 2021 08:32:12 +0000

A tale of growing up, and realizing your heroes have flaws

The Beginning

Almost nine years ago, when I was near the end of my BSc, I was sure that inheritance is the greatest thing an OO programmer has in their arsenal of tools. During most of my studies, we were taught the principles of OOP, why interfaces are important, and how inheritance is a great tool for reusing code, abstracting your code and design, and what a generally powerful tool it is. That's not incorrect. It is good for those things. But it took me a few years to realize that when I idolized those concepts, I tended to use inheritance where it shouldn't have been used.

We were shown the example where inheritance fails:

A circle, and an ellipse. Which is the abstraction of which?

But we were also told examples like this one:

class employee {
  pay() 
  generateHoursReport()
  assignVehicle()
}
class Manager extends Employee {
  assignEmployee()
}
class CEO extends Manager {

}

When you think about it in terms of technical abstraction, it makes perfect sense. A CEO is a manager. A Manager is an employee. A CEO is an employee.

All three share the basic functionality of the employee, and each extends those functions, and/or add new functionality. This solution is exactly the solution I would come up with, before I started to internalize the first of the SOLID principles, and learned more about the delegation/composition pattern.

Single Responsibility Principle

Robert C. Martin wrote about the SOLID principles in the Agile Software Development and the definition was that:

A module (class, interface, package,…) should do only one thing.

It was later refined by Martin to state

A module should have only one reason to change

I won't go into details on the meaning of the refinement, and you'll find a lot written about it already, especially in Martin's own blog. But let's apply this principle to our Employee example.

Looking at the Employee class, we can see that it handles three things: financials, motor department, and HR subjects. To discuss it with Martin’s refinement in mind, there are three departments that might give a reason to change something in the employee class.

Payroll department decides to update the way they calculate the salary of employees.
Motor department wants to make it so only managers can be assigned vehicles due to budget cuts.
HR replaced their reporting system and require a new template for the hourly report.

It took me a while to understand why this is even an issue. I would think that the Employee class knows how to handle the specifics for its own level, and once you travel up the chain, the Manager will probably override the things it needs, and so on.

But...

You implement the pay (finance) and generateHoursReport (HR), and quickly notice a piece of repetitive code between the two that counts the total amount of hours. You’ve read about code smells and you want to be DRY, so you refactor it out to a third, private method: getTotalHours.
Months later, HR wants to change the report so that it shows the remaining hours. The new developer is not familiar with the system and changes getTotalHours so that it returns the value times -1. generateHoursReport now works as expected, but pay is now very broken. The change requested in one method affected the other.

We have this structure, and now we want to introduce a new type of employee: Contractor. The contractor needs to generate a hours report, but not to your system, and your system does not need to pay him because you pay his company. You want to have Contractor extend Employee, but the new class should not include a pay method. Maybe have Employee extend the Contractor, but an employee is not a contractor.
So you start to introduce base classes that do not actually represent anything, it’s just a “base” class to extend from.
I promise you this: a year later, your simple inheritance structure became a huge inheritance tree, with classes like RootEmployee, ContractorWithVehicle, and TechnicalManager that cannot have any employees.

How can we change Employee and its structure to avoid these issues? How can we design the system to support all of these departments?

One way to do it, is to compartmentalize the logic to three different parts, and change the design to one of composition & delegation. This creates four pieces of code, each responsible for one topic: HR, Payroll, Motor, and the Employee that composites the three and interfaces with them by providing information for the relevant employee. With this modification, our design now looks like this:

interface IPaymentStrategy {
  pay()
}
class SalaryEmployeePayment implements IPaymentStrategy {
  pay() {...}
}
interface IReportFactory {
  generateHoursReport()
}
class HourlyEmployeeReportFactory implements IReportFactory {
  generateHoursReport() {...}
}
interface IVehicleDecorator {
  assignVehicle()
}
class LeasingVehicleDecorator {
  assignVehicle() {...}
}
class Employee {
  private IPaymentStrategy paymentStrategy
  private IReportFactory reportFactory
  private IVehicleDecorator vehicleDecorator
  public Employee(IPaymentStrategy p, IReportFactory r, IVehicleDecorator v) {
    ...
  }
  pay() { this.paymentStrategy.pay() }
  generateHoursReport() { this.reportFactory.generateHoursReport() }
  assignVehicle() { this.vehicleDecorator.assignVehicle() }
}
var employee = new Employee(
   new SalaryEmployeePayment(), 
   new HourlyEmployeeReportFacotry(), 
   new LeasingVehicleDecorator()
)
var manager = new Employee(
   new ManagerPayment(), 
   new HourlyEmployeeReportFactory(), 
   new LeasingVehicleDecorator()
)
var ceo = new Employee(
   new ManagerPayment(), 
   new HourlyEmployeeReportFactory(), 
   new CompensationVehicleDecorator()
)

With this separation, we only need one Employee class, and the inheritance is provided by giving it different behaviors for each part.
Replacing the behavior of one element has no effect on the other parts of the system, which is basically the core idea behind the single responsibility principle.
Inheritance may still be used, but it will be applied to a single element of behavior, and will not cause the ContractorWithVehicle type of problem or create a need for multiple inheritance.

Composition & Delegation

Composition does not necessarily mean delegation. The composition design pattern describes a way to define an object by its connection to other objects: TreeNode, or ListNode are the most basic of examples for it.

Delegation does not necessarily mean composition. Delegation (as is implied by the name) can be used when we want our object to fulfill a specific API, without being connected to the implementation of it.

The above example, and plenty of others like it (like GameObject) show how using inheritance can sometimes lead to poor design, especially when the inheritance is already implemented and more, different usages appear.

The composition design has more benefits:

It makes our design more testable, since each part of the code can be tested completely separately
It makes our design more aligned with the Liskov Substitution Principle
It makes our design more aligned with the Interface Segregation Principle

Composition and Delegation together, make us look at our classes in terms of behaviors, and not as only “X is a Y”.

Wrap up

I want to stress one final thing about “loving” a concept. In the past, thinking that a specific design is best often got me to a place where I tried to fit the problem to the design, and not the other way around.

Remember that you have more than a hammer. Don't see your problems as only nails.

Migrating your web application to NGXS state management

Yonatan Korem — Mon, 16 Aug 2021 07:53:54 +0000

What's state management?

State management is the concept of having the state of your application be decoupled from the logic or UI of your application, but also not having multiple copies of your data. That saves you from having to sync your data, and allows for an application that will be more consistent and have less defects.

One of the most popular state management patterns is REDUX which emphasizes reactive programming. With this pattern, you have a "single source of truth" and your application observes that data. When it changes, your application reacts to that change as needed by the specific component.

What's NGXS?

NGXS is a front end state management framework for Angular. It's similar to the popular NgRx framework, but offers a lower learning curve, and with it your code contains less boilerplate code - which is something that plagues NgRx.

In NGXS there are three basic concepts to understand before you start integrating it to your code.

Actions

The action is an object that represents a single notification to the store that something happened. For example, an action like SaveUserPreferences would be dispatched when the user clicks on the "Save" button.
An action also has an optional payload which will be readable by any action handler.

class SaveUserPreferences {
    static readonly type = '[Preferences] UserLevel.Save'
    public constructor(public payload: Preferences) {}
}

State

The state class is responsible for handling the partial state of the application. It contains an instance of the state model, and action handlers for whichever actions you want.
The action handlers can modify the state model and/or dispatch more actions.

The first part is the state model:

export interface PreferencesStateModel {
    userLevel: Preferences,
    systemLevel: Preferences
}

The second part is the state itself:


@State<PreferencesStateModel>({
    name: 'PreferencesState', // The name can be used to get the state
    defaults: { // The initial value of the state
        userLevel: {},
        systemLevel: {}
    }
})
export class PreferencesState {
  constructor(private prefService: PreferencesService) {}

  @Action(SaveUserPreferences)
  savePreferences(context, action) {
    return this.prefService.save(action.payload).pipe(
      tap(() => context.dispatch(new LogSuccessfulSave()))
    )
  }
}

Store

The store is an injectable singleton that will be the interface of your application with the state. Your template will observe parts of the state, and your logic will dispatch actions through it.

class Component {
    @Select(PreferenceState) state$: Observable<PreferenceStateModel>

    constructor(private store: Store) { }

    clickHandler() {
        this.store.dispatch(new SaveUserPreferences(this.preferences));
}

Introduction over, let's get to work

tl;dr - The end result

There I was, a new framework all to myself, but nothing to do with it yet. Cue cartoon lightbulb: instead of doing a bunch of work just to setup some mock website, instead I could migrate something to NGXS. What better for an Angular framework than the Angular tutorial - Tour of Heroes.

The app has three pages:

The Dashboard Containing a partial list of heroes and a search bar to find a hero by name.
The Heroes Containing the list of all heroes, the ability to delete them, and an input field to add new heroes to the database.
The Hero Displays the information of a specific hero, with the option to change it.

Each component has its own data, each loads "from the server" upon loading, each using the HeroService to perform actions.
One page even uses the Angular ActivatedRoute and Location objects to read the query parameters, and to navigate to other URLs.

I wanted to reach the following:

All data of the website is contained within the NGXS store.
All components use only Actions for the actions the user can perform.

My method is simple: work incrementally, and continuously test against the current implementation. I will take one component and slowly rip out its data and service usage, and replace it with the store and its actions.

How did it go?

I picked the Heroes page, since it is the most straightforward. A list of heroes, add a hero, and delete a hero.
Originally, when the component loads, it performs "GET" via a service, and stores the result locally.
Instead, I've defined a GetHeroes action:

// hero.actions.ts
export class GetHeroes {
    readonly type = '[Heroes] Get Heroes'
}

defined the Heroes state to include a list of heroes, and the action handler that performs the GET and stores the result in the state.

@State<HeroStateModel>({
  name: HERO_STATE_TOKEN,
  defaults: {
    heroes: []
  }
})
export class HeroState {
  constructor(private heroService: HeroService) {}

@Action(GetHeroes)
  getHeroes(ctx: StateContext<HeroStateModel>) {
    return this.heroService.getHeroes().pipe(
       tap(heroes => ctx.patchState({ heroes })
    );
  }

Now the component dispatches the action and "selects" the list from the store. The component template looks at the value of the observable and displays it.

export class HeroesComponent implements OnInit {
  @Select(HeroState.heroes) heroes$: Observable<Hero[]>;

  constructor(private store: Store) {}

  ngOnInit() {
      this.store.dispatch(new GetHeroes());
  }
}

BAM!

Did the same for the Add and Delete: Create the actions, dispatch from the component, handle by calling the service and updating the state according to the result.

BAM!

Without much work, the data and logic were completely decoupled from the component. The service wasn't changed at all, and each handler is incredibly focused on what it needs to do.
I then noticed that the Add handler and Get handler both write to the state. Not good! I created a StoreHeroes action and now the two handlers do even less. Call the API and dispatch a new action with the results.

DOUBLE BAM!

Up to here, using NGXS was amazingly simple, had very little boilerplate code, and resulted in highly decoupled code.

One down, two to go

The easiest page done, I decided to go with the Dashboard next. It will be similar to the Heroes page since it also takes the complete heroes list, only this one manipulates it a little.

Inject the store. Select the heroes stream. Create a secondary stream by mapping the complete list, to the first four items in it. Replace the *ngFor to iterate over the stream instead, and...

KABLAM!

The UI was already set to have it be clickable, and when clicking on a hero, it would route to it's page. That part just worked because I only changed the way the information was bound to the template. It was still the same exact structure. I didn't like having the template handling the route, but I decided to get to that later.

Next step was replacing the search. This would be the first time things were not trivial.
I thought: "Easy... I'll take the complete list and filter it with the search term".
But when I looked at the existing code, I noticed that the search is performed via a server call (or at least a mock server call since it is all in-memory).
Usually, I would take the search term, dispatch an action with it, and wait on the response to populate the state. Since the user can manipulate the search term before the response arrives, that means multiple actions can be dispatched. Luckily, NGXS allows to specify "abort this action handle if another action is dispatched".

@Action(HeroSearch, { cancelUncompleted: true })
  searchHero(ctx: StateContext<HeroStateModel>, action: HeroSearch) {
    if (!action.searchToken) {
      return ctx.dispatch(new ClearSearchResults());
    }
    return this.heroService.searchHeroes(action.searchToken).pipe(
      tap(heroes => ctx.patchState({ heroesSearchResults: heroes }))
   );
  }

Last one...

The hero details page was the most complex (which was not that complex) because it was the only one that enabled the user to modify fields of a single hero. That meant that I couldn't just use the value from the store directly.
I also didn't have the hero to display in the store yet.

The original component would read the hero ID from the route, fetch it with the service, and store it locally for modifications.
When you selected a hero via the search, dashboard, or heroes page, the template would route you to a different URL and put the requested hero ID in it.
But I don't want my components to do that stuff. They should be as "dumb" as possible.
Instead, all the places that changed the route would now dispatch a new action:

export class SelectHero {
   static readonly type = '[Hero] Select Hero';
   constructor(public heroId: number) {}
}

The action handler would fetch the hero with the existing service, save it to the store, and then navigate to the detailed view, same as it did before.

@Action(SelectHero)
selectHero(ctx: StateContext<HeroStateModel>, action: SelectHero) {
  return this.heroService.getHero(action.heroId).pipe(
    tap(hero => ctx.patchState({ selectedHero: hero })),
    tap(hero => this.router.navigate([`/detail/${hero.id}`]))
  );
}

Now the component doesn't need to load anything. The selected hero will already be in the store when the route is changed. All it needs to do is select it from the state. As I mentioned before, to enable editing, the component would need a copy of selected hero. To do that, I just need to subscribe to the stream and save a copy with a tap operator

this.hero$.pipe(
   tap(hero => this.heroCopy = { ...hero }) // shallow clone here is enough
).subscribe();

When the user edits the hero and clicks save, another action will be dispatched - UpdateHero. The action handler will do the actual work and the state will be updated accordingly.

Only one thing left: You could manually route directly into the detailed view without ever having the SelectHero action dispatched. To fix that, the component will still take the ID from the route and dispatch the SelectHero with it, but the action handler will ignore it if that ID is already the selected hero.

@Action(SelectHero)
  selectHero(ctx: StateContext<HeroStateModel>, action: SelectHero) {
    if (ctx.getState().selectedHero?.id === action.heroId) {
      return; // Ignore it. This hero is already selected
    }
    return this.heroService.getHero(action.heroId).pipe(
      tap(hero => ctx.patchState({ selectedHero: hero })),
      tap(hero => this.router.navigate([`/detail/${hero.id}`]))
    );
  }

KABLAMO!

With that, I was done. No component injected any service, all the operations were done via action dispatching, and the entire application state was in the store.
(There was a bit more that could have been done with the message logging, but that felt trivial at this point in the exercise)

Lessons Learned

The incremental approach to migration works well

Especially for state managements where you can slowly add to the state. Starting by defining the migration goals, studying the application, and defining a roadmap, made the process work great.

NGXS has a learning curve

But it is fairly slight curve. The straightforward usage of NGXS is simple and you can start using it pretty well. When you try to get complicated, you'll encounter the finer details of the framework. For example, the fact that the observable returned by the dispatch method will emit the state when the action completes, not the value from the async operation that happens in it.

The NGXS router plugin is limited (at the time of writing)

At some point, I wanted to get rid of the use of the ActivatedRoute and the Router and replace them with a NGXS plugin.
While it was great for navigation and getting parameters passed through the URL, the "back" functionality that exist in the "location" object did not. While it can be extended, I just felt it was not worth the trouble.

Tour of heroes is a good starting point, but...

There are a lot of other features in NGXS that proved unnecessary for this project. The entire action life cycle is a huge feature that does not exist at all in NgRx, that can save a lot of boilerplate code when you want to know if a specific action completed and did it succeed.

Hope you've found this article as helpful as I found it interesting to do.