DEV Community: Egor Pavlikhin

Dockerize Angular 9-10 App With Cached Dependencies

Egor Pavlikhin — Fri, 14 Aug 2020 02:12:47 +0000

I've had a perfectly fine Angular app built into a Docker container image, which became terribly slow to build after upgrading to Angular 9, with build process recompiling dependencies with ngcc every single build.

Compiling @angular/core : fesm5 as esm5

Here are the steps that you need to take to make your builds fast again.

1) Add .dockerignore file

node_modules
.git
.gitignorenpm

2) Add default.conf (this is for your nginx server)

server {
  listen 80;

  location / {
    root /usr/share/nginx/html;
    index index.html index.htm;
    try_files $uri$args $uri$args/ $uri $uri/ /index.html =404;
  }
}

3) Add Dockerfile

### STAGE 1: Build ###
FROM node:14.8.0-alpine AS build

WORKDIR /usr/src/app
ENV PATH=${PATH}:./node_modules/.bin
ENV NODE_PATH=/usr/src/app/node_modules
ADD package.json ./
ADD package-lock.json ./
RUN npm ci
RUN ngcc
ADD . .
RUN ng build --prod

### STAGE 2: Run ###
FROM nginx:1.17.1-alpine
COPY --from=build /usr/src/app/dist/web /usr/share/nginx/html
COPY default.conf /etc/nginx/conf.d/default.conf

4) Profit. Run docker build . and enjoy. NGCC will cache built dependencies now and subsequent builds will be as fast as usual.

6 "Things-I-Wish-I-Knew" After 15 Years As A Programmer

Egor Pavlikhin — Fri, 31 Jul 2020 16:04:27 +0000

1. Fundamentals are key

Everyone loves to try new and fancy technology and just start hacking. Even today I am still guilty of this. However, it's been proven over and over that learning fundamentals is far more productive. This applies both to general programming concepts and specific frameworks and libraries. If you understand how something works conceptually not only can you deduce how 80% of the thing works, but also fix issues far faster than desperately searching StackOverflow.

The thing about learning fundamentals is that it is ultimately boring. Often you have nothing to show for it for the longest time, and some concepts will be completely alien and useless to you. The important thing is to know what you don't know. All you need to do is remember where to go back to for information once you really need to. You are severely limiting yourself if you "don't know what you don't know".

Once you hit that "oh I know the perfect solution for this, I've read about it here and here!" there will be no further need to prove that fundamentals are key.

2. Know your own and your code's limits

Perfect code doesn't exist, just like perfect software doesn't exist. Software is never finished and neither is code. Therefore it's imperative to understand why your code is imperfect, how it could be better, and, most importantly, what the limits are. What can your code do really well and when does it perform poorly?

It is hard to predict how your code is going to be used in the future, but if you analyzed the corner cases it will be much easier to estimate how difficult future changes are going to be.

On the other hand don't be afraid to admin defeat when you don't know something well enough to implement in a reasonable time frame. When I was just starting out I tried building an interactive HTML tree for constructing a multi level menu of a website (that was before jQuery was a thing and IE had 90% browser share). It took me a week just to build the basic tree with drag-and-drop and one more week to realize the whole thing is way too brittle and will take months to bug fix. It would have save 2 weeks of my life to just admit that the task is too difficult and a simpler solution (such as limiting tree depth to 2 or 3) would be more appropriate.

3. Mature, respected libraries have bugs

Just recently I have encountered a bug in .Net tooling for web services that has existed for half a decade and is likely never going to be fixed. There are bugs in MySQL that still have not been fixed after more than a decade. Some bugs exist for 25 years before getting fixed.

These bugs are the most frustrating of all - you never suspect a mature library, you suspect your own code. These occurrences are not as frequent these days as 10 years ago, but they still happen and you should be ready for them. Sometimes the library doesn't work and you need to re-implement the same thing yourself (ideally make a pull request and fix the bug in the source).

Funnily enough, StackOverflow is littered with answers that recommend a library, when the original question originated because of bugs in said library. I have at least one such question.

4. Code quality is in the eyes of the beholder

No code is perfect. Some is really good, but perfection is too subjective. There is no point in blaming yourself or, worse, your colleagues for poor quality, unless you can produce a measurably better solution. And no, just because something looks better does not mean that it's a better solution.

Chasing perfection often leads to unnecessary abstractions that end up never being used by any other code, but the original use case it was designed for. Writing code that is easy to refactor later on is far more important. Follow a consistent style with your teammates, use abstractions already available in your framework of choice and stick to SOLID and in most cases that will be enough.

5. Complex subjects are easier than you think

We all learn "divide and conquer" as a strategy for solving programming tasks. This strategy works equally well for learning new things. A new tech or idea looks overwhelmingly complex? Split it into smaller chunks of unknowns. Make a list of small things that you need to learn first, before getting to bigger concepts. Each point should take no more than a day to learn.

These days it's a lot easier to do, with websites like Pluralsight, where content is broken down into streams, which are broken down into courses, which are further broken down into modules and chapters.

This might seem obvious - indeed educational content has been done like this since the beginning of time. However, it is too easy to overlook when you are educating yourself and not consuming a prepared course. Not every course or book is structured well, and they cannot know what your level of knowledge is, so be prepared to go on a search of your own - see point # 1.

6. Setup CI/CD first

Automated build process saves everyone so much time it should always be the first step in setting up a project. Even if it saves you just 5 minutes a day over a year it's about 3 working days saved every year. Reality is that not only it saves you 5 minutes doing the same thing over and over, but it also eliminates a lot of human errors and inevitable deployment bugs.

Automated deployment is even better, especially if you have multiple environments and automated verification process. OctopusDeploy is the best solution in this category. It's so good it should really be in its own category.

And don't forget that everything should be automated, including database deployment. Here my favorite solution is DbUp, it's quite specific to .Net world, but the general concept is amazing in its simplicity. I have tried several DB migration and change tracking tools over the years (DataGrip is one of the best for developers), but DbUp really nails the concept as it creates easy to follow, repeatable process that is completely transparent to everyone - developers, IT, DBAs.

Validation of nested form inputs (within components) in Angular

Egor Pavlikhin — Wed, 25 Mar 2020 01:01:58 +0000

Mostly writing this for myself, so I don't forget. Google articles for this topic usually show huge walls of text for more complicated cases, but that's not what you'd normally need.

Problem: when you have input fields within a component template the parent form will ignore these fields for validation purposes. So, a structure like this:

-- main.component.ts
<form #angularForm="ngForm">
  <app-nested-inputs></app-nested-inputs>
  <input type="submit" [disabled]="angularForm.form.invalid" />
</form>

-- nested-inputs.component.ts
<input name="xxx" [required]="true" />

The submit button will still be active, even when the form is invalid, unless you add the following to nested-inputs.component.ts Component directive

viewProviders: [ { provide: ControlContainer, useExisting: NgForm } ]

So the Component directive will looks like this:

@Component({
  selector: 'nested-inputs',
  templateUrl: './nested-inputs.component.html',
  styleUrls: ['./nested-inputs.component.scss'],
  viewProviders: [ { provide: ControlContainer, useExisting: NgForm } ]
})

Just a little tip to save you a few hours of headache :)

Map JSON object to Controller Action with multiple parameters

Egor Pavlikhin — Tue, 10 Mar 2020 05:47:06 +0000

Last week I was surprised to find out that .Net Core still doesn't support something that every JavaScript/TypeScript framework does: mapping JSON object to multiple action parameters just isn't supported. You are supposed to define a Model class, add it as a single parameter and mark it with [FromBody].

{ "a": "a", "b": "b" }

This just didn't sit right with me. Why declare a whole new class for every single Action with just a few parameters. What do I need to do to have a controller method as simple as this?

[HttpPost]
public async Task SometMethod(string a, string b) 
{
  return Ok();
}

I couldn't find an answer, so I wrote my own package to do that.
You can download it here or view source here.

It's very simple, and quite slow for now (as it re-parses the whole request for every parameter), but does the job for simple cases.

Usage

Given JSON

{ "a": "a", "b": { "c": "value" } }

Step 1.
Add attribute JsonParameters to your action

[HttpPost("two")]
[JsonParameters]
public async Task<IActionResult> TwoParameters(string a, dynamic b)
{
    Console.WriteLine(b.c);
    return Ok();
}

Step 2.
Enhoy simplified workflow.

Thanks

Thanks to tchivs for providing code to map to dynamic types

Postgres Arrays: JOIN table with matching ids

Egor Pavlikhin — Thu, 13 Feb 2020 22:04:39 +0000

Postgres has a wonderful feature that allows you to store any data type as an array. You can expand this to represent a one-to-many relationship and use it like so

SELECT u.*
FROM unnest(ARRAY[31,32,410]) user_id
LEFT JOIN user u on u.id = user_id

Be aware of potential issues when working with arrays though. While they are good if you want to store point-in-time data where you "write and forget", maintaining such data model can be troublesome:

No foreign keys support for arrays
Limited support in ORMs and libraries
To update an array you need to re-write all of it

Temporal tables or JSON storage

Egor Pavlikhin — Thu, 13 Feb 2020 04:00:55 +0000

Problem Definition

Imagine you run a small business and have a simple e-commerce system with Products, Orders and Customers: a customer places an order with a bunch of products inside. You ship the order and mark is as Done. The Customer then logs in and changes their shipping address. You go back to your owner's dashboard to look at past orders and see the new address as the shipment address, although it is clearly wrong.

I want to look at just one possible solution to this problem (in context of relational databases, specifically Postgres), which is to store all data relative to the order somewhere close to it.

Solution 1 - Add columns to Order table

First thing we could try is to denormalize the data and store everything related to an order directly in the Order table. This is quite straight forward and easy to find tools to support it with.

Advantages

Naturally aligns with well supported table structures.
Data is strongly typed and easily explorable - you always know what you will get back.
Easy to query and analyze.

Drawbacks

Hard to support changes to the way customer or product data is stored. Columns moved, deleted or added to the original tables might create problems in historic data.
Similarly, if product is deleted it might be hard or impossible to derive analytical data about it.
Only values are stored - schemas have to be versioned externally (this can be mitigated through Views).

Solution 2 - Temporal tables

Temporal tables are quite powerful, but unfortunately poorly supported in open-source database engines. There are extensions that exist for Postgres, but it's hard to say if they are production ready and there is definitely no official support.

The idea is to have a historic table next to data tables (so Customers and Customers_History) with the data table holding the current version of the data and historic tables storing all the rows that have ever existed with a START (CREATED) and END (CHANGED) dates.

It then becomes quite easy to link a row in the Orders table back to a specific Customer record in any point in time.

Advantages

Normalized data. Everything lives where you expect it to live.
You can get a snapshot of the entire data set at any point in time or exclude certain periods, which can be very powerful for analytical purposes.
Audit and historic data retention is the natural purpose of temporal tables.

Drawbacks

Hard to support changes to the way customer or product data is stored. Columns moved, deleted or added to the original tables might create problems in historic data.
Poor or no support in Postgres - you will have to support this manually and there is no perfect option.
Querying data becomes more complex.
Only values are stored as historical data - schemas have to be versioned externally (this can be mitigated through Views).

Solution 3 - JSON blobs

At last we arrive at my favorite solution - storing snapshots of data as JSON. The possibilities are endless here, but essentially the proposition is to have one column per each data type (Customer, Address, Product) and store all related data in plain JSON object. JSON data can even be versioned or you can store JSON Schema next to it.

Advantages

Postgres has excellent support for JSON data in jsonb type. It's fairly easy to query.
Data can be versioned directly inside the columns.
All relevant data is immediately available on the relevant row. If your front end is using JSON you can pass this data directly from DB with no processing.
To simplify analytics virtual columns / views can be easily created.

Drawbacks

Data in JSON blobs can be inconsistent. You will have to look after it yourself. Most likely you will need to store metadata about each version externally.
Queries will be slower. Much slower. And somewhat harder.
Analytics are harder and most out of the box solutions will likely not work.
DB size will grow significantly, since data will be essentially duplicated for each order. And hopefully you have a lot more orders that products and customers.

Conclusion

For me the clear winner is JSON - my back-end is in NodeJS / NestJS and front-end is in Angular, so manipulating JSON data comes naturally, but I can see how the first option might be attractive to traditional applications.

What would you choose?