DEV Community: Meinert Schwartau

Why TypeScripts never type is super useful

Meinert Schwartau — Sat, 12 Feb 2022 07:29:55 +0000

At first glance, the never type does not sound very useful for everyday coding. But actually, the following property of the type comes in very handy:

[…] The never type is assignable to every type; however, no type is assignable to never (except never itself) […]

With a small hack, this can be used to increase the type safety of a code base with no risk of breaking existing code and without any runtime overhead.

Problem

Maybe you have lots of places in your code, where you use some data like an userId, an email, a firstname or whatever. You could just use the standard types for that (string and so on):

getUserByUserId(userId: string): Observable<User>;

But this approach has some problems:

You could accidentally call this method with the wrong parameters, e.g. with an email instead of an userId.
This error would only be found during runtime, not compile time.
Sometimes during refactoring your codebase you want to find all places where a field like a userId is used. This is very hard if it has a simple type, but very easy if it has a custom type.

Solution

The problems described above disappears, if we don’t use the built in string type but create a custom type for the userId:

getUserByUserId(userId: UserId): Observable<User>;

Now it’s easy to find all places where this UserId type is used. And is not possible to accidentally call this method with an email because the latter has another type.

Where does the never type come into play?

How to implement this new UserId type? The naive solution does not work out:

export type UserId = string;
getUserByUserId('some string'); // no compiler errors here :-(

The problem is that for typescript string and UserId are the compatible (because at runtime they are the same). We need to change the UserId type, so that they are (at least at compile time) not compatible anymore in order to get compiler errors. We could create a new UserId class to achieve that. But this might break existing code, which you forgot to migrate and which still expects to get a string, at runtime. But with the help of the never type you can define this new type like this:

export type UserId = string & {
  // name of the following field needs to be unique. If it 
  // were not unique and you would reuse it for another type, 
  // both types would be compatible!
  ____doesNotMatter_UserId: never;
};

// you can't create new values of type UserId by hand, because 
// you cannot assign anything to never. so you'll need this
// converter function:
export const toUserId = (userId: string) => userId as UserId;

This solution has some nice characteristics:

It’s not possible to assign string values to variables of type UserId anymore, because they are missing the ____doesNotMatter_UserId attribute. The compiler will complain about that if you give it a try.
To convert a string to the new type, you need to cast latter to it. Creating a value of this type another way is impossible because you cannot assign anything to never. Normally you’ll have to do this in your tests, during url parameter parsing or (maybe) when converting your REST resources.
I write a factory method like this for the casting normally: export const toUserId = (userId: string) => userId as UserId;. This factory method is nice, because now you can search for the places where UserId values are created.
A big advantage is that this trick does not change the runtime behavior of your code. The sanity check is done only during compile time. Because it’s still a string during runtime, nothing bad will happen if you forget to migrate a place in your code (and which expects a string in our example above).
It can be used for other types too, not only strings.

So let’s see this type in action:

export type UserId = string & {
  // name of the following field needs to be unique. If it 
  // were not unique and you would reuse it for another type, 
  // both types would be compatible!
  ____doesNotMatter_UserId: never;
};

// you can't create new values of type UserId, so you'll need 
// this converter function:
export const toUserId = (userId: string) => userId as UserId;

// an example function which expects our new type
export const isSuperUser = (userId: UserId) => 
                                   userId === 'superuser';

// a function we forgot to migrate and which still get's a string
export const unmigratedIsGuestUserFn = (userId: string) 
                                       => userId === 'guest';

// some examples
const aUserId: UserId = toUserId('u1111');

isSuperUser(aUserId); // works and return false

isSuperUser('superuser'); // Compile time error: Argument of type
 // 'string' is not assignable to parameter of type 'UserId'. 
 // Type 'string' is not assignable to type '{ 
 // ____doesNotMatter_UserId: never; }'.

unmigratedIsGuestUserFn(aUserId); // no compile time errors 
// because at runtime our new type is just a string :-). 
// So this type refactoring is very safe

// warning - don't reuse your ___doesNotMatter attributes:
export type Email = string & {
  // DO NOT DO THIS:
  ____doesNotMatter_UserId: never;
};

//  Email and UserId are compatible because they have the same
//  signature, so we don't get compiler errors here:
const email: Email = toUserId('superuser')
isSuperUser(email);

// So -> always use unique names for these 
// ____doesNotMatter attributes

Conclusion

I found this approach to be especially useful, if values of a type are used in many places but are created only in few cases. For instance, I expect that I would only need to call the factory function toUserId to convert strings to the new UserId type during:

converting a rest resource to our domain model (should be one place)
parsing a url parameter (should be few places)
in tests But there might be lots of places, were a function gets a value our new type as parameter and just passes it around. So through this approach I get the former described benefits of a richer type model.

Acknowledgements

I learned about this trick during an Angular meetup held by Thiele Leonard in 2020. I found a recording of the same „Angular at scale“ talk hold on a different occasion, which contains many more tricks, so have a look.

Functional programming + DDD = ❤

Meinert Schwartau — Thu, 25 Feb 2021 18:13:08 +0000

Using Domain Driven Design (in short DDD) it is common to follow the principles of hexagonal, onion or ports and adapters architecture. This means that your domain logic should be independent from how it is connected to the outside world. For instance you shouldn’t see any messaging or REST specific logic in domain logic. Even though this sounds easy, in past projects I came across loads of messy code when connecting the outside world (like a REST Endpoint) with the domain. Doing it in a more functional way using functions solves a lot of problems and leads to cleaner code. I want to share this in this blog post.

The problem

Data always has to be mapped. There often exists something like the following:

Especially if your are using java and JPA you have to differentiate between creating new objects or updating existing ones. This is necessary because in the same JPA context there should exist only one object with the same id.
When your domain object has a list of other objects (e.g. your order may have a list of orderpositions), you don’t want to throw away the whole list and create a new one which may lead to lots of deletes and updates just because the customer wanted to increase the amount. Instead, only the relevant orderposition entry should be updated in the database and the others left unchanged.
When your domain object has a list of other objects (e.g. your order may have a list of orderpositions), you don’t want to throw away the whole list and create a new one which may lead to lots of deletes and updates just because the customer wanted to increase the amount. Instead, only the relevant orderposition entry should be updated in the database and the others left unchanged.

So what’s the problem? The problem (at least in Java) is how to convert between your adapter model (REST resource or message) to your domain model. Having to read the domain object from the database in a class which converts the REST resource to your domain object is very cumbersome. Furthermore you may have duplicated code in other parts of your code, e.g. where messages are parsed and converted to your domain object. Last but not least I normally don’t want to care whether the objects exist or not when I write such converters. The domain logic should handle that.

Some messy attempts

In the past I saw the following approaches:

Writing intermediate objects. Instead of mapping something like your REST Resource to your domain object, you map your REST object to an intermediate object and pass this to your business logic. The intermediate object can be created by different adapters. This leads to writing lots of mapper code and you are unable to use the fancy domain methods which you have written. Furthermore naming this intermediate objects is often hard which is a bad sign.
You pass both, the domain object and the REST Resource, to your converter. This leads to an ugly / more complicated interface of your services.
During the transformation, the converters lookup the domain objects in the database itself to handle the case of updating versus inserting them. This leads to the disadvantages mentioned before. In some cases the code is not too bad, but can become very ugly. At least this lookup code may be duplicated in different converters.

The better solution

The solution is simple, just pass functions. Your converters in the different adapters don’t create or update the domain object immediately. Instead they return a function which expects the domain object as input and updates it. This function is then passed to the domain logic which then applies it and hence updates the domain object. Such a converter may look like this:

// these lombok annotations if you don't know them are generating 
// gettes and builder methods ...
@Builder 
@Getter 
public class CatalogArticleUpdate {
     // the functions which update the domain object, may never be applied!
     // You create an own interface which extends Function<> if you like
     private final Function<Article, Article> articleUpdate;
     // some other fields which the domain services needs to figure out to which 
     // domain object, the update needs to be applied  
     private final Catalogue catalogue;  
     private final ExternalDocId externalDocId;  
}


public class CatalogueAricleChangedMessageConverter {

    // this method does not convert the message immediately but instead returns
    // a function which does this when it is applied
    public CatalogArticleUpdate convert(final CatalogueAricleChangedMessage message) {        
        return CatalogArticleUpdate.builder()
                .catalogue(Catalogue.valueOf(message.getCatalogueId())) 
                .externalDocId(ExternalDocId.valueOf(message.getDocId()))
                .articleUpdate(article -> {
                    // if the domain service calls this method, it provides the domain 
                    // object (see article parameter) so we don't need to lookup 
                    // it ourselves :-)
                    article.updateTitle(ArticleTitle.valueOf(message.getTitle())):
                    article.updateCataloguePrices(convertePrices(message));
                    // much more updates ....
                     return article;
                })
                .build();
    }
}

This approach has several advantages:

More business logic can be put in the domain layer.
The domain layer can even decide to ignore the update. In that case the update function is not executed. For instance, in my current project we get a lot of price and metadata updates from a catalogue which contains lots of articles. For our service, only a small amount of these updates are relevant. So the domain service may decide to ignore an update. Or it may decide to create a new one. Or it may decide to update several domain objects in the database because there exist different ones for different tenants. The point is, this logic how to apply the update belongs into the domain layer
Code in the domain layer may be reused. The example above creates a CatalogArticleUpdate for a message. A batch job, for instance, may load updates via feed and converts the result from the feed to an instance of CatalogArticleUpdate and calling the same domain service.
No unnecessary intermediate objects are needed so there is no need to write mapping code for them.
Very flexible, all of your fancy domain methods can be used.
Easy to test, just apply the function to something.
The resulting code is not so cumbersome.

In my current project we have used this style to just return functions instead of doing the conversion in the converters immediately for some time and it turned out nicely.

Troubleshooting karma (node) performance problems

Meinert Schwartau — Mon, 25 Nov 2019 20:25:03 +0000

After some months of nothing but backend development I started to work on my customers Angular frontend application again. I noticed that starting a single karma test took way longer than it used to be. A simple test for a pure function in headless chrome took about half a minute, in which the actual test only took some milliseconds. That was really annoying, because in my current refactoring task I have to execute lots of tests very often. Searching, browsing the karma issues and so on didn’t help, so I want to describe, how I found out what has been causing the problems.

1. Find out where the time is spend

At first it is necessary to figure out, where the time is spend. Fortunately node has a really good built in profiler. Their simple profiling guide is quite good and describes how to find out the problematic calls. I won’t repeat its content here but assume that you read it or are familiar with profiling.

So I followed their guide and added the --prof flag to the node options (in intellij this option can be added in the run configuration):

If you are using the command line, you could add it there too:

node --prof ./node_modules/@angular/cli/bin/ng test ...

After execution of the test a isolate-<some integer>-v8.log file is created in the project folder. This can be feed into node to create a report from it:

❯ node --prof-process ./isolate-0x10264e000-v8.log
...

ticks parent name
1327 44.1% T _fcntl$NOCANCEL

728 24.2% T v8::internal::IncrementalMarking::Step(unsigned long, 
v8::internal::IncrementalMarking::CompletionAction, v8::internal::StepOrigin, 
...

The simple profiling guide in the documentation gave me some tips how to interpret this output. But basically we see here that 44,1% of the time is spend in a C++ method called _fcntl$NOCANCEL.

2. What is the method doing?

Now we need to find out what this heavy load method is doing. Fortunately, this is a very specific search term, so using Google I found a stack overflow answer which said that _fcntl$NOCANCEL is related to file system operations.

3. Why is the method being called so often?

Such a simple test shouldn’t lead to lots of file system operations. Now we need to find out which files are accessed so heavily. Unfortunately the karma logging isn’t really helpful and we don’t know if we can trust it. Fortunately for linux there exists strace and for mac os (which I’m using) there exist dtrace and dtruss. Basically these programs just show you the system calls your program executes and their parameters. Hereby you don’t have to rely on the logging of the program but you can see what is really going on. This is not a strace or dtrace guide, but there exist a lot of how to guides out there (like this one for dtrace / dtruss for mac os).

There are two dtruss options which we will be using:

-f <command> When this option is provided, dtruss follows child processes. We’ll use this option to start the node process. Even when the node process starts child processes, we’ll see the system calls they execute.
-t <syscall> We know that file system operations slow down the test execution. If we would log all system calls our process is executing, we would probably see nothing. So we’ll use this flag to only examine the problematic file open_nocancel system call.

To only execute the single test, we annotate it with fdescribe. Afterwards we start the test via dtruss like this and see at lot of accesses to scss files (we even see which are accessed):

❯ sudo dtruss -t open_nocancel -f node ./node_modules/@angular/cli/bin/ng test lisa
# ...
82846/0x11de5b: open_nocancel(".../generated_material_theme.scss\0", 0x0, 0x1B6) = 29 0
# ....

Other than that, only the test files are accessed. So our stylesheet scss files seems to be the cause for the problem. That the tests even needs the scss Stylesheet files is unexpected, because our unit test just tests the code and the html template. So there ist no reason why the Saas scss files should be compiled.

4. Verification

Now we need to test, if the performance will be faster without these Saas scss files. Even though they aren’t needed by the tests they are referenced by the Angular components so we can’t simply delete them. The fastest way to verify our hypothesis that they are causing the problems is to overwrite them with empty files. This can be done by using the following bash command:

find . -iname "*.scss" -exec sh -c 'echo {}' \;

Now I compared the execution time of the tests:

Before with the original scss
- Executing all tests via command line took 6min and 20 secs
- Executing a single test via Intellij took 20 secs
With empty scss files
- Executing all tests via command line took 2min and 30 secs
- Executing a single test via Intellij took 6 secs

So it is three times faster. The single test execution in Intellij is acceptable now.

5. The quick fix

I already spent too much time on the issue. As a quick fix I wrote one function which just overwrites the scss files so that they are empty. And another one to restore the scss files, because I need them if I want to run the Angular application. I added these functions to my .zshrc (or .bashrc if you are using bash instead of zsh) so that I can execute them in my terminal:

function scss_empty {
  local scssFileName
  for scssFileName in `find . -iname "*.scss"`
  do
    local newFileName=${scssFileName//\.scss/.original_non_empty_scss}
    mv -v $scssFileName $newFileName
    echo > $scssFileName
  done
  echo "scss renamed. restore original files via scss_restore."
}

function scss_restore {
  local fileName
  for fileName in `find . -iname "*.original_non_empty_scss"`
  do
    local newFileName=${fileName//\.original_non_empty_scss/.scss}
    mv -v $fileName $newFileName
  done
  echo "scss restored. "
}

Before test driving my code, I execute the scss_empty function to overwrite the scss files. Before committing or before starting the Angular application, I excecute the scss_restore function. Our build pipeline now always overwrites the scss files before executing the unit tests, which speeds up the build by several minutes.

There might exist other solutions. I found no easy one like just setting one option, but in one stackoverflow thread someone suggested to change the resolver for scss files to return only empty strings if tests are executed. But that we’ll try out in future in a dedicated task. The quick fix solves my immediate problem so that I can execute the tests fast enough and often.

6. Conclusion

Thanks to the tools provided by node and thanks to the good simple profiling guide, finding the culprit method is very easy. Thanks to strace for Linux or dtrace / dtruss for mac os is a very easy way to see what’s going on – even when the logging of the node program like karma is not sufficient. It’s nice that you can even see the parameters of the system calls so that, for example, it is obvious which files are accessed. Especially the strace or dtrace / dtruss tools are tools to have in you sleeve.