DEV Community: Adrian

Do you really need Dependency Injection? OOP roundtrip journey.

Adrian — Sat, 17 Feb 2024 15:10:05 +0000

What is the idea?

Through my development career I've seen many codebases written in various ways. I remember struggling with code architecture and slowly learning what belongs where and why. Once I reached the full object oriented programming way of doing web development I thought that this is it, this is the way forward now.

But after some time I started having ideas for improvements, noticing things that are very rarely done but are enginnered to make possible. Turns out a lot of the OOP weight can be cut down, and it can go as far as removing all OOP altogether.

Let's see what we can trim down, but first, some background.

TLDR

Here's the simplified architecture without DI:
https://github.com/adrian-afl/oop-to-fn-experiment-blog-result

How it was before Nest.js (or Spring, Symfony) OOP web dev

Before OOP, my approach to architecture was very messy. I think code like that can be found everywhere, basically things like executing domain logic or persistence actions at the controller level, or scattering the code in various utility files with no general idea what should go where.

Code in this stage is very hard to adjust. If one day a requirement appear to add CLI interface to the app, or freeze the api for some external app, can make the team very frustrated about the architecture.

Main issues here are tightly coupled code layers, inflexibility and no idea about the code layout structure, where files are and what should be contained inside.

How it was like with the full OOP

After some time I discovered frameworks that at least try to guide their users about how to lay down the files and what are the general thing to do in most scenarios.

There is an essential tools in most of those framework, and that also makes them pretty similar, and it is dependency injection.

Let's analyze how dependency injection, for short, DI, is so awesome and why it looks like such a good idea.

The idea is that all code that do things is saved in form of classes, then classes are automatically instantiated and used. DI framework will take care of instantiating all classes in the correct order.

But the crucial thing is that if you want to use code from another class, for example, using user data access in some user logic class, you put it in the constructor as a parameter. You never call this constructor yourself, this is the DI job to read the content of the constructor and then provide correct dependencies based on this, all automatic.

Another crucial thing is that, in most cases, injecting via an interface is possible as well. This works well for alternate implementations, for example, various email sending services. If there's only one class implementing the interface in the app, it's usually automatically injected, if there are more you need to specify which will be used.

Looks awesome, but we will see later how it is in real life.

Usually, code in this stage is structured in a way, that there is a separate api layer, and separate logic layer. For example, a typical feature module would look like this:

userModule:
  controllers:
    UserController class:
      constructor(UserService)
      getUser(...)
      createUser(...)
      updateUser(...)
      ...
  dtos:
    UserDto interface
    CreateUserDto interface
  services:
    UserService class:
      getUser(...)
      createUser(...)
      updateUser(...)

In this layout there is a slight problem that the user controller and user service actually does multiple things at once. This interestingly causes another problem, confusion what should be where when the user logic grows more.

For example, you can end up with a method in user service that takes 150 lines to do the thing needed, and should you extract it? Based on what metric, where is the threshold when doing many things is too many already, and move to what? Another service? This is problematic because developer needs to think about it, rather than just know where to put it in the first place.

Going very strict with SRP

Multiple things done by services problem can be mitigated by strictly following the Single Responsibility Principle.

This will change the code a bit, because now a class is allowed to do only one main thing and handle all other related stuff.

So now we have separate classes for each endpoints, separate classes for each logical action, and dtos are gone because those are part of controllers now:

userModule:
  controllers:
    GetUserController class
    CreateUserController class
    UpdateUserController class
      ...
  logic:
    GetUser class
    CreateUser class
    UpdateUser class

This also means that most of the times, almost always, the classes will have only 1 public method, and you might want to call it the same always, for example execute.

This code also puts the related data interfaces for each class inside the class file, so you get the input UpdateUser data interface inside the file.

Now the developer knows where to put the code at first glance, everything is structured in a way that there is no doubt where something should go.

This is mostly very nice, is such a pleasure to test and generally code like that looks good and is easy to follow.

But I had some ideas...

Domain classes simplified

Now this part will be a bit more related to languages that use duck typing and imports, like TypeScript, but otherwise no worries, it applies to other languages too.

To understand what will happen we need to talk about some observations with DI first:

in DI, most of the times all of the instantiated classes are singletons, there exist only one instance always
swapping implementations around with DI is very rarely done in real like, it's often implemented and handled correctly but then never used, so the effort is wasted
adds a lot of verbosity in the multiple classes approach
injecting by interfaces, apart from swapping implementations is almost never done too especially for domain code, and if done, would involve immense amount of verbosity by requiring interfaces for every single class

So, now we have a lot of classes that have dependencies and only 1 public method.

And here is the main idea.

What if the dependencies were imports and the public method was the only one exported method, and the class would be gone?

This way a class turns into a method, and imports do the injecting. And suddenly if all classes are rewritten this way, no more need for DI for domain code.

Tearing down the api code

After this transformation was done for logic code, I tackled the controllers as well, and this went a bit technical, but thanks for Fastify and Zod I managed to create a very simple api layer with automatic validation that doesn't require DI as well.

And at this point, my backend code works like it is using a framework, but there is no framework involved, there is not even something like "our own framework", rather the code is just gone completely, not needed anymore.

Final minimalistic result

To test this hypothetical architecture approach, I decided to adjust my bootstrap-like user management project backend, this is a simple app that manages users, sends emails, verifies, etc.
This is the result:
https://github.com/adrian-afl/oop-to-fn-experiment-blog-result

It's similar to the last one but the classes became files with functions

userModule:
  controllers:
    getUserController fn
    createUserController fn
    updateUserController fn
      ...
  logic:
    getUser fn
    createUser fn
    updateUser fn

Pros of the final result

Removal of several big framework dependencies
Code greatly simplified
Architecture is very clear to follow and use
More control over what is happening
Less abstraction layers on api level - no framework involved in request handling adding unnecessary complexity or abstractions
Can apply layer boundaries with linter restricting imports

Cons of the final result

Sometimes import order will matter if the code inside executes immediately, for example, config files that read env variables immediately, but this is easily resolved anyway
Swapping implementations is no longer that easy and requires more work, but as I mentioned, this rarely happens anyway, and ever if needed would be very simple to add
There is some code needed to instantiate the api or other entrypoints that needs to be maintaned, but its mostly simple and gives much more control over handling anyway
Mocking can be problematic depending on language, but inside Node.js and using Jest this is not a problem at all, just mock the function and done

Final Remarks

I think this is good way to structure the code and to remove unnecessary dependencies. Give it a try in your next prototype and tell me what you think!

Are pre-commit git hooks a good idea? I don't think so.

Adrian — Sun, 17 Dec 2023 16:36:05 +0000

Here's a little rant from a technical owner perspective.

I've seen this at various places. A senior developer sets up a repository, everything is well and all right, the main branch is protected and there are CICD pipelines set up. Everything is fine. And then, with a little thought given to it, pre-commit git hooks are set up.

Obviously, I mean the hooks that run on your very machine, before you can actually commit stuff.

Is it a good idea? Maybe it's a bad and micro-manager-like way to power flex? Well, let's see.

The good intentions

I get it. This option looks attractive on the surface. Here are points that look all right at the first glance.

Git history looks cleaner

This point is stupid because you will squash pull requests anyway so any weird commits not passing tests are removed.

Save time on CI

Fair point, you will save some time, but are you really that tight on budget that you can't spare some seconds?

If this is a problem, maybe your pipelines can be configured better. Follow the fail-early method, run linter first, then units, then integration, and so on.

Ultimately, the developer working time is much more expensive, and with pre-commit hooks you lose a bit of this time.

Force developers to test before pushing

This point is distrust in your workers, just disguised. If you don't trust your developers, fire them, or fire yourself.

The bad reality

Usually, pre-commit hooks are a terrible idea. To clarify why, here are some my thoughts on this topic

Most likely your tests suite doesn't run in seconds

Tests can be speed up (see my other articles) but usually if you are working on non-trivial code base, the tests will take a considerable amount of time. Maybe it's 5 minutes, maybe 10, who knows, but you really do not want to do it before commit.

You already have CICD that will run exactly the same suite, why bother running it locally? Use the resources you yourself set up.

Developers will be either afraid of commiting often or will just skip this stupidity with --no-verify

This is a very important point, maybe even the most. You want your developers to commit often. This allows more experimentation, which can be reverted if they want, at any time. With long hook any commit is a long investment of time, nothing can be done in this time as well.

It will hinder your developers performance because of that. They will make less commits, make big commits, and will not be able to tests various solutions to a problem with easy way to revert them.

Are you sure tests can be run locally, or you only pretend?

Sure there can be projects that are set up correctly that everything can be run locally.

But even more often, nowadays, you will encounter absolutely rubbish setups that require, for example, AWS credentials to run tests.

Not only this makes it impossible to work locally, for example in a plane or on a train without internet, but also means that developers that are joining the project will need to wait for accesses to remote systems before even trying to commit anything! Hilarious situation, and complete waste of time.

Most likely one developer will not completely overhaul the whole system, in a way that every single test must be run locally when developing that refactor. Most of the time developer will work on one feature, quickly iterate with tests just for this feature, and once it's ready, will commit and push. Only then on CICD the whole suite will run, and your pipelines are already well configured for this task.

I should probably write a checklist style article about how to set up a project correctly. Follow me, it will happen soon.

Scope of testing in very big projects

Sometimes you encounter a monorepo with 10 subprojects inside it. Why the heck would a developer even want to run tests of completely unrelated projects when developing a small change in one of them? Again, waste of time. Anyway, CICD will validate stuff in the end.

Not to mention such monorepos are a stupid idea too, why group unrelated projects inside one single repository? There is no good answer to that.

Trust issues

Do you really want to show that you do not trust your developers to do the right thing? You should want them to experiment, push half working stuff to their branches, see the outcomes, who cares whats going on in their experimental branches.

The end goal is the same, working code, with passed pipeline tests, merged into main/dev/whatever.

By micromanaging your developers by enforcing pre-commit hooks you show that you don't trust them. What's the next step? Watching them code via screen sharing? Believe or not, I've seen that.

Final words

Just don't. If you already made a mistake and set up this feature already, remove it. Everyone already does --no-verify anyway.

Detect what calls your Node.js code

Adrian — Mon, 11 Dec 2023 16:30:02 +0000

While most of the time it's actively discouraged to detect and react to how your code is called, there are times when you would really want to know how the execution came around calling it.

Examples when this would be very useful are:

Logging libraries that autodetect and describe the caller
Invisible caching for example if you were to reimplement useEffect on the backend

I'm sure you will find more creative ways to utilize this madness. Let's jump in and see how to do it.

How is this possible

The trick is to utilize V8 stack capturing functionality. No errors are thrown here, just a simple V8 function is being called to get the stack, and then some parsing happens to make the result useful.

Prepare

Let's first set up a function with following signature:

export function getExecutionPath(
  filterDir = process.cwd(),
  limit = 10
)

filterDir - will be used to filter out calls from outside your code, for example, from node_modules or linked libraries
limit - how far into the execution history we will reach

Capture the stack

Now get the most important part, a fake Error stack trace:

const inobj = { stack: "" } satisfies { stack: string };

const oldLimit = Error.stackTraceLimit;
Error.stackTraceLimit = limit;
Error.captureStackTrace(inobj);
Error.stackTraceLimit = oldLimit;

This code sets the stack limit from the function parameter, captures the stack trace at this very point and restores the original limit.

The result is in pretty rough format:

Error
    at getExecutionPath (C:\Code\server-side-state\src\state\getExecutionPath.ts:23:9)
    at new SynchronizedStateful (C:\Code\server-side-state\src\state\state.ts:27:33)
    at state (C:\Code\server-side-state\src\state\state.ts:90:24)
    at Index (C:\Code\server-side-state\src\test\Index.tsx:7:20)
    at C:\Code\server-side-state\src\server.ts:18:25
    at Layer.handle [as handle_request] (C:\Code\server-side-state\node_modules\express\lib\router\layer.js:95:5)
    at next (C:\Code\server-side-state\node_modules\express\lib\router\route.js:144:13)
    at Route.dispatch (C:\Code\server-side-state\node_modules\express\lib\router\route.js:114:3)
    at Layer.handle [as handle_request] (C:\Code\server-side-state\node_modules\express\lib\router\layer.js:95:5)
    at C:\Code\server-side-state\node_modules\express\lib\router\index.js:284:15

We need to process that to make it useful.

Parse the stack

First run some regex on the lines to get only lines starting with at and extract the identifier and its location. At the end remove the call of our detection method from the results and reverse the order.

const detectorRegex = /at (.+?) \((.+?)\)/;

const stack = inobj.stack
  .split("\n")
  .map((x) => x.trim())
  .filter((x) => x.startsWith("at"))
  .map((x) => x.match(detectorRegex))
  .filter((x) => !!x) as RegExpMatchArray[];
stack.shift();
stack.reverse();

Then, transform the array of matches by parsing those strings inside. Get the identifier, and parse the second parameter to get the path, line and column in mre useful form.

const detections: StackEntry[] = stack.map((x) => {
  const identifier = x[1];
  const pathlinecol = x[2].split(":");
  const column =
    pathlinecol.length > 2 ? parseInt(pathlinecol.pop() ?? "0") : 0;
  const line =
    pathlinecol.length > 1 ? parseInt(pathlinecol.pop() ?? "0") : 0;
  const p = pathlinecol.join(":");
  return { identifier, path: p, line, column };
});

After those steps, the detections array looks like this:

[
  {
    identifier: 'Layer.handle [as handle_request]',
    path: 'C:\\Code\\server-side-state\\node_modules\\express\\lib\\router\\layer.js',
    line: 95,
    column: 5
  },
  {
    identifier: 'Route.dispatch',
    path: 'C:\\Code\\server-side-state\\node_modules\\express\\lib\\router\\route.js',
    line: 114,
    column: 3
  },
  {
    identifier: 'next',
    path: 'C:\\Code\\server-side-state\\node_modules\\express\\lib\\router\\route.js',
    line: 144,
    column: 13
  },
  {
    identifier: 'Layer.handle [as handle_request]',
    path: 'C:\\Code\\server-side-state\\node_modules\\express\\lib\\router\\layer.js',
    line: 95,
    column: 5
  },
  {
    identifier: 'Index',
    path: 'C:\\Code\\server-side-state\\src\\test\\Index.tsx',
    line: 9,
    column: 7
  },
  {
    identifier: 'once',
    path: 'C:\\Code\\server-side-state\\src\\state\\once.ts',
    line: 40,
    column: 26
  },
  {
    identifier: 'OnceRunner.produce',
    path: 'C:\\Code\\server-side-state\\src\\state\\once.ts',
    line: 23,
    column: 21
  }
]

Filter the stack

We see we got some calls that are outside our code, let's filter them out and make the paths relative to our base directory:

const local = detections.filter(
  (x) =>
    x.path.startsWith(path.resolve(filterDir)) &&
    !x.path.includes("node_modules")
);

For breadcrumbs, we still need to filter out anonymous functions and some other stuff, but better to put it in a separate variable named, because the things inside local can be very useful too.

  const named = local
    .filter((x) => !x.identifier.includes("<anonymous>"))
    .filter((x) => !x.identifier.startsWith("Object."))
    .filter((x) => !x.identifier.startsWith("Module."));

Final results

And now finally make the breadcrumbs string:

const breadcrumbs = named.map((x) => x.identifier).join("/");

And that's it! In this particular example I used, the breadcrumbs will look like this:
Index/once/OnceRunner.produce

and the local and named:

local: [
  {
    identifier: 'Index',
    path: 'C:\\Code\\server-side-state\\src\\test\\Index.tsx',
    line: 9,
    column: 7
  },
  {
    identifier: 'once',
    path: 'C:\\Code\\server-side-state\\src\\state\\once.ts',
    line: 40,
    column: 26
  },
  {
    identifier: 'OnceRunner.produce',
    path: 'C:\\Code\\server-side-state\\src\\state\\once.ts',
    line: 23,
    column: 33
  }
],
named: [
  {
    identifier: 'Index',
    path: 'C:\\Code\\server-side-state\\src\\test\\Index.tsx',
    line: 9,
    column: 7
  },
  {
    identifier: 'once',
    path: 'C:\\Code\\server-side-state\\src\\state\\once.ts',
    line: 40,
    column: 26
  },
  {
    identifier: 'OnceRunner.produce',
    path: 'C:\\Code\\server-side-state\\src\\state\\once.ts',
    line: 23,
    column: 33
  }
]

Cheers and happy coding as always!

How to render JSX to whatever you want with a custom JSX Renderer

Adrian — Mon, 04 Dec 2023 18:21:02 +0000

Most of the time, you don't need to write your own JSX renderer. But what if you needed to? Well, this guide will show you how to do it.

It's actually surprisingly simple! So let's dive in.

TLDR: If you want to see the code straight away, here is the repository: https://github.com/adrian-afl/render-jsx-reupload

What will be needed

You will need TypeScript to follow this guide, and that's it, no external libraries, just your code.

Preparation

Create a project - I mean, with package.json - and install TypeScript.

Then create a tsconfig file, configure it however you want, but in compilerOptions include those two values:

"jsx": "react-jsx",
"jsxImportSource": "./src/app",

jsx set to react-jsx tells the compiler that we want JSX, but we will be using a different implementation than React, in fact, we will write our own. jsxImportSource set to ./src/app tells the compiler that this is the place where to find the necessary exported stuff that makes handling JSX possible. Beware, this is a path, and TypeScript will append to it a postfix /jsx-runtime and will import that file.

Once this is done, let's go to the implementation.

The JSX namespace

Create a file ./src/app/jsx-runtime.ts. This is the file that TypeScript will actually load to do the JSX stuff.

TypeScript requires you to export very specific things from this file. Those are:

JSX namespace
- IntrinsicElements type - an object type, with keys being the allowed tags, and values being the type of attributes for tags
- Element type - can be absolutely anything, defines the type that is the result of the JSX node rendering
jsx function - a function which needs a specific arguments and return value type, this is the function that is called by the runtime when rendering JSX nodes.
jsxs and jsxDEV which for now can be aliased to point to the same function as jsx

This is also the place where you can limit what tags are allowed and what attributes are allowed. To limit that:

To limit what tags are allowed, narrow the type of IntrinsicElements key type.
To limit what attributes are allowed, narrow the type of IntrinsicElements value type

This limiting is usually done by making IntrinsicElements an interface looking like that:

export interface IntrinsicElements {
  a: AnhorAttributes;
  div: DivAttributes;
  span: SpanAttibutes;
  ... and so on
}

And the attribute types are also interfaces, for example

interface AnhorAttributes {
  href: string;
  target?: string;
}

I just made it in a way that all tags are allowed - by setting the IntrinsicElements key type to a string, and all attributes are allowed, by setting the IntrinsicElements value type to Record<string, JSXNode | undefined>.

Example in this article renders the JSX nodes to objects with only one field, a string containing the rendered HTML.

Here's how it looks like in this file:

// Set the attributes to allow any keys and very permissive values
export type HTMLAttributes = Record<string, JSXNode | undefined> &
        JSXChildren;

namespace JSX {
  // Allow any HTML tag
  export type IntrinsicElements = Record<string, HTMLAttributes>;

  // Declare the shape of JSX rendering result
  // This is required so the return types of components can be inferred
  export type Element = RenderedNode;
}

// Export the main namespace
export { JSX };

// Export factories
export const jsx = renderJSX;
export const jsxs = renderJSX;
export const jsxDEV = renderJSX;

You can see there are names that are not yet defined, RenderedNode, JSXNode and JSXChildren. Let's explain what they are.

Types

RenderedNode is the result shape of the rendering, jsx function returns values that are of this type. I decided to make it an object, actually a class, because if I went with just a string, I wouldn't be able to do proper escaping of the HTML. This is because I would not be able to differentiate between rendered HTML and a string literal. Anyway, here's how it looks like:

export class RenderedNode {
  public constructor(public readonly string: string) {}
}

Now, onto JSXChildren. It defines how a children property looks like. This is the thing that defines what is inside the JSX tags, for example in this JSX:

<div>
    Hello
</div>

the string Hello is the children prop.

export interface JSXChildren {
    children?: JSXNode | JSXNode[] | undefined;
}

And now, the big one. JSXNode defines the values inside JSX attributes and JSX children. I made it pretty permissive about what can be rendered. You can define separate types for values available for attributes and separate for children, I used the same type for both, which allowed me to use the same serializer function.

export type JSXNode =
    | RenderedNode
    | RawContentNode
    | (() => JSXNode)
    | boolean
    | number
    | bigint
    | string
    | null
    | undefined;

While we are on types, let's define 2 additional types that will come in handy soon:

interface RawContentNode {
    htmlContent: string;
}

export type FunctionComponent = (
    props: Record<string, unknown>
) => RenderedNode;

RawContentNode is a special object that if passed as a value will be rendered to a string but without any escaping. Handy for CSS and style tags, for example. Obviously we need to handle it, and this is not something standard, I just thought it's necessary, for example to handle <script> and <style> tags, and decided this is a cool way to specify it.

FunctionComponent is a type for a function component, will be used during the rendering.

Rendering

The jsx function and its friends are mapped to the renderJSX function. Here's how it looks like:

export function renderJSX(
  tag: string | FunctionComponent | undefined,
  props: JSX.HTMLAttributes,
  _key?: string
): JSX.Element {
  if (typeof tag === "function") {
    // handling Function Components
    return tag(props);
  } else if (tag === undefined) {
    // handling <></>
    return new RenderedNode(renderChildren(props));
  } else {
    // handling plain HTML codes
    return new RenderedNode(
      renderTag(tag, renderAttributes(props), renderChildren(props))
    );
  }
}

Not that complicated, in renderJSX 3 execution paths are possible:

If tag is a function, we got a function component to handle. It's already a function, and we got the props that should be passed to it, so just call it with the props and return whatever it returns.
If tag is undefined, we have a <></> fragment. It cannot take props, and can only have one or more children. Rendering it boils down to rendering every child inside, we will see soon how it's done.
If tag is a string, it's a plain HTML element. Render it using the renderTag method that I will describe below.

We see that we need 3 helper functions so far, those will again have helper functions, but bear with me.

function renderTag(tag: string, attributes: string, children: string): string {
  const tagWithAttributes = [tag, attributes].join(" ").trim();
  if (children.length !== 0) {
    // render open and close tags
    return `<${tagWithAttributes}>${children}</${tag}>`;
  } else {
    // render only one self-closing tag
    return `<${tagWithAttributes}/>`;
  }
}

This renderTag function is straightforward, renders a plain HTML element to a string, using the parameters passed.

function renderChildren(attributes: JSX.HTMLAttributes): string {
  const children = attributes.children;
  if (!children) {
    return "";
  }
  const childrenArray = !Array.isArray(children) ? [children] : children;
  return childrenArray.map((c) => serialize(c, escapeHTML)).join("");
}

renderChildren functions gets a special attribute named children from the props object that jsx got. This attribute, if present, holds whatever is between the JSX tags, like in the example before, <div>Hello<div> would contain Hello as children. This might be just one element, then it's not an array, or an array, then it comes in as an array. To handle it easier, transform the value to always be an array, then serialize all elements to strings and join them together.

function renderAttributes(attributes: JSX.HTMLAttributes): string {
  return Object.entries(attributes)
    .filter((prop) => prop[0] !== "children")
    .map((prop) => {
      const value = serialize(prop[1], escapeProp);
      return `${prop[0]}="${value}"`;
    })
    .join(" ");
}

This renderAttributes is responsible for rendering the HTML attributes string that will ultimately make its way into an HTML tag. First, filter out the special children property, it was handled with renderChildren. Then with the remaining elements, serialize the value to a string and render the attribute string. Finally, join the resulting strings, but this time with a space, because we need a space between the attributes in final HTML string.

As you can see, there are some other helper functions. First of all we need the serialize function, its helper type and helper error, here it is:

export class SerializationError extends Error {
  public constructor(public readonly invalidValue: unknown) {
    super("Invalid value");
  }
}

interface RawContentNodeTest {
  htmlContent?: string | undefined;
}

export function serialize(
  value: JSXNode,
  escaper: (value: string) => string
): string {
  // Null and undefined handling
  if (value === null || value === undefined) {
    return "";
  }
  // String node handling
  if (typeof value === "string") {
    return escaper(value);
  }
  // Number node handling
  if (typeof value === "number") {
    return value.toString();
  }
  if (typeof value === "bigint") {
    return value.toString();
  }
  // Boolean node handling
  if (typeof value === "boolean") {
    return value ? "true" : "false";
  }
  // Function node handling
  if (typeof value === "function") {
    return serialize(value(), escaper);
  }
  // RenderedNode node handling
  if (value instanceof RenderedNode) {
    return value.string;
  }
  // Dangerous string handling
  if (
    typeof value === "object" &&
    typeof (value as RawContentNodeTest).htmlContent === "string"
  ) {
    return value.htmlContent;
  }

  throw new SerializationError(value);
}

The error is only for semantics, I like to make custom errors for various occasions. The RawContentNodeTest is a helper type to test if the value coming in is the special one that should be rendered without escaping special characters.

You can also notice that the function takes a second parameter that will do the escaping. It's a simple function signature that takes a string and returns a string. I decided to make the serializer be the same for HTML content and attributes, but both require slightly different escaping, that's why the second parameter is needed.

So the following happen in this function:

If the value is null or undefined, return an empty string
If the value is a string, escape special characters and return it
If value is a number, bigint or boolean, convert it to string and return it
If value is a function, call it, and pass the result via serialize again, then return it
If value is already a RenderedNode, extract the content of it and return it
If value is of that special RawContentNode type, which is checked by checking that value is an object and then the existence and type of value under the htmlContent, then return it as is, without escaping
Otherwise, throw and error

What is left to make this implementation complete are the functions that escape special characters inside HTML content and HTML attributes. Those are very simple, look like that:

export function escapeProp(value: string): string {
  return value
    .replaceAll("&", "&amp;")
    .replaceAll('"', "&quot;")
    .replaceAll("<", "&lt;")
    .replaceAll(">", "&gt;")
    .replaceAll("\n", "&#10;")
    .trim();
}

export function escapeHTML(value: string): string {
  return value
    .replaceAll("&", "&amp;")
    .replaceAll('"', "&quot;")
    .replaceAll("'", "&#39;")
    .replaceAll("<", "&lt;")
    .replaceAll(">", "&gt;")
    .replaceAll("\n", "<br/>")
    .trim();
}

Those just replace some stuff inside the strings and return them.

Usage

And here we have it! It will work from now automatically without any special imports or libraries. Just this TypeScript code.

To use it, for example, create a Test.tsx file and put inside:

const ComponentA = (): RenderedNode => <div>Hello</div>;

const ComponentB = (): RenderedNode => (
  <span>
    <ComponentA />
  </span>
);

const rendered = <ComponentB />;

console.log(rendered);

Running this code you will see this printed out:

<div><span>Hello</span></div>

Why would anyone want to do that?

I know preact to string exists, but it is not a fit for some usages because it still assumes you are targeting real HTML. For example, only HTML tags are allowed, and the values and names of attributes are type-checked to be correct HTML. It's not desired sometimes. With this code you don't need to worry about it, you can render the JSX to whatever you want.

I personally need this for an upcoming terminal based UI library that will use JSX to define its content, but that's a story
for another time :)

Thanks for reading, and happy rendering!

Here you can find the code from this article: https://github.com/adrian-afl/render-jsx-reupload

Prisma VS TypeORM - description and comparison

Adrian — Mon, 15 May 2023 09:23:49 +0000

You probably encounter such discussion in the wild often, mostly probably if you are a backend developer. Which ORM to use? There are options and you cannot decide?

Well, let me help! But first...

What is an ORM?

To make this article accessible for most developers, I will explain very quickly why you would want to have the ORM:

Mostly a simplification, but truth nonetheless: It writes SQL for you!
Maps objects in the code to what and how is saved in your relational database (hence the name, object <-> relational mapping)
Most of libraries also expose some useful tools to keep the database in sync with the code, or other way around.

Why would you want to use one, you ask? Why not use raw connection with let's say PostgreSQL and manage data this way?
Well, you certainly could, and most people did that decade or two decades ago, and shock-you-not - it worked! But there were some disadvantages of doing this:

You need to keep the database up to date with the changes manually or spin up some tool to do that
If migrations mechanics are not employed, database state is not versioned or may be versioned in less than ideal way
You need to write code that maps raw stuff from the database into your code objects and vice-versa, map your code objects to database raw data.

So in this case, you write mapping by yourself, migrations by yourself, and then only SQL is left for you to do manually and you are quickly approaching the inevitable implict outcome - you will build your own ORM this way.

There is a reason there are so many new ORM appearing for the node ecosystem recently, many probably came into life this way, someone was writing some db connectivity convenience layer and it turned into an ORM. Fun activity, but don't do that when you code for the money in your current project.

Right, with this out of our way...

Tell me about TypeORM!

I'm glad you asked. I used this library for years now professionally. Which is the main reason I started looking into Prisma, but let's not get ahead of ourselves..

TypeORM key concept is that usually uses the code-first paradigm.

What does that mean? It means you write your code first, entity classes to be precise, then add decorators to those classes and fields inside that serve as a description of this class layout to the ORM, and ORM then uses that data, provided by the decorators, to map your classes into database data.

This is similar to what Doctrine (PHP and Symfony world) and Hibernate (Java and Spring world) usually do as well, and TypeORM also can do the mapping in separate files if needed, but in general using decorators is encouraged by its docs.

So summarizing, your write code, decorate it, and that's how you declare your mapping.

And now, Prisma

I'm experimenting with Prisma for some time now, but not yet switched in professional jobs, but this is on my horizon. But when it comes to my personal hobby projects - I use Prisma all the way.

Prisma has a very different approach to defining your mapping, and you do that in just one file, the schema file. This file is called usually schema.prisma and can be anywhere in your repo.

Now comes the brilliant part: Once you have your mapping written in that schema.prisma file, Prisma will generate all entity and repository classes and functions with all possible typing variations for you.

This means Prisma is schema-first, you write your schema, and code is generated for you. Along with the types!

TypeORM Setup

After installing the library, on npm called typeorm, let's create a file with the connection. This is fairly standard and is needed for typeorm cli to work, but we will also use it later in the code. Call the file data-source.ts:

As you can see, all of the database credentials are loaded from the environment. There could also be some bit of validation for those values, but let's keep things simple now (looking at you zod, later).

Now let's create 2 very simple entities

So this is it. This is how you do the mapping in the TypeORM. You can see there is a top level decorator making the class as an entity, and decorators for fields telling how to manage them. We can also see how relations are mapped, and the autogenerated uuid id field.

Now that we have our entities, we need to create a migration that will apply our changes to the database. There is a CLI tool typeorm exposes to do that, so let's use it, this way:

yarn typeorm-ts-node-commonjs migration:generate -d ./data-source.ts "./src/migrations/CreateUserAndAgency"

And if it works: we will be presented with following code:

You can see there are up and down methods, and that the code is in fact a class. You can write any code there, which is a blessing but also a curse. More on that later.

And finally, apply the migration!
yarn typeorm-ts-node-commonjs migration:run -d ./data-source.ts

If all went well, we can now use the ORM to do something:

Prisma setup

When it comes to installing prisma, we need 2 libraries:
@prisma/client in production dependencies and prisma in dev dependencies
Once installed, let's go straight to writing our schema!

And that's it! This is how you define your mapping in Prisma. There are several pros of this we will talk on later, but notice now that there is just 1 type of the field, not "type from code" and "type from db" dualism.

Once we have the schema ready, now it's time for the biggest trick, client code generation. Use this to do it:
prisma generate --schema=./schema.prisma
And generated! By default, the client code is saved to ./node_modules/.prisma

Now, just as in TypeORM, we will create a migration, using the following command:
prisma migrate dev --schema=./schema.prisma --create-only --name UserAndAgency

We can now see a big different - Prisma migrations are just SQL files! This can be a curse or a blessing, more on that later.

Now apply migrations, but now be careful:

on your dev environments where you don't care that much of data but care a lot about bug catching: prisma migrate dev --schema=./schema.prisma
on your prod environment where you care a lot of your data: prisma migrate deploy --schema=./schema.prisma

This is very important! Because Prisma in the dev migrate utilizes a powerful mechanism called Shadow Schema.
Prisma will basically create a new database side by side, migrate it to the point where original db is migrated to, check for schema drift, and report on it. This will allow you to catch bugs. But you don't want this done on the production.

And now, let's use the Prisma in the simpliest way possible:

Side by Side comparison

Fetching by relations:

In this scenario we will see how to fetch something based on something related, particularly users belonging to an agency.

In Prisma:

In TypeORM:

Very similar, the repositories is a different approach between those ORMs. In TypeORM every entity has its own repository class configured to take care of that entity. In Prisma similar thing exists, but as subfields of the client object, like prismClient.user.

Fetching with relations:

In this scenario, we want to get and user but also its agency along with it.

Prisma:

TypeORM:

This looks very similar, but there are some differences under the hood:

TypeORM will execute 1 query with a join clause to get the results
Prisma will execute 1 query to get the user, and then 1 query to get the agency. If multiple users were fetched, there would still be 2 queries, 1 for users, and 2 for all agencies of fetched users. This is not n+1, rather 1+1 :)

TypeORM typing in this situation shows its biggest weakness. If we did not include agency in relations field, we would get an User field, which type suggest there is a non nullable Agency field in it. And compilator would not know any better that TypeORM will happily ignore that and return you an User with agency set to undefined, despite the types saying its not undefined. You can't rely on types to be sure what your entity really contains, you need to know its path from being fetched to being used to understand it's format. This is really painful.

Prisma, on the other hand, has types prepared for every situation, so in the example above it will return the result of type of a user with agency field, but if we removed the agency from include clause, the type returned would reflect that, and agency field wouldn't be there! This is perfect.

Inserting new data

In Prisma:

In TypeORM:

Some interesting points: look at this select clause in prisma api call. We put only 'id' there, and so the result will be typed to only have this one field and nothing else. This is how you get the ID back if it's generated by the database, usually.
In TypeORM example, pay attention that the create method doesn't write anything to database, it just creates an object instance, of type User, with fields set to that. This is needed so the repository knows that this object is in fact of the user entity type. The objects is then saved using the save method that works like an upsert - inserts if not exist, updates if exists. Confusing? Now tell me what if I put the id in the body? Will it update? Insert new? Override ID generation? How knows, and docs would not help.

Update existing entity

In Prisma:

In TypeORM:

Prisma and first TypeORM examples are simple and clear.
But the second TypeORM example shows something different, if you already have the entity in your code, you can change it directly and save it, the result will be the same.

Running code in transaction

In Prisma:

In TypeORM:

This type of transactions are called interactive because arbirtary code can be put inside. Beware there are timeouts, in prisma, the defualt timeout in this transaction block is 5000ms but can be changed easily.
Be sure to use the parameter of the callback method you pass to the transaction method instead!

Dockerization

When it comes to TypeORM, it's fairly easy because no code generation so I won't go into details here.

But when it comes to Prisma, here is an example of a Dockerfile how to approach it:

Let me explain.
In the first step, production node modules are downloaded
In next step, application is build, and prisma client code is generated
In last step, production node modules are copied back but also autogenerated prisma client code from build stage
And that should be enough to get it up and running.

Conclusion!

TypeORM

Pros:

Similar to other well known tooks - Doctrine, Hibernate
Can be more modular fitting in more complex project architectures - for example giving one module access to only some entities
I think integration with NestJS is very good, but Prisma integrates just as well in my opinion
More optimal queries at times
Migrations are code - Yay! Custom logic!

Cons:

Very weird bugs happening at times
Most usually your bug solving will result in you reading tons of github issues of the typeorm repo, instead of docs
Types are commonly invalid at runtime
Data-source file is not documented anywhere well while being crucial part of cli
No idea why, but it looks like schema drift happens very quick
Migrations are code - Oh no, people will use real entities in there and break migrations - happens too often and best is to disallow imports from app into migrations - you don't want a junior import and Enum and iterate over it in the migration file...
Migrations are less readable, especially keys names
At times error handling is very not helpful

Prisma

Pros:

So fresh approach
Very good type safety
Much better error reporting
Nice documentation
Looks like the community around is more helpful
Handy schema language - I recommend a JetBrains plugin for that, makes working with the file so easy
Code generation saves a lot of time
Can create and update multiple and nested entities in one api call in a migration - see nested writes

Cons:

At times relations are not obvious but the JetBrains plugin helps
No query builder at all, either high level management or low level queies
At times type errors are very verbose but at least are clear
Especially at the beginning - easy to forget to rebuild client - best to have it as part of the build and dev commands

So we got it.

I hope this article will help to make some educated decision on your next project! I will personally switch to Prisma.

Happy mapping!

Dependency Injection without decorators in TypeScript

Adrian — Mon, 13 Feb 2023 19:34:52 +0000

Decorators will get a big revamp in TypeScript 5, and while decorators are cool, what about getting rid of them? And what if we can make the DI much better along the way?

As I worked in other languages as well with more established DI ecosystems, I see that a good DI framework should:

Be transparent - your domain should not import anything related to the DI, and it should not know about the DI at all
Support interfaces without silly decorators with the interface name
Automatically provide the implementation for interfaces that are implemented only once
Should work just fine when compiled down to a production package
Autowires - means it will automatically create everything it can with no or very minimal configuration

Matching those points using TypeScript is not trivial. I will ignore decorators for now and their experimental reflection features, and focus on what we will have on hand with TypeScript 5.

There are a few obstacles along the way:

TypeScript provides no reflection support whatsoever
Due to lack of reflection, getting classes constructor parameter's types is impossible
Runtime will also not preload all your classes so you would need to import them somewhere so that the class declaration runs and the class is defined
Interfaces are erased during compilation and have zero meaning in the running code
Getting a list of implemented interfaces by a given class is impossible

Rough. But it turns out that overcoming all of those problems and matching all of the key features of a good DI implementation is possible.

The key here is Reflection.

Fine, I will do it myself

How to even approach this? We need to get information about every class in the application and a lot of information from it, including implemented interfaces and constructor parameters, without decorators. Right...

What if we could scan our codebase and read every file, find class declarations in them, parse those and extract necessary data from it? Surely there must be a way.

Enter TypeScript compiler API.

The Compiler API can be used to get a tokenized representation of the source code.
So we can do the following to get what we want:

Scan over the source directory for typescript files
For each file:
- Parse it to get the abstract syntax tree - a tree of tokens
- Recursively find all nodes that are class declarations, then for each found:
- Read the node to find the class name, extends, implements
- Find the constructor node and read parameters from it
- Push found data to some array

After this is completed, we will be left with an array of class reflection data objects. Using that, we can generate a typescript source file with that metadata and store it in the source folder itself.

After that, we have class metadata, ready to use.

What now?

In my implementation, after generating the metadata, the saved result is an array of the following objects:

  fqcn: string; // Fully qualified class name - path and name
  name: string; // Class name
  ctor: Promise<Constructor> | null; // Constructor for that the class - null if not public
  implementsInterfaces: string[]; // Interfaces implemented by the class
  extendsClass: string | null; // Parent of the class - null if not extending
  constructorParameters: ParameterData[]; // names and types of constructor parameters
  constructorVisibility: "public" | "protected" | "private"; // Constructor visibility

We can see that we have everything needed to create a working DI.

Now, given each class in the metadata:

We how its name, so we can find it by name when needed
We know the interfaces it implements, so we can find it by interface names
We have its constructor function, so we can create instances
We know what parameters that constructor takes, its names, and its type names

Autowiring will be done recursively, we only need to know the name of the type:

Find metadata of the class by the provided name
If not found, find metadata of the class implementing the provided name
For each constructor parameter in the metadata:
- Autowire the parameter by type name the same way (recursively)
With ready parameters array, construct the class
Return the constructed instance

This method will work just as fine with classes and interfaces, so both can be used in parameter names and during resolving

And that's it! We successfully implemented a working DI without decorators.

Of course, there are other details that need to be taken into consideration to make the DI fully functional, but those are easy things now that the difficult part is done.

Happy autowiring!

Use Mocha instead of Jest and boost your tests speed

Adrian — Wed, 08 Feb 2023 18:15:39 +0000

Don't you want to save time?

I think it is common knowledge that testing applications is a crucial part of software development. It is also very helpful when it comes to developing separate features.

But what if your tests are very slow? Maybe a single test case takes 30 seconds to execute? Or your local test setup is so cumbersome you want to avoid it? You will end up not using them much, and this will harm your velocity.

Here are some traits of good tests in a project:

You can easily run just one test
Running just one test is very quick
You don't need to do any changes to the environment to run tests locally
You can run tests repeatedly without reinitializing the database
You can run locally the same test suite that is run on your pipeline
Even running the whole test suite is fast

In this post, I will focus on the points related to speed.

We really need speed

Just one time reiterating on this so it's clear - the speed of tests runtime will directly affect your delivery speed. And delivery is crucial.

But what we can do about our tests? Turns out there is one key change that can be applied to a lot of projects, is almost a drop-in replacement and usually results in an extreme speed boost.

How Jest does it

Most projects use Jest to run their tests. It's the default runtime for React apps and NestJS apps.

Assume we have the following NestJS application:

Is Typescript based
Integration tests start up the NestJS application module
Supertest is used to hit the API
Nest build takes 5 seconds
Nest app startup takes 1 second

Let's see how Jest runs tests:

Start a Typescript compiler instance and compile and run any global setup defined
Find all files containing tests, most likely files ending with .spec.ts
For every of found files:
- Start a Typescript compiler instance and compile the file, which in turns compiles the whole application because NestJS is imported to create the application to test
- Run the tests in the file

If we had 10 test files, it will result in at least 10 * 5, so 50 seconds wasted on recompiling the application over and over again 10 times, and then another 10 seconds to reinitialize the application 10 times.
The more files there are, the slower it gets.

Why does Jest do that? Probably due to sandboxing, maybe it makes sense. Parallelization works fine and improves on that but be prepared to bump your runners 1 or 2 tiers up to make them handle parallelized compilation over and over again.

The secret sauce - Mocha

Mocha (https://mochajs.org/) is a test runtime just like Jest. But it does things a bit differently.
Assuming the same setup we had in Jest example, here is what Mocha would do:

Find all files containing tests, most likely files ending with .spec.ts
Start a Typescript compiler instance and compile all tests at once like all of them were included in the tsconfig project
Run your global setup
Find all describe blocks in compiled tests
For every describe:
- Run the describe block, of course effectively running the tests

The key here is that compilation is happening only once. If you move your application setup to global setup you can also initialize the application just one time and reuse it.
This will bring the same test runtime down to 5 + 1 seconds, making it over 8x faster. And adding more files won't affect the performance much.

How to migrate if I already use Jest?

This boils down to several steps:

Install Mocha, Chai and Sinon - Chai is for assertions and Sinon is for mocking, you can skip Chai if you want to keep using Jest expect, which is possible
Create mocha config file, you can use a template I made: https://pastebin.com/m2x3H2Mr
Convert all of your beforeAll and afterAll calls to before and after - this is just a name change you can easily regex replace in all your tests
Convert all your Jest mocks to use Sinon, the usage is very similar
If you want to use Jest's expect, you can import it in every test from Jest package or export it globally somewhere, otherwise rewrite your assertions to use one of Chai apis, I think assert one is the most similar to what Jest offers

2024 update

Chai is no longer recommended, best to use Jest Expect module instead!

The biggest issue would be to convert all your Jest mocks to sinon. Maybe there is a way to use Jest's mocks but rewriting all of them is pretty simple, and many cases can be regex replaced.

And that's it! After that, you are ready to run mocha and enjoy your tests running much faster than before.

Happy testing!