DEV Community: Alex Blokh

Best TypeScript ORM just got better

Alex Blokh — Sun, 21 May 2023 10:11:24 +0000

Drizzle ORM v0.26 is out and it changes the game

Well, the last 3 months were wild. We went public with Drizzle ORM and it gained quite some adoption. The first YouTube video about us came out and it blew through the roof. As of now, we sit on 5.6k GitHub stars and actively growing 🤯

3.4k developers subscribed to us on Twitter and we have 960 community members in Discord and boy oh boy we’ve got quite some feedback, we’ve listened and now we got some big news 👀

Drizzle ORM is designed as a thin TypeScript layer atop SQL, it’s meant to solve your problems, empower and not interfere, it has remarkable performance and fully fledged joins. Drizzle lets developers love SQL again.

Yet we’ve received a lot of feedback from devs struggling with relational queries, doing massive amounts of joins and mapping complex results. Well, we have a solution now, meet Relational Queries.

We’ve designed Relational Queries to be an extension to existing Drizzle ORM API, all you need to do is to define relations alongside schema declaration and you’re ready to go.

We’ve spent quite some time to make sure you have both best-in-class developer experience and performance 🚀

Regardless of how many nested relations you query - Drizzle will always make exactly one SQL query to the database, it makes it extremely explicit and easy to tune performance with indexes.

We support nested limit, you no longer have to fetch users and all of their posts from the database, you can just fetch the last 5.

You can include and exclude columns to omit unnecessary data transfer, you can add extra custom fields like lower(first_name + last_name) and you have all the power of where operator at any nested level.

And yes, you can prepare statements and gain up to 5x massive performance improvements!

We’ve been undercover for several weeks, building everything through blood and tears and we’re thrilled to hear your feedback.
Check out the docs and give it a spin!

Docs, by the way… Lets talk docs

Docs were probably #1 requested feature since day one and yes, we do have docs now — orm.drizzle.team

Those are not raindrops, no no. Those are teardrops. We’ve cried a lot.
We’ve handcrafted every page and we’d love to hear your feedback and make it better!
And yes, we have comprehensive docs for Drizzle Kit and you can finally read how to use db push!

Don’t hesitate to hit us a star on GitHub and subscribe on Twitter!

Hanji v0.0.1

Alex Blokh — Mon, 29 Aug 2022 12:22:51 +0000

I've been building CLI tools for 2 of my open-source projects for over a year now. I've started with Ink since I have React developers on team and we were under impression that's gonna let us rapidly build stuff and that turned out to be misleading to say the list.

We've tried Inquirer, Enquirer, Prompts and Ink and for some reason there's not a single library to let you customise stuff. Every library is hardly focused on a particular scenario without any room for you to move around.

I've spent some time digging around libraries internals and found out that core of the library is pretty simple.

As a library developer - I can handle 95% of behind the scenes stuff and let user render text the way he wants leaving room for boundless CLI design.

This is how you obtain stdin, stdout and readline

  const stdin = process.stdin;
  const stdout = process.stdout;

  const readline = require("readline");
  const rl = readline.createInterface({
    input: stdin,
    escapeCodeTimeout: 50,
  });

  readline.emitKeypressEvents(stdin, rl);

now we can listen to all the keypress events

// keystrokes are typed
type Key = {
  sequence: string;
  name: string | undefined;
  ctrl: boolean;
  meta: boolean;
  shift: boolean;
};

const keypress = (str: string | undefined, key: Key) => {
  // handle keypresses
}

stdin.on("keypress", keypress);

// whenever you're done, you just close readline
readline.close()

now to render text we just output it to stdout

let previousText = "";
stdout.write(clear(previousText, stdout.columns));

stdout.write(string);
previousText = string;

// here's how you clear cli
const strip = (str: string) => {
  const pattern = [
    "[\\u001B\\u009B][[\\]()#;?]*(?:(?:(?:(?:;[-a-zA-Z\\d\\/#&.:=?%@~_]+)*|[a-zA-Z\\d]+(?:;[-a-zA-Z\\d\\/#&.:=?%@~_]*)*)?\\u0007)",
    "(?:(?:\\d{1,4}(?:;\\d{0,4})*)?[\\dA-PRZcf-ntqry=><~]))",
  ].join("|");

  const RGX = new RegExp(pattern, "g");
  return typeof str === "string" ? str.replace(RGX, "") : str;
};

const stringWidth = (str: string) => [...strip(str)].length;

export const clear = function (prompt: string, perLine: number) {
  if (!perLine) return erase.line + cursor.to(0);

  let rows = 0;
  const lines = prompt.split(/\r?\n/);
  for (let line of lines) {
    rows += 1 + Math.floor(Math.max(stringWidth(line) - 1, 0) / perLine);
  }

  return erase.lines(rows);
};

While building a design-less toolkit I'm still keen to provide user as much utility as possible, so I decided to implement StateWrappers for domain specific kind of Inputs like select

this is how a select state wrapper would look like. The one below is for simple strings array, it handles up and down keypresses and keeps track of selected index and loops it whenever it's out of bound:

export class SelectState {
  public selectedIdx = 0;
  constructor(public readonly items: string[]) {}

  consume(str: string | undefined, key: AnyKey): boolean {
    if (!key) return false;

    if (key.name === "down") {
      this.selectedIdx = (this.selectedIdx + 1) % this.items.length;
      return true;
    }

    if (key.name === "up") {
      this.selectedIdx -= 1;
      this.selectedIdx =
        this.selectedIdx < 0 ? this.items.length - 1 : this.selectedIdx;
      return true;
    }

    return false;
  }
}

This is library defined Prompt API

export abstract class Prompt<RESULT> {
  protected terminal: ITerminal | undefined;

  protected requestLayout() {
    this.terminal!.requestLayout();
  }

  attach(terminal: ITerminal) {
    this.terminal = terminal;
    this.onAttach(terminal);
  }

  detach(terminal: ITerminal) {
    this.onDetach(terminal);
    this.terminal = undefined;
  }

  onInput(str: string | undefined, key: AnyKey) {}

  abstract result(): RESULT;
  abstract onAttach(terminal: ITerminal): void;
  abstract onDetach(terminal: ITerminal): void;
  abstract render(status: "idle" | "submitted" | "aborted"): string;
}

Now all you as a developer have to do is to define how to render select element and not worry about state and keypress managements, you just leave it to the library and replace it with custom implementations whenever needed.

export class Select extends Prompt<{ index: number; value: string }> {
  private readonly data: SelectState;

  constructor(items: string[]) {
    super();
    this.data = new SelectState(items);
  }

  onAttach(terminal: ITerminal) {
    terminal.toggleCursor("hide");
  }

  onDetach(terminal: ITerminal) {
    terminal.toggleCursor("show");
  }

  override onInput(str: string | undefined, key: any) {
    super.onInput(str, key);
    const invlidate = this.data.consume(str, key);
    if (invlidate) {
      this.requestLayout();
      return;
    }
  }

  render(status: "idle" | "submitted" | "aborted"): string {
    if (status === "submitted" || status === "aborted") {
      return "";
    }

    let text = "";
    this.data.items.forEach((it, idx) => {
      text +=
        idx === this.data.selectedIdx ? `${color.green("❯ " + it)}` : `  ${it}`;
      text += idx != this.data.items.length - 1 ? "\n" : "";
    });
    return text;
  }

  result() {
    return {
      index: this.data.selectedIdx,
      value: this.data.items[this.data.selectedIdx]!,
    };
  }
}

Now we just render and wait for user input

const result = await render(new Select(["user1", "user2" ...]))

I've spent some time over the weekend and published a v0.0.1
you can give it a try - https://www.npmjs.com/package/hanji

I'm going to drop v0.0.2 soon with proper CTRL+C support and API simplifications.

you can stay tuned on twitter - https://twitter.com/_alexblokh

Universe will collapse before you need BIGINT for milliseconds in JavaScript

Alex Blokh — Sun, 24 Jul 2022 07:34:00 +0000

In SQL databases you can store whole numbers in primarily as SMALLINT 2¹⁵-1, INTEGER 2³¹-1 and BIGINT 2⁶³-1, always wondered why not just INT🧐. SMALLINT is too small, INTEGER for milliseconds is just 49.7 days, so we're left with BIGINT and here come the tricky part - JavaScript numbers.

The MAX_SAFE_INTEGER constant has a value of 9007199254740991 (9,007,199,254,740,991 or ~9 quadrillion). The reasoning behind that number is that JavaScript uses double-precision floating-point format numbers as specified in IEEE 754 and can only safely represent integers between -2⁵³+1 and 2⁵³–1.
For larger integers, consider using BigInt.

Who likes bigint in JS? You have to transform numbers to bigints before you can operate with them, you can't just pass bigint to an API JSON response, they're tricky. Those are exact problems my teammate faced on the project recently. So what to do?

We can assume that we'll never surpass a 2⁵³ milliseconds in any foreseeable future. I was roughly calculating with him on the phone call and I was under impression it's going to be more than a lifespan of the universe, which is ~13.4 billion years, but when I got to the keyboard I've found out that it's just 285 427 years. This number it's that dramatic, but it's more than enough to let people of the future debug the code searching for why your service messes up dates on the webpage.

That is exactly why we created a configuration for bigints and bigserials in DrizzleORM so you can automatically enjoy the numbers-land

bigserial("will_be_number", maxBytes: "max_bytes_53");
bigserial("will_be_bigint", maxBytes: "max_bytes_64");

thanks!

DrizzleORM v0.11 | 18 months into development

Alex Blokh — Wed, 20 Jul 2022 13:23:21 +0000

DrizzleORM - is an open source TypeScript ORM, supports PostgreSQL and about to have MySQL and SQLite support in couple of weeks. We've decided it's time to share it with public.

With drizzle you have a fully typed SQL schema in-code which benefits you in multiple different major ways, which I'll cover later

// declaring enum in database
export const popularityEnum = createEnum({ alias: 'popularity', values: ['unknown', 'known', 'popular'] });

export class CountriesTable extends PgTable<CountriesTable> {
  id = this.serial("id").primaryKey();
  name = this.varchar("name", { size: 256 })

  // declaring index
  nameIndex = this.uniqueIndex(this.name)

  public tableName(): string {
    return 'countries';
  }
}

export class CitiesTable extends PgTable<CitiesTable> {
  id = this.serial("id").primaryKey();
  name = this.varchar("name", { size: 256 })
  countryId = this.int("country_id").foreignKey(CountriesTable, (country) => country.id)

  // declaring enum column in table
  popularity = this.type(popularityEnum, "popularity")

  public tableName(): string {
    return 'cities';
  }
}

This is quick start example of how you connect to the database and make your first query with typed result

import { drizzle, PgTable } from 'drizzle-orm'

export class UsersTable extends PgTable<UsersTable> {
  public id = this.serial('id').primaryKey();
  public fullName = this.text('full_name');
  public phone = this.varchar('phone', { size: 256 });

  public tableName(): string {
    return 'users';
  }
}
export type User = InferType<UsersTable>

const db = await drizzle.connect("postgres://user:password@host:port/db");
const usersTable = new UsersTable(db);

const users: User[] = await usersTable.select().execute();

This is how you use WHERE statement with filters, run partial select queries, use limit/offset and orderBy

await table.select().where(
  eq(table.id, 42)
).execute();

// you can combine filters with eq(...) or or(...)
await table.select().where(
  and([eq(table.id, 42), eq(table.name, "Dan")])
).execute();

await table.select().where(
  or([eq(table.id, 42), eq(table.id, 1)])
).execute();

// partial select
const result = await table.select({
     mapped1: table.id,
     mapped2: table.name,
}).execute();
const { mapped1, mapped2 } = result[0];

// limit offset & order by
await table.select().limit(10).offset(10).execute()
await table.select().orderBy((table) => table.name, Order.ASC)
await table.select().orderBy((table) => table.name, Order.DESC)

This is how you run inserts, updates and deletes

const result = await usersTable.insert({
  name: "Andrew",
  createdAt: new Date(),
}).execute();

const result = await usersTable.insertMany([{
  name: "Andrew",
  createdAt: new Date(),
}, {
  name: "Dan",
  createdAt: new Date(),
}]).execute();

await usersTable.update()
  .where(eq(usersTable.name, 'Dan'))
  .set({ name: 'Mr. Dan' })
  .execute();

await usersTable.delete()
  .where(eq(usersTable.name, 'Dan'))
  .execute();

One of the most powerful features we have in our ORM are fully typed joins, compiler won't let you make a mistake

const usersTable = new UsersTable(db);
const citiesTable = new CitiesTable(db);

const result = await citiesTable.select()
  .leftJoin(usersTable, (cities, users) => eq(cities.userId, users.id))
  .where((cities, users) => eq(cities.id, 1))
  .execute();

const citiesWithUsers: { city: City, user: User }[] = result.map((city, user) => ({ city, user }));

Here's a many to many relationship example

export class UsersTable extends PgTable<UsersTable> {
  id = this.serial("id").primaryKey();
    name = this.varchar("name");
}

export class ChatGroupsTable extends PgTable<ChatGroupsTable> {
  id = this.serial("id").primaryKey();
}

export class ManyToManyTable extends PgTable<ManyToManyTable> {
  userId = this.int('user_id').foreignKey(UsersTable, (table) => table.id, { onDelete: 'CASCADE' });
  groupId = this.int('group_id').foreignKey(ChatGroupsTable, (table) => table.id, { onDelete: 'CASCADE' });
}

...
const usersTable = new UsersTable(db);
const chatGroupsTable = new ChatGroupsTable(db);
const manyToManyTable = new ManyToManyTable(db);

// querying user group with id 1 and all the participants(users)
const usersWithUserGroups = await manyToManyTable.select()
  .leftJoin(usersTable, (manyToMany, users) => eq(manyToManyTable.userId, users.id))
  .leftJoin(chatGroupsTable, (manyToMany, _users, chatGroups) => eq(manyToManyTable.groupId, chatGroups.id))
  .where((manyToMany, _users, userGroups) => eq(userGroups.id, 1))
  .execute();

Last but not least are migrations. We've implemented a CLI tool for automatic migrations generation, which does handle renames and deletes by prompting you to resolve.

For a typescript schema below

import { PgTable } from "drizzle-orm";

export class UsersTable extends PgTable <UsersTable> {
  public id = this.serial("id").primaryKey();
  public fullName = this.varchar("full_name", { size: 256 });

  public fullNameIndex = this.index(this.fullName);

  public tableName(): string {
    return "users";
  }
}

export class AuthOtpTable extends PgTable <AuthOtpTable> {
  public id = this.serial("id").primaryKey();
  public phone = this.varchar("phone", { size: 256 });
  public userId = this.int("user_id").foreignKey(UsersTable, (t) => t.id);

  public tableName(): string {
    return "auth_otp";
  }
}

It will automatically generate SQL migration

CREATE TABLE IF NOT EXISTS auth_otp (
    "id" SERIAL PRIMARY KEY,
    "phone" character varying(256),
    "user_id" INT
);

CREATE TABLE IF NOT EXISTS users (
    "id" SERIAL PRIMARY KEY,
    "full_name" character varying(256)
);

DO $$ BEGIN
 ALTER TABLE auth_otp ADD CONSTRAINT auth_otp_user_id_fkey FOREIGN KEY ("user_id") REFERENCES users(id);
EXCEPTION
 WHEN duplicate_object THEN null;
END $$;

CREATE INDEX IF NOT EXISTS users_full_name_index ON users (full_name);

Feel free to give it a try - https://www.npmjs.com/package/drizzle-orm

If you have any questions or feature request - reach me out on tweeter

thanks!

What is your TypeScript ORM of choice and why?

Alex Blokh — Mon, 20 Jun 2022 19:41:49 +0000

DxDD - developer experience driven development

Alex Blokh — Mon, 20 Jun 2022 19:21:16 +0000

Basically all OSS libraries are built by developers for developers and developers tend to think solution first while I always advocate to think API first. Let me explain.

By solution first I mean designing APIs of the library to an implementation. You wrap interface around working core which is a fundamental flaw. You lose an opportunity to make it the best version of itself, you lose freedom of thought, you are now bounded by a solution core you've already made. Sometimes it might work, but most of the time it's not.

I've noticed that phenomena when we were building chatbots. When you give developer Telegram Bot API and business demands - they usually produce a bot that works like a terminal application. Same for the clients when they used Google Sheets for business demands.
When I asked developer to use ChatFuel interface to build a chat flow for a bot immediately I got an output of a humanised version of a conversation with stickers, photos and GIFs. That led me to an idea of building chat bot design tool, hopefully we'll build someday, but this is another story.

The API first design is when you have all the freedom, you play around and fantasise on what would be the most useful and clean and compact and explicit API of the library and then and only then you try to design solution based on the API design. And do your best to fit, weigh compromises. Treat your library as a product and developers who gonna use it as customers.

Thanks!

Building TypeScript ORM

Alex Blokh — Mon, 20 Jun 2022 09:59:47 +0000

We've spent last year writing the next big thing in the world of ORMs, specifically in the domain of TypeScript. I decided to share our initial 12 fundamental design decisions.

1. Don't interfere:
It is a library, not a framework. Be an opt in all the time. Provide features without any tradeoffs. Let developer opt out at any given time and place. Help, enrich, not interfere.

2. Simple things stay simple:
It is hard to do things simple. Avoid adding features if they break simplicity. Work them out until they can be integrated seamlessly. Remove or rework things that make library harder to deal with.

3. SQL-like syntax, little to none learning curve:
Learn SQL, not ORM libraries and frameworks.

4. Predictability:
Things should act the way you'd expect them. Everything should be explicit. Ask for permission if you need to act on behalf o a developer. APIs have to be explorable.

5. Types:
Types ground top. Developer should have no room to make a mistake. If something's wrong - it does not compile. Provide a way to write custom types. The only unsafe place is in raw SQL.

6. Feature support:
Spend 95% of the time on 95% of most used features.

7. Joins:
They have to be perfect. They have to be type safe, explicit and infinitely flexible.

8. Migrations:
That is easily the hardest one. Provide an opt in opinionated way to build semi-automatic, type-safe robust migrations. Provide an opt in opinionated way to run migrations.

9. Stay slim:
Little to none runtime overhead. Zero dependencies, stay serverless friendly. Embrace native database drivers, don't build our own. Transactions degrade performance and have to be opt it, not opt out.

10. Production, Tests and Documentation
Usage in real world projects. Extensive tests on multiple database versions. Extensive documentation.

11. Communicate:
Show your work, let people glimpse at what you're doing and how. Provide an easy to use communication and feedback channel

12. Love:
Share some love

Thanks for reading! Ping me out if you're interested in accessing a beta version of the ORM!

Dots, Strings and JavaScript pt. 1

Alex Blokh — Wed, 24 Jun 2020 08:23:05 +0000

It is a very nifty task to pump up algorithmic thinking and clarity into complexity, execution time and memory usage.

On the input you have a string of symbols, on the output you get an array of all possible strings with dots placed inside the string between characters.

> abc
[a.bc ab.c a.b.c]

> abcd
[a.bcd, ab.cd, a.b.cd, abc.d ...]

Newcomers in our team who googled the solution on combinatoric forums usually see a connection with permutations. The task above is about combinations(permutations with repetition) if to be precise. We have exactly two choices '.' and '' to fill slots between characters and we have N-1 slots, where N is number of characters. With a brief calculation you can find out that number of combinations is going to be 2 to the power of N-1.

const text = 'abc'
const totalCombinations = Math.pow(2, text.length - 1)

Now we have to decide where to put dots on every iteration. The first thing developers lean to do is to convert an index of a loop to a binary representation and then use it as a mask to apply to an input string

for (let i = 0; i < totalCombinations; i++) { 
    const mask = i.toString(2)
    ...
}

then apply that mask to the input string and place dots where we have 1 in our binary mask

...
    const mask = i.toString(2)

    let result = ''

    for (let j = 0; j < text.length; j++){
        result += text[j]

        if(j === mask.length) break; 
        // we've filled all slots at this point
        // we can also omit the check above, it'll still work
        // yet we prefer clarity over code lines

        result += mask[j] === '1' ? '.' : ''
    }

the code above is almost correct, you might've noticed that we didn't prepend a leading zeros to our binary mask and instead of having '00' '01'.. we're going to have 0, 1, 10. Let's fix that.

A brief google search on adding leading zeros to binary numbers will lead you to

var n = num.toString(2);
n = "00000000".substr(n.length) + n;

or

function pad(n, width, z) {
  z = z || '0';
  n = n + '';
  return n.length >= width ? n : new Array(width - n.length + 1).join(z) + n;
}

or somthing like 

function pad(num, size) {
    var s = num+"";
    while (s.length < size) s = "0" + s;
    return s;
}

etc...

Yet we can just use a native String.prototype.padStart()

const mask = i.toString(2).padStart(text.length - 1, '0')

Let's put everything together

const text = "abcde";
const total = Math.pow(2, text.length - 1);

const output = [];
for (let i = 0; i < total; i++) {
  const mask = i.toString(2).padStart(text.length - 1, "0");
  let result = "";

  for (let j = 0; j < text.length; j++) {
    result += text[j];

    if (j === mask.length) break;
    result += mask[j] === "1" ? "." : "";
  }

  output.push(result)
}

console.log(output)

And give it a run

node test.js 
[
  'abcde',    'abcd.e',
  'abc.de',   'abc.d.e',
  'ab.cde',   'ab.cd.e',
  'ab.c.de',  'ab.c.d.e',
  'a.bcde',   'a.bcd.e',
  'a.bc.de',  'a.bc.d.e',
  'a.b.cde',  'a.b.cd.e',
  'a.b.c.de', 'a.b.c.d.e'
]

Great, everything works as expected. When our developers come to that stage of resolving the problem we give them the improvement task, let's have a look at those in the next post.