GraphQL Subscriptions and MikroOrm in 2021

👋 Hello, welcome to my first post! If you're here because you have been trying to get subscriptions working with express and GraphQL, you can go ahead and wipe the tears out of your eyes!

Okay but seriously, if you've fallen down the rabbit hole of Apollo docs pointing you towards one library (subscription-transport-ws) which then points you to another (graphql-ws) , and so on and so forth, then hopefully this helps pull you out.

Overview ✨

At the end of this, you will have a GraphQL server, capable of subscriptions, that uses Mikro-Orm for managing your database.

Buckle in, this will be a long one!

Project Structure 🧬

graphql-mikro-subscriptions
├── db.sqlite
├── package.json
├── src
│   ├── application.ts
│   ├── assets
│   │   └── playground.html
│   ├── entities
│   │   ├── base.entity.ts
│   │   └── message.entity.ts
│   ├── enums
│   │   └── SubscriptionEvent.ts
│   ├── index.ts
│   ├── interfaces
│   │   └── ReqContext.ts
│   └── resolvers
│       └── message.resolver.ts
├── tsconfig.json
└── yarn.lock

6 directories, 12 files

I'll be walking you through how I set up the project, but in case you're antsy 👉 the GitHub link

Setup 🔨

I'll be breaking the setup into multiple steps. Please keep in mind that throughout this tutorial I will be making assumptions about your knowledge. I am assuming you already know the basics of GraphQL, understand express and cors, have familiarity with ORMs in general, know your way around a terminal, etc. If you don't, that's totally okay! I would just suggest you spend some time reading through docs before making your way back here 🙂

Step 1: Initialize the project

Go ahead and create a new project folder and initialize it with a package.json.

mkdir graphql-mikro-subscriptions
cd graphql-mikro-subscriptions && yarn init -y

You'll want to then generate a tsconfig.json

npx tsc --init

This will generate a basic tsconfig with comments for all the available fields. I have pasted my tsconfig below for simplicity:

{
  "compilerOptions": {
    "target": "es2018",
    "module": "commonjs",
    "lib": ["dom", "es6", "es2018", "esnext.asynciterable"],
    "outDir": "./dist",
    "strict": true,
    "noUnusedLocals": true,
    "noUnusedParameters": true,
    "noImplicitReturns": true,
    "noFallthroughCasesInSwitch": true,
    "esModuleInterop": true,
    "resolveJsonModule": true,
    "experimentalDecorators": true,
    "emitDecoratorMetadata": true,
    "allowJs": true,
    "skipLibCheck": true,
    "allowSyntheticDefaultImports": true,
    "forceConsistentCasingInFileNames": true,
    "moduleResolution": "node",
    "isolatedModules": true,
    "sourceMap": true,
    "removeComments": true,
    "noImplicitAny": true,
    "strictNullChecks": true,
    "strictFunctionTypes": true,
    "noImplicitThis": true
  },
  "exclude": ["node_modules"]
}

At this point, we can go ahead and install all the dependencies we will need for the project. I'll explain a few of these afterwards. Let's start with the normal dependencies:

yarn add @mikro-orm/core @mikro-orm/sqlite @mikro-orm/sql-highlighter apollo-server-express class-validator cors express express-graphql graphql graphql-ws ts-node type-graphql ws

The main thing to point out here is that I am installing the sqlite mikro-orm package. Typically I use postgres, however for the sake of simplicity this project will use sqlite.

Now let's install some devDependencies:

yarn add --dev @types/express @types/node ts-node-dev typescript @types/ws

With the dependencies installed, let's add a quick script to our scripts field in the package.json

"scripts": {
    "dev": "ts-node-dev ./src/index.ts"
  },

Okay, with all that out of the way we can actually write some code - almost. Here's where I put a small disclaimer: the project structure I create is just my preferred structure for an express app. If you don't like it this way, change it! The most important pieces for getting the server running will be in the application.ts file we will create - so take the logic and rearrange it however you're most comfortable.

Step 2: Setup the express server

Make some subdirectories and files to be on our way to start coding!

mkdir src
mkdir src/entities
mkdir src/resolvers
mkdir src/enums
mkdir src/interfaces
mkdir src/assets

touch src/index.ts
touch src/application.ts

# we will talk about this one later
touch src/assets/playground.html

index.ts

This will be the entry point to the application. Add the following to index.ts:

import Application from './application';
import 'reflect-metadata';

export const PRODUCTION = process.env.NODE_ENV === 'production';

export let application: Application;

async function main() {
  application = new Application();
  await application.connect();
  // await application.seedDb();
  await application.init();
}

main();

Quick explainer: I have an async function to bootstrap everything. Internally, this will instantiate an Application object, which will handle connecting to the database with Mikro-Orm and then starting up the server. GraphQL depends on the import of reflect-metadata, so I typically do it here at the root.

This will throw some errors since we haven't implemented the import from application.ts, so let's go ahead and do that next.

application.ts

I usually opt to wrap my express app in a class. Add the following to application.ts

import express from 'express';
import { createServer, Server } from 'http';
import cors from 'cors';
import { Connection, IDatabaseDriver, MikroORM } from '@mikro-orm/core';

export default class Application {
  public orm!: MikroORM<IDatabaseDriver<Connection>>;
  public expressApp!: express.Application;
  public httpServer!: Server;

  public async connect() {
    // TODO: connect with mikro-orm
  }

  public async seedDb() {
    // TODO: populate database with test data
  }

  public async init() {
    this.expressApp = express();
    this.httpServer = createServer(this.expressApp);

    const corsOptions = {
      origin: '*', // FIXME: change me to fit your configuration
    };

    this.expressApp.use(cors(corsOptions));

    this.expressApp.get('/', (_req, res) => res.send('Hello, World!'));

    const port = process.env.PORT || 5000;
    this.httpServer.listen(port, () =>
      console.log(`httpServer listening at http://localhost:${port}`)
    );
  }
}

To test what we have so far go ahead and run the script you added to the package.json:

yarn dev

This will compile and run the code, and watch for changes. Think of it like nodemon. Go to your browser and load up localhost:5000 and you should now see this:

🎉 We are now one step closer!

Step 3: Configure Mikro-Orm

Remember for this guide we are using sqlite, refer to the official documentation to see instructions for connecting to varying drivers. (P.S. it's one of the best docs I have read through).

Entities

To start, let's create our first entities:

touch src/entities/base.entity.ts
touch src/entities/message.entity.ts

I'll briefly explain the decorators after we create these two entities. Let's start with base.entity.ts, add the following:

import { PrimaryKey, Property } from '@mikro-orm/core';

export default abstract class BaseEntity {
  @PrimaryKey()
  id!: number;

  @Property()
  createdAt = new Date();

  @Property({ onUpdate: () => new Date() })
  updatedAt = new Date();
}

Mini discussion: notice this is an abstract class! The contents of the BaseEntity class you just created could just be dropped into each entity we create, however now we can just have other entity classes extend this one, reducing the repeating code we write. In this regard, base.entity.ts is really optional.

Now, let's add the following to message.entity.ts:

import { Entity, Property } from '@mikro-orm/core';
import BaseEntity from './base.entity';

// the comments are optional and are just for organization :)
@Entity()
export default class Message extends BaseEntity {
  //====== PROPERTIES ======//
  @Property()
  from!: string;

  @Property()
  content!: string;
  //====== RELATIONS ======//
  //====== METHODS ======//
  //====== GETTERS ======//
  //====== MUTATORS ======//
  //====== CONSTRUCTORS ======//
  constructor(from: string, content: string) {
    // call the constructor for BaseEntity
    super();

    // assign the properties
    this.from = from;
    this.content = content;
  }
}

Okay let's discuss. For a more in-depth overview of the decorators, see the documentation. The @Entity() decorator marks the class as being an entity. The @PrimaryKey() decorator identifies the property as being the primary key for this entity. The @Property() decorator defines a property for the entity, which can be thought of as analogous to a database column (in SQL).

You'll notice I passed in additional arguments to the @Property() decorator in the BaseEntity.updatedAt property. This will trigger each time the entity is updated.

For the Message entity constructor, things are a little more straight forward. I assign the properties the values passed to the constructor as you would any other class constructor.

Configure the orm

Now that we have a manageable entity, let's configure the orm in application.ts. Add the following to application.ts:

// Add these imports
import { PRODUCTION } from '.';
import { SqlHighlighter } from '@mikro-orm/sql-highlighter';
import Message from './entities/message.entity';

// Add the following body to the connect method
public async connect() {
  try {
    this.orm = await MikroORM.init({
      entities: [Message],
      type: 'sqlite',
      dbName: 'db',
      debug: !PRODUCTION,
      highlighter: !PRODUCTION ? new SqlHighlighter() : undefined,
    });
  } catch (error) {
    console.error('📌 Could not connect to the database', error);
  }
}

Running the application again, you should see something like this:

Seed the database

Let's create a really small seed function to test that everything is working properly. Add the following to application.ts:

// Add the following body to the seedDb method
public async seedDb() {
  const generator = this.orm.getSchemaGenerator();

  await generator.dropSchema(); // drops all the tables
  await generator.createSchema(); // creates all the tables

  const testMessage = new Message('Aaron', 'Hello, World!');

  await this.orm.em
    .persistAndFlush(testMessage)
    .then(() => console.log('💪 message persisted to database'))
    .catch((err) => console.log('😱 something went wrong!:', err));
}

Save your changes, or rerun the app and you should now see this:

Inspecting the sqlite file with a sqlite database browser, you can see that it worked!

Step 4: Adding GraphQL

Now that we have the express server running and the orm configured, let's set up the GraphQL layer.

Making our entities accessible for GraphQL

In order for GraphQL to generate the schema from our entities, we need to annotate our entity classes with additional decorators from type-graphql. As with the Mikro decorators, I won't go too in-depth in my explanations. Review the type-graphql documentation for a better understanding of what's going on.

In general, for this application we will only need to use ObjectType() and Field decorators from type-graphql. Wherever we have @Entity(), we will be adding @ObjectType(), and wherever we have Property(), we will be adding Field() (unless we didn't want GraphQL to be able to query on certain entity properties, e.g. password hashes). These will automagically create the type-definitions for us.

Since BaseEntity is an abstract class, base.entity.ts should now look like this:

import { PrimaryKey, Property } from '@mikro-orm/core';
import { Field, ID, ObjectType } from 'type-graphql';

@ObjectType({ isAbstract: true })
export default abstract class BaseEntity {
  @Field(() => ID)
  @PrimaryKey()
  id!: number;

  @Field(() => Date)
  @Property()
  createdAt = new Date();

  @Field(() => Date)
  @Property({ onUpdate: () => new Date() })
  updatedAt = new Date();
}

Similarly, message.entity.ts should now look like this:

import { Entity, Property } from '@mikro-orm/core';
import { Field, ObjectType } from 'type-graphql';
import BaseEntity from './base.entity';

@ObjectType()
@Entity()
export default class Message extends BaseEntity {
  //====== PROPERTIES ======//
  @Field()
  @Property()
  from!: string;

  @Field()
  @Property()
  content!: string;
  //====== RELATIONS ======//
  //====== METHODS ======//
  //====== GETTERS ======//
  //====== MUTATORS ======//
  //====== CONSTRUCTORS ======//
  constructor(from: string, content: string) {
    // call the constructor for BaseEntity
    super();

    // assign the properties
    this.from = from;
    this.content = content;
  }
}

GraphQL Resolvers

Now that our entities are GraphQL friendly, let's create a resolver for our Message entity.

touch src/resolvers/message.resolver.ts

Add the following to message.resolver.ts:

import { Resolver, Query } from 'type-graphql';
import { application } from '..';
import Message from '../entities/message.entity';

@Resolver()
export class MessageResolver {
  @Query(() => [Message])
  async getMessages(): Promise<Message[]> {
    return application.orm.em.find(Message, {});
  }
}

The @Resolver() decorator let's GraphQL know that this is a resolver, and, likewise, the @Query() decorator let's it know that the method is a query. Later, we'll add a mutation and subscription.

Run the Apollo Server

Now that we have the entities GraphQL friendly and we have a nice little resolver, we can generate the GraphQL schema needed for the apollo server.

Add the following to application.ts

// new imports
import { buildSchema } from 'type-graphql';
import { MessageResolver } from './resolvers/message.resolver';
import { ApolloServer } from 'apollo-server-express';

// add a new class property
public apolloServer!: ApolloServer;

// the init method should now look like this
public async init() {
  this.expressApp = express();
  this.httpServer = createServer(this.expressApp);

  const corsOptions = {
    origin: '*', // FIXME: change me to fit your configuration
  };

  this.expressApp.use(cors(corsOptions));

  this.expressApp.get('/', (_req, res) => res.send('Hello, World!'));

  // generate the graphql schema
  const schema = await buildSchema({
    resolvers: [MessageResolver],
  });

  // initalize the apollo server, passing in the schema and then
  // defining the context each query/mutation will have access to
  this.apolloServer = new ApolloServer({
    schema,
    context: ({ req, res }) => ({
      req,
      res,
      // I am injecting the entity manager into my context. This will let me
      // use it directly by extracting it from the context of my queries/mutations.
      em: this.orm.em.fork(),
    }),
  });

  // you need to start the server BEFORE applying middleware
  await this.apolloServer.start();
  // pass the express app and the cors config to the middleware
  this.apolloServer.applyMiddleware({
    app: this.expressApp,
    cors: corsOptions,
  });

  const port = process.env.PORT || 5000;
  this.httpServer.listen(port, () =>
    console.log(`httpServer listening at http://localhost:${port}`)
  );
}

Now when you start the app and go to localhost:5000/graphql you should see the GraphQL playground. Run the following query:

query {
  getMessages {
    id
    from
    content
  }
}

🎉 And now GraphQL is added! We are really close to the end (I swear 😭)

Step 5: GraphQL Subscriptions

Adding the websocket server

We use graphql-ws here to add a websocket server to handle our subscriptions. Add the following to your application.ts:

// new imports
import ws from 'ws';
import {useServer} from 'graphql-ws/lib/use/ws';

// add a new class property
public subscriptionServer!: ws.Server;

// the body of the init method should now look like this
public async init() {
  this.expressApp = express();
  this.httpServer = createServer(this.expressApp);

  const corsOptions = {
    origin: '*', // FIXME: change me to fit your configuration
  };

  this.expressApp.use(cors(corsOptions));

  this.expressApp.get('/', (_req, res) => res.send('Hello, World!'));

  // generate the graphql schema
  const schema = await buildSchema({
    resolvers: [MessageResolver],
  });

  // initialize the ws server to handle subscriptions
  this.subscriptionServer = new ws.Server({
    server: this.httpServer,
    path: '/graphql',
  });

  // initalize the apollo server, passing in the schema and then
  // defining the context each query/mutation will have access to
  this.apolloServer = new ApolloServer({
    schema,
    context: ({ req, res }) => ({
      req,
      res,
      // I am injecting the entity manager into my context. This will let me
      // use it directly by extracting it from the context of my queries/mutations.
      em: this.orm.em.fork(),
    }),
    plugins: [
      // we need to use a callback here since the subscriptionServer is scoped
      // to the class and would not exist otherwise in the plugin definition
      (subscriptionServer = this.subscriptionServer) => {
        return {
          async serverWillStart() {
            return {
              async drainServer() {
                subscriptionServer.close();
              },
            };
          },
        };
      },
    ],
  });

  // you need to start the server BEFORE applying middleware
  await this.apolloServer.start();
  // pass the express app and the cors config to the middleware
  this.apolloServer.applyMiddleware({
    app: this.expressApp,
    cors: corsOptions,
  });

  const port = process.env.PORT || 5000;
  this.httpServer.listen(port, () => {
    // pass in the schema and then the subscription server
    useServer({ schema }, this.subscriptionServer);
    console.log(`httpServer listening at http://localhost:${port}`);
  });
}

Set up the new playground

One caveat of subscriptions this way is that GraphQL's playground does not use graphql-ws and so therefore does not work. Thankfully, there is a work around. Copy the contents of this html file to the src/assets/playground.html file we created. Be sure to also change the port in the html file:

// around line 60
const wsClient = graphqlWs.createClient({
  url: 'ws://localhost:5000/graphql',
  lazy: false, // connect as soon as the page opens
});

Then add the following to application.ts:

// new import
import path from 'path'

// add this line to the init method below the test get request
this.expressApp.get('/graphql', (_req, res) => {
      res.sendFile(path.join(__dirname, './assets/playground.html'));
    });

Adding the new line will override apollo's configuration for the /graphql path, so be sure to add it before the apollo server is created.

Now, when you go to localhost:500/graphql and run the same query you should see this:

PubSub Engine

If you're from Florida, no this is not the famous pubsub (although I have been missing those badly)! Tldr; it's a publish/subscribe engine.

For simplicity, we are going to use the default PubSub engine that comes bundled with graphql-subscriptions, however there are other implementations you can use.

Add the following updates to the init method in application.ts:

// create the PubSub (not the sandwich 😩)
const pubSub = new PubSub();

// generate the graphql schema
const schema = await buildSchema({
  resolvers: [MessageResolver],
  pubSub,
});

Creating a subscription

We're close! Before we go back to message.resolver.ts. There are three important things we need to quickly address:

1. If you look back at the apollo server we created, you'll see we injected the entity manager. Using the @Ctx() decorator, we can now access the entity manager from context rather than importing the application.

2. To access the PubSub we passed to buildSchema, we can use the @PubSub() decorator. We can then issue publish events!

3. To access the payloads from publish events, we can use the Root() decorator.

We'll have to type our context in order to access it, so let's create the interface for that:

touch src/interfaces/ReqContext.ts

Add the following to ReqContext.ts

import { Request, Response } from 'express';
import { EntityManager } from '@mikro-orm/sqlite';

export default interface ReqContext {
  req: Request;
  res: Response;
  em: EntityManager;
}

We'll also need to define our subscription events. I typically use an enum for this, with multiple values, so even though there is only one subscription type I'm going to keep that format.

touch src/enums/SubscriptionEvent.ts

And add the following to SubscriptionEvent.ts:

enum SubscriptionEvent {
  NEW_MESSAGE = 'NEW_MESSAGE',
}

export default SubscriptionEvent;

Now we can go to message.resolver.ts and make it look like this:

import { PubSubEngine } from 'graphql-subscriptions';
import {
  Resolver,
  Query,
  Mutation,
  Ctx,
  Subscription,
  Root,
  PubSub,
} from 'type-graphql';
import Message from '../entities/message.entity';
import SubscriptionEvent from '../enums/SubscriptionEvent';
import ReqContext from '../interfaces/ReqContext';

@Resolver()
export class MessageResolver {
  @Query(() => [Message])
  async getMessages(@Ctx() ctx: ReqContext): Promise<Message[]> {
    return ctx.em.find(Message, {});
  }

  @Mutation(() => Message)
  async createMessage(
    @Ctx() ctx: ReqContext,
    @PubSub() pubSub: PubSubEngine
  ): Promise<Message> {
    // grab the entity manager from context
    const { em } = ctx;

    const message = new Message('GraphQL', 'Hello, Aaron!');

    // persist the new message to the database
    await em.persistAndFlush(message);

    // publish the message
    await pubSub.publish(SubscriptionEvent.NEW_MESSAGE, message);

    // return the message
    return message;
  }

  @Subscription(() => Message, { topics: SubscriptionEvent.NEW_MESSAGE })
  async createdMessage(@Root() payload: Message) {
    return payload;
  }
}

Now let's go to localhost:500/graphql and run that same query just to make sure everything is good and you should see....

An error?? Yep. I know I know, not to be that person who intentionally leaves out a key step, but I thought this was important. We provided context for our apollo server, but we didn't do that with our websocket server. Here's the quick fix!

At the end of the init method in application.ts, change the last few lines to this:

this.httpServer.listen(port, () => {
  // pass in the schema and then the subscription server
  useServer(
    { schema, context: { em: this.orm.em.fork() } },
    this.subscriptionServer
  );
  console.log(`httpServer listening at http://localhost:${port}`);
});

Now, retry that query!

Alriiiight! Now let's see what happens we try the new subscription! You'll want to run the subscription in one tab, then run the mutation in a new tab, and then check back to the tab with the subscription:

Tada! We're done! 🎉

Thanks for bearing with me! Let me know if this was helpful or if you have any notes (I love notes!)

Again, here is the GitHub link for everything we just did.

Comments

[kieronjmckenna]

Thank you so much... the ecosystem around GraphQL subscriptions seems to be such a mess at the minute, but your solution makes it so simple. You've saved me what could have been another week of deciding on what packages and setup to go with, but now I will be putting this into action.