DEV Community: Elie

Inside the Open-Source Novu Notification Engine

Elie — Thu, 14 Mar 2024 15:03:00 +0000

In this post we'll dive into what Novu is, why you'd want to use it, and then jump into the code behind it.

You don't need to know how it works to use it, but it will help you understand it at a deeper level, and will teach you how to use Redis BullMQ on your own projects.

If you prefer video, here's my video on the same topic:

Novu, is open-source notification infrastructure. It simplifies the process of integrating notifications into your app, supporting various channels like email, SMS, and push notifications. Novu has 31,000 stars on GitHub!

Before we begin, I also run a YouTube channel on open source, and am building my an open source email app called Inbox Zero that helps people clean up and automate their inbox.

So Why Novu?

Why use Novu? Why not send the email notifications to your users yourself using something like SES or Resend?

As your app grows, notifications become more complex.

If you're building an app like Twitter and you want to notify users when they receive messages, you wouldn't want to bombard the most popular user with individual messages every few seconds.

Novu offers a smarter solution. Instead of sending out notifications immediately it does the following:

Aggregate notifications: group notifications together based on a specific timeframe (e.g., 30 minutes). This prevents users from being overwhelmed by a constant stream of individual messages.
Optimize delivery: choose the most appropriate channel for each notification (email, push notification, etc.), ensuring users receive them in a way they prefer.
Scale efficiently: Novu is built to handle high volumes of notifications without compromising performance.

Basic Usage

To integrate Novu into your project, you have to utilize their API, for example using Novu node package:



import { Novu } from '@novu/node';

const novu = new Novu(process.env.NOVU_API_KEY);

await novu.trigger('<WORKFLOW_TRIGGER_ID>',
  {
    to: {
      subscriberId: '<UNIQUE_SUBSCRIBER_IDENTIFIER>',
      email: 'john@doemail.com',
      firstName: 'John',
      lastName: 'Doe',
    },
    payload: {
      name: "Hello World",
      organization: {
        logo: 'https://happycorp.com/logo.png',
      },
    },
  }
);

You trigger a notification by providing the workflow ID, along with an object containing the subscriber ID for who you want to send this message to, and the payload, which represents the message to be send to the user.

Once you've triggered the notification, you can easily display it in your project's UI. Novu provides packages for UI, as shown in the image above, although you have the option to build your own. This makes it simpler to integrate notifications directly into your project.

Providers

There are a lot of different providers you can connect to. For example, if you're sending email, you could use Sendgrid, Mailgun, and SES. You could send messages via SMS using Twilio. For more details, you can check the list of Novu Providers here.

A Glimpse into Novu Cloud

This is what Novu cloud-hosted offering looks like. Let's set up a new workflow for a mention in a comment.

The first step is to call the trigger, after calling the trigger the user gets an in-app notification immediately. This notification tells the user that commenterName mentioned them in commentSnippet which are variables that are passed in the payload when the trigger is called.



{{commenterName}} has mentioned you in <b> "{{commentSnippet}}" </b>

The third step is creating a digest. A digest gathers notifications to use them later on. In this template, we will wait for 30 minutes to gather all received notifications.

The last step is sending an email. The email content varies depending on the number of mentions received.



{{#if step.digest}}
  {{step.events.0.mentionedUser}} and {{step.total_count}} others mentioned you in a comment.
{{else}}
  {{mentionedUser}} mentioned you in a comment.
{{/if}}

The Power Behind Novu

Novu uses NestJS a framework within NodeJS, which brings a solid structure to it. It uses MongoDB and Redis as its databases.

Novu serves major clients, operating at a large scale and handling hundreds of millions of events monthly. It's built for scalability, utilizing Kubernetes and AWS auto-scaling for the management, enabling it to scale horizontally.

Introduction to BullMQ: A Core Component

A core piece of Novu's back-end is BullMQ, which is used as a message broker. It's a queue where you add tasks to it and there are workers that listens to jobs.



import { Queue } from 'bullmq';

const myQueue = new Queue('foo');

async function addJobs() {
  await myQueue.add('myJobNAme', { foo: 'bar'});
  await myQueue.add('myJob NAme', {qux: 'baz'});
}

await addJobs();

You might want to use queues if you have long running operations. Imagine an API that compresses images. Each image takes one minute to compres. You don't want your API to take a minute to respond.

The solution?

Asynchronous processing:

You wait for someone to call your API to compress an image.
Immediately return a success response.
In the background, the job is added the to a queue.
The workers are then going to do the image compression.

This also makes the system highly scalable. You can have many background workers handling the compression and can scale out these workers separately from the rest of your app.

Here, you can see a worker being set up, which is listening for events.



import { Worker } from 'bullmq';

const worker = new Worker('foo', async job => {
  // Will print { foo: 'bar'} for the first job
  // and { qux: 'baz' } for the second.
  console.log(job.data);
});

You can have a hundred different workers working in parallel and they're all managing different items in the queue. And this is a great way to scale your app and tasks.

Deep Dive into Novu's Inner Workings

Every notification begins with a trigger. To understand this process, let's delve into the backend. Navigate to the events folder within Novu's repository, and open the events controller file. Here, you'll find an endpoint dedicated to handling POST requests, labeled /trigger.

Once the trigger is initiated, the function trackEvent serves as a handler for incoming trigger events, it takes two parameters UserSession and Body. Inside the function a method is called to process the received data.

This method creates a command object with various details extracted from the user's session and the trigger event, once the command is ready, it's executed, and returns the result as a TriggerEventResponseDto.



// novu/apps/api/src/app/events/events.controller.ts
@Post('/trigger')
@ApiResponse(TriggerEventResponseDto, 201)
async trackEvent(
  @UserSession() user: IJwtPayload,
  @Body() body: TriggerEventRequestDto
): Promise<TriggerEventResponseDto> {
  const result = await this.parseEventRequest.execute(
    ParseEventRequestMulticastCommand.create({
      // parameters
    })
  );
  return result as unknown as TriggerEventResponseDto;
}

Inside the parseEventRequest, this class is doing all sorts of validation, making sure there are workflow steps and active workflow steps. Upon successful validation, this is the point where we actually handle the request and get the job data and add it to a workflow queue service.



// novu/apps/api/src/app/events/usecases/parse-event-request/parse-event request.usecase.ts
if (command.payload && Array.isArray(command.payload.attachments)) {
  this.modifyAttachments(command);
  await this.storageHelperService.uploadAttachments(command.payload.attachments);
  command.payload.attachments = command.payload.attachments.map(({ file, ...attachment }) => attachment);
}

const defaultPayload = this.verifyPayload.execute(
  VerifyPayloadCommand.create({
    payload: command.payload,
    template,
  })
);

command.payload = merge({}, defaultPayload, command.payload);

const jobData: IWorkflowDataDto = {
  ...command,
  actor: command.actor,
  transactionId,
};

await this.workflowQueueService.add({ name: transactionId, data: jobData, groupId: command.organizationId });

Inside the workflow queue service. All it does is add something to the queue or if we get an array of jobs it will bulk add them to the queue. As mentioned earlier, this is how the BullMQ is implemented. When you call the Novu API to trigger a notification, it's adding a notification to the BullMQ queue.



// novu/packages/application-generic/src/services/queues/workflow-queue.service.ts
public async add(data: IWorkflowJobDto) {
  return await super.add(data);
}

public async addBulk(data: IWorkflowBulkJobDto[]) {
  return await super.addBulk(data);
}

After submitting events to the queue, the next step involves running a worker to process those events on the receiving end. createWorker constructs a BullMQ worker, specifying how to process tasks from a queue by providing the processing function, the topic name, and worker options. initWorker simplifies worker setup, logging initialization and calling createWorker to start processing tasks efficiently.



// novu/packages/application-generic/src/services/workers/worker-base.services.ts
public initWorker(processor: WorkerProcessor, options?: WorkerOptions): void {
  Logger.log(`Worker ${this.topic} initialized`, LOG_CONTEXT);
  this.createWorker(processor, options);
}

public createWorker(
  processor: WorkerProcessor,
  options: WorkerOptions
): void {
  this.instance.createWorker(this.topic, processor, options);
}

If I jump back to standard worker, you'll see that I have this.initWorker and over here getWorkerProcessor() is where I'm returning the function.



// novu/apps/worker/src/app/workflow/services/standard.worker.ts
export class StandardWorker extends StandardWorkerService {
  constructor(
    private runJob: RunJob,
    // constructor parameters
  ) {
    super(new BullMqService(workflowInMemoryProviderService));
    this.initWorker(this.getWorkerProcessor(), this.getWorkerOptions());

    this.worker.on('failed', async (job: Job<IStandardDataDto, void, string>, error: Error): Promise<void> => {
      await this.jobHasFailed(job, error);
    });
  }

  private getWorkerOptions(): WorkerOptions {
    return {
      ...getStandardWorkerOptions(),
      settings: {
        backoffStrategy: this.getBackoffStrategies(),
      },
    };
  }
}

Within the getWorkerProcess definition, you'll see that it's calling runJob to execute while also parsing the minimalJobData that we've received.



// novu/apps/worker/src/app/workflow/services/standard.worker.ts
storage.run(new Store(PinoLogger.root), () => {
  _this.runJob
    .execute(RunJobCommand.create(minimalJobData))
    .then(resolve)
    .catch((error) => {
      Logger.error(
        error,
        `Failed to run the job ${minimalJobData.jobId} during worker processing`,
        LOG_CONTEXT
      );
      return reject(error);
    })
    .finally(() => {
      transaction.end();
    });
});

If you look inside the runJob, you'll see that it's calling sendMessage.execute. This command directs the sendMessage class to dispatch various messages, which can be sent via SMS or email.



// novu/apps/worker/src/app/workflow/usecases/run-job/run-job.usecase.ts
await this.sendMessage.execute(
  SendMessageCommand.create({
    // parameters
  })
);

We can jump to the sendMessageEmail class and check the important part, which is the sendMessage. Inside this, it creates a mailFactory and gets the handler, and what this is doing is getting the provider that we need.



// novu/apps/worker/src/app/workflow/usecases/send-message/send-message email.usecase.ts
private async sendMessage(
  integration: IntegrationEntity,
  mailData: IEmailOptions,
  message: MessageEntity,
  command: SendMessageCommand
) {
  const mailFactory = new MailFactory();
  const mailHandler = mailFactory.getHandler(this.buildFactoryIntegration(integration), mailData.from);
  // ...rest of the code
}

In mail.factory.ts, you'll see different provider handlers. If we jump into one of the providers, specifically the Mailgun provider, you can see all it does is call the API that actually sends an email via Mailgun. Once Mailgun successfully delivers the message to the user's inbox, the notification sending process ends.



// novu/providers/mailgun/src/lib/mailgun.provider.ts

this.mailgunClient = mailgun.client({

  username: config.username,

  key: config.apiKey,

  url: config.baseUrl || 'https://api.mailgun.net',

});

Final Thoughts

This exploration has scratched the surface of Novu's capabilities. While there's more to discover, you now grasp the essential flow: triggering events (via UI or API) creates DTOs, pushes them to a queue, and triggers workers to process and send notifications (SMS, email, etc.).

Remember, this is just a glimpse into Novu's broad range of functionalities. Hopefully, you've gained a solid understanding of the core workflow. If you're interested in learning more, I highly recommend exploring the project's GitHub repository.

Stay in touch

Follow me on Twitter
Subscribe to my YouTube channel
Join my Learn from Open Source newsletter
Star Inbox Zero on GitHub

AI assistant for managing your email

Elie — Wed, 06 Mar 2024 19:10:00 +0000

Inbox Zero is the best way to create an AI assistant to manage your email for you automatically.

You give it rules in plain English.

You can run it on autopilot or manually approve each action the AI suggests for you.

It's truly magical.

I highly recommend checking it out.

The app is fully open source too!

The site: https://getinboxzero.com
The GitHub: https://getinboxzero.com/github
Original post: https://telegra.ph/AI-assistant-for-managing-your-email-03-06

The AI magic behind open v0 ✨ - an open source React Tailwind generator

Elie — Tue, 26 Sep 2023 20:17:54 +0000

Last week Vercel announced v0 💎

It gave us a glimpse into the future of UI dev 👨‍💻

A few days later an open source alternative called openv0 was dropped 📖

In this week's video I jump into the AI magic behind it ✨

The Code Behind OpenStatus and how it uses Turborepo

Elie — Sun, 03 Sep 2023 18:39:00 +0000

OpenStatus is an open source monitoring service! It’s built with Next.js, Turso, Tinybird, Turborepo, Shadcn UI, and Tailwind!

In this video I cover the code behind OpenStatus and how to use Turborepo to build a monorepo Next.js app.

To learn from more open source projects, subscribe to my YouTube channel: https://youtube.com/elie2222

Best Admin solutions for Node.js apps.

Elie — Mon, 18 Jan 2021 21:49:19 +0000

Have you ever wasted time on building a custom admin?

There are a lot of out the box solutions can speed things up for you. Some worth looking at are:

Forest Admin
Retool
React Admin

Forest Admin will generate an admin for you in under 5 minutes. It's not great if you need a lot of customisation although it can be customised to what you need most of the time.

Retool is great too. You can write any database queries you need to grab data direct and display it in Retool components.

The last tool is React Admin. Here you actually write frontend code but many of the components and frontend logic has been written for you.

Special mention to Airtable which can be used with care in specific cases. Be careful not to duplicate a lot of information from your real database. If things get out of sync it can be annoying to know what's accurate. I'd recommend not duplicating any data.

Final note: any third part service will help you get going quickly. But if you need a lot of customisation it may be worth to build your admin from scratch. Note that most of these solutions are for internal use, and not for client facing admin views.

Lastly, if you do choose to use a third party admin tool you can always build custom tools alongside it. You're not stuck with any specific tool, and can even use many of them in conjunction. There's no reason Retool and Forest Admin can't run side by side and you use whichever is easier for you at that moment.

Want to learn more? Here's the full video:

Link:
https://youtu.be/a4vXsrM67Rs

How I Built This: IsraelVC — Gatsby, Google Sheets, Now Serverless

Elie — Wed, 29 May 2019 23:06:26 +0000

I recently launched a mini side project, IsraelVC.co, that catalogues the latest investments in Israeli high tech.

It’s a simple site: one page containing a list of the 20 most recent investments that have happened in Israel. It also has a Google Sheet connected to it which acts as the database for the site. The database contains 4000 investments. In the near future the site will show all the investments too.

This post is about how I built it.

Tech Stack

Gatsby (React)
Google Sheets
Now Serverless
Netlify
Mailchimp

Gatsby

Gatsby is a React based static site generator. It allows you to easily create sites that are lightning fast and SEO friendly.

Hosting a Gatsby site is very simple. You can use a service such as Netlify or Now to host the site for free. Free doesn’t mean these services are bad. These services are the best available. They will serve your content from a CDN located near your users. You can’t ask for much better speed and is infinitely scalable without needing to maintain any servers. Win. Win. Win.

Netlify and Now both have continuous deployment set up. What this means is that you can push to your GitHub repo and your site will automatically updated. If you update and push a commit on a development branch, a new deploy will be published at its own url for that branch. There is even a published site for every commit. It’s a great all round developer experience. If you use Now you can also deploy with their command line tool, by simply typing now.

Google Sheets

I use Google Sheets as the database for the project. A typical site would use PostgreSQL, MongoDB or some other database. This project was small enough that I could rely on Sheets to do the job.

Using Sheets as a database may come as a surprise to some people. That’s not how you develop I hear you shout!

So why use Sheets?

For one it’s super simple to use. Anyone can use it. To add information you just open it up and start typing. Compare that to a traditional database. You have to be a developer to update it or read information in it.

Sheets provides permission based read/write access out the box. No need to build an admin panel.

You don’t have to worry about server maintenance or scaling. That’s Google’s job. If I had to list millions of investments I may have run into issues, but that wasn’t a case I had to deal with.

In this specific case I also wanted to give users easy access to the data so that they could play with it as they wish. The spreadsheet itself is public.
Sheets has a tonne of functionality built in. For example, being able to sort or search through thousands of investments. This saved me the trouble of building out the functionality myself.

One other thing you can do is connect a Google Forms to Sheets, with the form responses going straight into the spreadsheet.

Traditional databases have their benefits. Most web or mobile apps need a
backend with a database. Performance, large scale, schemas, backups, triggers, joins, fine grained user access control. The list goes on. You could probably figure out a way to implement some of these features using a spreadsheet, but ultimately this probably isn’t a good idea. Luckily, IsraelVC didn’t need these benefits.

Alternatives

There are some alternatives to Sheets. One is simply storing the information in the repo as JSON. Another option is storing Markdown files in the repo containing the data. This is how a typical Gatsby blog is built. For example, https://overreacted.io/ by Dan Abramov, but there’s an endless list of tech blogs just like it.

The first iteration of IsraelVC actually did this. I used Netlify CMS to help. It gives you a simple admin interface in which you can update your blog posts (or any other data on the site). Once you hit “Publish” it commits the changes to your git repo and if you’re using a CD tool will update your site automatically.

For a blog or landing page where you want to update data easily, Netlify CMS is a great option. You can write straight to the repo and for many developers this is a good solution, but if you want to allow less technical members of your team to update items without needing your help, a CMS is a better solution.

Netlify CMS has its downsides. It’s a little slow to publish and update posts. For the occasional blog post this doesn’t matter, but if you’re entering a lot of data often this can be annoying.

Other headless CMS systems include Contentful, Prismic, and GraphCMS. Wordpress also offers a headless CMS system. I haven’t used any of these so I won’t comment on them, but I assume they provide a more expansive solution than Netlify CMS and will be better a choice for many teams.

Now Serverless

I use Now as the API for my site. All the API does is query my Google sheet of investments and return it.

You can see the code for it on GitHub here:

All in all it’s 60 lines of code and you can deploy it with a single command:

now

It’s also possible to set up continuous deployment for Now with a few clicks, but I have yet to do that.

There are a few pieces of magic I’d like to talk about here.

I don’t have to maintain a server, it’s infinitely scalable, and free (well I pay a minimal amount for it, but you can go with the free plan too and get the same functionality).

It’s fast. Now recently launched serverless pre-rendering
(SPR). What it does is cache responses from your serverless function. My function makes a call to Google Sheets API which can take a few seconds to respond. I don’t want my users waiting a few seconds for a response. They have more important things to do like investing in startups.

SPR caches the response at the edge on CDN and responds immediately. In the background, the function grabs the latest sheets data or whatever else it’s doing and updates the CDN for the next users that visit the site.

To enable SPR, you have to add a single line of code to your function:

res.setHeader(‘Cache-Control’, ‘s-maxage=30, stale-while-revalidate=3600, max-age=0’)

s-maxage=30— tells the CDN to cache the response for 30 seconds.

stale-while-revalidate=3600 — tells the CDN that if the data is stale (i.e. the cached data was updated more than 30 seconds ago), then the stale version of the site should be served. And the cache will be updated in the background so that the data is fresh for the next user that comes to the site. If someone hasn’t been to the site in a long time and the stale data is more than 3600 seconds old (1 hour), the CDN is instructed to not serve the stale data, but instead fetch the latest data from Sheets and serve that to the client directly.

max-age=0— tells the client not to cache the response. The client will always request the latest data from our API.

To test the serverless function locally while developing I used now dev. It’s a new tool and has its problems. It randomly crashes every so often and hot reloading can take a while. The Now team has been working on it and releasing updates. Recently it has worked better, but it’s still not the best experience.

Netlify also provides serverless. It’s called Netlify Functions. I had a little more trouble setting this up. The settings needed for it in the netlify.toml file are confusing, and netlify dev seems even buggier than now dev.

Both netlify dev and now dev are very new, so I will cut them some slack. These are two amazing companies if I haven’t already made it clear how I feel about them :)

Netlify

I currently use Netlify for frontend hosting. I love using Netlify for static deployments. It’s so easy to use. 10/10 on that front. It’s free and even has features like automatic lossless image compression that Now is missing. Having CD setup out the box is a joy to use, especially when developing a project on a team.

There is a downside to having the frontend hosted on Netlify and the serverless backend on Now. In the future I will likely move the frontend to Now as well. The main reason is that I’d like to avoid an extra DNS lookup which happens when the API and frontend are at different domains. This is a bit of a micro optimisation that isn’t strictly necessary, but having seen the DNS lookup for the API take 2.5 seconds one time I’d like to cut that out. Another way to solve the problem is to move the backend to Netlify Functions, but I prefer Now Serverless.

UPDATE:

After posting this the team at Gatsby pointed out another way of integrating Gatsby with Sheets without the need for the Now API server.

There is a Gatsby Sheets plugin:
https://www.gatsbyjs.org/packages/gatsby-source-google-sheets/

This plugin grabs the data from your spreadsheet at build time. You use Gatsby graphql queries to grab the data you want from the sheet.

Taking this approach, you need to trigger a rebuild of your site whenever the Sheets data changes. You can do this by going to your Netlify admin, using the Netlify API, or by adding a button to the sheet that calls the Netlify API and rebuilds the site on click. This third approach is cool and documented here.

Mailchimp

Not much to write here. I will be sending out a weekly email newsletter with that week’s investments so I added a simple Mailchimp form to the site. It’s free up to 2000 users and no hassle. For now I will be manually sending the emails. In the future I may automate this process and use a solution such as Mailgun or SendGrid to do the sending.

Summary

Some great tools exist to massively speed up development, minimise maintenance and lower costs.

Handling servers, scaling, and performance can all be headaches. Paying a few dollars per month for every mini side project you launch is also annoying. Nowadays you can use the best of the best for free.

The stack I covered is simple, but extremely powerful. As a freelance developer I deal with databases and deployments on a weekly basis. I also maintain a site with hundreds of thousands of users. I didn’t choose to use this stack for lack of other options. I chose it because it’s an awesome solution. Deploying to Now or Netlify feels like cheating, but it’s not. It’s the smart choice in many cases.

I’d love to hear your thoughts or any questions in the comments below.

If you enjoyed this post and would like to read more like it in the future, be sure to smash the follow button below. Feel free to reach out to me on Twitter @elie2222:

Or at my website: elie.tech.

4 Techniques for Fixing NPM Packages

Elie — Tue, 05 Mar 2019 10:05:46 +0000

If you’re a Javascript you likely make use of many npm packages. There are times a package has a bug or doesn’t do exactly what you want. This post details a few different ways to work around this.

I have often found myself wasting hours trying to make a one line fix, so I hope the strategies listed here will be of help to others. Many of the tools mentioned are also useful when developing a package.

Solutions

We’ll start with the obvious. The most basic thing you can do to solve your problems is to use a different package. If it’s a small piece of functionality, you can rewrite it yourself and avoid having to use the package.

This sometimes works, but if the package contains a lot of code this solution may not be practical. Beyond completely rewriting from scratch, you could copy and paste the contents of the package into your local project and edit as you need. The disadvantage of such an approach is that you won’t be able to easily follow future updates made by others to the package. Likely not the best solution in many instances.

Perhaps the first thing you should do before trying to fix a package yourself is post an issue in the GitHub repo. The functionality you are trying to add may already exist, or you are misusing the library and the package maintainers can point you in the right direction. If it really is a bug the package maintainers are likely the ones best suited to fix it.

Fix Locally

npm/yarn link

The first way to fix a package is to fork and clone it locally. You can then use npm link or yarn link to use it in your project. Any changes you make to the forked package will be reflected in your project.

You need to run two commands to link. Using yarn:

yarn link # in the root of the forked package
yarn link forked-package # in the root of your project

You will also need to run yarn install on the forked package and run any build/prepublish steps. In the past I have sometimes found the build/prepublish step to be annoying to get working. Your mileage may vary and it depends on how the package was written.

Another problem with this solution is that it only creates a local symlink to the package. This means that you’ve only solved the problem locally, but not for team members.

Once you’ve fixed the problem locally, you will want to publish it to GitHub and either rely on your forked version of the package, make a pull request to the main package, or publish the package to npm under your name.

yalc

Yalc is similar to npm link, but it publishes the forked package locally rather than creating a symlink. I have found that it often works better than npm link as it handles the full build process a package maintainer uses to publish to npm including npm pack.

Using yalc is similar to using npm/yarn link:

yalc publish # in the root of the forked package
yalc add forked-package # in the root of your project

One problem I’ve run into using yalc is when you want to fix a package that is depended on by another package. For more discussion on that situation see this issue.

You can commit yalc changes to git to share the fix with other team members. This is useful to make a quick fix, although likely should not be used as a long term solution.

patch-package

Another solution is to use patch-package. To make use of it you don’t need to fork the buggy package. You can simply edit your node_modules directory with the changes you want.

To use patch-package:

# add to your package.json:
 "scripts": {
   "postinstall": "patch-package"
 }

# fix the broken file in your project:
# e.g. node_modules/some-package/brokenFile.js

You can add the patches to your git repo to share the changes with the rest of the team. Patch-package applies these changes to the project itself by creating a patch file with the changes.

There are some problems with patch-package however. When a package author publishes a project to npm, many of the original files are lost in the process.

For example, if the project was written in TypeScript/ES6+, the original files may not have been published to npm, just the final transpiled version. This is the case if the package.json contains the files field, or the project contains a .npmignore file. Then not all the files in the project will be downloaded to your node_modules folder. Usually this is a good thing as it reduces the amount of data that needs to be downloaded, but it makes it difficult to use patch-package.

You can still edit the final ES5 or minified version using patch-package, but this is usually an uncomfortable experience.

Also, even if the src folder has been published to npm, you will still need to run the build steps locally. If some of the files necessary to build the project have been excluded, this will not be possible unless you grab the missing files from GitHub. At this stage it may just be easier to clone the package locally in a separate folder.

Beyond this, any fixes you make with patch-package are not shared with the wider community and anyone else using this package that may benefit from your changes.

Fork on GitHub

Another solution is to make the changes you need and publish them to GitHub/GitLab (or any other source control platform you use).

This can often be done in conjunction with the steps above. You may first make the fixes using npm/yarn link or yalc, check that all works as expected and then push the changes to your own Git repo. You can then run one of the following to add the package to your project and make it accessible to others (changing the url as appropriate):

yarn add https://github.com/fancyapps/fancybox [remote url]

yarn add https://github.com/fancyapps/fancybox#3.0  [branch/tag]

yarn add https://github.com/fancyapps/fancybox#5cda5b529ce3fb6c167a55d42ee5a316e921d95f [commit]

If you need to do any automated builds, you can take a look at this StackOverflow post.

At this point you can also make a Pull Request to the main repo to get the changes merged in for everyone using the package. Once the changes have been merged in you can go back to using the original package.

Along these lines, you can also republish the package under your own npm account and then install it as you would any npm package.

Final Words

None of the solutions above are perfect. Either patch-package or a yalc followed by committing the changes to GitHub is my favoured solution of those listed above. But all the solutions have problems. After all these years, we are still missing the holy grail for simple package fixes.

If you know of some solutions that I’ve missed and know a better way to fix problems I would love to know in the comments below!

If you enjoyed this article be sure to give me like and follow for similar content in the future :). You can follow me on Twitter at: elie2222.

Some Lesser Known TypeScript Features

Elie — Fri, 15 Feb 2019 15:06:20 +0000

In the past few years, TypeScript has become a popular way to write JavaScript apps. The language is immense and can do a lot of things.

Here’s a short list of some rarer TypeScript features you may not be aware of to help with your development:

You can write numbers with underscores. For example, 1000000000 can be written as 1_000_000_000. It will be compiled to the regular number in Javascript, but using this syntax makes things easier to read. The underscores can be placed anywhere in the number although writing 25 is more readable than 2_5.
If you know a variable is defined you can add a ! after it to access it without first checking if the value is undefined. For example, you may be injecting props into a React component using a decorator (e.g. @inject(router) or @inject(store)). You can now write this.props.store!.someValue even if the definition of store is store?: Store.

Be careful not to overuse this feature, but there are times where it comes in handy. You can read here for further discussion of this problem. (You could always use an if statement too, but that’s more verbose for cases you know the if statement is truthy).

You can set a variable to have type never. This means it can’t be set to anything. How is this useful? One place it can be useful is to check you’ve handled every possibility of a switch statement. For example:

const someFunction = (temperature: 'hot' | 'cold' | 'warm') => {
  switch (temperature) {
    'hot': console.log("it's hot outside!"); break
    'cold': console.log("it's cold outside!"); break
    default: const x: never = temperature;
  }
}

The above code will throw a compile-time error, because x can be reached. If you add a switch case for warm with a break after it, the compile error will go away as the default block can no longer be reached.

Another lesser used type you can use is unknown. This just means we have no idea what the item we have is. This may happen if we’re calling a third party API for example. If we try do x.substr(0, 2) it will throw a compile error. We can use an unknown variable by first doing a type check. For example, writing if (typeof x === “string”) x.substr(0, 2) would no longer throw an error.

No any other useful TypeScript tricks that should be included? Feel free to mention them in the comments!