DEV Community: ilinieja

Serve Next.js with Fastify

ilinieja — Thu, 11 Jul 2024 14:14:31 +0000

Next.js is an exceptional framework for React applications that comes with a lot of bells and whistles for Server-Side Rendering and Static Site Generation. One of the quickest ways to start writing production-ready React without spending the time on setup.

Next.js comes with its own server that can be used out-of-the-box when starting a brand-new projects.
But what if you need to serve Next.js app from an existing Node server? Or maybe you want to have more flexibility additional flexibility for integrating middleware, handling custom routes, etc?

If that's the case - this post is for you, it covers the setup of custom Next.js server with Fastify, solution for Express.js or plain Node.js server will be similar.

Example project used here is also available as a template on Github.

Initial setup

So imagine you have an existing Fastify project. For the sake of example I have a simple Fastify API here. It's initialized from this great Fastify template and has a couple of endpoints returning mock data:

/_health - server status
/api/pokemons - Pokemons list
/api/stats - list of Pokemon stats

// src/app.ts

import { fastify as Fastify, FastifyServerOptions } from "fastify";
import { POKEMONS, STATS } from "./mocks";

export default (opts?: FastifyServerOptions) => {
  const fastify = Fastify(opts);

  fastify.get("/_health", async (request, reply) => {
    return { status: "OK" };
  });

  fastify.get("/api/pokemons", async (request, reply) => {
    return POKEMONS;
  });

  fastify.get("/api/stats", async (request, reply) => {
    return STATS;
  });

  return fastify;
};

Adding Next.js app

It's as easy as just generating a new Next.js project using create-next-app, I'll do it in ./src directory:

cd ./src && npx create-next-app nextjs-app

Handling requests using Next.js

To allow Next.js render pages Fastify needs to pass requests to it.

For this example, I want Next.js to handle all routes under /nextjs-app

// Path Next.js app is served at.
const NEXTJS_APP_ROOT = "/nextjs-app";
fastify.all(`${NEXTJS_APP_ROOT}*`, (request, reply) => {
    // Remove prefix to let Next.js handle request
    // like it was made directly to it.
    const nextjsAppUrl = parse(
      request.url.replace(NEXTJS_APP_ROOT, "") || "/",
      true
    );

    nextjsHandler(request.raw, reply.raw, nextjsAppUrl).then(() => {
      reply.hijack();
      reply.raw.end();
    });
});

Next.js also makes requests to get static, client code chunks etc. on /_next/* routes, need to pass requests from Fastify to it:

// Let Next.js handle its static etc.
fastify.all("/_next*", (request, reply) => {
  nextjsHandler(request.raw, reply.raw).then(() => {
    reply.hijack();
    reply.raw.end();
  });
});

As a result, complete Fastify routing would look like this:

// src/fastify-app.ts

import { fastify as Fastify, FastifyServerOptions } from "fastify";
import { POKEMONS, STATS } from "./mocks";
import nextjsApp from "./nextjs-app";
import { parse } from "url";

const nextjsHandler = nextjsApp.getRequestHandler();

export default (opts?: FastifyServerOptions) => {
  const fastify = Fastify(opts);

  fastify.get("/_health", async (request, reply) => {
    return { status: "OK" };
  });

  fastify.get("/api/pokemons", async (request, reply) => {
    return POKEMONS;
  });

  fastify.get("/api/stats", async (request, reply) => {
    return STATS;
  });

  // Path Next.js app is served at.
  const NEXTJS_APP_ROOT = "/nextjs-app";
  fastify.all(`${NEXTJS_APP_ROOT}*`, (request, reply) => {
    // Remove prefix to make URL relative to let Next.js handle request
    // like it was made directly to it.
    const nextjsAppUrl = parse(
      request.url.replace(NEXTJS_APP_ROOT, "") || "/",
      true
    );

    nextjsHandler(request.raw, reply.raw, nextjsAppUrl).then(() => {
      reply.hijack();
      reply.raw.end();
    });
  });

  // Let Next.js handle its static etc.
  fastify.all("/_next*", (request, reply) => {
    nextjsHandler(request.raw, reply.raw).then(() => {
      reply.hijack();
      reply.raw.end();
    });
  });

  return fastify;
};

Where the nextjsApp comes from Next.js initialization here:

// src/nextjs-app.ts

import next from "next";
import env from "./env";

export default next({
  dev: import.meta.env.DEV,
  hostname: env.HOST,
  port: env.PORT,
  // Next.js project directory relative to project root
  dir: "./src/nextjs-app",
});

And last but not the least - Next.js app needs to be initialized before starting the server:

nextjsApp.prepare().then(() => {
  fastifyApp.listen({ port: env.PORT as number, host: env.HOST });
  fastifyApp.log.info(`Server started on ${env.HOST}:${env.PORT}`);
});

Full server init will look like this:

// src/server.ts

import fastify from "./fastify-app";
import logger from "./logger";
import env from "./env";
import nextjsApp from "./nextjs-app";

const fastifyApp = fastify({
  logger,
  pluginTimeout: 50000,
  bodyLimit: 15485760,
});

try {
  nextjsApp.prepare().then(() => {
    fastifyApp.listen({ port: env.PORT as number, host: env.HOST });
    fastifyApp.log.info(`Server started on ${env.HOST}:${env.PORT}`);
  });
} catch (err) {
  fastifyApp.log.error(err);
  process.exit(1);
}

Build updates

Now Next.js app needs to be built before starting the server, so a couple updates in package.json:

  "scripts": {
    "build": "concurrently \"npm:build:fastify\" \"npm:build:nextjs\"",
    "build:fastify": "vite build --outDir build --ssr src/server.ts",
    "build:nextjs": "cd ./src/nextjs-app && npm run build",
    "start": "pnpm run build && node build/server.mjs",
    ...

Result

With these changes applied, Fastify keeps handling all the routes it initially had:

/_health - server status
/api/pokemons - Pokemons list
/api/stats - list of Pokemon stats

And everything under /nextjs-app is handled by Next.js:

/nextjs-app - main page of the new Next.js app, renders a list of Pokemons using the same data API does

Note on limitations

Vite HMR for the Fastify server became problematic after adding Next.js app - Next.js has separate build setup and it doesn't play well with Vite Node plugin out of the box.
However, HMR for Next.js app works fine and can be used with next dev inside Next.js project.

As Next.js docs mention, using custom server disables automatic static optimizations and doesn't allow Vercel deploys.

MongoDB Materialized Views in NodeJS + Mongoose

ilinieja — Tue, 11 Jul 2023 22:32:21 +0000

What are materialized views?

Materialized views are database objects containing the result of a query. They're useful when we want to quickly retrieve the output of a heavy aggregate query without running it for each request.

In MongoDB, materialized views are just like any other collection. In our case, we want to create materialized views to store multiple time-bucketed analytical views from a large events collection. Depending on a use case, we can update views on a daily, weekly, monthly or any other timespan basis. We can also trigger updates on-demand.

Working example code for this article is available on GitHub.

Original collection

We're going to use an example collection of Orders - large collection of orders made in an online shop. Here's a simplified schema for it.

// Order.ts

const Order = new mongoose.Schema({
  customer: {
    fullName: { type: String, required: true },
    email: { type: String, required: true },
  },
  items: [
    {
      item: {
        name: { type: String, required: true },
        price: { type: Number, required: true },
      },
      quantity: { type: Number, required: true },
    },
  ],
  date: { type: Date, required: true },
});

// This prevents Mongoose from recompiling the model.
export default mongoose.models?.Order || mongoose.model("Order", Order);

Materialized view

We're going to aggregate daily stats of all orders without grouping. That'll reduce out huge event collection to steady-growing 365-documents-a-year collection that's inexpensive to query and store.

This view is expensive to create though since we're aggregating from a whole Orders collection. One of optimizations here can be to only aggregate from the Orders that were created later than the last entry in this materialized view.

It's convenient to store aggregation pipeline for a materialized view model next to it to avoid updating them separately.

// DayStats.ts

const DAY_STATS_COLLECTION_NAME = "day_stats";

// Schema of the aggregate documents stored in materialized view.
const DayStats = new mongoose.Schema(
  {
    _id: { type: String, required: true },
    date: { type: Date, required: true },
    revenue: { type: Number, required: true },
    orders: { type: Number, required: true },
    averageOrderRevenue: { type: Number, required: true },
  },
  { collection: DAY_STATS_COLLECTION_NAME, versionKey: false }
);

// Aggregation pipeline for the materialized view.
export async function calculateDayStats() {
  await Order.aggregate([
    // Flattens items array to calculate sums later.
    { $unwind: { path: "$items" } },
    // Actual calculations for totals and averages.
    {
      $group: {
        _id: { $dateToString: { format: "%Y-%m-%d", date: "$date" } },
        revenue: {
          $sum: { $multiply: ["$items.quantity", "$items.item.price"] },
        },
        orders: { $sum: 1 },
        averageOrderRevenue: {
          $avg: { $multiply: ["$items.quantity", "$items.item.price"] },
        },
      },
    },
    // Adding date object after grouping for querying 
    // the materialized view in future (to use date comparison).
    {
      $addFields: {
        date: {
          $dateFromString: { dateString: "$_id", format: "%Y-%m-%d" },
        },
      },
    },
    // Rounding calculated values.
    {
      $set: {
        revenue: { $round: ["$revenue", 5] },
        orders: { $round: ["$orders", 5] }, 
        averageOrderRevenue: { $round: ["$averageOrderRevenue", 5] },
      },
    },
    // Saving to materialized view (writing into collection).
    { $merge: { into: DAY_STATS_COLLECTION_NAME, whenMatched: "replace" } },
  ]);
}

// This prevents Mongoose from recompiling the model.
export default mongoose.models?.DayStats ||
  mongoose.model("DayStats", DayStats);

Updating the materialized view

Now we have the materialized view and we need to update it daily. For this, we'll use Agenda, a light-weight job scheduling library for Node.js with Mongo backed persistence layer.

We'll have a worker script that'll set up the queue, schedule updates and update the materialized view based on that schedule.

Worker script needs to be always running - it regularly checks the scheduled jobs and triggers them when needed.

// worker.ts

import * as dotenv from "dotenv";
dotenv.config();

import setupQueue from "./setup-queue";
import scheduleDbViewUpdates from "./db-view-updates";

(async function () {
  const queue = await setupQueue();
  const jobs = await scheduleDbViewUpdates(queue);

  console.info("Jobs worker started");
})();

setup-queue is just going to initialize Agenda and point it to MongoDB.

// setup-queue.ts

import { Agenda } from "@hokify/agenda";

const DATABASE_URI = process.env.DATABASE_URI;
const DATABASE_COLLECTION = process.env.JOBS_DATABASE_COLLECTION;

if (!DATABASE_URI || !DATABASE_COLLECTION) {
  throw new Error(
    "DATABASE_URI and JOBS_DATABASE_COLLECTION are missing in env"
  );
}

export default async function setupQueue() {
  const agenda = new Agenda({
    db: {
      address: DATABASE_URI,
      collection: DATABASE_COLLECTION,
    },
    processEvery: "5 minutes",
  });

  agenda.on("start", (job) => {
    console.info(`Job ${job.attrs.name} started`);
  });

  agenda.on("complete", (job) => {
    console.info(`Job ${job.attrs.name} completed`);
  });

  agenda.on("fail", (err, job) => {
    console.error(`Job ${job.attrs.name} failed`, err);
  });

  await agenda.start();

  return agenda;
}

And db-view-updates will schedule our view updates.

// db-view-updates.ts

import { Agenda } from "@hokify/agenda";

import dbConnection from "../shared/dbConnection";
import { calculateDayStats } from "../models/DayStats";

const JOB_NAMES = {
  CALCULATE_DAY_STATS: "CALCULATE_DAY_STATS",
  CALCULATE_DAY_GEO_BUCKET_STATS: "CALCULATE_DAY_GEO_BUCKET_STATS",
  CALCULATE_DAY_PLATFORM_STATS: "CALCULATE_DAY_PLATFORM_STATS",
};

export default async function scheduleDbViewUpdates(agenda: Agenda) {
  const jobs = [];

  agenda.define(JOB_NAMES.CALCULATE_DAY_STATS, async (job) => {
    const connection = await dbConnection();

    await calculateDayStats();

    connection.disconnect();
  });

  jobs.push(await agenda.every("1 day", JOB_NAMES.CALCULATE_DAY_STATS));

  return jobs;
}

Conclusion

This simple setup allows adding arbitrary number of materialized views for performance-intensive MongoDB aggregations. It can also be used with aggregations joining multiple collections.

When extending this setup it's important to keep an eye on aggregations performance and optimize them using indexes, only recalculating for new documents and running aggregations on secondary replica.

Working example code is available on GitHub.

Web Security Refresher for Frontend Developers

ilinieja — Thu, 08 Jun 2023 17:07:20 +0000

Security is extremely important when developing for the Web.
And unfortunately, it's often overlooked when building frontend developer skillset.

This article gives a learning path and links to some great materials to build or refresh your Web security knowledge.

The Basics of Web Application Security by Martin Fowler

Great long-read giving an overview of risk areas in a Web application and guidance for how to mitigate them.

Perfect starting point or knowledge gap filler - you can dig deeper into each of the topics using the links provided and googling the threats mentioned.

OWASP Top Ten and Cheat Sheets

OWASP Top Ten threats are affecting all layers of Web apps, not only the frontend.

It's good to know and monitor the Top Ten, but the following cheat sheets are must-read and must-remember for a Frontend developer:

HTML5 Security
XSS Prevention
DOM based XSS Prevention
XSS Filter Evasion (reference for XSS mitigation testing)
CSRF Prevention
CSP
Secure Headers Project

Kontra OWASP Top Ten Training

Interactive trainings for the common Web vulnerabilities based on real-world cases. The format is perfect for understanding how the attacks can be carried out against your app.

Note: There's a paywall appearing after passing a couple of trainings. You'll need to make a post on LinkedIn to get free access to the rest of the trainings.

Web Application Security Course - Mike North (FrontendMasters)💸

Practical guide using example NodeJS app to show common vulnerabilities and their fixes. You'll get most out of the practical part if you use NodeJS and render server-side. The tech stack of the example is a bit old, but the vulnerabilities shown are still relevant.

There's also his talk on Web Security available on YouTube, it touches the topics from the course, but without the practical part.

React Security Fundamentals

Project-based tutorial showing you how to build a secure Web application featuring Authorization and Role Based Access Control. Most useful if you're building a React app and NodeJS API and need some directions on how to organize it.

Conclusion

There's a lot of vulnerabilities and information on them on the Web and it's easy to get lost figuring out how to mitigate the risks in your application.
I hope this article makes it a bit easier to build the knowledge necessary to protect your users.

Suggestions for other materials are most welcome.