DEV Community: Callis Ezenwaka

Building a Multi-Service Application with a Shared Database Layer.

Callis Ezenwaka — Tue, 25 Mar 2025 09:55:25 +0000

In today's post, I'll walk through how to build a multi-service application that shares a common database layer. This architecture pattern is particularly useful when you're breaking a monolith into microservices or when you want to separate concerns while maintaining data consistency. All code examples and the complete implementation are available on GitHub at https://github.com/callezenwaka/database.

Project Structure

Let's look at the architecture we'll be building:

.
├── LICENSE
├── README.md
├── backend
│   ├── Dockerfile
│   ├── README.md
│   ├── src
│   └── tsconfig.json
├── database
│   ├── Dockerfile
│   ├── README.md
│   ├── src
│   └── tsconfig.json
├── docker-compose.yml
└── server
    ├── Dockerfile
    ├── README.md
    ├── src
    └── tsconfig.json

This structure follows a clean separation of concerns:

database: Contains shared database functionality
backend: Application backend service
server: Application server service

The Shared Database Package

The core of our architecture is the shared database package which provides:

A unified connection mechanism
Migration management
Consistent APIs for database operations
Entity definitions shared across services

Database Package Overview

The database package is designed to be consumed by other services within our application. It's not published to npm but referenced locally by other packages.
Here's what makes it special:

Automatically builds TypeScript code when installed by other services
Provides a consistent interface for database operations
Handles connection pooling and lifecycle management
Centralizes migration and seeding logic

Implementation Details

Setting Up the Database Package

The database package uses TypeORM to interface with PostgreSQL. Here's how to use it:

import { getDataSource, closeDatabase } from '@database/database';

// Connect to the database
async function connectToDatabase() {
  const dataSource = await getDataSource();

  // Use the dataSource for database operations

  // When done, close the connection
  await closeDatabase();
}

Creating a Resilient Service

Let's examine a real-world implementation from our backend service's entry point:

// backend/src/index.ts
... // other codes

async function bootstrap() {
  try {
    // Start the server first
    const server: Server = app.listen(PORT, () => {
      logger.info(`Backend running on http://localhost:${PORT}`);
    });

    // Check Redis connection
    try {
      const pingResult = await redisClient.ping();
      app.locals.redisAvailable = pingResult === 'PONG';
      logger.info('Redis connection established:', app.locals.redisAvailable);
    } catch (redisError) {
      app.locals.redisAvailable = false;
      logger.warn('Redis connection failed:',
        redisError instanceof Error ? redisError.message : String(redisError));

      // Redis will automatically attempt reconnection based on its config
      redisClient.on('connect', () => {
        app.locals.redisAvailable = true;
        logger.info('Redis connection restored');
      });

      redisClient.on('error', () => {
        app.locals.redisAvailable = false;
      });
    }

    // Then attempt database connection
    try {
      await getDataSource('backend');
      logger.info('Database connection established. Full functionality available.');
      app.locals.databaseAvailable = true;
    } catch (dbError) {
      logger.error(String(dbError));
      let databaseError = dbError instanceof Error ? dbError.message : String(dbError);
      logger.warn('Database connection failed. Operating in limited functionality mode:', databaseError);
      app.locals.databaseAvailable = false;

      // Attempt periodic reconnection with attempt tracking
      let reconnectAttempts = 0;
      const MAX_RECONNECT_ATTEMPTS = 10; // Set to -1 for unlimited attempts

      const reconnectInterval = setInterval(async () => {
        reconnectAttempts++;

        try {
          await getDataSource('backend');
          logger.info(`Database connection established after ${reconnectAttempts} attempts. Full functionality restored.`);
          app.locals.databaseAvailable = true;
          app.locals.reconnectAttempts = reconnectAttempts;
          clearInterval(reconnectInterval);
        } catch (error) {
          // Log with attempt count
          logger.debug(`Database reconnection attempt ${reconnectAttempts} failed`);

          // Check if we've reached the max attempts (if not unlimited)
          if (MAX_RECONNECT_ATTEMPTS > 0 && reconnectAttempts >= MAX_RECONNECT_ATTEMPTS) {
            logger.error(`Maximum reconnection attempts (${MAX_RECONNECT_ATTEMPTS}) reached. Giving up on database connection.`);
            app.locals.dbReconnectionExhausted = true;
            clearInterval(reconnectInterval);
          }
        }
      }, 30000); // try every 30 seconds
    }

    // Handle graceful shutdown
    const shutdown = async () => {
      logger.info('Shutting down server...');
      // Shutdown logic for server, Redis, and database
      // ...
    };

    // Handle termination signals
    process.on('SIGTERM', shutdown);
    process.on('SIGINT', shutdown);

  } catch (error) {
    logger.error('Failed to start server:',
      error instanceof Error ? error.message : String(error));
    process.exit(1);
  }
}

bootstrap();

Using the Database Package in Services

To use the database package in your services, add it as a local dependency:

{
  "dependencies": {
    "@database/database": "file:../database"
  }
}

This creates a symbolic link to the database package, allowing your service to import its functionality.

Local Development

Starting the Development Environment

Our project includes Docker configurations for easy local development:

# Start docker components
docker-compose down -v && docker-compose up -d

# Confirm database is running
docker exec -it database-app-postgres-1 psql -U app_user -d app_db

# Check service logs
docker logs --tail 20 database-service-name

Running Services

If you decide to run the servers locally, you could use the command:

# Start server
npm run dev # npm run start:dev - for initialization

Database Management

Running Migrations

The database package includes scripts for managing database migrations, which are defined in the package.json file:

{
  "name": "@database/database",
  "version": "1.0.0",
  "scripts": {
    "migrate": "ts-node src/run-migrations.ts",
    "typeorm": "typeorm-ts-node-commonjs",
    "migration:run": "typeorm-ts-node-commonjs migration:run -d src/index.ts",
    "migration:generate": "typeorm-ts-node-commonjs migration:generate -d src/index.ts -n",
    "migration:create": "typeorm-ts-node-commonjs migration:create src/migrations/",
    "seed": "ts-node src/seeds/run-seeds.ts",
    "reset": "npm run migration:run && npm run seed"
  }
}

These scripts make database management convenient:

# Run all pending migrations
npm run migration:run

# Generate a new migration based on entity changes
npm run migration:generate -- MigrationName

# Create a new empty migration
npm run migration:create

Seeding Data

For development and testing, you can seed your database using the predefined scripts:

# Run seed scripts
npm run seed

# Reset database (run migrations and seeds)
npm run reset

The reset command is particularly useful for local development as it runs all migrations and then seeds the database with test data in one step.

Resilient Connection Management

A robust multi-service application needs to handle connection issues gracefully. Let's break down the key components of our resilient connection management:

Connection States

Our backend service tracks connection states in the application:

app.locals.databaseAvailable = true; // Database connection status
app.locals.redisAvailable = true;    // Redis connection status

This allows the application to provide limited functionality even when connections fail.

Automatic Reconnection

For database connections:

The service attempts reconnection with an exponential backoff strategy
Reconnection attempts are tracked and can be limited
The application maintains state awareness throughout

For Redis:

We leverage Redis's built-in reconnection capabilities
Event listeners track connection state changes
The application adapts functionality based on availability

Graceful Shutdown

The shutdown process:

Stops accepting new requests
Closes the HTTP server
Terminates Redis connections
Closes database connections
Exits the process with appropriate status code

This ensures all resources are properly released when the application terminates.

Architecture Benefits

This architecture offers several advantages:

Code reuse: Database logic is written once and shared across services
Consistency: All services interact with the database in the same way
Maintenance: Database changes affect a single package, not multiple services
Separation of concerns: Each service focuses on its core functionality
Resilience: Services can handle connection failures gracefully
Self-healing: Automatic reconnection attempts restore full functionality

Testing

We can test the full implementation by spinning up docker compose with docker command docker-compose down -v && docker-compose up -d.
Then, we could use postman to make API calls. To add a new user, make a POST request to the backend POST endpoint.

{
    "username": "User Test",
    "email": "user@test.com",
    "password": "password"
}

While to add a new blog, make a POST API call to server endpoint:

{
    "title": "Getting Started with Coding",
    "description": "A new beginner's guide to TypeScript and its powerful features.",
    "author": "User Test",
    "isActive": true
}

Troubleshooting

If you encounter issues importing from the database package, ensure:

The package has been built (dist directory exists)
The service has the correct reference in package.json
TypeScript paths are configured correctly in the consuming service

Conclusion

A shared database layer can simplify development across multiple services while maintaining consistency and reducing duplication. By centralizing database logic, you create a more maintainable system that's easier to evolve over time.

If you would like to explore this architecture further or use it as a starting point for your own projects, check out the complete implementation at https://github.com/callezenwaka/database.

The repository includes all the code we have discussed in this post, plus additional examples and configurations that can help you get started quickly.

Have you implemented a similar architecture? I'd love to hear about your experiences in the comments!

Using Edge Functions to Mitigate Cost and Latency for Distributed Users.

Callis Ezenwaka — Wed, 26 Apr 2023 15:43:16 +0000

Introduction

Edge functions is a piece of code that executes on an “edge network” or a “Content Delivery Network” (CDN). Edge Networks, which consist of servers that sit on the "edge" of multiple networks, act as bridges to route traffic between nodes or networks across the CDN (1).

Edge networks are networks of servers, distributed around the world, working together to serve cached content to users from the closest server location possible. Edge functions are serverless functions while all serverless functions are not edge functions.

Serverless functions are codes deployed mostly on the cloud and executed on demand, without need for provisioning their own servers, or scaling their application to meet traffic spikes. They are also available in broad geographic regions.

In contrast, Edge functions are serverless functions executed in the location closest to the user’s request. Edge functions are not constrained to specific pieces of code or function. They have the capacity to run just any other business applications.

Since all the business logic is deployed to an edge function, when the user sends a request to a node, it is the function that determines how to respond to the browser. The following cloud providers offer edge functions or a variant of it: Netlify, Vercel, AWS, Azion, Cloudflare, Azure etc.

Benefits of Edge Functions

Edge functions deliver content to users with a fast and personalized experience, and has the following additional benefits (2):
• Decreased latency.
• Reduction in cold start boot times.
• Conditional routing.
• Adding a custom HTTP response header.
• A/B testing.
• Setting a cookie.
• Personalizing by geolocation.
• Authentication.

Common Drawbacks

However, while edge functions offer users the above benefits, they nevertheless have the following drawbacks:
• Latency due to data source location or origin.
• Low execution time.
• Limited memory and functions size.
• Capped invocation limits.
• Unavailable and unsupported native APIs.
• Maximum initial response time.

Conclusion

With the exception of drawbacks, some of which were listed above, edge functions have the potential to reduce cost as it will reduce the latency between a user request and the response time.

They have reduced cold start boot time and thus, could intercept user requests faster and deliver personalized responses based on the user location. Such use of geolocation for content delivery and data management could help mitigate compliance or regulatory challenges such as GDPR.

Of the different cloud providers researched, Amazon CloudFront appears to be a more dominant player in this field. Not only that it has two different flavors for edge networks, but it also has the capability to serve video on demand or live stream real-time events, encrypt specific fields throughout system processing, and serve private content using edge customizations (3).

Since supporting real-time communication between distributed users is an operation that has value only during the interaction time. Hence, round tripping to the main data center could introduce latency and its associated cost.

Edge networks have far reached advantages, most of which were outlined above. The use of edge networks could suffice and deliver more secure, reliable, effective, and affordable services to distributed users.

Resources

https://whitep4nth3r.com/blog/what-is-the-edge-serverless-functions/#edge-functions-are-serverless-functions-at-the-edge
https://vercel.com/docs/concepts/functions/edge-functions/limitations
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/IntroductionUseCases.html
https://www.tigera.io/blog/containerized-air-gapped-edge-platform-architecture/
https://www.netlify.com/blog/edge-functions-explained/
https://www.azion.com/en/documentation/products/edge-application/edge-functions/
https://developers.cloudflare.com/workers/learning/using-websockets/
https://azure.microsoft.com/en-us/blog/microsoft-partners-with-the-industry-to-unlock-new-5g-scenarios-with-azure-edge-zones/

Real-Time Data Visualization with D3.js and Vue.js.

Callis Ezenwaka — Sat, 22 Apr 2023 22:48:24 +0000

Many web applications and solutions are becoming more dynamic and responsive. This is, in part, due to high and scalable computing capacity. More web applications are deployed on various cloud platforms as a result of high reliability, availability and security.

To add to this, is the end users' continual demand for real time experience. As organizations becomes more data-driven, many are using data visualizations to either tell their stories or share real time feedback with their customers.

This tutorial will use D3.js, a JavaScript library for manipulating documents based on data and Vue.js, a progressive JavaScript framework used for building user interfaces, to demonstrate real time visualization.

Firstly, we will create a vue.js project using vite.js, a build tool for modern web development that is designed to be fast and efficient. If you do not have vite already installed, run the following command npm create vite@latest.

Then, navigate to project directory and run npm init vue@latest and follow the prompt by picking a project name, selecting TypeScript and other relevant features.

✔ Project name: … <your-project-name>
✔ Add TypeScript? … No / Yes
✔ Add JSX Support? … No / Yes
✔ Add Vue Router for Single Page Application development? … No / Yes
✔ Add Pinia for state management? … No / Yes
✔ Add Vitest for Unit testing? … No / Yes
✔ Add Cypress for both Unit and End-to-End testing? … No / Yes
✔ Add ESLint for code quality? … No / Yes
✔ Add Prettier for code formatting? … No / Yes

Scaffolding project in ./<your-project-name>...
Done.

This will scaffold a vue.js project with similar project tree:

├── README.md
├── env.d.ts
├── index.html
├── node_modules
├── package-lock.json
├── package.json
├── public
│   └── favicon.ico
├── src
│   ├── App.vue
│   ├── assets
│   ├── components
│   ├── main.ts
│   ├── router
│   ├── types
│   └── views
├── tsconfig.app.json
├── tsconfig.config.json
├── tsconfig.json
├── tsconfig.vitest.json
└── vite.config.ts

Afterwards, confirm that your package.json file has the following dependencies, else copy and paste (more reliably install them individually if the latest version is your thing) them. Make sure that you are inside your project directory. Otherwise, change to your project directory and run npm install.

  "dependencies": {
    "@types/d3": "^7.4.0",
    "@types/d3-shape": "^3.1.0",
    "d3": "^7.6.1",
    "d3-shape": "^3.1.0",
    "vue": "^3.2.45",
    "vue-router": "^4.1.6",
    "vuex": "^4.1.0"
  },
  "devDependencies": {
    "@rushstack/eslint-patch": "^1.1.4",
    "@types/jsdom": "^20.0.1",
    "@types/node": "^18.11.12",
    "@vitejs/plugin-vue": "^4.0.0",
    "@vue/eslint-config-prettier": "^7.0.0",
    "@vue/eslint-config-typescript": "^11.0.0",
    "@vue/test-utils": "^2.2.6",
    "@vue/tsconfig": "^0.1.3",
    "eslint": "^8.22.0",
    "eslint-plugin-vue": "^9.3.0",
    "jsdom": "^20.0.3",
    "npm-run-all": "^4.1.5",
    "prettier": "^2.7.1",
    "typescript": "~4.7.4",
    "vite": "^4.0.0",
    "vitest": "^0.25.6",
    "vue-tsc": "^1.0.12"
  }

Navigate to the components directory and create the chart folder (Barchart for this example) and a BarChart.vue file as shown below:

├── BarChart
│   └── BarChart.vue

Open the .vue file and paste the following code:

<template>
  <div class="" ref="barChartRef"></div>
</template>

This is a vue.js code template with a ref?: VNodeRef property called barChartRef. We will define the ref inside the script section. Next create a script section and import the D3.js library and other Vue.js dependencies.

For demonstration purposes, we will be skipping some codes here but are available in the repository.

<script setup lang="ts">
// @ is an alias to /src
import * as d3 from "d3";
import { onMounted, onBeforeUnmount, computed, ref, reactive, watch, } from "vue";
...

</script>

We can also define our data, data interface (schema) and other variables for the chart.

interface Input {
  label: string;
  value: number;
}

const barChartRef = ref<HTMLElement | null>(null);
const margin = reactive({ top: 30, right: 20, bottom: 30, left: 40 });
const height = ref(360);
const width = ref(
  (barChartRef?.value?.offsetWidth || 300) - margin.left - margin.right
);

const revenue = ref([
  {
    label: "2013-06-30",
    value: 660,
  },
  {
    label: "2014-12-31",
    value: 814,
  },
  {
    label: "2015-06-30",
    value: 1131,
  },
  {
    label: "2016-12-31",
    value: 1267,
  },
  {
    label: "2017-06-30",
    value: 1514,
  },
  {
    label: "2018-12-31",
    value: 1763,
  },
  {
    label: "2019-06-30",
    value: 2653,
  },
  {
    label: "2020-12-31",
    value: 6148,
  },
  {
    label: "2021-06-30",
    value: 12394,
  },
  {
    label: "2022-12-31",
    value: 2162,
  },
]);

Then, we will transform our data to the right format as below using a computed function:

const data = computed(() => {
  return revenue.value.map((d: Input) => {
    return {
      label: d.label,
      value: d.value,
    };
  });
});

We are going to use other Vue.js native API functions to control how our chart is rendered on the page.

onMounted(async () => {
  window.addEventListener("resize", handleResize);
  handleResize();
  handleDraw();
});

onBeforeUnmount(() => {
  window.removeEventListener("resize", handleResize);
});

watch(
  () => width.value,
  () => {
    remove();
    handleDraw();
  }
);

To avoid having multiple charts when the page re-renders, we can create a function that checks if the number of charts on the page exceeds one. If true, it will purge all except one.

const remove = () => {
  // TODO: Get svg reference
  const svg = d3.select(barChartRef.value).selectAll("svg");

  // check the number of existing elements, if greater than 0; remove all existing ones
  if (svg.size()) svg.remove().exit();
};

We can make the chart to be responsive for every device screen by checking the user's screen size and displaying a chart within the width of the screen.

const handleResize = () => {
  if (barChartRef?.value?.offsetWidth) {
    width.value = barChartRef.value.offsetWidth - margin.left - margin.right;
    return;
  }

  if (window.innerWidth < 400) {
    width.value = 300 - margin.left - margin.right;
    return;
  }

  width.value = 550 - margin.left - margin.right;
  return;
};

Finally, let us create a function that with generate our chart every time the page renders.

const handleDraw = async () => {
  // append the svg object to the body of the page
  const svg = d3
    .select(barChartRef.value)
    .append("svg")
    .attr("width", width.value + margin.left + margin.right)
    .attr("height", height.value + margin.top + margin.bottom)
    .append("g")
    .attr("transform", `translate(${margin.left},${margin.top})`);

  // Add X axis
  const x = d3
    .scaleBand()
    .domain(data.value.map((d) => d.label))
    .range([0, width.value])
    .padding(0.2);

  svg
    .append("g")
    .attr("transform", `translate(0, ${height.value})`)
    .call(d3.axisBottom(x))
    .selectAll("text")
    .attr("transform", "translate(-10,0)rotate(-45)")
    .style("text-anchor", "end");

  // Add Y axis
  const y = d3
    .scaleLinear()
    .domain([0, d3.max(data.value, (d): number => d.value)] as number[])
    .range([height.value, 0]);

  svg.append("g").call(d3.axisLeft(y));

  // Bars
  svg
    .selectAll("mybar")
    .data(data.value)
    .enter()
    .append("rect")
    .attr("x", (d) => x(d.label) as number)
    .attr("y", (d) => y(d.value))
    .attr("width", x.bandwidth())
    .attr("height", (d) => height.value - y(d.value))
    .attr("fill", "#4682b4");

  svg
    .append("text")
    .attr("class", "title")
    .attr("x", width.value / 2)
    .attr("y", 0 - margin.top / 2)
    .attr("text-anchor", "middle")
    .text("Company Revenue in USD ($)");
};

In the above code, we added both the x-axis and y-axis components. First, we defined both axes, then we formatted them to fit into our chart using the following code snippet:

  // Add X axis
  const x = d3
    .scaleBand()
    .domain(data.value.map((d) => d.label))
    .range([0, width.value])
    .padding(0.2);

  svg
    .append("g")
    .attr("transform", `translate(0, ${height.value})`)
    .call(d3.axisBottom(x))
    .selectAll("text")
    .attr("transform", "translate(-10,0)rotate(-45)")
    .style("text-anchor", "end");

  // Add Y axis
  const y = d3
    .scaleLinear()
    .domain([0, d3.max(data.value, (d): number => d.value)] as number[])
    .range([height.value, 0]);

  svg.append("g").call(d3.axisLeft(y));

A sample output should look like the image below:

That is all we need to do in order to create a real time responsive and dynamic visualization using D3.js and Vue.js. The repository for this tutorial is on GitHub.

If you like the article, do like and share with friends.

Deploy Node.js and Vue.js as Multi-Service Applications on the same Google App Engine.

Callis Ezenwaka — Mon, 10 Apr 2023 17:48:58 +0000

Many startups fall into the trap of premature optimization, investing excessive resources in complex infrastructure like microservices before validating their business model. This misalignment of priorities diverts precious engineering time away from customer acquisition and revenue generation, often leading to technically impressive but commercially unsuccessful ventures.

Google App Engine (GAE) offers a compelling alternative as a serverless platform that handles scalability, security, and reliability concerns automatically. GAE enables organizations to focus primarily on their business logic and core value proposition, rather than operational complexities by abstracting away infrastructure management.

The challenge becomes particularly evident in polyglot architectures where applications are developed using different programming languages and frameworks. Traditional approaches might require manually managing containers or virtual machines for each component, adding significant operational overhead and complexity.

GAE provides an elegant solution by allowing multiple applications—such as a Vue.js client interface, React dashboards, Python analytics and Node.js backendl, to be deployed within a single project. This approach maintains technological flexibility while dramatically reducing the operational burden, even as the architecture expands to include recommendation engines, analytics services, and other specialized components.

For early-stage businesses, the priority should be building "extensible enough" infrastructure that supports rapid iteration and learning until the business model is proven. Only after achieving product-market fit and sustainable revenue does it make sense to invest heavily in more sophisticated technical foundations—and even then, managed platforms often remain the most efficient solution.

Let us start by virtualizing our client project tree:

.
├── README.md
├── app.yaml
├── index.html
├── node_modules
├── package-lock.json
├── package.json
├── public
│   └── vite.svg
├── src
└── vite.config.js

The node_modules directory is redacted to maximize space. If we are looking at the project tree, there is an app.yaml file that contains the GAE commands and required application parameters as shown below:

# Runtime engine nodejs 18 LTS
runtime: nodejs18
service: default
handlers:
  # Serve all static files with urls ending with a file extension
- url: /(.*\..+)$ 
  static_files: dist/\1
  upload: dist/(.*\..+)$
  # catch all handler to index.html
- url: /.*
  static_files: dist/index.html
  upload: dist/index.html

It has the runtime, service and handler for routing request. It is worth mentioning that the first deployed application takes default as it's service name and might throw an error if this is skipped. You can read documentation for more information.

Navigate to the google cloud console and configure the cloudbuild service account xxxxxxxxxxxxxx@cloudbuild.gserviceaccount.com by add App Engine Deployer, App Engine Deployer, and Storage Object Viewer roles.

Next, let's build our Vue.js project by running the command npm run build. Then, we will deploy the application by running the following command gcloud app deploy --project <project-name> from the root directory. Once the deploy is successful, navigate to https://<project-name>.<region_id>.r.appspot.com/ to access the default application.

To deploy the node.js server service within the same project, we follow the same process. First, the project tree looks like the following:

.
├── README.md
├── app.yaml
├── controllers
│   └── user.js
├── database
│   └── index.js
├── index.js
├── lib
│   └── auth.js
├── migrate.js
├── node_modules
├── package-lock.json
├── package.json
└── routes
    └── user.js

This is a simple and typical Node.js application and our interest is the app.yaml file which contains the following key-value pairs runtime: nodejs18 and service: server. As above, we will deploy the application by running the following command gcloud app deploy --project <project-name> from the root directory. And visit the application by navigating to https://<service-name>-dot-<project-name>.<region_id>.r.appspot.com/.

One final thing that we might need to add is mapping a domain name to these services. To do that, we can a dispatch.yaml file to either the client or server application.

dispatch:
- url: "www.client.com/*"
  service: default
- url: "client.com/*"
  service: default
- url: "server.com/*"
  service: server

Client application is preferable as it is the default application. Worthy to mention is that we can separate the content of the dispatch file into respective application, the choice is yours. Ensure that you have added all the domain and subdomain in the custom domain tab of the App Engine.

Then deploy the dispatch.yaml file by running the command gcloud app deploy dispatch.yaml. If all deployments are successful, visiting any of the urls in the dispatch file should open any of the mapped application.

Again, we just deployed only two services, but it could have been more to fit our peculiarities. The repository for this tutorial is on GitHub.

If you like the article, do like and share with friends.

Use Firebase Auth to Manage User Permissions and Enforce Principle of Least Privilege on API Endpoints. Part 2.

Callis Ezenwaka — Mon, 10 Apr 2023 11:45:31 +0000

In the part 1 of User Permissions and the Principle of Least Privilege on API Endpoints using Firebase, we explained the need for managing user authentication and authorization using roles and permissions.

We also applied the principle to various server endpoints. Here, we are jumping straight into client implementation using Vue.js, An approachable, performant and versatile framework for building web user interfaces.

A quick display of the project tree is depicted below:

.
├── README.md
├── index.html
├── package-lock.json
├── package.json
├── public
│   └── vite.svg
├── src
│   ├── App.vue
│   ├── assets
│   │   ├── avatar.png
│   │   └── vue.svg
│   ├── components
│   │   ├── core
│   │   │   └── Header.vue
│   │   ├── generics
│   │   └── user
│   │       ├── User.vue
│   │       ├── UserCard copy.vue
│   │       ├── UserCard.vue
│   │       ├── UserCreate.vue
│   │       ├── UserSkeleton.vue
│   │       ├── UserUpdate.vue
│   │       └── Users.vue
│   ├── config
│   │   └── index.js
│   ├── database
│   │   └── index.js
│   ├── firebase
│   │   └── index.js
│   ├── main.js
│   ├── router
│   │   └── index.js
│   ├── service
│   │   └── index.js
│   ├── store
│   │   ├── index.js
│   │   └── modules
│   │       └── user.js
│   ├── style.css
│   ├── utils
│   │   └── index.js
│   └── views
│       ├── Dashboard.vue
│       ├── Login.vue
│       ├── NotFound.vue
│       └── Register.vue
└── vite.config.js

Unlike in the server side, the client side is straight forward as most of the heavy lifted has already been performed on the server. However, few directories deserve a mention: components, config, firebase, store, and utils.

The component directory has all the various pages for performing CRUD operations on the client side. The config directory has axios implementation for making requests and getting responses from the server.

export const request = async (url, method, token, payload, query) => {
  return await axios({
    method: method,
    url: `${url}`,
    data: payload,
    params: query,
    headers: {
      'Accept': 'application/json',
      'Authorization': `Bearer ${store.getters.idToken}`,
    },
    json: true,
  });
};

Then, the firebase directory has the Firebase configuration object containing keys and identifiers for our application.
The store directory has, well, the Veux store that manages our data state and help centralize the implementation of requests to API endpoints. We will come back to Veux store shortly.

And finally, the utils directory that has our custom defined functions. Within this file, we defined the isAuhorised method that together with firebase onAuthStateChanged supports the re-validation of user token and claims.

/**
 * check user authorisation
 * @param {string} action 
 * @returns {boolean}  
 */
export const isAuhorised = (action) => {
  // TODO: get current user permissions
  const { permissions } = store.getters.profile;
  // TODO: verify if user has permission for the intended action
  return permissions.some(permission => permission.name === action && permission.value === true);
};

The main.js file serves as the entry point to our client application. Here, we use the onAuthStateChanged method from the firebase/auth to identify and setup the current signed in user, authentication status, role and permissions custom claims, and id token.

onAuthStateChanged(auth, async user => {
  if (user && user.emailVerified) {
    const idTokenResult = await user.getIdTokenResult();
    const { claims: { entity, name, permissions } } = await user.getIdTokenResult();

    if (idTokenResult && idTokenResult.claims) {
      await store.dispatch('setProfile', { ...user, entity, name, permissions });
      await store.dispatch('setClaims', idTokenResult.claims);
      await store.dispatch('setIsAuthenticated', true);
      await store.dispatch('setIdToken', idTokenResult.token);
      await store.dispatch('setCurrentUser', idTokenResult.claims.entity);
    }
  }
});

Finally, within the Veux store, we will use user.js as the entry point for invoking the user endpoints. To be sure that a user is granted only the necessary access needed to execute a task, we are going to use the isAuhorised function. The necessary action is passed as a parameter which either allow or deny requests to the server.

if(!isAuhorised(action)) return;

So, when a user tries to make an API call to the getUsers endpoint, we can check if the user has enough privilege to continue:

  async getUsers(context, payload) {
    try {
      // TODO: check user permission
      if(!isAuhorised('readAll')) return;
      // TODO: api call
      await context.dispatch('setLoading', true);
      if (!payload && context.state.users && !!context.state.users.length) {
        await context.dispatch('setLoading', false);
        return context.state.users;
      }
      const { data } = await userApi.getUsers(context.rootGetters.idToken);
      if (!Array.isArray(data)) return;
      await context.dispatch('setLoading', false);
      context.commit('SET_USERS', data);
      return data;
    } catch (error) {
      console.log('Error: ', error);
      return;
    }
  },

This way, we are making sure that users do not attempt a request unless they have the permission to do so. The repository for this tutorial is on GitHub.

If you like the article, do like and share with friends.

Reference:
https://www.cyberark.com/what-is/least-privilege/

Use Firebase Auth to Manage User Permissions and Enforce Principle of Least Privilege on API Endpoints. Part 1.

Callis Ezenwaka — Mon, 10 Apr 2023 10:35:47 +0000

The principle of least privilege is a foundational component of zero trust frameworks. These sets of frameworks advocate that organizations should not automatically trust activities within or outside their infrastructure perimeters.

It demands that organizations setup and deploy stringent authentication and authorization protocols and grant only the necessary access needed for executing a task to devices that connect to either their remote or on-premises systems.

Many organizations are accelerating their digital transformation strategies, thus shifting from traditional perimeter security approaches to the Zero Trust framework to protect their most sensitive networks.

This piece will focus on fine-grain permissions of server endpoints using Firebase, an app development platform that helps you build and grow apps and games users love. We have Node.js®, an open-source, cross-platform JavaScript runtime environment. Check out the part 2 here.

One might ask, why is the Principle of Least Privilege important? Some answers are provided below:

It reduces the cyber-attack surface.
It stops the spread of malware.
It improves end-user productivity.
It helps streamline compliance and audits.

A simple project setup will be used to make the tutorial as basic as possible. However, the same approach could be extended and applied to more complex Enterprise SaaS applications. The code tree is as shown below:

.
├── README.md
├── controllers
│   └── user.js
├── database
│   └── index.js
├── index.js
├── lib
│   └── auth.js
├── migrate.js
├── package-lock.json
├── package.json
├── permission.json
└── routes
    └── user.js

Here, we have four directories viz: controllers, routes, databaseand lib. As already mentioned, the setup is simplified by using a single user route. The controller directory has the user.js file with the endpoint callback functions. While the routes directory has the various endpoints to perform CRUD operations.

The lib directory has the auth.js file for the authentication related functions. Other files are the index.js file, which as we know is the entry point for the project. The permission.json is another important file as it contains the firebase service account settings.

The database has the mock data with a user role. The user role is assigned to a user using the setCustomUserClaims of the firebase getAuth method. The mock data consist of user object with the following schema:

{
    id: '1',
    username: 'janedoe',
    displayName: 'Jane Doe',
    email: 'jane.doe@permission.com',
    password: 'password',
    phoneNumber: '+2348030004001',
    photoURL: 'http://avatar.png',
    emailVerified: true,
    disabled: false,
    role: {
      name: "User",
      entity: "isUser",
      permissions: [
        { name: "create", value: false, },
        { name: "readAll", value: false },
        { name: "read", value: true, },
        { name: "update", value: true, },
        { name: "delete", value: false, },
      ],
    },
    created_at: 1673431871000,
    updated_at: 1673431871000,
  }

As we can see, the role variable has the name, entity, permissions etc. The permissions variable is also an array of key-value pairs of name and value, the fine-grained access permissions. The name is the intended action while the value is the privilege to perform such action.

Then, there is the migration.js file which is used to seed data to firebase authentication table. And finally, the package.json file, which has the script object for starting and migrating the mock data. To migrate the mock data, run the command npm run migration up. (N.B: Make sure that you do not skip the last up word in the command).

  "scripts": {
    "start": "node index.js",
    "dev": "nodemon -r dotenv/config index.js",
    "migration": "node migrate.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },

The project set has five endpoints: getUsers, getUser, addUser, updateUser, and deleteUser. Each of these endpoints has a corresponding action: readAll, read, create, update and delete respectively. As previously stated, the simplicity of this implementation is for demonstration purposes.

The auth.js file has two major function definitions. The isAuthenticated function for authentication and the isUser function for authorization. This role is added to the account custom claim during the account creation step.

/**
 * [START CHECK USER]
 * @param {object} req Express request context.
 * @param {object} res Express response context.
 * @param {object} next Express next context.
 */
exports.isUser = (action) => (req, res, next) => {
    getAuthToken(req, res, async () => {
        try {
            // TODO: get and verify authToken
            const { authToken } = req;
            const userInfo = await getAuth().verifyIdToken(authToken);
            // TODO: user has permission required to perform action on any API method
            const allow = userInfo.permissions.some(permission => permission.name === action && permission.value === true);
            // TODO: user not allowed to take action
            if (!allow) return res.status(403).json('Forbidden access!!');
            // TODO: user allowed to take action
            req.user = userInfo;
            return next();
        }
        catch (error) {
            return res.status(401).json('Unauthorized access!');
        }
    });
}
// [END CHECK USER]

This, I am certain, is very explanatory. The action parameter is used to determine the intended task and if a user making a request or expecting a response has the right permission. These permissions and the corresponding authentication function is used to authenticate and authorize requests and responses.

The isAuthenticated is passed as application middleware within index.js to authenticate requests as below:

// Verify request middleware
app.use('/', isAuthenticated);
// Private routes
app.use('/api/v1/user', user);

While the isUser role with its associated permission is passed before the callback function as a middleware on every route endpoint to authorize user actions.

'use strict';

// import packages and dependencies
const user = require('../controllers/user');
const { isUser } = require('../lib/auth');
const express = require('express');
const router = express();

router.get('/', isUser('readAll'), user.getUsers); 

router.get('/:id', isUser('read'), user.getUser);

router.post('/', isUser('create'), user.addUser);

router.put('/', isUser('update'), user.updateUser);

router.delete('/:id', isUser('delete'), user.deleteUser);

module.exports = router;

That is much about setting up the Principle of Least Privilege on API Endpoints on the server. Next, we will cover how to hook it up with client to authenticate and authorize a user to perform an action. The repository for this tutorial is on GitHub.

If you like the article, do like and share with friends.

Reference:
https://www.cyberark.com/what-is/least-privilege/

Automating Blockchain Build and Integration with Node.js.

Callis Ezenwaka — Wed, 29 Mar 2023 19:01:36 +0000

When implementing a blockchain solution using Node.js and Solidity, developers often need to copy the ABI code during deployment. This process often happens before commit to a remote repository.

In this article, we shall see a quick trick that achieves the same result without breaking a sweat. Before delving into the details, let's run a quick project setup with hardhat. For more info, see.

For a start, run the basic installation of hardhat npm install --save-dev hardhat if not previously done. Setup a new project with this command npx hardhat and follow the prompt.

The use-case for your blockchain application will differ from the one elaborate here. Be sure to run npm install <package name> to install packages that are peculiar to your use-case.

The project folder structure or tree should look like below image after setup:

Here, we can see all the directories especially the scripts and src/abi. Both directories will be handy in our implementations. Ensure that the file(s) for your solidity code are inside the contract folder.

Now, this is where the magic happens. Create a new script file files.ts inside the scripts directory and add the below code:

import fs from "fs";
import path from "path";

const folderList: string[] = [];
function output(folderPaths: string[]) {
  folderPaths.forEach(folderPath => {
    const results = fs.readdirSync(folderPath);
    const folders = results.filter((res: any) => fs.lstatSync(path.resolve(folderPath, res)).isDirectory());
    const innerFolderPaths = folders.map((folder: any) => path.resolve(folderPath, folder));
    if(innerFolderPaths.length === 0) {
      return;
    };
    innerFolderPaths.forEach((innerFolderPath: string) => {
      fs.readdirSync(innerFolderPath).forEach((file: string) => {
        if (file.split('.')[1] === 'json') {
          fs.rename(`${innerFolderPath}/${file}`, `./src/abi/${file}`, function (err: any) {
            if (err) {
              throw err;
            }
          })
        }
      });
      folderList.push(innerFolderPath)
    });
    output(innerFolderPaths);
  });
}

output([path.resolve(__dirname, "../artifacts/contracts")]);

The above code uses recursion and fileSystem native API to loop through all the directories in a given directory, in this case ../artifacts/contracts. Then, open the package.json file and add the following piece of code inside the scripts object:

"files": "ts-node ./scripts/files.ts",
"compile": "npx hardhat compile && npm run files",

What happens here is that once you run npm run compile command to compile your solidity smart contract, it will run the script ./scripts/files.ts. Afterward, it will copy the ABI codes for all the solidity code found inside the contract directory to the src/abi directory as shown below.

With this simple automation, there is no need to manually copy the ABIs compiled smart contracts to your project. The repository for this tutorial is on GitHub.

If you like the article, do like and share with friends.

Understanding Blockchain using JavaScript Class

Callis Ezenwaka — Mon, 20 Mar 2023 19:08:59 +0000

Introduction

This tutorial will demonstrate how to simulate and run a blockchain system using JavaScript. Often times, whenever blockchain and to some extent smart contract is described, what comes to minds is an opaque black-box. It could be that the predominant language used for writing most smart contract codes contributes to this.

This piece will show a detailed step-by-step approach to implementing a smart contract using Node.js and vanilla JavaScript. A test suite will also be added to test the whole application. The source code for the article can be found in this repository. Let's get started!

Here, we have three parameters, two arrays representing initialBalances and transactions and a third integer representing the blockSize. We are going to create a blockchain smart contract that includes all valid pending transactions in the order in which they are given. And provide functionality to get the balance of a specific account on the blockchain.

Installation of Dependencies

First, clone the above repository and run npm install and skip over to the Implementation section below. If you are otherwise coding along, initialize your node.js project by running npm init -y. Then, create two files blockchain.js and blockchain.test.js. Install the following dependencies by running npm i sha1 chai mocha.

The files in the directory should look like the below image:

Then add the following codes to the blockchain.js file blockchain.js

Save the file and add the following code to the blockchain.test.js file blockchain.test.js

Save the file also and open the package.json file. Here, you need to make minor changes to the script commands. First, the following code to the script part of the package.json:

    "start": "node blockchain.js",
    "test": "mocha --timeout 10000"

The --timeout 10000 will ensure that all the test will pass during testing.

Implementation

There are two interfaces viz: init and getAccountBalance. But before that, let's define some parameters.

  const balances = [200, 250, 500];
  const transactions = [[0, 1, 50], [1, 2, 80], [1, 0, 100], [1, 2, 600], [2, 0, 150], [1, 0, 50], [2, 1, 60], [0, 1, 55], [1, 2, 40], [1, 0, 70]];
  const blockSize = 3;
  const index = 1;
  const pendingTransactions = [[0,1,50], [1,2,80]];
  const prevBlockHash = '548bc92f827bd41ad97243cc3ca4b765f8c68a2c';

We are going to execute the code and run the test. Run the following code sequentially inside the root directory of the project (Ignore the undefined output after executing any of the commands):

Activate the Node REPL with node
Initialize the constructor by running const Blockchain = require('./blockchain');
Instantiate an object by running const blockchain = new Blockchain();
Initialize an object by running blockchain.init(balances, transactions, blockSize);
Retrieve account balance by running blockchain.getAccountBalance(index);

The Node REPL should look like the following image if everything checks out:

Testing the Codes

Exit the Node REPL and return to the root directory. Run the test by executing npm test ./blockchain.test.js. The output should look like below image:

You have successfully implemented and tested a blockchain code using JavaScript. You can take some time to go through the code in order to understand how each part of the code interact with each other.

If you like the article, do like and share with friends.