DEV Community: Narcisa

[Micro frontends] NX + Angular Module Federation + React MFE using ChatGPT

Narcisa — Fri, 17 Mar 2023 15:36:05 +0000

Let's have a quick discussion about the things we've built so far and what's new as part of the micro frontends series.

Recap
What's new
Challenges
Conclusion

Recap

We started from the Angular Tour of Heroes and migrated the monolithic architecture to micro frontends, by creating a host and remote applications as part of the same NX workspace. In terms of applications:

the shell application is based on Angular framework
2 remote applications use Angular
1 remote application uses Vue.js

What's new

So far so good, but I also wanted to plug in a React remote application and see how it behaves in terms of integration. Well, instead of going through a step-by-step tutorial, let's see the challenges I faced during the development.
From the code perspective, the entire configuration, app development and deployments can be found here:
https://github.com/blminami/tour-of-heroes-microfrontends.

Considering it's a POC, the react application consists of a single page, where we display the React logo and a message. I'm going to use this app to actual code something in React, but for the purpose of today's article, it's enough.

Check out the react configuration here: https://github.com/blminami/tour-of-heroes-microfrontends/tree/master/apps/calendar-microapp

Tour of Heroes App: https://master--tour-of-heroes-mfe.netlify.app/

The questions I wanted to answer this week:

How easy is it to create a react application and integrate it as part of the mfe architecture?
Would ChatGPT help me with this project, considering I have minimal experience with React + Webpack 5 and Module Federation?
How far can I get without using Stack Overflow or official documentations myself (using only ChatGPT and my own knowledge)?

Challenges

🔸 How easy is it to create a react application and integrate it as part of the mfe architecture?

Overall, pretty easy, considering that:

NX supports React without custom configuration, there is a plugin for generating react applications as part of an NX workspace
I already have experience configuring a Vue.js app as a web-component, and React is no different
the configuration of the shell application allows the usage of remote modules, no matter the framework / library it's based upon

🔸 What about ChatGPT as part of this experiment?

Considering I already had a direction, I was curious if ChatGPT would make it even easier for me to do the configuration.

✨ Short answer: it was not straight-forward and I had to input multiple prompts (which implied some knowledge about the process on my side), but I was able to integrate it from scratch, without using any external resources myself. Which is great, right?

Let's get into details, shall we?

ChatGPT would be helpful on its own to some extent, but it'd require some knowledge on the topic and architecture to actually make it work in this context:

There are multiple approaches when it comes to micro frontends. So, prompts like integrate react micro frontend in nx workspace and angular shell app won't work. been there, done that - in this case, it instructs to create a react application, build and export it, create NX library that uses the application and import the library as part of the Angular shell application. Which is valid, but not at all consistent with the remote modules architecture in place. integrate react micro frontend in nx workspace and angular shell app with webpack module federation is more appropriate for this goal, cuz the steps provided are based on webpack and module federation.
The prompt above generated the steps that one should follow to integrate a react application. Those contained bits of documentation (only relevant parts), code snippets and cli commands. Even though it seemed pretty straight-forward at first glance, some commands generated errors and I had to explicitly request webpack 5 version (some libraries were outdated).
I continued with requesting a webpack config file that uses the module federation plugin. The one provided was useful, tho quite generic. Great start, but once I ran the app based on that webpack config file, I got errors (OF COURSE, I was expecting them 😈).
Based on the errors I received, adjustments to use the right paths + additional configurations and rules for different file formats were required. I could have asked ChatGPT for them (considering I already knew what was missing), but I decided to speed-up the process and add them myself.🙂

Same thing, just faster. 😄
Which leads me to the next question.

🔸 How far can I get without using Stack Overflow or official documentations myself and using only ChatGPT and my own knowledge?

Simply said, I made it work solely on ChatGPT and my knowledge, but check my comments above. It was not easy peasy lemon squeezy, but not that hard either. ☯️

Conclusion

In the end, I declare myself satisfied with my experiment, I got what I wanted. My curiosity can take a break now, hehe ^_^.
Next time, I'm going to play a bit with the ChatGPT API.💥

What are your thoughts and which are your experiences with ChatGPT and coding?

Until next time, thank you for reading! 🐾

[Tailwind CSS] From CSS to Tailwind CSS

Narcisa — Fri, 03 Mar 2023 12:02:12 +0000

As part of the Micro Frontends series, we developed some micro applications based on Angular (dashboard micro application and heroes micro application). We started from the Angular Tour of Heroes tutorial and until now, we didn't care about anything related to styling.

Check-out the repository for the source code https://github.com/blminami/tour-of-heroes-microfrontends

Today, we're going to add Tailwind CSS to our NX workspace and style the dashboard application. Have a look at the screenshots below (before and after) to see what we want to achieve.

Before

After

Let's start by running the following command which will add tailwindcss config to the workspace:

npx nx generate @nrwl/angular:setup-tailwind dashboard-microapp

According to the NX documentation, the command will install the dependencies and add the configuration files.

// tailwind.config.js

const { createGlobPatternsForDependencies } = require('@nrwl/angular/tailwind');
const { join } = require('path');

/** @type {import('tailwindcss').Config} */
module.exports = {
  content: [
    join(__dirname, 'src/**/!(*.stories|*.spec).{ts,html}'),
    ...createGlobPatternsForDependencies(__dirname),
  ],
  theme: {
    extend: {
      colors: {
        primary: '#E4B363',
        secondary: '#074F57',
        cordovan: '#c45663',
        'dark-bg': '#242424',
      },
    },
  },
  plugins: [],
};

As part of tailwind.config.js file, we can extend the default theme by adding custom colors and other properties. For demo purposes, I updated the color palette to include some new colors.

/* dashboard.component.css */

@tailwind base;
@tailwind components;
@tailwind utilities;

Also, we have to import TailwindCSS bases style. In order to work in the micro frontend context, the styles have to be added at component level, instead of styles.css file.

Now that the configuration is done, we can switch to the html files and use the pre-defined classes to style the components.

First thing, let's have a look at the hero-search component, which exposes the functionality to search a hero by name.

<div class="flex flex-col items-center mb-4">
  <input
    #searchBox
    id="search-box"
    (input)="search(searchBox.value)"
    placeholder="Search for a hero"
    class="shadow appearance-none border rounded w-2/5 py-2 px-3 text-gray-700 leading-tight focus:outline-none focus:shadow-outline"
  />
  <ul
    *ngIf="heroes$ | async as heroes; else null"
    class="shadow appearance-none border bg-white rounded w-2/5 py-2 px-3 text-gray-700 leading-tight mt-px cursor-pointer"
  >
    <li *ngFor="let hero of heroes" class="mt-1">
      <a routerLink="/heroes/detail/{{ hero.id }}">
        {{ hero.name }}
      </a>
    </li>
  </ul>
</div>

It consists of an input element and a list of heroes that meet the search criteria. We would like to have the 2 elements on separated rows and centered. To achieve it, we add a div container with the following classes class="flex flex-col items-center mb-4" which applies flex display, centers the items and add a margin bottom.
Let's also have a look at the input element and the classes class="shadow appearance-none border rounded w-2/5 py-2 px-3 text-gray-700 leading-tight focus:outline-none focus:shadow-outline"
What we got: an element with shadow, rounded border (border-radius: 0.25rem; /* 4px */), width 40%, vertical and horizontal padding (py-2 px-3) and gray-700 text color.
For more details, check out the Tailwind documentation

Next, let's give life to our heroes! Instead of displaying boring hero cards, let's have an image, a name and a description of their super powers. We display the top heroes, after all, they deserve some recognition for their powers, right?😄

In the in-memory-data.service.ts, update the heroes array and add the following properties to each object: id, name, description, image. Find the new array of data here.

{
        id: 13,
        name: 'Bombasto',
        description:
          'Telekinesis: The hero can move objects with their mind, allowing them to lift heavy objects or even people',
        image: 'https://i.postimg.cc/3wZMGqn8/11-2-anime-png-picture.png',
},

For each hero, we're going to have a container that has rounded corners, box-shadow, cursor pointer and a gradient as background color (you can be creative with the gradients, here is a interesting list of options https://hypercolor.dev/)
class="rounded flex shadow-lg mb-2 mr-2 bg-gradient-to-br from-gray-200 via-gray-400 to-gray-600 cursor-pointer".
Also, we're going to display the name, image and description of the hero, in 2 different columns.

<div
      *ngFor="let hero of heroes"
      class="rounded flex shadow-lg mb-2 mr-2 bg-gradient-to-br from-gray-200 via-gray-400 to-gray-600 cursor-pointer"
      routerLink="/heroes/detail/{{ hero.id }}"
    >
      <img
        class="object-contain h-52 w-40 self-center"
        [src]="hero.image || defaultImage"
        alt="hero image"
      />
      <div class="px-6 py-4">
        <div class="font-bold text-xl mb-2">{{ hero.name }}</div>
        <p class="text-test text-base">
          {{ hero.description }}
        </p>
      </div>
    </div>

Results:

Easy-peasy, right? 🥳

✨Check out the Github repo for the entire configuration!

Until next time, thank you for reading! 🐾

[Micro frontends] NX + Angular Module Federation + Angular MFE + VueJs MFE

Narcisa — Fri, 24 Feb 2023 12:42:11 +0000

This article is part of a micro frontends series and a continuation of [Micro frontends] Getting Started, where we covered what we want to build, the application architecture, the initial NX workspace, Angular micro frontends and Angular Module Federation.

The aim of this article is to create a VueJs micro frontend and integrate it in the NX workspace.

The starting point is a VueJs application added to our workspace - for the record, NX doesn't support VueJs app generation, so simply generate a web application using webpack and babel compiler and add the vue dependencies (vue/cli, vue-loader, etc.).

The application is configured on port 4203. By starting the application, we can see the VueJs application, which uses Webpack and Typescript.

npx nx run messages-microapp:serve

Source code: https://github.com/blminami/tour-of-heroes-microfrontends/tree/nx-angular-vue-mfe-css/apps/messages-microapp

How do we setup the messages application as a micro frontend?

We're going to wrap our application in Web Components. Check out the code below to see how a VueJs component can be wrapped:

// bootstrap.ts

import { createApp } from 'vue';
import Messages from './app/components/Messages.vue';
import './styles.css';

export class MfeVue extends HTMLElement {
  connectedCallback() {
    createApp(Messages).mount(this);
  }
}

customElements.define('messages-element', MfeVue);

In the main.ts file, add the following import to bootstrap the application async:

import('./bootstrap');

And finally, configure the webpack.config.js file by adding the ModuleFederationPlugin. By doing so, we instruct webpack which files to make available for remote consumption:

new ModuleFederationPlugin({
      name: 'messages_microapp',
      library: { type: 'var', name: 'messages_microapp' },
      filename: 'remoteEntry.js',
      exposes: {
        './messages-wc': 'apps/messages-microapp/src/bootstrap.ts',
      },
      shared: {
        ...dependencies,
      },
    })

Once the setup is done, run the serve command and navigate to http://localhost:4203/remoteEntry.js, where the remote module is available and ready to be used.

Let's setup the shell application, which uses Angular framework and Angular Module Federation. There is a plugin available for loading remote web components: https://www.npmjs.com/package/@angular-architects/module-federation-tools

npm i @angular-architects/module-federation-tools

For this example, we're going to directly load the messages web component like this:

<mft-wc-wrapper [options]="item"></mft-wc-wrapper>

where

item: WebComponentWrapperOptions = {
    type: 'script',
    remoteEntry: 'http://localhost:4203/remoteEntry.js',
    exposedModule: './messages-wc',
    remoteName: 'messages_microapp',
    elementName: 'messages-element',
  };

remoteEntry: where the remote application is deployed (in our case, localhost, port 4203)
exposedModule: configured in Micro Frontend's webpack configuration
remoteName: configured in Micro Frontend's webpack configuration
elementName: configured in Micro Frontend's bootstrap file
WebComponentWrapperOptions - imported from '@angular-architects/module-federation-tools'

For it to work, add ModuleFederationToolsModule to app.module.ts.

In case of loading the web component by route, there is an example in the app-routing.module.ts, where navigating to /messages will load the remote web component.

Let's run the application and see what we got:

run:all command was updated to start the messages application as well:

"run:all": "nx run-many --targets=serve  
--maxParallel=4 
--projects=dashboard-microapp,heroes-microapp,messages-microapp,tour-of-heroes"

✨It's super easy to plugin a new Micro-Frontend into an existing workspace by using Module Federation and Webpack 5!

Check out the Github repo for the entire configuration!

Until next time, thank you for reading! 🐾

[Micro frontends] Resources

Narcisa — Fri, 17 Feb 2023 09:31:45 +0000

As I'm focusing more and more on micro frontends and combining different libraries and frameworks in the same application, I want to share some great resources that I found on the topic:

Github repositories:

Articles:

Courses:

Microfrontends Architecture

I'm going to keep this article up-to-date with all the interesting stuff that I'm finding!!!

Until next time, thank you for reading! 🐾

[Micro frontends] Getting Started

Narcisa — Fri, 10 Feb 2023 08:09:38 +0000

At the beginning of this week, I took a course on micro frontends architecture topic. It's such a buzzword when it comes to frontend development, but I've never attempted anything in practice until this week.

Introduction
Application architecture
Prerequisites
Step-by-step tutorial
Conclusion
Additional resources

Introduction

When it comes to microservices, the backend services have this architecture in place for several years already. So why not taking advantage of all the benefits and start developing micro applications / micro components that can be owned, developed, managed and maintained by different teams. Of course, that's only one of the advantages (vertical teams that will handle a feature end-to-end), but we can also talk about the ability to scale the product by working on different features / modules in parallel, speed of delivery, support for innovation and new technologies, more reliable testing and maximum decoupling.

Let's see how it goes in practice!

Application architecture

I've decided to start from the well-known Angular Tour of Heroes tutorial and split the codebase into micro apps. There are a few approaches when it comes to splitting monolithic applications into micro frontends, I chose to do it by feature. The application provides two major features:

dashboard: top heroes + search heroes capability
heroes: list of heroes + hero management capabilities (view here, add hero, delete hero)

Considering the split by feature, I've decided on the following architecture of my application:

where:

shell application is the app that will host all the remote micro applications (dashboard and heroes) and will handle everything related to the navigation (navigation component and routing logic)
dashboard micro application is one of the remote applications, targeting the dashboard feature
heroes micro application is the second remote application, targeting the heroes feature

Prerequisites for Angular micro frontends:

NodeJs 10.13.0+
Webpack 5
Angular 12+
NX

Step-by-step tutorial

🔸 download the Angular Tour of Heroes application
🔸 open terminal to the root directory and run

npm i && npm install -g nx

🔸 transform the Angular cli workspace to an integrated NX monorepository (all the micro frontends will be part of the NX monorepo for simplicity)

ng add @nrwl/angular@<version_number>

🔸 get rid of all the unused files, configs, packages and components to keep it simple and focus on the micro apps (remove the message component and e2e files)
🔸 run the application locally to check that all the changes made so far didn't affect the app

npx nx build tour-of-heroes

So far, you should be able to navigate to http://localhost:4200 and check the application in the browser.

Let's keep having fun:

🔸 generate NX Angular application for dashboard-microapp and hero-microapp using nx cli

npx nx generate @nrwl/angular:app dashboard-microapp

npx nx generate @nrwl/angular:app heroes-microapp

🔸 add angular module federation plugin to all apps (shell, dashboard, hero)

npx nx g @angular-architects/module-federation:init 
--project tour-of-heroes --port 4200 --type host

npx nx g @angular-architects/module-federation:init 
--project dashboard-microapp --port 4201 --type remote

npx nx g @angular-architects/module-federation:init 
--project heroes-microapp --port 4202 --type remote

This command will do a couple of things: add webpack.config.js, webpack.prod.config.js, bootstrap.js files and additional configurations (more details here)

The apps folder should look like this:

🔸 let's consider the dashboard-microapp (same will apply to hero-microapp). At this point, in src folder, you have an Angular app configuration - with app.module, app.component.ts, app.component.html and app.component.css. We're going to create a new component, module and routing for the Dashboard feature

ng generate module dashboard --project=dashboard-microapp

🔸 move the logic from the initial application (dashboard and hero-search components) to the dashboard application (you can copy paste the components and update the imports in case the IDE doesn't handle it)
🔸 configure the dashboard.module.ts as follow (by importing modules, declaring routes and components):

export const remoteRoutes: Route[] = [ 
  { path: '', component: DashboardComponent }, 
];

@NgModule({ 
  imports: [ 
    CommonModule, 
    HttpClientModule, 
    FormsModule, 
    RouterModule.forChild(remoteRoutes), 
  ], 
  declarations: [DashboardComponent, HeroSearchComponent], 
}) 
export class DashboardModule {}

🔸 expose DashboardModule in webpack.config.js:

const { 
  withModuleFederationPlugin, 
} = require('@angular-architects/module-federation/webpack');

module.exports = withModuleFederationPlugin({ 
  name: 'dashboard-microapp', 
  filename: 'remoteEntry.js', 
  exposes: { 
    './Module': 
      './apps/dashboard-microapp/src/app/dashboard/dashboard.module.ts', 
  }, 
});

🔸 now that you got the functionality moved to the micro-app and the module configured for this feature, update the app.module.ts:

@NgModule({ 
  declarations: [AppComponent], 
  imports: [ 
    BrowserModule, 
    HttpClientModule, 
    // optional: setting in-memory-web-api for microapps so we can test them separately 
    HttpClientInMemoryWebApiModule.forRoot(InMemoryDataService, { 
      dataEncapsulation: false, 
    }), 
    RouterModule.forRoot( 
      [ 
        { 
          path: '', 
          loadChildren: () => 
            import('./dashboard/dashboard.module').then( 
              (m) => m.DashboardModule 
            ), 
        }, 
      ], 
      { initialNavigation: 'enabledBlocking' } 
    ), 
  ], 
  providers: [], 
  bootstrap: [AppComponent], 
}) 
export class AppModule {}

To test everything related to dashboard-microapp, let's run it in the browser and see what we get:

npm start dashboard-microapp

navigate to http://localhost:4201 and you should be able to see the dashboard app:

🔸 repeat the same steps for hero-microapp (generate module, move functionality, update hero.module.ts file and app.module.ts, run it in browser and check that everything works as expected).

So far, we have 2 microapps that are running independently in the browser, how do we integrate them into the shell application so we can create the illusion of a single application for the end user?

🔸 switch to shell application (tour-of-heroes) and adjust webpack.config.js:

const { 
  shareAll, 
  withModuleFederationPlugin, 
} = require('@angular-architects/module-federation/webpack');

module.exports = withModuleFederationPlugin({ 
  remotes: {}, 
  shared: { 
    ...shareAll({ 
      singleton: true, 
      strictVersion: true, 
      requiredVersion: 'auto', 
    }), 
  }, 
});

keep remotes: {} for dynamic consumption of micro applications

🔸 load the micro apps lazily when it's routed to:

const routes: Routes = [ 
  { 
    path: '', 
    loadChildren: () => 
      loadRemoteModule({ 
        remoteEntry: 'http://localhost:4201/remoteEntry.js', 
        type: 'module', 
        exposedModule: './Module', 
      }).then((m) => m.DashboardModule), 
  }, 
  { 
    path: 'heroes', 
    loadChildren: () => 
      loadRemoteModule({ 
        remoteEntry: 'http://localhost:4202/remoteEntry.js', 
        type: 'module', 
        exposedModule: './Module', 
      }).then((m) => m.HeroesModule), 
  }, 
];

🔸 run all the applications

npm run run:all

and navigate to http://localhost:4200. You should be able to see the Tour of heroes application - same as in the beginning, because nothing changed for the end user.

Hoooray! That was it! 🥳

Conclusion

This is my first attempt at micro frontends, I hope it clarifies the initial steps and configuration of the micro frontend architecture and encourages you to consider it for your future projects. I'll keep playing with it and document my stories while learning new stuff, so stay tuned! ^_^

Additional resources:

Until next time, thank you for reading! 🐾

[Discussion] 5 years of professional coding OR how I earn money while having fun

Narcisa — Wed, 01 Feb 2023 17:47:32 +0000

Considering I've always loved to share my own experiences and stories, it's time to a have a tour down the memory lane and reflect on my past (and first) 5 years of professional coding. If you're at the beginning of your own journey, you might find some useful tips to avoid some mistakes. After all, we're here to learn from one another and I encourage you to reflect on your path as well.

✨ Background ✨

I got my first job right after finishing the second year at university, a summer internship at a company in my town. I still remember the interview, it was on paper and consisted of many questions from object-oriented programming concepts, algorithms, data structure, AI, databases and exercises that required actual coding - quite normal for an internship / entry level position, as it covers general knowledge and logical thinking.

Once I got the job, I spent the first 2 months learning about the MEAN stack and developing an internal application based on those technologies. Next, I spent 1 year working on different projects (around 6 months each), covering all stages of an application - from the initial discussions with the clients to understand their needs and what should be built, to translating client's needs into technical requirements, choosing the technologies and frameworks, developing, deploying and testing the application.

Most of my expertise is around web technologies as 4 out of 5 years were spent on this area, but I also worked on server side for 1 year, building microservices using Go language, creating scripts for automating different processes, administrating databases and, generally, having fun in the terminal. It was one of the craziest years of my life: working for a start-up as a junior, as part of a team full of senior engineers and with little to no knowledge about the technologies I was using - deploying services to production for a world-wide application, well, never before, it was intimidating at first - to say the least 😅.

Once I got bored (yep, bored 💤) of the server side thingy, I switched back to front-end development where I had the chance to work on some legacy applications, while also developing and contributing to some big projects. It was quite different than anything I did before because:

〰️ I was working within a mixed team of front-end developers, back-end developers, designers and QA engineers;
〰️ there were many standards and quality checks in place to make sure everything is properly delivered;
〰️ tons of meetings (cuz hey, out of nowhere, I was part of the corporate life).

So, I got from working for some small clients as a full-stack developer, to working as a back-end engineer for a start-up (did I mention the extra extra extra hours ☠️), to being part of the corporate life as a senior software engineer. Pretty fun, right?

✨ What would I do the same? ✨

everything except some things, of course 🤣

Considering I was following tech courses at university and I was preparing for a career in the IT field, the fact that I got my first job as a developer helped me a lot (instead of spending time doing some other jobs).

Tip #1: if you know what you want, go and work hard for it, don't let yourself get distracted.

🔸start on smaller projects (chance to get first-hand experience on a software development cycle). In order to develop great software, it's important to understand every step of the process;
🔸work for a start-up - other than learning new things extremely fast and using the latest technologies, I was constantly out of my comfort zone as I was performing under pressure;

Tip #2: start as part of a small team, small project or dynamic environment to speed up the learning process

🔸struggle with some technical tasks on my own, before asking for help. When you're a junior (and not only), everything feels new, different and hard. It's easier to ask for help, especially if you're surrounded by people willing to guide you, but sometimes it's better to solve bugs on your own. Not only it'll force you to read and learn more, but it'll boost your confidence as well;
🔸give 100% of myself at the job, not because I'm asked to, but because of my mindset and the things I want to achieve;

Tip #3: go all the way

🔸mentor other people. Something I've always valued - give back and help others whenever you can.

✨ What would I do differently? ✨

🔸I would stress less and set some healthy boundaries at work. I didn't do it while working for the start-up (everyone knows that such a project requires a lot of dedication), though I maintain a reasonable workload nowadays;

Tip #4: Set boundaries at work to achieve a great work-life balance

🔸I would take more time off. Guess what, the projects will go on, even if you take a holiday every trimester for example. Also, I would disconnect entirely, no applications on my phone, no notifications, no messages during that time. It might sound extreme, but it's important for the mental health on the long run;
🔸I would say no more often. Just because you can do something or you have the time and resources, it doesn't mean you have to;
🔸I would document the technical challenges I face. Write articles, quick snippets of code or quick notes about everything and contribute to open-source;
🔸I would take part in more hackathons as it's a great opportunity to learn new technical skills, play with new technologies, network and meet people with similar interests.

✨ Recommendations ✨

🔸All tips above should be considered recommendations😉;
🔸Don't neglect open-source and writing about your journey, tech challenges and topics. Share as much as you can with others. You can bring huge contribution to the community while forcing yourself to get out of your comfort zone. This is how you grow;
🔸Challenge yourself with new projects, new technologies and keep learning new skills that will help you reach your goals. You're the one responsible for your own career, don't let it in the hands of someone else;
🔸Take guidance from your team and have mentors, but don't expect every challenge you face to be solved by others;
🔸Read a lot (no matter the topic) and take notes. Everything connects, and you never know how you wake up one day with an idea. Knowledge is power nowadays;
🔸Meet deadlines, learn how to prioritize tasks and how to estimate effort, maintain high code quality standards;
🔸Test your code ‼️;
🔸Read the documentation before attempting any implementation. It'll save you hours / days 📄;
🔸Be aware of every step in the process, even if it doesn't fall within your responsibilities;
🔸Stay curious and be resourceful.

✨ Conclusion ✨

I plan on continuing this series yearly, as I'll do a lot of growing meanwhile. If you have any recommendations or additional tips (and I'm sure there are plenty of things that can be mentioned), let's hear it in the comments section.

Until next time, thank you for reading! 🐾

[NestJs] Resolve Request rate is large - Azure CosmosDB error 16500

Narcisa — Mon, 14 Nov 2022 15:17:10 +0000

If you're familiar with Azure CosmosDB, you may have heard of / encountered the 16500 error - Request rate is large. Well, I knew about it, but I don't remember fixing it in the past, at least not using NestJS. What is it about? This error occurs when the request rate to the database is too high. In my case, I tried to populate a collection with some seed data (around 200 new records).

What's the solution? There are a couple of ways to solve it:

Increase the Request Unit (RU) value in Azure CosmosDB - which, of course, implies additional costs.
Batch requests - instead of making individual requests, you can batch your requests together. This will reduce the overall request rate and help resolve the issue.
Implement some sort of retry mechanism - by configuring the application to automatically retry requests that fail due to the Request rate is large error.

Let's see how we can implement the third solution. In our case, we're going to insert some data into a collection and retry until a). all records are inserted b). retry count is reached.

private async insertManyWithRetry(
    collection: Model<TestDocument>,
    data: any[],
    retries: number = 10,
    delay: number = 300
  ) {
    try {
      if (retries <= 0) {
        Logger.log('Done retrying');
        return;
      }
      return await collection.insertMany(data);
    } catch (error) {
      Logger.error(error);
      if (error.code !== 16500) {
        throw new InternalServerErrorException(error);
      }
      setTimeout(async () => {
        const processedRecordsIds = error.insertedDocs.map((elem) => elem.id);
        const unprocessedRecords = data.filter(
          (record) => !processedRecordsIds.includes(record.id)
        );
        await this.insertManyWithRetry(collection, unprocessedRecords, retries - 1);
      }, delay);
    }
  }

Let's have a look at the function insertManyWithRetry, it receives the following parameters:

collection - where we want to store data
data - array of JSON documents
retries - limit of retries, if none provided, default value is 10
delay - time to wait until next retry, if none provided, default value is 300ms

First, we're going to insert data into the collection. This request can be either successful or generate an error, which will be caught by the error block.

According to the MongoDB documentation, the insertMany method will return this sort of document in case of write error:

BulkWriteError({
   "writeErrors" : [
      {
         "index" : 24,
         "code" : 16500,
         "errmsg" : "Response status code does not indicate success: TooManyRequests (429)",
         "op" : {
            "_id" : 13,
         }
      }
   ],
   "writeConcernErrors" : [ ],
   "insertedDocs": [ ],
   "nInserted" : 1,
   "nUpserted" : 0,
   "nMatched" : 0,
   "nModified" : 0,
   "nRemoved" : 0,
   "upsert

This tells us all we need to know about our request - what's the error message, error code and which operation / document generated the error. It also provides some stats about the documents (how many were modified or upserted, etc.) and the array of the documents that were inserted.

In case of error:
1). if the error code is not 16500 => log and throw an internal server error exception.
2). if the error code is 16500 => make another attempt at inserting data, once the documents that were already inserted are filtered out.
In our case, unprocessed records are stored in the unprocessedRecords variable. We can invoke the function again, with the new values: await this.insertManyWithRetry(collection, unprocessedRecords, retries - 1);, which:

can be successful and insert all the remaining documents
can generate a new 16500 error => re-apply the same mechanism (check the error, filter out inserted documents and retry with a delay of 300 ms)

That's it. The implementation can be enhanced by logging more information, errors and retry attempts and using promises, but the logic remains the same.

Any thoughts?

Additional resources on the topic:

Thank you for reading! 🐾

[NestJs] Intercept axios responses and throw built-in HTTP errors for the exception filter

Narcisa — Tue, 25 Oct 2022 10:11:08 +0000

Context

How do we intercept the external http calls made by our NestJs server, so we can log and better handle the errors?
NestJs Interceptors are able to process the requests and the responses made via the controllers, not the external http calls that we might need as part of a controller request.

Solutions

One solution would be to add a catchError on the http call. NestJs documentation provides a full example and extensive information on HttpModule and HttpService from @nestjs/axios package.

What if we already have a couple of axios requests and we don't want to add catchError block on each request? Here comes the second solution: axios response interceptors and exception filters.

First, let's consider this call as part of our service, where (hypothetically) our http call will throw an error:

//app.service.ts
getCats(): Observable<any> {
    return this.httpService.get('http://localhost:3001/cats').pipe(
      map((data) => {
        console.log('Do something');
        return data;
      }),
    );
  }

Let's configure the axios response interceptor. We're going to add this snippet as part of the bootstrap function. All HttpService methods return an AxiosResponse wrapped in an Observable object, responses that we're going to intercept via this code:

// main.ts
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { HttpService } from '@nestjs/axios';
import { InternalServerErrorException } from '@nestjs/common';

async function bootstrap() {
  const httpService = new HttpService();
  const app = await NestFactory.create(AppModule);

  httpService.axiosRef.interceptors.response.use(
    (response) => {
      return response;
    },
    (error) => {
      console.error('Internal server error exception', error);
      throw new InternalServerErrorException();
    },
  );

  await app.listen(3000);
}

Responses with status 2** will be handled by the first function, where we simply return the response. In case of error, we log the error and throw a built-in HTTP exception, InternalServerErrorException. Of course, we can implement more advanced error handling here, throwing exceptions depending on the error that we get from the axios code and so on.

Next, what happens with our exception? It's going to be caught by the global http exception filter.

//http-exception.filter.ts
import { ArgumentsHost, Catch, ExceptionFilter, HttpException } from '@nestjs/common';
import { Request, Response } from 'express';

@Catch(HttpException)
export class HttpExceptionFilter implements ExceptionFilter {
  catch(exception: HttpException, host: ArgumentsHost) {
    const ctx = host.switchToHttp();
    const response = ctx.getResponse<Response>();
    const request = ctx.getRequest<Request>();
    const status = exception.getStatus();

    response.status(status).json({
      statusCode: status,
      timestamp: new Date().toISOString(),
      path: request.url
    });
  }
}

Now that we have the code for the exception filter, let's make the adjustments so we can use it:

//app.module.ts
providers: [
    {
      provide: APP_FILTER,
      useClass: HttpExceptionFilter
    }
  ]

//main.ts
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { HttpService } from '@nestjs/axios';
import { HttpExceptionFilter } from './http-exception.filter';
import { InternalServerErrorException } from '@nestjs/common';

async function bootstrap() {
  const httpService = new HttpService();
  const app = await NestFactory.create(AppModule);

  //http-exception as global filter
  app.useGlobalFilters(new HttpExceptionFilter());

  httpService.axiosRef.interceptors.response.use(
    (response) => {
      return response;
    },
    (error) => {
      console.error('Internal server error exception', error);
      throw new InternalServerErrorException();
    },
  );

  await app.listen(3000);
}

What we learned as part of this blogpost:

how to make an external http call via the HttpService from @nestJs/axios package
how to intercept the responses of the http calls and throw built-in http exceptions in case of error
how to intercept the http exceptions via the global http exception filter

In-depth documentation on error handling, axios interceptors and exception filters can be found here:

✨Github repo with the NestJs setup, exception filter and axios interceptor!

Thank you for reading! 🐾

DEV Community: Narcisa

[Micro frontends] NX + Angular Module Federation + React MFE using ChatGPT

Table of contents

Recap

What's new

Challenges

Conclusion

[Tailwind CSS] From CSS to Tailwind CSS

[Micro frontends] NX + Angular Module Federation + Angular MFE + VueJs MFE

How do we setup the messages application as a micro frontend?

[Micro frontends] Resources

Github repositories:

Articles:

Courses:

[Micro frontends] Getting Started

Table of contents

Introduction

Application architecture

Prerequisites for Angular micro frontends:

Step-by-step tutorial

Conclusion

Additional resources:

[Discussion] 5 years of professional coding OR how I earn money while having fun

✨ Background ✨

✨ What would I do the same? ✨

✨ What would I do differently? ✨

✨ Recommendations ✨

✨ Conclusion ✨

[NestJs] Resolve Request rate is large - Azure CosmosDB error 16500

[NestJs] Intercept axios responses and throw built-in HTTP errors for the exception filter

Context

Solutions

DEV Community: Narcisa

[Micro frontends] NX + Angular Module Federation + React MFE using ChatGPT

Table of contents

Recap

What's new

Challenges

Conclusion

[Tailwind CSS] From CSS to Tailwind CSS

[Micro frontends] NX + Angular Module Federation + Angular MFE + VueJs MFE

How do we setup the messages application as a micro frontend?

[Micro frontends] Resources

Github repositories:

Articles:

Courses:

[Micro frontends] Getting Started

Table of contents

Introduction

Application architecture

Prerequisites for Angular micro frontends:

Step-by-step tutorial

Conclusion

Additional resources:

[Discussion] 5 years of *professional* coding OR how I earn money while having fun

✨ Background ✨

✨ What would I do the same? ✨

✨ What would I do differently? ✨

✨ Recommendations ✨

✨ Conclusion ✨

[NestJs] Resolve Request rate is large - Azure CosmosDB error 16500

[NestJs] Intercept axios responses and throw built-in HTTP errors for the exception filter

Context

Solutions

[Discussion] 5 years of professional coding OR how I earn money while having fun