DEV Community: Mohamed Aymen Ourabi

Improving Integration/E2E testing using NestJS and TestContainers

Mohamed Aymen Ourabi — Fri, 23 Feb 2024 16:36:08 +0000

Introduction

Testing javascript based services in backend context is a required practise to ensure the consistency of our application, however testing the integrity with other services is most of the time done by mocking returned values of methods or by using some automated scripts or tests databases inside piplines for example.

Today we are going to walk you through this tutorial on how you can improve these kind of test cases using TestContainers

Setup

First we start by creating a Nest.js application :
1 - Under command line execute npm i -g @nestjs/cli to add Nest CLI
2- Create Nest project nest new project-name

3- Follow the steps and create your project
4- For this project we choose to use Prismaas our ORM , so under our directory run
npm install prisma @prisma/client --save-dev

Now we should be able to start Coding !!

Getting Started

Using command line run npx prisma init
This should create a prisma/schema.prisma file in your root folder
inside that folder we will create our entity Car



generator client {
  provider = "prisma-client-js"
  output   = "./generated/prisma-client-js"
}

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}


model Car {
  id    Int     @default(autoincrement()) @id
  model  String  @unique
  color  String?
}

No go to your .envfile and add your DATABASE_URL
then go to your command line and run



 $ npx prisma migrate dev --name init

A migration file should be generated under /primsa/migrations

Creating our Resource

After setting up our database now we can start generating our resource in Nestjs app
using CLI provided from Nestjs run



$ nest generate resource

Choose the name Carfor the resource then pick REST API

your resource should be generated under src

After that we should align our DTO's and Entitieswith the right types.
Hopefully Prisma client provides us with the generated types automatically from the generated model so the files should be aligned this way



//create-car.dto.ts
import { Prisma } from "@prisma/client";

export type CreateCarDto = Prisma.CarCreateInput;



//update-car.dto.ts
import { Prisma } from '@prisma/client';

export type UpdateCarDto = Prisma.CarUpdateInput



//car.entity.ts
import { Prisma } from "@prisma/client";

export type Car =  Prisma.CarSelect

Now let's create our PrismaService.
Under project directory run



$ nest generate service prisma

No go to Prisma.service.ts and add these lines



import { Injectable, OnModuleInit } from '@nestjs/common'
import { PrismaClient } from '@prisma/client'

@Injectable()
export class PrismaService extends PrismaClient
    implements OnModuleInit {

    async onModuleInit() {
        await this.$connect();
    }

}

let's now create our carService



//cars.service.ts
import { Injectable } from '@nestjs/common';
import { Car as CarModel, Prisma } from '@prisma/client'
import { PrismaService } from '../prisma/primsa.service';

@Injectable()
export class CarsService {

  constructor(private readonly prismaService: PrismaService) { }
  async create(data: Prisma.CarCreateInput): Promise<CarModel> {
    try {
      const car = await this.prismaService.car.create({ data })
      return car;
    } catch (error) {
      throw error
    }
  }

  async findAll(): Promise<CarModel[]> {
    try {
      return await this.prismaService.car.findMany()
    } catch (error) {
      throw error
    }
  }

  async findOne(where: Prisma.CarWhereUniqueInput) {
    try {
      return await this.prismaService.car.findUnique({ where })
    } catch (error) {
      throw error
    }
  }

  async update(where: Prisma.CarWhereUniqueInput, data: Prisma.CarUpdateInput) {
    try {
      return await this.prismaService.car.update({ data, where })
    } catch (error) {
      throw error
    }
  }

  async remove(where: Prisma.CarWhereUniqueInput) {
    try {
      return await this.prismaService.car.delete({ where })
    } catch (error) {
      throw error
    }
  }
}

and now our controller



import { Controller, Get, Post, Body, Patch, Param, Delete } from '@nestjs/common';
import { CarsService } from './cars.service';
import { CreateCarDto } from './dto/create-car.dto';
import { UpdateCarDto } from './dto/update-car.dto';

@Controller('cars')
export class CarsController {
  constructor(private readonly carsService: CarsService) { }

  @Post()
  create(@Body() createCarDto: CreateCarDto) {
    return this.carsService.create(createCarDto);
  }

  @Get()
  findAll() {
    return this.carsService.findAll();
  }

  @Get(':id')
  findOne(@Param('id') id: string) {
    return this.carsService.findOne({ id: Number(id) });
  }

  @Patch(':id')
  update(@Param('id') id: string, @Body() updateCarDto: UpdateCarDto) {
    return this.carsService.update({ id: Number(id) }, updateCarDto);
  }

  @Delete(':id')
  remove(@Param('id') id: string) {
    return this.carsService.remove({ id: Number(id) });
  }
}

And finally we should add PrismaService to cars.module.ts



import { Module } from '@nestjs/common';
import { CarsService } from './cars.service';
import { CarsController } from './cars.controller';
import { PrismaService } from '../prisma/primsa.service';


@Module({
  controllers: [CarsController],
  providers: [CarsService, PrismaService],
})
export class CarsModule { }

Before start testing we can try to test some endpoints,thus we can use some tools like Postman

Everything works fine

Setup TestContainers for testing

In order to start working in our tests we should first install testContainers



$ npm install --save-dev @testcontainers/postgresql testContainers

TestContainers is a npm dependency that allow us to run database images programatically inside docker containers, these databases are lighweight and super efficent for tests

NOTE: for this tutorial you need to have docker already installed

In order to start testing let's create a Setup file for our tests.
Inside /test create a file setupTests.e2e.ts.

NOTE: The idea of this file is to setup a database for our test environement and initialize a Prisma instance that will be used inside our tests. This database will be created just before running our tests and then will be stopped and unmounted once all tests are executed.
This way we make sure that we connect only once to our database and we will get rid of extra anoying lines of code inside our tests.



import { PostgreSqlContainer, StartedPostgreSqlContainer } from '@testcontainers/postgresql';
import { Client } from 'pg';
import { PrismaService } from '../src/prisma/primsa.service';
import { execSync } from 'child_process';

let postgresContainer: StartedPostgreSqlContainer;
let postgresClient: Client;
let prismaService: PrismaService;

beforeAll(async () => {
  //connect our container 
  postgresContainer = await new PostgreSqlContainer().start();

  postgresClient = new Client({
    host: postgresContainer.getHost(),
    port: postgresContainer.getPort(),
    database: postgresContainer.getDatabase(),
    user: postgresContainer.getUsername(),
    password: postgresContainer.getPassword(),
  });

  await postgresClient.connect();
   //Set new database Url 
  const databaseUrl = `postgresql://${postgresClient.user}:${postgresClient.password}@${postgresClient.host}:${postgresClient.port}/${postgresClient.database}`;
    // Execute Prisma migrations
    execSync('npx prisma migrate dev', { env: { DATABASE_URL: databaseUrl } });
//Set prisma instance
  prismaService = new PrismaService({
    datasources: {
      db: {
        url: databaseUrl,
      },
    },
    log: ['query']

  },
  );
  console.log('connected to test db...');
})

afterAll(async () => {
//Stop container as well as postgresClient 
  await postgresClient.end();
  await postgresContainer.stop();
  console.log('test db stopped...');
});
// add some timeout until containers are up and working 
jest.setTimeout(8000);
export { postgresClient, prismaService };

Now go to jest.e2e.ts and set our config



{
  "moduleFileExtensions": ["js", "json", "ts"],
  "rootDir": ".",
  "roots": ["../src"],
  "setupFilesAfterEnv": ["./setupTests.e2e.ts"],
  "testEnvironment": "node",
  "testRegex": ".e2e-spec.ts$",
  "transform": {
    "^.+\\.(t|j)s$": "ts-jest"
  }
}

Now let's try to create a test case for our car service
we need to make sure that SQL query to the database are performing in the right way.
create a file called car.e2e-spec.ts under /src/cars



import { Test, TestingModule } from '@nestjs/testing';
import { CarsService } from './cars.service';
import { PrismaService } from '../prisma/primsa.service';
import { postgresClient, prismaService } from '../../test/setupTests.e2e';


describe('CarsService', () => {
  let service: CarsService;

  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      providers: [CarsService, PrismaService],
    }).overrideProvider(PrismaService)
      .useValue(prismaService)
      .compile();

    service = module.get<CarsService>(CarsService);
  });

});

First using the createTestModule utility we should override the Prisma instance because currenty createTestingModule is using the real PrismaServicethat connects to the real database, and for that we need to import the new Prisma instance from ../../test/setupTests.e2e then using :



.overrideProvider(PrismaService)
      .useValue(prismaService)
      .compile();

We inject our new PrismaServiceto override the real Prisma.

Now let's create our first test.
So we will try to test the create method inside our service and here are the testing steps:

1- insert into the database using the createmethod

2- perform a SELECT query for the newly created car
3- test the result using jest assertion



it('should create a car', async () => {
    // Start a transaction
    await postgresClient.query('BEGIN');

    try {
      // Perform the create operation
      const createResult = await service.create({ model: "mercedes", color: "red" });

      // Commit the transaction
      await postgresClient.query('COMMIT');

      // Query the database for the newly created car
      const result = await postgresClient.query('SELECT * FROM "public"."Car"');

      // Log the results
      console.log(result.rows);

      // Assert the create result
      expect(createResult).toEqual({
        id: 1,
        model: "mercedes",
        color: "red"
      });
    } catch (error) {
      // Rollback the transaction in case of an error
      await postgresClient.query('ROLLBACK');
      throw error;
    }
  });

Now under our project directory run



$ npm run test:e2e

Once the command is running we can see that the containers are created and started automatically

And bingo our tests are working !!

Now let's create a test for our controller



import { Test, TestingModule } from '@nestjs/testing';
import { CarsController } from './cars.controller';
import { CarsService } from './cars.service';
import { PrismaService } from '../prisma/primsa.service';
import { prismaService, postgresClient } from '../../test/setupTests.e2e';
import * as request from 'supertest';
import { INestApplication } from '@nestjs/common';

describe('CarsController', () => {
  let controller: CarsController;
  let app: INestApplication;
  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      controllers: [CarsController],
      providers: [CarsService, PrismaService],
    }).overrideProvider(PrismaService)
      .useValue(prismaService)
      .compile();

    controller = module.get<CarsController>(CarsController);
    app = module.createNestApplication();
    await app.init();
  });

  it('should be defined', () => {
    expect(controller).toBeDefined();
  });

  it('should create a car', async () => {
    const createCarDto = { model: 'Mercedes', color: 'Red' };

    const response = await request(app.getHttpServer())
      .post('/cars')
      .send(createCarDto)
      .expect(201); 

    expect(response.body).toEqual({
      id: 1,
      model: 'Mercedes',
      color: 'Red',
    });
  });

});

For our controller we tried to use SuperTest in order to perform a real http request to the endpoint POST: /car that points to the method create()

And here our test passing.
For the rest of the CRUD operation it will follow the same strategy for services and also controllers

Summary

In this workaround we tried to show you how to work with TestContainers which is a greate tool for backend applications that can be used for integration tests as well as E2E and makes our tests more consistent and more real without going through mocks or third party tools

Hope you like this tutorial!
Any questions please let us know in your comments ?
Cheers !!

React unit testing using Vitest, RTL and MSW

Mohamed Aymen Ourabi — Thu, 15 Feb 2024 10:10:23 +0000

Introduction

Hello everyone, i'm finally back after almost 5 years since my last post in this community, i would like to apologize to everyone that i couldn't answer their questions, i completely lost access to this account a few years ago and it was a hell of a road to get it back.
Today i would like to share with you a quick tutorial about how to use MSW (Mock service worker) inside React.js application for both testing and development

Getting started

What is MSW ?

MSW or mock service worker is an api mocking library used mostly to mock api calls, it is mainly using browser Service workers to set up their system and to be able to intercept requests at network levels

Why using MSW ?
Well there are a couple of reasons why we use MSW.

For development it is pretty usefull for frontend developers to prevent them being blocked during their tasks in case there are some issues in backend side. We all know that these things occurs sometimes when the server is down or when there is something that should be fixed and we find that we depend on the data that should be returned to continue progressing. However using MSW we can create our mocks exactly with the same structure that we have in backend side and this will completly be on a separated box from what http client we are using either fetch or axios our bussiness logic will remain the same
For testing we all know that most of our projects rely on http requests and when creating unit test we find that we are mostly forced to mock these requests inside our tests



import { http } from "./api/myService";
jest.mock("./api/myService");

describe('Fetch data , () => {
    it('should render active user when isActive us true', async () => {
        http.mockResolvedValue(
            {
                id: '1',
                name: 'Mohamed Aymen',
                isActive: STATUS.Active,
            },
        )

        render(<MyComponent />)

        expect(screen.getByTestId('isActive')).toBeInTheDocument()
    })
}

In this example we can see that we mocked the returned value of an http request in order to test this behavior.
But what happen when we need this mocked value in other test cases ?
we will have to mock this request everytime we need to use it
and what if we need something reusable accros all of our tests?
*This is where the power of MSW comes in hand :)) *
With MSW the mocks will be created on a separted box and will run in parallel with our application logic as well as tests so we don't need to mock the returned values everytime as will get automatically the responses from our mocks in MSW

Setup MSW

for this tutorial we will need to create a React application first so under you project directory open terminal then run the following command

npm create vite@latest

we will use TypeScript template because i personally think TypeScript is just AWESOME :))

Install MSW and Axios
Under /yourdirectory/my-msw-app run this command

npm install msw --save-dev
and
npm install axios --save

Now that MSW is installed let's create our React component.
To keep this tutorial quick and simple we will try to create a simple component that allows us to send an http requests that return a user with a parameters status with that we can check if this user is Active or not

first create the componnet Profile/Profile.tsx under /src



import React, { useEffect, useState } from 'react'
import { STATUS } from './consts'
import axios, { AxiosError, AxiosResponse } from 'axios';

const Profile = () => {
    axios.defaults.baseURL = 'http://localhost:8000';
    const [status, setStatus] = useState<STATUS>(STATUS.OFFLINE);

    useEffect(() => {
        axios.get('api/getStatus').then((res: AxiosResponse<{isActive: STATUS }>) => {
            setStatus(res.data.isActive)
        }).catch((err: AxiosError) => {
            console.error({ err })
        })
    }, [])

    return (
        <div>
            {
                status === STATUS.ACTIVE ?
                    <div data-testid="user-active">
                        this user is active
                    </div>
                    :
                    <div data-testid="user-offline">
                        this user is offline
                    </div>
            }

        </div>
    )
}

export default Profile

then create a consts.ts file to declare our Enum



export enum STATUS {
    ACTIVE="ACTIVE",
    OFFLINE = "OFFLINE"
}

initialize our MSW
Under src create a folder called msw then create inside two files worker.ts and handlers.ts



//Worker.ts
import { setupWorker } from "msw/browser";
import { handlers } from "./handlers";

export const serviceWorker = setupWorker(...handlers);

In this file we initlize our workers and pass the handlers



//handlers.ts
import { http, HttpResponse } from "msw";
import { STATUS } from "../UserProfile/consts";
export const handlers = [
    http.get("http://localhost:8000/api/getStatus", () => {
        return HttpResponse.json({
            isActive: STATUS.ACTIVE,
        })
    }),
];

In this file we declare our handlers and setup our mocks
as you can see the endpoint /api/getStatus is basically the same used in axios so no need to override or rewrite anything as MSW will automatically intercept this request and return our mock

final step is to call the worker inside our index.tsx



import React from 'react';
import ReactDOM from 'react-dom/client';
import './index.css';
import App from './App';

const root = ReactDOM.createRoot(
  document.getElementById('root') as HTMLElement
);

async function enableMocking() {
  if (!Boolean(import.meta.env.VITE_ENABLE_MSW)) {
    return
  }

  const { worker } = await import('./msw/workers')

  // `worker.start()` returns a Promise that resolves
  // once the Service Worker is up and ready to intercept requests.
  return worker.start()
}
enableMocking().then(() => {
  root.render(
    <React.StrictMode>
      <App />
    </React.StrictMode>
  );
});

As you can see the msw will be triggered in one condition if the env variable VITE_ENABLE_MSW is set to true

for this we need to create a .env.local file and also update scripts in package.json



//.env.local
VITE_ENABLE_MSW=false



  "scripts": {
    "dev": "vite",
    "dev-msw": "set VITE_ENABLE_MSW=true && vite",
    "msw-init": "npx msw init ./public --save",
}

NOTE:
if we want msw to be used in our development env we can just run our project using npm run msw-dev but before that we need to execute npm run msw-init in order to regsiter our mockServiceWorker.jsinside /public so that the file will be accessible via http://localhost:3000/mockServiceWorker.js.

After running our command we should be able to access our React app at http://localhost:3000/ then if we want to make sure that msw is working fine we can check our chrome devTools under application=>service worker

now we are ready to use it !!

*Run the project *

Once our project running we can check our http://localhost:3000/to check our app

if eveything was fine we should see this "User is active" text is displayed !!
if we try to check our network in chrome devTools for example we should see that the request has been successfully intercepted by MSW and we get our mocked result returned to us

As we can see we can use MSW this way during development and specially when our backend is down or blocked we can at least mock some endpoints and continue progressing

MSW and React Testing

In order to setup MSW for unit test we should proceed as follow.
1- install React testing library and Vitest



npm install vitest jsdom @testing-library/react @testing-library/jest-dom --save-dev

2- go to vite.config.ts and add the following config



import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [react()],
  test: {
    globals: true,
    environment: 'jsdom',
    setupFiles: './src/setupTests.ts',
  },
})

3- update our msw/worker.ts
For test purposes we need to change our previous import from
import { setupWorker } from 'msw/browser'
to
import { setupServer } from 'msw/node'
thus the file will look like this



//Worker.ts
import { setupServer} from "msw/node";
import { handlers } from "./handlers";

export const serviceWorker = setupServer(...handlers);

4- create a file called src/setupTests.ts and add the following config



import { expect, afterEach, beforeAll, afterAll } from 'vitest';
import { cleanup } from '@testing-library/react';
import * as matchers from "@testing-library/jest-dom/matchers";
import { worker } from './msw/workers';
expect.extend(matchers);

afterEach(() => {
  cleanup();
});

// Start worker before all tests
beforeAll(() => { worker.listen() })

//  Close worker after all tests
afterAll(() => {worker.close()})

// Reset handlers after each test `important for test isolation`
afterEach(() => {worker.resetHandlers()})

Before each test worker will start in order to setup our handlers then after each test it will reset the handlers to keep our mocks consistent then once all tests are finished the worker will be stoped
5- update package.json



"scripts": {
    "test:unit": "vitest --root src/"
}

6- Now create a unit test for our React component
First let make sure that our component is rendering without any issue



import { expect, test, describe } from 'vitest'
import { render } from '@testing-library/react';
import Profile from './profile';

describe('profile Component', () => {
    test('should render without crashing', () => {
        const { getByTestId } = render(<Profile />);
        const root = getByTestId('root');
        expect(root).toBeInTheDocument()
    })

})

then run :
npm run test:unit

and tests are working fine!!

Now let's try to check if msw is working inside tests, we will try to test if the component is returning the text "this user is active" which mean that the http requests to the msw are returning 200 and that we get what we want.
For that we will use the testid's to check if the component is rendering



test('should render user is active when server retun Status:Active', async() => {
        render(<Profile />);
        await waitFor(() => {
            const section1 = screen.queryByTestId('user-active');
            const section2 = screen.queryByTestId('user-offline');
            expect(section1).toBeInTheDocument();
            expect(section2).toBeNull();
          });

    })

And Bingo !! our tests are working

the full content of the test file:



import { expect, test, describe } from 'vitest'

import { render , screen, waitFor } from '@testing-library/react';

import Profile from './profile';

describe('profile Component', () => {

    test('should render without crashing', () => {

        const { getByTestId } = render(<Profile />);

        const root = getByTestId('root');

        expect(root).toBeInTheDocument()

    })

    test('should render user is active when server retun Status:Active', async() => {

        render(<Profile />);

        await waitFor(() => {

            const section1 = screen.queryByTestId('user-active');

            const section2 = screen.queryByTestId('user-offline');

            expect(section1).toBeInTheDocument();

            expect(section2).toBeNull();

          });

<span class="p">})</span>



})

Conclusion

MSW is a powerfull tool to use in our projects for development as well as testing it allows us to have a central setup mocks for our tests and removes bunch of noizy mocks from our test files also it keeps them consistent.
Any questions or issues please let us know in your comments
Cheers!!

Creating a realtime chat app with android , NodeJs and Socket.io

Mohamed Aymen Ourabi — Tue, 17 Jul 2018 23:27:15 +0000

Introduction

WebSockets are very beautiful tools that allows us to establish a realtime communication in modern web applications. In fact this mechanism is so powerfull and it's used to build different kind of apps like realtime chat or notification system etc ..

In this article we will show you how to build a realtime chat app using android nodeJs and Socket.io

Getting started

Our chat app is divded into 2 part :

1- Server side : a node js server with the implementation of socket.io for server

2- Client side : creating the android app and implementing socket.io for client

Our NodeJs Server

well , to make things clear our project architecture will be composed of 2 files :
package.json which will handle all the depandencies for our node js app and index.js which will be our main server .

After creating the two files , we open the commande line under our project
directory and execute this command

npm install --save  express socket.io

now in our index.js file we will build our server and make all the configurations so that it will look like this

const express = require('express'),
http = require('http'),
app = express(),
server = http.createServer(app),
io = require('socket.io').listen(server);
app.get('/', (req, res) => {

res.send('Chat Server is running on port 3000')
});


server.listen(3000,()=>{

console.log('Node app is running on port 3000')

});

to make sure that our server is runing go to the command line under our project directory and execute this command

node index.js

NOTE: using node command we can run any server created with node environment but the problem is that we have to run the same commande every time we update our index.js file , so to make things more simple we can use nodemon command which will automatically restart our server every time we make changes

so to install nodemon go to your command line and run

npm install -g nodemon

to make sure that our project is running we should see this log in our console

now comes the best part !!

we will try now to implement some socket.io methods in our server to handle all the events of our chat app including users connection states and messages .

in our index.js file we add the first implementation that will detect if we have a user connected to our server

io.on('connection', (socket) => {

console.log('user connected')

socket.on('join', function(userNickname) {

        console.log(userNickname +" : has joined the chat "  )

        socket.broadcast.emit('userjoinedthechat',userNickname +" : has joined the chat ")
    });

});

Actually socket.io mechanism is based on listening and firing events , in this first implementation that we have done the (on) method which takes two parameters ('eventname',callback) defines a listener to an event called connection and this event will be fired from the client side so that node js can handle it , after that we defined a method that will listen to an emitted event called 'join' and will log the name of the user who has join the chat in the console .

Now when node js detect a user it fires an event to the client side called 'userjoinedthechat' using the method emit , note that socket.broadcast.emit will send the event to every single user connected to the server except the sender .

if we want to send the message to all users including the sender we just have to use io.emit() instead of socket.emit().

Now to handle messages we add these few lines and we can see that we have added extra argument to the callback function which are the user nickname and the message content , actually these informations will be sent from the client side when firing the event 'messagedetection'

 socket.on('messagedetection', (senderNickname,messageContent) => {

       //log the message in console 

       console.log(senderNickname+" :" +messageContent)
        //create a message object

      let  message = {"message":messageContent, "senderNickname":senderNickname}

// send the message to the client side  

       socket.emit('message', message )

      });

And finally when the user disconnect from the client side , the event will be handled by this implementation


 socket.on('disconnect', function() {
    console.log( 'user has left ')
    socket.broadcast.emit( "userdisconnect" ,' user has left')


});

Now that our server is ready the index.js file should look like this

const express = require('express'),
http = require('http'),
app = express(),
server = http.createServer(app),
io = require('socket.io').listen(server);
app.get('/', (req, res) => {

res.send('Chat Server is running on port 3000')
});
io.on('connection', (socket) => {

console.log('user connected')

socket.on('join', function(userNickname) {

        console.log(userNickname +" : has joined the chat "  );

        socket.broadcast.emit('userjoinedthechat',userNickname +" : has joined the chat ");
    })


socket.on('messagedetection', (senderNickname,messageContent) => {

       //log the message in console 

       console.log(senderNickname+" : " +messageContent)

      //create a message object 

      let  message = {"message":messageContent, "senderNickname":senderNickname}

       // send the message to all users including the sender  using io.emit() 

      io.emit('message', message )

      })

socket.on('disconnect', function() {

        console.log(userNickname +' has left ')

        socket.broadcast.emit( "userdisconnect" ,' user has left')




    })




})






server.listen(3000,()=>{

console.log('Node app is running on port 3000')

})

Our Android app (Socket client)

To start open android studio and create a new projet with an empty activity , after that open app build.gradle file and add these dependencies then synchronize your project.

compile 'com.android.support:recyclerview-v7:25.3.1'
compile('com.github.nkzawa:socket.io-client:0.5.0') {
    exclude group: 'org.json', module: 'json'
}

Now about these lines :

the first one is the recycler view that we will use to display the list of our messages and the second one is the library that will provid us with the implementation of socket.io for the client side so that we can fire or listen to events .

don't forget to enable INTERNET permission in you manifest.xml

<uses-permission android:name="android.permission.INTERNET" ></uses-permission>

In activity_main.xml we will add an EditText for the user to put his nickname and a button that allows him to enter the chatbox

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"xmlns:app="http://schemas.android.com/apk/res-auto"xmlns:tools="http://schemas.android.com/tools"android:layout_width="match_parent"android:layout_height="match_parent"tools:context="com.example.aymen.androidchat.MainActivity">

   <EditText 
      android:id="@+id/nickname"android:layout_centerInParent="true"android:textSize="30dp"android:hint="Enter your nickname !"android:layout_width="match_parent"android:layout_height="wrap_content" /><Buttonandroid:layout_below="@+id/nickname"android:id="@+id/enterchat"android:text="Go to chat "android:layout_width="match_parent"android:layout_height="wrap_content" />

 </RelativeLayout>

so that the preview will look like this

now your MainActivity.java should look like this

import android.content.Intent;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.view.View;
import android.widget.Button;
import android.widget.EditText;

public class MainActivity extends AppCompatActivity {


    private Button btn;
    private EditText nickname;
    public static final String NICKNAME = "usernickname";
    @Overrideprotected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        //call UI components  by id
        btn = (Button)findViewById(R.id.enterchat) ;
        nickname = (EditText) findViewById(R.id.nickname);

        btn.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                //if the nickname is not empty go to chatbox activity and add the nickname to the intent extra


    if(!nickname.getText().toString().isEmpty()){

              Intent i  = new Intent(MainActivity.this,ChatBoxActivity.class);

                     //retreive nickname from EditText and add it to intent extra
                     i.putExtra(NICKNAME,nickname.getText().toString());

                     startActivity(i);
                 }
            }
        });

    }
}

Now create a second empty activity called ChatBoxActivity and in activity_chat_box.xml add these lines

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"xmlns:app="http://schemas.android.com/apk/res-auto"xmlns:tools="http://schemas.android.com/tools"android:layout_width="match_parent"android:layout_height="match_parent"tools:context="com.example.aymen.androidchat.ChatBoxActivity">
<LinearLayout
android:weightSum="3"
android:orientation="vertical"
android:layout_width="match_parent"
android:layout_height="match_parent">
<android.support.v7.widget.RecyclerViewandroid:layout_weight="3"android:id="@+id/messagelist"android:layout_width="match_parent"android:layout_height="wrap_content"android:clipToPadding="false"android:scrollbars="vertical"/><Viewandroid:layout_marginTop="5mm"android:id="@+id/separator"android:layout_width="match_parent"android:layout_height="1dp"android:background="@android:color/darker_gray"/>

<LinearLayoutandroid:weightSum="3"android:orientation="horizontal"android:layout_width="match_parent"android:layout_height="wrap_content">

            <EditText

                android:id="@+id/message"android:layout_weight="3"android:layout_width="wrap_content"android:hint="your message"

                android:layout_height="match_parent" />

            <Button

                android:id="@+id/send"android:layout_width="wrap_content"android:layout_height="wrap_content"android:background="#00000000"android:text="send"
                /></LinearLayout>

    </LinearLayout>
</RelativeLayout>

your preview should look like this

Now before implementing the socket client we should create an adapter to handle and display our messages for that we need to create a file called item.xml and a java class called message which have two simple string properties (nickname,message) .

In our project directory along side with activites create a file called Message.java :

public class Message {

    private String nickname; 
    private String message ;

    public  Message(){

    }
    public Message(String nickname, String message) {
        this.nickname = nickname;
        this.message = message;
    }

    public String getNickname() {
        return nickname;
    }

    public void setNickname(String nickname) {
        this.nickname = nickname;
    }

    public String getMessage() {
        return message;
    }

    public void setMessage(String message) {
        this.message = message;
    }
}

now create a file called item.xml under layout directory and add these lines

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"android:orientation="horizontal" android:layout_width="match_parent"android:layout_height="match_parent"><TextViewandroid:id="@id/nickname"android:textSize="15dp"android:textStyle="bold"android:text="Nickname : "android:layout_width="wrap_content"android:layout_height="wrap_content" /><TextViewandroid:id="@id/message"android:textSize="15dp"android:text=" message "android:layout_width="wrap_content"android:layout_height="wrap_content" />
</LinearLayout>

create a file called ChatBoxAdapter.java and put these lines

package com.example.aymen.androidchat;

import android.support.annotation.NonNull;
import android.support.v7.app.AppCompatActivity;
import android.support.v7.widget.RecyclerView;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;
import android.widget.ImageView;
import android.widget.TextView;

import java.util.List;


public class ChatBoxAdapter  extends RecyclerView.Adapter<ChatBoxAdapter.MyViewHolder> {
    private List<Message> MessageList;

    public  class MyViewHolder extends RecyclerView.ViewHolder {
        public TextView nickname;
        public TextView message;


        public MyViewHolder(View view) {
            super(view);

            nickname = (TextView) view.findViewById(R.id.nickname);
            message = (TextView) view.findViewById(R.id.message);





        }
    }
// in this adaper constructor we add the list of messages as a parameter so that 
// we will passe  it when making an instance of the adapter object in our activity 



public ChatBoxAdapter(List<Message>MessagesList) {

        this.MessageList = MessagesList;


    }

    @Overridepublic int getItemCount() {
        return MessageList.size();
    }
    @Overridepublic ChatBoxAdapter.MyViewHolder onCreateViewHolder(ViewGroup parent, int viewType) {
        View itemView = LayoutInflater.from(parent.getContext())
                .inflate(R.layout.item, parent, false);



        return new ChatBoxAdapter.MyViewHolder(itemView);
    }

    @Overridepublic void onBindViewHolder(final ChatBoxAdapter.MyViewHolder holder, final int position) {

 //binding the data from our ArrayList of object to the item.xml using the viewholder 



        Message m = MessageList.get(position);
        holder.nickname.setText(m.getNickname());

        holder.message.setText(m.getMessage() );




    }



}

now with everything setup we can implement the socket client in our ChatBoxActivity.java so this is how we are going to proceed :

1.Get the nickname of the user from the intent extra

2.call and implement all the methods relative to recycler view including the adapter instanciation

3.declare and define the host for socket client to make connection with the server

4.handle all events fired from the server

5.emit events when user connect , disconnect or send a message

but before that let's check if everything is ok or not so in our ChatBoxActivity we will declare the socket object and add the socket connection in the method onCreate so that when the activity is called the socket client will directly fire the event connection

public class ChatBoxActivity extends AppCompatActivity {

    //declare socket object

private Socket socket;
private String Nickname ;

@Overrideprotected 
void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_chat_box);

// get the nickame of the user



  Nickname= (String)getIntent().getExtras().getString(MainActivity.NICKNAME);

//connect you socket client to the server

try {


//if you are using a phone device you should connect to same local network as your laptop and disable your pubic firewall as well 

 socket = IO.socket("http://yourlocalIPaddress:3000");

 //create connection 

socket.connect()

// emit the event join along side with the nickname

socket.emit('join',Nickname); 


        } catch (URISyntaxException e) {
            e.printStackTrace();

        }

    }
}

now run your emulator and enter a nickname in the first activity then click go to chat you will see a log in your server console that indicates that a user has successfully made a connection with the server and we can see that the listener for the fired event join in our server is working properly so that it logs the name of the user connected

now with everything working , we should not forget that when our server handle an event it broadcast other costum events as well and so those fired events should be handled in the client side , for that we will make the first listener for the event "userjoinedthechat" which is a custom event fired when the server handle the event "join".

in our ChatBoxActivity we will add these lines

socket.on("userjoinedthechat", new Emitter.Listener() {
    @Overridepublic void call(final Object... args) {
        runOnUiThread(new Runnable() {
            @Overridepublic void run() {
                String data = (String) args[0];
                // get the extra data from the fired event and display a toast 
                Toast.makeText(ChatBoxActivity.this,data,Toast.LENGTH_SHORT).show();

            }
        });
    }

now we run 2 emulator in the same time and we enter two different nicknames from both sides and as we can see that one of the two emulator indicated that a user has successfully joined the chat

now comes the best part of our app which is the chat messages :

to display the messages we have to proceed this way

1.add onclickListener to the button send and grab the message content from the EditText after that emit the event "messagedetection" using emit() method along side with the nickname of the sender and the message content

2.the event will be handled by the server and broadcasted to all users

3.adding a socket listener in android to listen for the event "message" fired by the server

4.extract the nickname and the message from the extra data and make a new instance of the object Message

5.adding the instance to the ArrayList of messages and notify the adapter to update the recycler view

But before that let's setup our recyler view , Adapter , message textfield and the button send.

Add the declarations below in ChatBoxActivity

public RecyclerView myRecylerView ;
public List<Message> MessageList ;
public ChatBoxAdapter chatBoxAdapter;
public  EditText messagetxt ;
public  Button send ;

in the method onCreate add these lines

messagetxt = (EditText) findViewById(R.id.message) ;
send = (Button)findViewById(R.id.send);
MessageList = new ArrayList<>();
myRecylerView = (RecyclerView) findViewById(R.id.messagelist);
RecyclerView.LayoutManager mLayoutManager = new LinearLayoutManager(getApplicationContext());
myRecylerView.setLayoutManager(mLayoutManager);
myRecylerView.setItemAnimator(new DefaultItemAnimator());

Now in your ChatBoxActivity the button action should look like this

send.setOnClickListener(new View.OnClickListener() {
    @Overridepublic void onClick(View v) {
        //retrieve the nickname and the message content and fire the event messagedetection


  if(!messagetxt.getText().toString().isEmpty()){

            socket.emit("messagedetection",Nickname,messagetxt.getText().toString());

            messagetxt.setText(" ");         
    }




    }
});

and the listener should look like this

socket.on("message", new Emitter.Listener() {
    @Overridepublic void call(final Object... args) {
        runOnUiThread(new Runnable() {
            @Overridepublic void run() {
                JSONObject data = (JSONObject) args[0];
                try {
                    //extract data from fired event

              String nickname = data.getString("senderNickname");
              String message = data.getString("message");

           // make instance of message

          Message m = new Message(nickname,message);


          //add the message to the messageList

          MessageList.add(m);

          // add the new updated list to the adapter 
          chatBoxAdapter = new ChatBoxAdapter(MessageList);

           // notify the adapter to update the recycler view

          chatBoxAdapter.notifyDataSetChanged();

           //set the adapter for the recycler view 

          myRecylerView.setAdapter(chatBoxAdapter);


                } catch (JSONException e) {
                    e.printStackTrace();
                }


            }
        });
    }
});

as we can see in the screenshot below everything is working properly :)) and the messages are displaying from both sides , note that we can connect with many other users but we just have to run other emulators and enter nicknames to join the chatbox

before ending this tutorial we have to make our last functionnality which detects if the user has disconnected from the chatbox .

In our ChatBoxActivity override the method onDestroy() and add these lines

@Override
protected void onDestroy() {
    super.onDestroy();
    socket.disconnect(); 
}

and for the listener

socket.on("userdisconnect", new Emitter.Listener() {
    @Overridepublic void call(final Object... args) {
        runOnUiThread(new Runnable() {
            @Overridepublic void run() {
                String data = (String) args[0];

                Toast.makeText(ChatBoxActivity.this,data,Toast.LENGTH_SHORT).show();

            }
        });
    }
});

finally our ChatBoxActivity will look like this

import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.support.v7.widget.DefaultItemAnimator;
import android.support.v7.widget.LinearLayoutManager;
import android.support.v7.widget.RecyclerView;
import android.view.View;
import android.widget.Button;
import android.widget.EditText;
import android.widget.Toast;

import com.github.nkzawa.emitter.Emitter;
import com.github.nkzawa.socketio.client.IO;
import com.github.nkzawa.socketio.client.Socket;

import org.json.JSONException;
import org.json.JSONObject;

import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.List;

public class ChatBoxActivity extends AppCompatActivity {
    public RecyclerView myRecylerView ;
    public List<Message> MessageList ;
    public ChatBoxAdapter chatBoxAdapter;
    public  EditText messagetxt ;
    public  Button send ;
    //declare socket objectprivate Socket socket;

    public String Nickname ;
    @Overrideprotected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_chat_box);

        messagetxt = (EditText) findViewById(R.id.message) ;
        send = (Button)findViewById(R.id.send);
        // get the nickame of the user
        Nickname= (String)getIntent().getExtras().getString(MainActivity.NICKNAME);
        //connect you socket client to the servertry {
            socket = IO.socket("http://yourlocalIPaddress:3000");
            socket.connect();
            socket.emit("join", Nickname);
        } catch (URISyntaxException e) {
            e.printStackTrace();

        }
       //setting up recyler
        MessageList = new ArrayList<>();
        myRecylerView = (RecyclerView) findViewById(R.id.messagelist);
        RecyclerView.LayoutManager mLayoutManager = new LinearLayoutManager(getApplicationContext());
        myRecylerView.setLayoutManager(mLayoutManager);
        myRecylerView.setItemAnimator(new DefaultItemAnimator());



        // message send action
        send.setOnClickListener(new View.OnClickListener() {
            @Overridepublic void onClick(View v) {
                //retrieve the nickname and the message content and fire the event messagedetectionif(!messagetxt.getText().toString().isEmpty()){
                    socket.emit("messagedetection",Nickname,messagetxt.getText().toString());

                    messagetxt.setText(" ");
                }


            }
        });

        //implementing socket listeners
        socket.on("userjoinedthechat", new Emitter.Listener() {
            @Overridepublic void call(final Object... args) {
                runOnUiThread(new Runnable() {
                    @Overridepublic void run() {
                        String data = (String) args[0];

                        Toast.makeText(ChatBoxActivity.this,data,Toast.LENGTH_SHORT).show();

                    }
                });
            }
        });
        socket.on("userdisconnect", new Emitter.Listener() {
            @Overridepublic void call(final Object... args) {
                runOnUiThread(new Runnable() {
                    @Overridepublic void run() {
                        String data = (String) args[0];

                        Toast.makeText(ChatBoxActivity.this,data,Toast.LENGTH_SHORT).show();

                    }
                });
            }
        });
        socket.on("message", new Emitter.Listener() {
            @Overridepublic void call(final Object... args) {
                runOnUiThread(new Runnable() {
                    @Overridepublic void run() {
                        JSONObject data = (JSONObject) args[0];
                        try {
                            //extract data from fired event

                            String nickname = data.getString("senderNickname");
                            String message = data.getString("message");

                            // make instance of message

                            Message m = new Message(nickname,message);


                            //add the message to the messageList

                            MessageList.add(m);

                            // add the new updated list to the dapter
                            chatBoxAdapter = new ChatBoxAdapter(MessageList);

                            // notify the adapter to update the recycler view

                            chatBoxAdapter.notifyDataSetChanged();

                            //set the adapter for the recycler view

                            myRecylerView.setAdapter(chatBoxAdapter);

                        } catch (JSONException e) {
                            e.printStackTrace();
                        }


                    }
                });
            }
        });
    }

    @Override

protected void onDestroy() {
        super.onDestroy();

        socket.disconnect(); 
    }
}

Conclusion

In this exemple we had a great look at the usage of socket.io along side with node js and android , we tried as well to explain some basics and understand the mechanism of socket.io and how to establish a bi-directional communications between a client and a server , note that there are other tools in socket.io like rooms and namespaces which could be very helpfull to make beautifull web and mobile apps .

find in these related links the two projects :

client side :https://github.com/medaymenTN/AndroidChat

server side: https://github.com/medaymenTN/NodeJSChatServer

Securing your Node js api with JSON Web Token

Mohamed Aymen Ourabi — Sun, 15 Jul 2018 14:47:49 +0000

Introduction

Nowadays REST ( Representational state transfer) has become the most used style in web architecture due to their simple syntax and flexibility . REST allows users to make their apps extensible, the fact that a client from different frontend platforms can perform requests to the server using http protocol in a simple way and exchange data which could be encoded in JSON or XML format.

Now with all the features that comes up with Restful architecture it still have some problems specially with their security .

From the many security approaches that are used to secure Restful api's is token based authentication

So what is token based authentication ?

let's make this easy :))

The general concept behind a token-based authentication system is simple.

Allow users to use their username and password in order to obtain a token which allows them to access a specific resource without using every time their natural credentials.

Once their token has been obtained, the user can use that token to access a specific resource in a server for a time period to the remote site.

How it works ??

well the process of using jwt is composed of 6 steps

1- authenticate using credentials

2- once authentication is granted the server generate a random string which contains the json web token

3- return the token to the client side

4- storing the token in the client side

5- sending the token with every single http request from the client to the server

6- the server check whether the token is valid or not and grant access to the specified resource

What we are going to build ?

well in this article we are going to build an API with Node js and Express.js and we will test it with postman so let's get started :))

first let's take a look at our project structure

-/configurations
             ->/config.js

-package.json

-index.js

now that our project is dived and ready to go let's install our packages.

open your command line under your project directory and write this command

npm install --save  express body-parser morgan jsonwebtoken

so let's explain the packages that we have installed

Express : the famous node js framework

body-parser: allow us to get the data from the body of the requests

morgan : logs the requests in the console

jsonwebtoken : the package that allows us to generate jwt and build our middleware to check whether the token is valid or not .

Now let's go the config.js

config.js

well accually this file is used to setup some configurations that most users need to do in order to better organise their projects.

They can setup configuration for databases or for other purposes , in our case we are going to use this file to setup our secret which will be used when creating our jwt so the file should look like this

module.exports = {

    secret : "heymynameismohamedaymen"
}

Now let's go to our index.js which is the most important file in our app .

index.js

const express = require('express'),
bodyParser = require('body-parser'),
morgan      = require('morgan'),
jwt    = require('jsonwebtoken'),
config = require('./configurations/config'),
app = express(); 

//set secret
app.set('Secret', config.secret);

// use morgan to log requests to the console
app.use(morgan('dev'));

// parse application/x-www-form-urlencoded
app.use(bodyParser.urlencoded({ extended: true }));

// parse application/json
app.use(bodyParser.json());

app.listen(3000,()=>{

 console.log('server is running on port 3000') 

});
app.get('/', function(req, res) {
    res.send('Hello world  app is running on http://localhost:3000/');
});

to check if everything is ok go to the command line and run this command


node index.js

open your browser on http://localhost:3000/

well everything looks fine !!

we can see also that the request is logged in our console thanks to morgan package

Setting up the authentication system

Now comes the best part of our app :)) .

Well in this app we are not really going to work with real models which are stored in database but we are going to setup a static login and password to check whether the user exists or not ,because what we want to focus on in this article is the usage of JWT, so simply you can change the code that we are about to write .

so in our index.js let's create the authenticate route

here we choose aymen as a username and 123 as a password

app.post('/authenticate',(req,res)=>{

    if(req.body.username==="aymen"){

        if(req.body.password===123){
             //if eveything is okey let's create our token 

        const payload = {

            check:  true

          };

          var token = jwt.sign(payload, app.get('Secret'), {
                expiresIn: 1440 // expires in 24 hours

          });


          res.json({
            message: 'authentication done ',
            token: token
          });

        }else{
            res.json({message:"please check your password !"})
        }

    }else{

        res.json({message:"user not found !"})

    }

})

now that the route is build we can get back our token .. so let's make a test with postman

now that we have the token, as a client we have first to store that token somehow and there are many tools to do that , for exemple if we are using our browsers we can use localstorage or if we are using android to create a mobile app we can use sharedpreferences

Setting up the middleware

For the moment we have our token and we can make http requests to the server , but we must also build our middleware that will handle every http request , search for the token and check if it's valide or not .

But before creating the middleware we must create the routes that will be protected with it , so in our index.js the protected routes should look like this

const  ProtectedRoutes = express.Router(); 

app.use('/api', ProtectedRoutes);

Now every route under /api will be a protected route by the middleware and to get access to the resource that uses /api as a parent route we have to provide the right token for it .

To send the token to the server along side with some data we usually store the token in the header of every request after that the middleware will handle the http request and extract the token from the header .

so in our index.js let's write the code that will do that

ProtectedRoutes.use((req, res, next) =>{


    // check header for the token
    var token = req.headers['access-token'];

    // decode token
    if (token) {

      // verifies secret and checks if the token is expired
      jwt.verify(token, app.get('Secret'), (err, decoded) =>{      
        if (err) {
          return res.json({ message: 'invalid token' });    
        } else {
          // if everything is good, save to request for use in other routes
          req.decoded = decoded;    
          next();
        }
      });

    } else {

      // if there is no token  

      res.send({ 

          message: 'No token provided.' 
      });

    }
  });

To check if our middleware works or not we are going to create another route in which we are going to return an array of products and we are going to set this route under /api using ProtectedRoutes, after that we will make a GET request to get the data back .

first let's create the route

ProtectedRoutes.get('/getAllProducts',(req,res)=>{
 let products = [
     {
         id: 1,
         name:"cheese"
     },
     {
        id: 2,
        name:"carottes"
    }
 ]

 res.json(products)

})

now let's try to get the list of products without providing the token and see what happens

this time the middleware did not return the data because we did not provide the token or in other word the api did not recognize us and thought that we are the bad guys that want to grab some informations .

so now using postman we are going to put the token in the header of our request and let's perform another request

Now everthing looks great we get the data back ;)).

Conclusion

In this exemple we had a great look at the usage of JWT and their importance for the security of the Restful Api's , note that this is not the only approach for the security of node js apps but there are a lot of tools that could be very usefull .

we hope that this look has given a better understunding about how the tokens are created , how the routes are protected and how all of this if managed inside a node js app .

you can find the project in the link below :

(https://github.com/medaymenTN/JWTNodeJS)