DEV Community: Bruno

Full Stack setup from scratch - Node.js, Express.js, React.js and Lowdb (Part 3)

Bruno — Fri, 20 Mar 2020 14:28:45 +0000

To deploy to Heroku, you have create an account at Heroku.

Heroku is a cloud platform that lets companies build, deliver, monitor and scale apps — we're the fastest way to go from idea to URL, bypassing all those infrastructure headaches.

After creating and logging in into your account, download and install the Heroku cli. Then, open your terminal and type the following command to login Heroku services:

heroku login

Next, create your app, typing:

heroku create filter-pdf

Then, in package.json in the root project, add the script:

"heroku-postbuild": "NPM_CONFIG_PRODUCTION=false npm install --prefix front && npm run build --prefix front"

Now, commit that change and push it to Heroku:

git push heroku master

If you want deploy other branch, so:

git push heroku otherBranch:master

Then, go to the heroku dashboard page, access your app and check the app's overview section. That is where Heroku will display some building logs.

Once you get a "Deployed" message, you can click on the "Open app" button and see the application running.

I hope this tutorial helps. Thanks.

Full Stack setup from scratch - Node.js, Express.js, React.js and Lowdb (Part 2)

Bruno — Fri, 20 Mar 2020 14:27:32 +0000

The purpose of this post is to learn how to upload a pdf file and validate the file that will be sent.

Following Brazilian labor law, employees must register the punch in and the punch out of the office, and lunch breaks. A simple problem we have faced is that we have several offices in Brazil and there is a single pdf document with all employees (> 160).

To avoid wasting paper by printing all sheets or wasting time selecting only employees from 1 office, each employee needs to print their own time sheet.

The application works as follows:

Launch the application displaying the list of employees (first and last name), allowing to update the list by adding, editing or removing employees;
Select the time sheet pdf document and upload it;
After the upload, the employees and the corresponding page in the pdf are listed;
And finally just click on a button to display the list with ranges of pages to be printed;

The first step in the process was to find employee information that corresponded to the document.
In one of the company's systems I filtered the employees from Fortaleza-CE, but only the first and last names were useful.

At first I couldn't find a lib for the frontend that would read the pdf content, so I found the pdf-parse for the backend.

First, let's install the dependencies in package.json, in the root project:

npm install --save multer pdf-parse

multer is a node.js middleware for handling multipart/form-data, which is primarily used for uploading files.
pdf-parse is a pure javascript cross-platform module to extract texts from PDFs.

Then, let's add uploadRoutes.js to folder /routes and add the content:

const low = require('lowdb');
const multer = require('multer');
const fs = require('fs');
const pdf = require('pdf-parse');

const FileSync = require('lowdb/adapters/FileSync');

const adapter = new FileSync('db.json');
const db = low(adapter);

let storage = multer.diskStorage({
  destination: function (req, file, cb) {
    cb(null, 'files')
  },
  filename: function (req, file, cb) {
    cb(null, file.originalname )
  }
})
let upload = multer({ storage }).single('file');

let usersToPrint = async (file) => {
  let dataBuffer = fs.readFileSync(file.path);

  const data = await pdf(dataBuffer);
  const pages = data.text.split('CONTROLE DE FREQÜÊNCIA');
  const namesPDF = pages.map(page => {
    let startIndex = page.lastIndexOf("Empregado:");
    if (startIndex === -1) {
      startIndex = page.lastIndexOf("Estagiário:") + 1;
    }
    return page.substring(startIndex + 17, page.lastIndexOf("CTPS: ") - 18);
  });
  namesPDF.shift();
  const toPrint = [];
  const users = db.get('users').value();
  users.map(user => {
    // user -> [BRUNO, BEZERRA]
    // name -> BRUNO BEZERRA CHAVES
    namesPDF.find((name, index) => {
      const nameList = name.split(' ');
      if (nameList.includes(user.name) && nameList.includes(user.lastName)) {
        toPrint.push({
          nameComplete: name,
          page: index + 1,
          checked: true,
          ...user
        });
      }
    });
  });
  toPrint.sort((a, b) => a.page > b.page ? 1 : -1);
  return toPrint;
}

module.exports = (app) => {

  app.post(`/api/upload`, (req, res) => {
    upload(req, res, async function (err) {
      if (err instanceof multer.MulterError) {
        return res.status(500).json(err);
      } else if (err) {
        return res.status(500).json(err);
      }

      const users = await usersToPrint(req.file);
      return res.status(200).send({ users });
    });
  })

}

The files loaded (upload) in the client, go to the /files folder. The function usersToPrint is used in /api/upload to read the pdf (full name) and filter users according users (name and last name) in database.
So, add the folder in the root project:

mkdir files

So, let's import the uploadRoutes.js file to index.js:

require('./routes/uploadRoutes')(app);

in frontend, let's install the dependencies:

npm install --save bootstrap reactstrap react-toastify

bootstrap is an open source toolkit for developing with HTML, CSS, and JS.
reactstrap is easy to use React Bootstrap 4 components.
react-toastify allow you to add notification to your app with ease.

Then, let's create a upload.service.js to upload pdf:

import axios from 'axios';

export default {
  upload: async (data, progress) => {
    let res = await axios.post(`/api/upload`, data, progress);
    return res.data || [];
  }
}

Let's create 4 files in folder front/src/components to use in application:

InputFile.js to handle pdf to upload and some validations.

import React from 'react';
import { toast } from 'react-toastify';
import { Input } from 'reactstrap';

const InputFile = (props) => {
  const maxSelectFile = (event) => {
    let files = event.target.files; // create file object
    if (files.length > 1) { 
      const msg = 'Only 1 pdf can be uploaded at a time';
      event.target.value = null; // discard selected file
      toast.error(msg);
      return false;
    }
    return true;
  }

  const checkMimeType = (event) => {
    let files = event.target.files;
    let err = '';
    const types = ['application/pdf'];
    for(let x = 0; x<files.length; x++) {
      if (types.every(type => files[x].type !== type)) {
        err += files[x].type + ' is not a supported format\n';
      }
    };

    for(var z = 0; z<err.length; z++) {
      event.target.value = null;
      toast.error(err[z]);
    }
    return true; 
  }

  const checkFileSize = (event) => {
    let files = event.target.files;
    let size = 20000000;
    let err = ''; 
    for(let x = 0; x<files.length; x++) {
      if (files[x].size > size) {
        err += files[x].type + 'is too large, please pick a smaller file\n';
      }
    }
    for(let z = 0; z<err.length; z++) {
      toast.error(err[z]);
      event.target.value = null;
    }
    return true; 
  }

  const onChangeHandler = async (event) => {
    console.log(event.target.files[0]);
    if (maxSelectFile(event) && checkMimeType(event) && checkFileSize(event)) {
      props.selectedFile(event.target.files[0]);
    }
  }

  return (
    <Input className="mt15" type="file" name="file" onChange={onChangeHandler}/>
    );
  }

  export default InputFile;

TableRow.js to show user name, user lastName, page in pdf and button to edit.

import React from 'react';
import { Button } from 'reactstrap';

const TableRow = (props) => {
  const { id, user, edit } = props;
  const name = user.name.charAt(0) + user.name.toLowerCase().slice(1);
  const lastName = user.lastName.charAt(0) + user.lastName.toLowerCase().slice(1);

  return (
    <tr key={id} className="list__item user" title={user.nameComplete}>
      <td>{user.page ? (
        <input
          type="checkbox"
          id={id}
          name={id}
          value={user.checked}
          defaultChecked={user.checked}
          onClick={() => {user.checked = !user.checked}}
          ></input>
        ) : ''}
      </td>
      <td>{name}</td>
      <td>{lastName}</td>
      <td>{user.page ? user.page  : ''}</td>
      <td>
        <Button color="info" onClick={() => edit(user)}>Edit</Button>
      </td>
    </tr>
  );
}

export default TableRow;

ModalPdf.js to show pages to print.

import React, { useState } from 'react';
import { Modal, ModalHeader, ModalBody, ModalFooter, Button } from 'reactstrap';

const ModalPdf = (props) => {
  const {
    onClose
  } = props;

  const [modal, setModal] = useState(true);

  const toggle = () => {
    onClose();
    setModal(!modal);
  }

  const copyToClipboard = () => {
    navigator.clipboard.writeText(props.children);
  }

  return (
    <div>
      <Modal isOpen={modal} toggle={toggle}>
        <ModalHeader toggle={toggle}>Pages</ModalHeader>
        <ModalBody>
          {props.children}
        </ModalBody>
        <ModalFooter>
          <Button onClick={copyToClipboard}>Copy To Clipboard</Button>
        </ModalFooter>
      </Modal>
    </div>
  );
}

export default ModalPdf;

ModalDuplicated.js to show users that contain the same name and last name.

import React, { useState } from 'react';
import { Modal, ModalHeader, ModalBody } from 'reactstrap';

const ModalDuplicated = (props) => {
  const {
    onClose
  } = props;

  const [modal, setModal] = useState(true);

  const toggle = () => {
    onClose();
    setModal(!modal);
  }

  return (
    <div>
      <Modal isOpen={modal} toggle={toggle}>
        <ModalHeader toggle={toggle}>Duplicates</ModalHeader>
        <ModalBody>
          {props.children}
        </ModalBody>
      </Modal>
    </div>
  );
}

export default ModalDuplicated;

Then, let's add the code in front/src/index.css:

.mt15 {
  margin-top: 15px;
}

.scroll-table {
  display: block;
  height: 400px;
  overflow-y: scroll;
}

And finally, import bootstrap and react-toastfy to front/src/index.js:

import 'bootstrap/dist/css/bootstrap.min.css';
import 'react-toastify/dist/ReactToastify.css';

Run application with command npm run dev and follow the steps:
Choose File -> Upload -> Select users to print -> Pages to print -> Copy To Clipboard

Finally, we can print only pages according employees of the office (employees saved in the database).
In this quick tutorial we saw how to upload a pdf, manipulate the content, and run validations for these situations: too many images to upload, upload an image with wrong file extension and sending an image file that is too large.

We can make many improvements and apply other good practices to the project. For example, better modularize the components, simplify the Modals, etc ... but that is for future posts.

The source code can be found in node-react-project in branch filter-pdf.

Full Stack setup from scratch - Node.js, Express.js, React.js and Lowdb (Part 1)

Bruno — Fri, 20 Mar 2020 14:26:04 +0000

First, in command line, let's create a project folder and a package.json file:

mkdir node-react-project
cd node-react-project
npm init -y

package.json

name defines the name of the application or package;

version indicates the current version;

description is a summary of your application/package;

main defines the application entry point;

private (true) prevents your application from being accidentally published to npm;

scripts defines a set of Node scripts for you to execute;

dependencies defines a list of installed npm packages as dependencies;

devDependencies defines a list of installed npm packages as development dependencies;

engines defines which Node versions this package /application works;

browserslist is used to tell which browsers (and versions) you want to support;

Now, let's install the project dependencies:

npm install --save express body-parser lowdb

Express.js is a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications.
body-parser Node.js body parsing middleware. Parse incoming request bodies in a middleware before your handlers, available under the req.body property.
lowdb is a small JSON database for Node, Electron and the browser.

Install the development dependencies:

npm install --save-dev nodemon concurrently

nodemon is a tool that helps develop node.js based applications by automatically restarting the node application when file changes in the directory are detected.
concurrently run multiple commands concurrently.

Then, we have package.json like this:

Let's create the project scructure:

mkdir routes
touch index.js

In the index.js we will add the following code:

const express = require('express');
const bodyParser = require('body-parser');

const app = express();

app.use(bodyParser.json());

const PORT = process.env.PORT || 5000;
app.listen(PORT, () => {
  console.log(`app running on port ${PORT}`)
});

So, add the "server" entry into the "scripts" section of your package.json:

"server": "nodemon index.js"

Then, we can execute that script to run backend:

npm run server

Before starting version control, let's create a .gitignore file in the project's root folder and add the following line:

node_modules

After that, create a new repository on GitHub and upload our source code to that.

git init
git add .
git commit -m "backend configuration"
git remote add origin https://github.com/brunobc/node-react-project.git
git push -u origin master

Now, let's create a frontend with create-react-app and add dependencies:

npx create-react-app front
cd front
npm install --save http-proxy-middleware axios

http-proxy-middleware is a node.js proxying from frontend to backend.
axios is a promise based HTTP client for the browser and node.js.

In client directory /front/src, let's add the file setupProxy.js to proxy from frontend to backend.

const { createProxyMiddleware } = require('http-proxy-middleware')

module.exports = function(app) {
  app.use('/api/', createProxyMiddleware({
    target: 'http://localhost:5000',
    changeOrigin: true
  }));
}

In the package.json, in the root project, add the scripts:

"client": "npm run start --prefix front",
"server": "nodemon index.js",
"dev": "concurrently --kill-others-on-fail \"npm run server\" \"npm run client\"",
"start": "node index.js"

To run the project in production, we need to add the following code after app.use(bodyParser.json()), in your index.js file. That will redirect all requests from the frontend.

if (process.env.NODE_ENV === 'production') {
  app.use(express.static('front/build'));

  const path = require('path');
  app.get('*', (req,res) => {
    res.sendFile(path.resolve(__dirname, 'front', 'build', 'index.html'));
  });
}

Let's create the database file, as required by LowDB, naming it as db.json in the root project, then let's use shortid to create the user' ids.

touch db.json
npm install --save shortid

In db.json:

{
  "users": []
}

Now let's create the routes for our backend API.
We added the userRoutes.js file to the /routes directory. Then, add the following snippet:

const low = require('lowdb');
const FileSync = require('lowdb/adapters/FileSync');
const shortid = require('shortid');

const adapter = new FileSync('db.json');
const db = low(adapter);

module.exports = (app) => {

  app.get(`/api/users`, async (req, res) => {
    const users = db.get('users').value();
    return res.status(200).send(users);
  });

  app.post(`/api/user`, async (req, res) => {
    const { name, lastName } = req.body;
    const id = shortid.generate();
    const users = db
      .get('users')
      .push({ id, name, lastName })
      .write();

    const user = db.get('users')
      .find({ id })
      .value();

    return res.status(201).send({
      error: false,
      user
    });
  })

  app.put(`/api/user`, async (req, res) => {
    const { name, lastName, id } = req.body;

    let users = db.get('users')
        .find({ id })
        .assign({ name, lastName })
        .write();

    const user = db.get('users')
      .find({ id })
      .value();

    return res.status(202).send({
      error: false,
      user
    });
  });

  app.delete(`/api/user/:id`, async (req, res) => {
    const { id } = req.params;
    console.log(id);

    db.get('users')
      .remove({ id })
      .write()

    return res.status(202).send({
      error: false
    })

  })

}

Finally, open the index,js file, and after app.use(bodyParser.json()), configure the api routes:

require('./routes/userRoutes')(app);

Now we can run the backend using the url:http://localhost:5000/api/users and do get, insert, update e delete a user.

We can use Postman to add new users. Make a POST to http://localhost:5000/api/user with the json using this payload:

{
  "name": "Bruno",
  "lastName": "Bezerra"
}

To facilitate the use of requests, just import the following collecion in postman.

Now on the frontend (react) we will add a service in the /front/src/services folder called user.service.js to make requests.

import axios from 'axios';

export default {
  getAll: async () => {
    let res = await axios.get(`/api/users`);
    return res.data || [];
  },
  add: async (name, lastName) => {
    let res = await axios.post(`/api/user/`, { name, lastName })
    return res.data || {};
  },
  edit: async (name, lastName, id) => {
    let res = await axios.put(`/api/user/`, { name, lastName, id })
    return res.data || {};
  },
  delete: async (id) => {
    let res = await axios.delete(`/api/user/${id}`);
    return res.data || [];   
  }
}

Edit the App.js to list users:

import React, { useState, useEffect } from "react";

import userService from './services/user.service';

function App() {
  const [ users, setUsers ] = useState(null);

  useEffect(() => {
    if (!users) {
      getUsers();
    }
  })

  const getUsers = async () => {
    let res = await userService.getAll();
    setUsers(res);
  }

  const renderUser = user => {
    return (
      <li key={user._id} className="list__item user">
        <h3 className="user__name">{user.name}</h3>
        <h3 className="user__lastName">{user.lastName}</h3>
      </li>
    );
  };

  return (
    <div className="App">
      <ul className="list">
        {(users && users.length > 0) ? (
          users.map(user => renderUser(user))
        ) : (
          <p>No users found</p>
        )}
      </ul>
    </div>
  );
}

export default App;

And finally, we can run the command at the root of the project to run the backend and frontend:

npm run dev

I hope this application is useful and serves as a starting point for future projects. :)
The source code can be found in node-react-project.