Read data from Gmail using Node JS

#node #webdev #devops #javascript

I've struggled a lot with this topic. Finally, after a lot of searches and referring documentation, I'm finally writing this article. I hope it helps everyone. In this article, the code will be written in NodeJS. This app will include 2 parts, the first is creating and setting up a project on google console and the next is the coding part.

1) Creating a project on Google Console

Open https://console.developers.google.com/ website and you need to create a project. On the top left corner, there will be a list of your previous projects or option select a project.
A modal will pop up. Select on the option NEW PROJECT.

select -> NEW PROJECT Enter a project name and create the project

2) Enable Gmail API

Ensure that you've selected that project, then in the search box search Gmail API. Enable Gmail API.

3) Configure Project

You need credentials but before creating credentials you need to configure credentials. On the left-hand side, you'll find OAuth consent Screen. Click on it.

User Type -> External
Enter app information ie, app name, supporting email, developer contact information.
Scope page save and continue
Test User Tab: Click on add user and you can add up to 100 emails. Add your email for testing. Save and continue.
Finally, after setting up, click on Credentials.

4) Create Credentials

After landing on Credentials, on the top click on CREATE CREDENTIALS. Click on OAuth client ID. Select your application type. As we're using NodeJS, it is a web application. Add URI's as
http://localhost:3000. Create and you'll get your credentials.

Now comes part two, the coding part

5) Code set up:

Go to this page, Node JS quickstart.

Copy Paste the sample code and then copy-paste in a file named index.js.
In the folder, where you created this file, the terminal add the command

npm init

and initializes

package.json
You need to install some dependencies with the command

npm i googleapis cheerio mailparser js-base64 open
Go to google developers console in your project. Navigate to the credentials part. In OAuth 2.0 Client IDs, you'll find a small download icon, download your credentials file from there and add to your folder where you've created this project. Name this file as

credentials.json
Run your code in your terminal. When you run for the first time, you'll get something like this

Authorize this app by visiting this url: https://accounts.google.com/o/oauth2/v2/auth?access_type=offline&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fgmail.readonly&response_type=code&client_id=479559853488-050lms0ffusprdhh938s954q0s36kg4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob
‍Enter the code from that page here:

Click on that URL and enter the code.
Now, in order to be able to manage the labels of the messages, you need to change the initial scope from gmail.readonly to gmail.modify.

const SCOPES = ['https://www.googleapis.com/auth/gmail.modify'];

Delete token.json from your working directory.

One error that some of you might get. The reference code has credentials.installed but it should be credentials.web. When you check the file, credentials.json you'll find everything is inside web object. So, if you get that error just check your creddentials.json file once and replace installed accordingly.

function authorize(credentials, callback) {
  const {client_secret, client_id, redirect_uris} = credentials.installed;
...
}

If you're facing any other issue regarding app verification then navigate to verification process and follow them.

6)Final Output:

Then, go to the listLabels function and add the label id in the console log.

functionlistLabels(auth) {  
  ...      
    labels.forEach((label) => {        
      console.log(`- ${label.name} : ${label.id}`);      
    });  
  ...}

Finally, run the application by using the command

npm run server

to authorize the app again and update the access permissions. You should be able to see the label list but now each label with its corresponding id.

Now, to read mail box,

function listMessages(auth, query){
  query = 'unnatibamania8@gmail.com';
  return new Promise((resolve, reject) => {    
    const gmail = google.gmail({version: 'v1', auth});
    gmail.users.messages.list(      
      {        
        userId: 'me',  
        q:query,      
        maxResults:5     
      },            (err, res) => {        
        if (err) {                    reject(err);          
          return;        
        }        
        if (!res.data.messages) {                    resolve([]);          
          return;        
        }                resolve(res.data);  

                         getMail(res.data.messages[0].id, auth);
      }    
    );  
  })
}

Here, the function listMessages takes 2 parameters auth and query, you can call this function

fs.readFile('credentials.json', (err, content) => {
  if (err) return console.log('Error loading client secret file:', err);
  // Authorize a client with credentials, then call the Gmail API.
  authorize(JSON.parse(content), listMessages);

});

after console logging res.data.messages, you'll get threadId and id. Pass the id into getMail function.

function getMail(msgId, auth){
  console.log(msgId)
  const gmail = google.gmail({version: 'v1', auth});
  //This api call will fetch the mailbody.
  gmail.users.messages.get({
      userId:'me',
      id: msgId ,
  }, (err, res) => {
    console.log(res.data.labelIds.INBOX)
      if(!err){
        console.log("no error")
          var body = res.data.payload.parts[0].body.data;

          var htmlBody = base64.decode(body.replace(/-/g, '+').replace(/_/g, '/'));
          console.log(htmlBody)
          var mailparser = new Mailparser();

          mailparser.on("end", (err,res) => {
              console.log("res",res);
          })

          mailparser.on('data', (dat) => {
              if(dat.type === 'text'){
                  const $ = cheerio.load(dat.textAsHtml);
                  var links = [];
                  var modLinks = [];
                  $('a').each(function(i) {
                      links[i] = $(this).attr('href');
                  });

                  //Regular Expression to filter out an array of urls.
                  var pat = /------[0-9]-[0-9][0-9]/;

                  //A new array modLinks is created which stores the urls.
                  modLinks = links.filter(li => {
                      if(li.match(pat) !== null){
                          return true;
                      }
                      else{
                          return false;
                      }
                  });
                  console.log(modLinks);

                  //This function is called to open all links in the array.

              }
          })

          mailparser.write(htmlBody);
          mailparser.end();

      }
  });
}

Final Output

Don't read all the messages all together it might crash the application. Hence, I've read only 1 mail at a time. You can just play around with the code and find a lot of other stuff!
I hope this article helps.