How I have created my own technological survey with node.js and Gitlab CI/CD.

Disclaimer: It's my first article, and I'm not yet proud of my English level. Nice read at all !

Hello guys, how are you ? Do you subscribes to newsletters for your daily technological survey and don't read any articles ?

I realized that some time ago. So I have decided to create my own "private" flux of news/articles.

In theory.

The theory is really simple:

1. We need to find some rss flux of what you want to read.
2. After this search, we need to make a script who send an email every morning at your favorite email address.
3. And at the end, you can run this script on a server as a cron or you can also use Gitlab CI/CD.

Notice that I have make the minimum at the moment. We can add more rss flux, and clear articles/news before send an email to improve your technological survey.

Let's code !

I want to improve my javascript level so I have pick this technology, but you can do the same thing with another techno..

I will not explain a lot my code because you can do what you want, and you will probably understand my code easily !

My gitlab repository.

For my example I have take a Reddit rss like this :

let feed = await parser.parseURL('https://www.reddit.com/r/softwaredevelopment/.rss');

Sending email with nodemailer and handlebars.

Sending email with node.js is really simple with nodemailer !
First, we need to define the transport for nodemailer, in my case I have use gmail, but you can take mailgun, or whatever, but gmail is easy to set up.

var transporter = nodemailer.createTransport({
  service: 'gmail',
  auth: {
    user: '[e-mail]',
    pass: process.env.MAIL_PASS
  }
});

Next, we have to feed our HTML with our news previously collected on the rss flux.

For that, I have used Handlebars. Handlebars is a simple templating language. It uses a template and an input object to generate HTML or other text formats.

It's working like this :

 readHTMLFile(__dirname + '/email.html', function (err, html) {
    var template = handlebars.compile(html);
    var replacements = {
      posts: posts
    };
    var htmlToSend = template(replacements);

In the htmlToSend variable we have the template with our news. (My repository have an example of an email template).

Running the srcipt as a cron

In my case, I have choose GitLab CI/CD.
GitLab CI/CD is a tool built into GitLab for software development through the continuous methodologies:

• Continuous Integration (CI)
• Continuous Delivery (CD)
• Continuous Deployment (CD)

It's perfect when you need to run a scrips as a cron, it's easy to use.
So, first step, we need to add a .gitlab-ci.yml file in our repository.

default:
  image: node:latest

stages:
  - run

cron run:
  stage: run
  script:
    - npm i
    - node cron.js

The first line is facultive, in fact if we remove this, Gitlab will take the default image for your docker container, but I have decided to take a node image (more coherent with my project).

Second step, we should whrite a stage and a job. In the job we should set the list of actions to setup the project. For a node project, we need to install packages with npm i and run the cron node cron.js.

After that, if you push you will see in CI/CD settings of your project that a pipeline as been created.

Final step, go to your schedules settings in CI/CD and set up your cron, with the target branch, timezone, recurrence ..

And it's done !

I hope you have enjoyed my article, I did it with pleasure ❤️.