DEV Community: reymon359

Managing the infrastructure of a reusable ecommerce platform with Terraform

reymon359 — Thu, 03 Jun 2021 06:18:51 +0000

Setting up and maintaining the infrastructure for a reusable ecommerce platform with a microservice architecture is not an easy task. You have to provide every service with resources, keep track of them and update them when needed.

Our team uses Azure as our cloud provider to manage all those resources. Every service uses different resources related to the business logic they handle. We use resources like Azure Service Bus to handle the asynchronous communication between them and Azure Key Vault to store the secrets and environment variables.

In the ones where we need a persistence layer, we rely on the resources Azure Cosmos DB or Azure Database for PostgreSQL. Other services provide an API to search among a catalog of products with Azure Cognitive Search. As I will explain later, we work with different environments, therefore, creating and updating the resources across them becomes a harder task.

If you want to understand better the context of the reusable ecommerce platform I mention in this lecture, I recommend you to take a look at this article. Although the architecture has changed a bit, the core concept of the reusable ecommerce platform remains the same, making this article easier to understand.

So, how do you keep track of all the services resources that are being used across them?

This is where Terraform comes in handy.

What is Terraform
- Modules
- Resources
- Terraform Plan
- Terraform Apply
- State
How we use it
- Version Control System
- Terraform Variables
- Environments and CICD
Conclusions

What is Terraform

Terraform logo by HashiCorp

Terraform is an open-source infrastructure as code software tool developed by HashiCorp. It provides a consistent CLI workflow to manage hundreds of cloud services in configuration files. Here is the official guide on how to install it.

Terraform is a tool that helps you manage various cloud infrastructure services coding configuration files where you can configure the ones needed. It has support for the main cloud providers out there like AWS, Azure, or GCP.

I will enumerate the main features it provides. There are plenty more, however, these are the minimum ones required to understand the tool and this article.

Modules

Terraform provides modules that help us reuse our Terraform code. Our complex infrastructure is broken into multiple modules making it more reusable and simpler to maintain.

It is very easy to convert a given Terraform configuration into modules and Terraform has its eco-system for pre-built modules.

Resources

Resources are the key element in Terraform. Each of them describes a cloud service such as a computed instance, persistence system, or DNS record.

Here is a simple example of how we would define a resource we want to use in our infrastructure:

terraform {
  required_providers {
    food = {
      source  = "hashicorp/food"
      version = "~> 1.0.4"
    }
  }
}

resource "food_tapas" "spanish_omelet" {
  ingredient_1 = "eggs"
  ingredient_2 = "onions"
  Ingredient_3 = "potatoes"
}

The language used in Terraform is the HashiCorp Configuration Language (HCL). In the example above we first specify the cloud provider food. Then define a resource of the type food_tapas with the name spanish_omelet. Inside the resource block, we have the configuration arguments which are the ingredients for this delicious resource.

Terraform Plan

Once all the infrastructure resources have been defined, a Plan is run. Here Terraform checks the code for syntax errors, state verification, API authentication, and then outputs a summary of the changes.

Running terraform plan allows you to check if the plan to be applied for such configuration is the one you were expecting before changing the infrastructure.

Here is an example of the output summary we would get when running terraform plan with the example resource

$ terraform plan
An execution plan has been generated and is shown below.

Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # food_tapas.spanish_omelet will be created
  + resource "food_tapas" "spanish_omelet" {
        + id                = (known after apply)
        + ingredient_1      = "eggs"
        + ingredient_2      = "onions"
        + ingredient_3      = "potatoes"
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Terraform Apply

Once confirmed, the summary of changes are the ones wanted, we execute them with terraform apply.

After running it successfully the console shows this output.

 Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

And that is an overall view of defining and creating your infrastructure resources in an automated way. All through config files removing the manual process.

Terraform resource Spanish omelet from Casa Paco, Madrid

State

The last thing to mention would be the Terraform State. Its purpose is to store the relation between the objects in a remote system and the resource instances declared in your configuration.

When the remote object is created its identity is recorded against a particular resource instance.

Future configuration changes will update the state and make terraform update or delete that object.

How we use it

Going back to our reusable ecommerce platform. Here is how we take advantage of Terraform’s features to improve our workflow.

Version Control System

Having all the infrastructure defined in code files means we can use a VCS (Version Control System). We opt for Git and host it in a GitHub repository. This way we can keep a history control of the resources we have been using.

In addition, the integration with GitHub allows us to work with pull requests in order to add, change or destroy resources from the Infrastructure.

Although the Terraform State is stored locally by default in a file named terraform.tfstate, we store it remotely outside the VCS. This is a good practice when working in a team environment.

Terraform Variables

Being a reusable platform means its infrastructure has to be the same in every shop we create. With terraform, we accomplish this by defining a Terraform input variable called shops_to_apply.

We then pass this variable through the modules and resources where we want to use it. This way when we apply the plan the changes will happen in each of the shops.

Here is an example of how we would define the variable if we had to set up the infrastructure for three different shops.

variable "shops_to_apply" {
  default = {
   happy  = "happyReadings"
   world  = "worldOfWaves"   
   healt  = "healthyFruits"
  }
}

Then in each resource, we use a Terraform for_each meta-argument to iterate them.

resource "azurerm_postgresql_database" "orders-api-service" {
  for_each            = var.shops_to_apply
  name                = "${each.key}-orders-api-service"
  ... other config
}

Environments and CICD

We work with three environments:

Development: A testing environment where all services new features, refactors, and bug fixes get released once merged to the main repository branch.
Staging: A mirror of the Production environment. When the changes have been properly implemented in Development they are promoted to this environment. In this environment, we can see how they will work in a production-similar situation.
Production: The environment the user has access to. The final environment where the changes get promoted from Staging.

Not all the changes are visible at once in each environment. We use ConfigCat as our system of feature flags to make most of the changes visible or not.

We use GitHub Actions and Azure DevOps for our Continuous Integration and Continuous Delivery system. When the changes on the Terraform infrastructure repository are added to the main branch, the CICD system gets triggered and the plan is executed on our development environments. When the logs from the plan are the expected ones we manually run the apply.

Conclusions

Having the infrastructure defined as code has brought us nothing but improvements to our platform. Here is a list of them:

Reduce human error and increase automation. The fewer resources you set up manually, the fewer errors you get in the long term.
It is easier to maintain since all the infrastructure is coded in config files. Plus we are using GitHub repositories and Pull Requests to collaborate all together.
The infrastructure is easier to change compared to traditional manual ways.
All the configuration files work as documentation of the resources we are using.
Thanks to the use of variables we speed up the process of setting up new ecommerce platforms.
Updating and deleting resources is a delicate topic. Having a proper CICD allows us to predict and make changes to the infrastructure in the safest way possible.
We can effortlessly provision our development, staging, and production environments with the same consistent configuration.

7 lessons learned from The Pragmatic Programmer and The Clean Coder

reymon359 — Wed, 13 Jan 2021 10:21:58 +0000

I recently finished reading both books: The Pragmatic Programmer 20th Anniversary Edition (2019) and The Clean Coder (2011) which are found on almost every “top 10 Software Development books” lists out there. My goal was to learn, improve, and get something a teacher from mine used to say: “knowledge that can’t be obtained from just reading articles.”

When you are developing software you may get stuck at a point where Youtube videos and Stackoverflow answers don’t help. You end up reaching for the official docs, or the source code of that technology to find the answer. The same thing happens when you want to understand the subject properly. Articles fall short and reading well-known books is the best approach.

These books focus not only on how to write code but on teaching you best practices for developing software and even useful life lessons. I’ll share some lessons I learned from them.

Taking Responsibility
Testing
Teamwork
Estimating
Tracer bullets
Handling pressure
Refactoring
Main differences
Conclusions
Resources

Taking Responsibility

As a software developer, you are responsible for the code you create and must ensure that it not only works now but will do in the best possible way for a long time (nothing lasts forever). The best way to make sure the code won’t fail is by testing it, having automated tests that you run each time you write new lines to be sure everything is still working.

Take Responsibility. Responsibility is something you actively agree to.
— The Pragmatic Programmer

Responsibility is not only related to coding, you have to take responsibility for improving yourself too and get better by scheduling time for it.

Professionals spend time caring for their profession. Presumably, you became a software developer because you are passionate about software and your desire to be a professional is motivated by that passion.
— The Clean Coder

Your knowledge and experience are your most important day-to-day professional assets.
— The Pragmatic Programmer

Testing

The gravity of testing in software development is such that both books focus on this topic.

You have to look at tests as the first users of your code, so they are the best feedback that guides your coding.

Practice TDD Test Driven Development. How? Three steps explanation:

1 Choose a feature to add and write a test that will pass after implementing it. Now, all tests but the new one should pass.

2 Write the code required to pass it.

3 Refactor the code and check all tests still pass.

Thus said, it is important to look at the big picture and not to miss the main goal by writing too many tests.

It is easy to become seduced by the green "tests passed" message, writing lots of code that doesn’t actually get you closer to a solution.
— The Pragmatic Programmer

There are three ways of testing: First, During, and Never. First (TDD) is the best one, During is a fallback when First is not useful, and Never is often called “Test Later” but sadly in most cases Later means Never.

The need to test first forces you to think about "good design."
— The Clean Coder

Having tests give you the confidence to refactor code more often because you have the tests to check if they still pass after the changes are done.

Tests should be run as frequently as possible to provide maximum feedback and to ensure that the system remains continuously clean.
— The Clean Coder

Use acceptance tests to define when a requirement is done collaborating with the stakeholders.

Developers take the responsibility to ensure that tests are always automated for a simple reason: cost.

Developers have to aim “QA should find nothing” as a goal by implementing different kinds of tests, in different measures from unit to exploratory ones.

Teamwork

When working on a team you have to be a "team player", communicate frequently, keep an eye out for your teammates, and execute your responsibilities as well as possible.

Good communication is key to avoiding these problems. And by "good" we mean instant and frictionless. Frictionless means it’s easy and low-ceremony to ask questions, share your progress, your problems, your insights and learnings, and to stay aware of what your teammates are doing.
— The Pragmatic Programmer

Teams should be small, less than 10-12 members, where everyone knows and trusts each other. This team environment is hard to achieve therefore once you get it you have to care for it changing the projects they work on rather than the members.

As team size grows, communication paths grow at the rate of O(n^2), where n is the number of team members. On larger teams, communication begins to break down and becomes ineffective.
— The Pragmatic Programmer

To form teams around projects is a foolish approach. Individuals are only on the project for a short time and therefore never learn how to deal with each other. Teams are harder to build than projects. Therefore, it is better to form persistent teams that move together from one project to the next and can take on more than one project at a time.
— The Clean Coder

Furthermore, great teams will face and solve issues together where each individual will provide their best. They get things done as a unit. In the end, they will be known for their performance and quality of work.

Quality can come only from the individual contributions of all team members. Quality is built in, not bolted on.
— The Pragmatic Programmer

A gelled team can work miracles, anticipate each other, cover for each other, support each other, and demand the best from each other. They make things happen.
— The Clean Coder

Great project teams have a distinct personality. People look forward to meetings with them, because they know that they’ll see a well-prepared performance that makes everyone feel good. The documentation they produce is crisp, accurate, and consistent.
— The Pragmatic Programmer

Estimating

This lesson, like the majority from both books, is as important in software development as in real life. The more you practice and develop it, the more intuitive will be your ability to determine the feasibility of any task.

First, I want to clarify what estimating means by sharing The Clean Coder definition of an estimate which refers to it as a probability distribution.

An estimate is not a number. An estimate is a probability distribution, the likelihood of completion.
— The Clean Coder

To help you understand it, here is a figure of the likelihood of completion of a task for the next 11 days.

One of the basics to make great estimations is to have proper knowledge of what you are estimating.

The first part of any estimation exercise is building an understanding of what’s being asked. You need to have a grasp of the scope of the domain.
— The Pragmatic Programmer

Don’t estimate alone, communicate with other people to be as accurate as possible.

The most important estimation resource you have are the people around you. They can see things that you don’t. They can help you estimate your tasks more accurately than you can estimate them on your own.
— The Clean Coder

A basic estimating trick that always gives good answers: ask someone who’s already done it.
— The Pragmatic Programmer

When asked for an estimation choose the units that better reflect the accuracy you intend to convey. This Estimation times scale from The Pragmatic Programmer may help you.

Duration	Quote estimate in
1–15 days	Days
3–6 weeks	Weeks
8–20 weeks	Months
20+ weeks	Think hard before giving an estimate

Although Business likes to view estimates as commitments, remember that an estimate is just a guess therefore no commitment is implied.

A commitment is something you must achieve. If you commit to getting something done by a certain date, then you simply have to get it done by that date. Professionals don’t make commitments unless they know they can achieve them. Missing a commitment is an act of dishonesty only slightly less onerous than an overt lie.
— The Clean Coder

Thus said, to help Business measuring requirements and making appropriate plans you have to remove the requirement’s ambiguity before estimating. Afterwards, keep them updated about the progress.

The trick to managing lateness is early detection and transparency. Regularly measure your progress against your goal. Be as honest as you can about all dates. Do not incorporate hope into your estimates!
— The Clean Coder

Do not reinvent the wheel, use well-known estimation techniques for tasks. Here I made a summary of some techniques mentioned in both books.

The more experience you have on a certain project the better you will estimate its tasks. Therefore don’t worry if the first estimates you make are not as accurate as they could be. It is an incremental process as with every long term goal you want to achieve.

Like one of my favorite quotes states:

There is only one way to eat an elephant: a bite at a time.
— Desmond Tutu

However, there is no way you can eat such cuteness

Tracer bullets

Tracer bullets are a special kind of bullet used in movies to mark the path they’ve taken as feedback for the shooter to aim better the next time. Therefore, the main goal of Tracer Bullet Development is to "shoot" new features into the project and get quick feedback to "aim" better on the next ones.

Tracer development is consistent with the idea that a project is never finished: there will always be changes required and functions to add. It is an incremental approach.
— The Pragmatic Programmer

This method helps developers to focus on the main features to be implemented so others can be built on. In addition, it serves as proof the architecture is compatible and feasible by providing a functional and demonstrable skeleton to work from the beginning of the development process.

Look for the important requirements, the ones that define the system. Look for the areas where you have doubts, and where you see the biggest risks. Then prioritize your development so that these are the first areas you code.
— The Pragmatic Programmer

Lastly, Tracer Bullet Development should not be confused with prototyping. The code from prototypes isn’t supposed to be part of the project whereas the code from the tracer bullets isn’t thrown away. It works and is improved each iteration with new features.

Prototyping generates disposable code. Tracer code is lean but complete, and forms part of the skeleton of the final system. Think of prototyping as the reconnaissance and intelligence gathering that takes place before a single tracer bullet is fired.
— The Pragmatic Programmer

Handling pressure

I liked this one as well since it helps you outside software development too. Soon or later you will be under pressure, and the best tricks to handling it are to avoid it when you can, and weather it when you can’t.

The best way to stay calm under pressure is to avoid the situations that cause pressure.
— The Clean Coder

You mainly avoid it by managing commitments, keeping clean, and following your disciplines.

The best way to manage commitments is by saying no to those which deadlines you aren’t sure to meet. Keeping clean basically means to have no mess on your systems, code, and design.

The way to go fast, and to keep the deadlines at bay, is to stay clean. Professionals do not succumb to the temptation to create a mess in order to move quickly. “Quick and dirty” is an oxymoron. Dirty always means slow!
— The Clean Coder

Follow disciplines you truly believe in and stick to them all the time no matter the situation. Crisis times will come, and it’s when you have to pay attention to how you behave because if you follow your disciplines means you believe in them. Changing your behavior and not following them would mean you don’t truly believe in your normal behavior, and you have to shift those disciplines to improve it.

If you keep your code clean during normal times but make messes in a crisis, then you don’t really believe that messes slow you down. If you pair in a crisis but don’t normally pair, then you believe pairing is more efficient than non-pairing.
— The Clean Coder

Choose disciplines that you feel comfortable following in a crisis. Then follow them all the time. Following these disciplines is the best way to avoid getting into a crisis. Don’t change your behavior when the crunch comes. If your disciplines are the best way to work, then they should be followed even in the depths of a crisis.

However, you can’t avoid pressure always so you have to learn how to get through it. You weather it by staying calm, communicating, following your disciplines, and getting help.

To stay calm don’t panic, manage your stress, and think the problem through to find the best possible outcome. Then go for it at a steady pace, eating an elephant way- communicating all the time with your team and superiors to let them know when you are in trouble to get input and guidance. This way there won’t be any unexpected surprises in the end.

Avoid creating surprises. Nothing makes people more angry and less rational than surprises. Surprises multiply the pressure by ten.
— The Clean Coder

In the same way you relied on your disciplines to avoid pressure, you should also rely on them when the moment gets tough. In fact, these times you have to pay special attention to them and neither question nor abandon them.

The communication tip includes asking for help from teammates to pair, superiors, or internet sites and forums.

Don't forget to be there for others too when they are under pressure and need help.

When the heat is on, find an associate who is willing to pair program with you. You will get done faster, with fewer defects. Your pair partner will help you hold on to your disciplines and keep you from panicking.
— The Clean Coder

Refactoring

The term Refactoring is defined by Martin Fowler as a:

Disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.
— Martin Fowler

Sometimes you will find code that does not feel right and should be fixed or improved, and you have to keep in mind the best moment to do so is now, when you find it.

It is inevitable, a program’s code needs to grow, evolve, and improve. To do so, some decisions must be rethought, and the code has to be changed therefore be sure to have it covered with automated tests to guarantee that the external behavior does not change.

Rather than construction, software is more like gardening—it is more organic than concrete. You plant many things in a garden according to an initial plan and conditions. You constantly monitor the health of the garden, and make adjustments as needed.
— The Pragmatic Programmer

When should you refactor the code? Here is a list of situations that qualify to do so:

Remove code duplication.
Making some parts of the code more orthogonal.
Update code and/or documentation outdated.
To improve performance.

And here are Martin Fowler tips on how to refactor without doing more harm than good:

Don’t try to refactor and add functionality at the same time.
Make sure you have good tests before you begin refactoring. Run the tests as often as possible.
Take short, deliberate steps: move a field from one class to another, split a method, rename a variable. Refactoring often involves making many localized changes that result in a larger-scale change.

The most important thing to keep in mind is that refactoring is not a certain task, it is a habit that, as with most things in life, is easier to do while the issues are small, as an ongoing activity while coding. The less you refactor now the greater time you’ll have to invest to fix the problem down the road.

Refactoring as "a growth." Removing it requires invasive surgery. You can take it out while it is still small. Or, you could wait while it grows and spreads—but removing it then will be both more expensive and more dangerous. Wait even longer, and you may lose the patient entirely.
— The Pragmatic Programmer

Main differences

I did not want this lecture to be a comparison but although they focus on similar subjects, the content, and the way they are narrated are not the same. Here are the main impressions I got from them not related to the content itself that can help you have an idea of what to expect while reading them.

The Clean Coder speaks about the developer in a more day by day at work way, with references to common situations that are given in such an environment like the relationship with sales or business people, working in a team, or saying no to clients. The developer at the Pragmatic Programmer isn’t that much pictured into job situations, rather it grants an overview of the field structuring the book on advice: topics and tips, for any situation.
The Clean Coder refers to the figure of the Software Developer as a Professional Programmer while The Pragmatic Programmer uses the totally unexpected term Pragmatic Programmer.
In general terms, The Clean Coder has a more subjective sense since it tells more personal experiences from the author whereas The Pragmatic Programmer feels more objective focusing mostly on the advice themselves.
The Pragmatic Programmer contains more code examples in different programming languages than the Clean Coder which helps you understand the concepts discussed.

Conclusions

Keep in mind that these are just the personal impressions and lessons I got from reading the two titles. The books have much more to offer, and the best thing you can do is read both yourself to form your own opinion and conclusions.

If you are interested in software development, want to get better but think they are too similar and don’t have time to read both, think twice, each will bring you different knowledge and best practices for your career.

Having said that, if you still want to go the lazy way, I made this open-source project with the core sentences from them.

Resources

The main resources are both books which you can find easily on the internet and the project where I noted down the sentences I found most important from them.

Thank you for reading! You can read it too on my site along with others!

Using GitHub Actions and Renovate App to safely update dependencies automatically

reymon359 — Mon, 21 Dec 2020 16:27:51 +0000

In Software Development keeping up to date with technology updates is crucial not only for developers who have to be always learning and renewing their set of skills but also for the projects they work on and have to be maintained.

When you start a project from the ground you normally set it up with the libraries' latest stable versions. Then time goes by, and the size of the project grows, new features are added, new libraries too, and the version of the libraries and packages usually remains the same and almost never gets updated. And yes, why would you update them if the project works perfectly with the current ones?

Here are some reasons why you should keep your dependencies updated:

Solving problems from old versions.
Adding vulnerability fixes.
Increase the overall performance.
Add new features from them.
...

When you keep the dependencies updated you are solving problems from older versions, improving the performance with new optimizations, and you are also able to use new features that other developers added to them. All of these contribute to increasing the maintainability of the code, and the overall project.

We all have worked on projects where the dependencies have never (or rarely) been updated.

So how do we do it?

Well, first you can run npm outdated to see the latest releases of the packages you are currently using.

You can then run npm update to update them (it will not update them to the major versions) but how do you know which ones will make the project break and which ones won't?

Then there is the when, when should you update them? When should you check for updates? every day? every week? ...month?

This is why I made this project, to learn about GitHub Actions and use it to have a safe way to automatically update the dependencies without making the project fail.

The short explanation of this tutorial is about adding the Renovate app to check for dependencies updates and then submit Pull Requests to update them, this way we will abstract ourselves on checking for updates, so we can focus on more important things.

The point of using GitHub Actions is to set up a workflow and trigger it with every Pull Request to check that the build and tests pass with the updated dependencies before adding them to the project.

Getting Started
Set up GitHub Actions Workflow
Add Renovate
Conclusion
Useful Resources

Getting started

Although this approach can be applied to any project we will use a React project made with Create React App as it will give us a basic project with everything ready to work on. By the way, if you do not have Node.js installed here is the link to do so.

Also, in this repository is the resulting project you will obtain after following all the steps below in case you want to have a look before starting.

So let's begin running

npx create-react-app my-app
cd my-app
npm start

If you use npm 5.1 or earlier, you can't use npx. Instead, install create-react-app globally:

npm install -g create-react-app

And then run:

create-react-app my-app

Set up Github Actions Workflow

Now we will proceed to define a GitHub Actions Workflow in our repository to automate the process.

GitHub Actions is a Github Feature that helps your software development workflows automating from simple tasks to custom end-to-end continuous integration (CI) and continuous deployment (CD) capabilities in your repository.

In our root folder, we will create a new folder and name it .github and inside of it a workflows one. This is how your project should look after these steps:

📁 my-app
├── 📁 .github
│   └── 📁 workflows
├── ...
...

Here is where we will create and add our Workflows. Github Actions Workflows are the Continuous Integration automated processes we want to run in our project.

Workflows are composed of jobs that contain a set of steps and to explain them in a clearer way let's create our own workflow explaining it step by step.

In the .github/workflows directory, add a .yml or .yaml file and name it main.yml. I chose that name to keep things simple, but you can give it any other name like build-test.yml or continuous-integration-workflow.yml.

📁 my-app
├── 📁 .github
│   └── 📁 workflows
│       └── 📄 main.yml
├── ...
...

Here is how the workflow will look in the end in case you just want to copy it and add it directly before the explanation.

name: Build and Test

on:
  push:
    branches: [master]
  pull_request:
    branches: [master]

jobs:
  build_and_test:
    runs-on: ubuntu-latest

    strategy:
      matrix:
        node: [10, 12]

    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v1
        with:
          node-version: ${{ matrix.node-version }}
      - name: Install project
        run: npm install
      - name: Build the project
        run: npm run build --if-present
      - name: Run tests
        run: npm test

The first param of our workflow will be its name.

name: Build and Test

The second param is the trigger.

We can choose if the workflow is triggered by an event like a push or pull request to a specific branch, or we can even schedule a cron to automatically trigger it every defined amount of time!.

In our project we will want to trigger it when pushing to the master branch and when the Renovate app submits a Pull Request to update a dependence:

on:
  push:
    branches: [master]
  pull_request:
    branches: [master]

Next, we define the jobs.

In this example, there will only be one job which will be the one to build and test the project, and chose the virtual machine where the job will be run on.

jobs:
  build_and_test:
    runs-on: ubuntu-latest

Then it comes to the matrix where we will configure the combination of versions and systems we want to run out Workflow. In our case, we will run it on Node.js 10 and 12.

    strategy:
      matrix:
        node-version: [10, 12]

Finally, the Workflow's steps. The first one is the checkout action which is a standard action that you must include in your workflow when you need a copy of your repository to run the workflow.

Then you can run other actions and processes, in our app, we will use the setup-node action with the matrix we defined before. Then we will add steps to install the project, build it, and run the tests.

    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v1
        with:
          node-version: ${{ matrix.node-version }}
      - name: Install project
        run: npm install
      - name: Build the project
        run: npm run build --if-present
      - name: Run tests
        run: npm test

Now Create a GitHub Repository for the project, commit the local changes made, and push them to it.

Quick tip, If you want to create it faster go to repo.new or github.new. You can use gist.new for gists too!

Once you push your changes the Workflow will run, and you will be able to see how it went in the Actions tab from the GitHub Project.

Add Renovate

Renovate is a free, open-source, customizable app to automate the update of our dependencies in software projects by receiving pull requests. It is used by software companies like Google, Mozilla, or Uber and you can use it on GitHub, Gitlab, Bitbucket, Azure DevOps, and Gitea.

Going to the point what we will do is add a bot that will submit pull requests to our repository when there are updates in our project dependencies.

The cool thing, and the whole point of our project, is that we have previously defined in our workflow to run the tests with the pull requests so when renovate submits one we will automatically check if the updates proposed will break the project or not before merging them to the master branch.

To add Renovate to our project we have to install its app into the project's repository. Be careful when selecting the repository you want to add Renovate to and choose the one created before. If by mistake you want to reconfigure it you can do it in the Personal Settings' Applications tab from your account.

After some minutes we will have to accept and merge the onboarding Pull Request that we will receive.

Once we have it integrated we will configure it updating the renovate.json file on the project root. Remember to pull the changes after merging the Pull Request for it to appear.

We can use the default configuration where renovate will submit the pull requests whenever it finds out about updates and wait for us to merge them:

{
  "extends": ["config:base"]
}

Or we can adapt it to the requirements of our project like the one used by Renovate itself. To avoid falling short and to learn a little more about the tool, we will use a configuration with some of its most useful features. If you want to learn more about its configuration here are the docs for it.

This will be our renovate.json file. Have a look at it and I will explain it after.

{
  "extends": [
    "config:base"
  ],
  "packageRules": [
    {
      "updateTypes": [
        "minor",
        "patch"
      ],
      "automerge": true
    }
  ],
  "timezone": "Europe/Madrid",
  "schedule": [
    "after 10pm every weekday",
    "before 5am every weekday",
    "every weekend"
  ]
}

In the first part, we are telling renovate that our configuration will be an extension from the default one.

{
  "extends": [
    "config:base"
  ],

Then we have the packageRules. After some months using it I realized that going (from time to time) through the pull requests checking and accepting them if the tests passed was a major waste of time. This is why the automerge is set to true, so Renovate automatically merges the pull request if the workflow passed successfully.

To restrict Renovate's freedom a bit, we define that he only can perform the automerge when it is a minor or patch update. This way, if it is a major or another kind of update, we will be the ones to check whether that update should be added or not.

Here you can find more information about the update types available.

  "packageRules": [
    {
      "updateTypes": [
        "minor",
        "patch"
      ],
      "automerge": true
    }
  ],

Lastly, we have the time schedule. If you work alone or in a team at certain hours it is nice to have the updates done at times when you are not working to avoid unnecessary distractions.

We select our timezone and add a custom schedule for it. You can find the valid timezone names here.

  "timezone": "Europe/Madrid",
  "schedule": [
    "after 10pm every weekday",
    "before 5am every weekday",
    "every weekend"
  ],

Anyway, if you do not care about the time the pull requests will be submitted, or the people that contribute to the code are in different timezones then you can remove this part.

Once we have updated the configuration we push the changes to GitHub to have the Renovate app adapted to the new configuration.

Now you finally have the project dependencies safely up-to-date without you having to check for them. Here is the resulting project after following all the steps mentioned above.

Remember that if you added the time schedule part you will not get the pull request merged automatically until it complies with that configuration.

Conclusion

Although there are other ways to keep the dependencies updated in an automated way if you use GitHub to host your code you should take advantage and make the most of its awesome free features.

If you are wondering what else you are able to do and automate with the GitHub apps and actions just have a look at its Marketplace.

In addition, you can have a look at a project I made where I am working on from time to time and on which I based myself for this tutorial. It is a bit more complex and has more features than the one from this tutorial.

I hope you enjoyed the tutorial and learned about GitHub Actions and its Apps. If you've got any questions, suggestions, or feedback in general, don't hesitate to reach out on any of the social networks from my site or by mail.

Useful Resources

Here is a collection of links and resources which I think can be useful to improve and learn more about GitHub Actions and Apps.

Tutorial project. - The resulting project from this tutorial.
GitHub Marketplace. - The place to find all GitHub Actions and Apps.
GitHub Actions Workflow Configuration - The full documentation on how to set up a workflow on Github Actions.
Renovate GitHub app - The Renovate App main page on the GitHub Marketplace.
GitHub Actions project Workflow. - The Workflow used in this tutorial.
Renovate App's configuration file. - Renovate App's custom configuration file from the tutorial.
Up to Date React Template. - A Personal project that uses the approach described in this tutorial.

Thank you for reading! You can read it too on my site along with others!

Getting started with GraphQL and Node.js

reymon359 — Wed, 25 Nov 2020 10:35:53 +0000

The main purpose of this server-client Node.js project is to help other people understand how GraphQL exposes the data from the Server and the Client fetches it. I have tried to make it as simple as possible- if you want to dive into the code of the project you can find it here.

Now, straight to the point: GraphQL is a query language for APIs developed and open-sourced by Facebook to speed up the request process.

While REST has been a popular way to expose data from a server, instead of having multiple endpoints that return fixed data structures, GraphQL just has a single endpoint and it is the client's job to specify what data it needs from it.

Getting started
Define the Schema
Add Resolver function
Set up the Server
Set up the Client
Fetching data from the server
Displaying the data
Conclusion
Useful resources
- Docs 📚
- Learn 📝
- Tools 🔧
- IDEs 💻
- Extras 🍍

Getting started

The first step is to download and install Node.js in case you haven't already. Once you have it installed let's begin with the directory structure. The project will be composed of two directories, one for the Client and another for the Server. I have chosen to keep both inside the project root directory but then you can split it into two separate projects or any way you want.

📁 project
├── 📁 client
└── 📁 server

Now we will initialize the project in the server directory. Change the location to the server folder in your terminal and run npm init to fill in the project info and generate the package.json file. Or npm init -y which tells the generator to use the defaults (instead of asking questions and simply generate an empty npm project without going through an interactive process).

The next step will be to install GraphQL.js and Apollo Server to our server. GraphQL.js will provide two important capabilities:

Building a type schema, which we will do in the next step.
Serving queries against that type schema.

To install it just run npm install graphql. I am assuming you are using a version of NPM equal or higher than 5.0.0 so you do not need to add --save when installing a dependency to be saved in the package.json.

Apollo Server, on the other hand, will help us to implement the GraphQL functionalities. It is part of the Apollo Data Graph Platform.

Apollo is a platform for building a data graph, a communication layer that seamlessly connects your application clients (such as React and iOS apps) to your back-end services. Is an implementation of GraphQL designed for the needs of product engineering teams building modern, data-driven applications. - Apollo Documentation

What you need to know about Apollo, at least for now, is it’s a community that builds on top of GraphQL and provides different tools to help you build your projects. The tools provided by Apollo are mainly 2: Client and Server.

Apollo Client helps your Frontend communicate with a GraphQL API. It has support for the most popular frameworks such as React, Vue, or Angular and native development on iOS and Android.
Apollo Server is the GraphQL server layer in your backend that delivers the responses back to the client requests.

Now that you understand Apollo better and why we will use it, let's continue setting up GraphQL.

Define the Schema

A GraphQL Schema is at the core of any GraphQL server implementation. It describes the shape of your data, defining it with a hierarchy of types with fields that are populated from your data source and specifies which queries and mutations are available, so the client knows about the information that can be requested or sent.

For example, if we wanted to build a music application, our simplest schema, usually defined in a schema.graphql file, would contain two Object types: Song and Author like this

type Song {
  title: String
  author: Author
}

type Author {
  name: String
  songs: [Song]
}

Then we would have a Query type to define the available queries: getSongs and getAuthors, each returning a list of the corresponding type.

type Query {
  getSongs: [Song]
  getAuthors: [Author]
}

To keep it as simple as possible our schema will have just a single Query type which will return a String.

type Query {
  greeting: String
}

We can use any programming language to create a GraphQL schema and build an interface around it but as I explained before we will use Apollo server to execute GraphQL queries. So we create a new server.js file in the server directory to define the Schema on it.

📁 project
├── 📁 client
└── 📁 server
    └── 📄 server.js

Now we install apollo-server running npm install apollo-server

We have to import the tag function gql from apollo-server to parse the schema this way: const {gql} = require('apollo-server'); and then declare a typeDefs constant which is an abstract syntax tree of the Graphql code.

When a GraphQL server receives a query to process it generally comes in as a String. This string must be tokenized and parsed into a representation that the machine understands. This representation is called an abstract syntax tree.

If you want to learn more about abstract syntax trees AST Explorer is an online tool that lets you explore the syntax tree created by a chosen language as a parser.

The server.js file would look like this.

const { gql } = require('apollo-server');

const typeDefs = gql`
  type Query {
    greeting: String
  }
`;

Add Resolver function

Now we have defined our Schema we need a way to answer the client requests for that data; the resolvers.
A resolver is a function that handles the data for each one of the fields of your schema. You can send that data to the client by fetching a back-end database or a third-party API among others.

They have to match the type definitions of the Schema, in our case, we just have one type definition Query which returns a greeting of type String, so we will define a resolver for the greeting field, like so:

const resolvers = {
  Query: {
    greeting: () => 'Hello GraphQL world!👋',
  },
};

As I explained at the beginning we will keep this example as simple as possible but keep in mind that in a real case here is where you have to make the queries to the database, external API, or from wherever which you intend to extract the query data.

Set up the Server

In the same server.js, we define and create a new ApolloServer object passing the Schema (typeDefs) and resolvers as parameters.

const { ApolloServer, gql } = require('apollo-server');

const server = new ApolloServer({ typeDefs, resolvers });

Then calling the listen method we start the server on the port that we specify in the params.

server
  .listen({ port: 9000 })
  .then(serverInfo => console.log(`Server running at ${serverInfo.url}`));

We can also destructure the ServerInfo url when logging it.

server
  .listen({ port: 9000 })
  .then(({ url }) => console.log(`Server running at ${url}`));

The server.js file should look like this right now.

const { ApolloServer, gql } = require('apollo-server');

const typeDefs = gql`
  type Query {
    greeting: String
  }
`;

const resolvers = {
  Query: {
    greeting: () => 'Hello GraphQL world!👋',
  },
};

const server = new ApolloServer({ typeDefs, resolvers });
server
  .listen({ port: 9000 })
  .then(({ url }) => console.log(`Server running at ${url}`));

Now if we run node server/server.js we will finally have our GraphQL server up and running🎉!

You can go and check it on http://localhost:9000/

~/graphql-hello-world-server
> node server/server.js
Server running at http://localhost:9000/

For the first time using GraphQL, you may be thinking what is this application I am seeing in front of me if we have not written a single line of client code?.

The answer to that question is the GraphQL Playground.

GraphQL Playground is a graphical, interactive, in-browser GraphQL IDE, created by Prisma and based on GraphiQL. - Apollo docs

But what does that mean? It means that this is an environment where we can perform Queries, Mutations, or Subscriptions to our schema and interact with its data.

If you have worked with RESTful requests before this would be some kind of equivalent to Postman, just that here you do not have to download and configure anything, it just comes by default with Apollo!

So let's try it!

On the left panel write the greeting query we defined in our schema.
Then press the ▶ button that is in the middle.
And Voila! On the right panel appears the data we defined in our resolver to return.

Set up the Client

Now we have our server up and running let's focus on the client part. We will start creating a client.html file inside our client folder.

📁 project
├── 📁 client
|   └── 📄 client.html
└── 📁 server
    └── 📄 server.js

The index.html file will have the basics of any HTML file and a loading header <h1>Loading...</h1> to show the user something while we request the data from the server.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Hello World GraphQL Client</title>
  </head>

  <body>
    <h1>Loading...</h1>

    <script src="app.js"></script>
  </body>
</html>

Fetching data from the server

First off, in the same client folder, we create an app.js file where we will write the client logic to fetch the data from the server.

📁 project
├── 📁 client
|   └── 📄 client.html
|   └── 📄 app.js
└── 📁 server
    └── 📄 server.js

Inside it, we set the server URL to which we will make the request.

const GRAPHQL_URL = 'http://localhost:9000/';

Next, we define our async function fetchGreeting() to, fetch the greeting from the server. We will use the fetch API to perform the HTTP request which by default returns a promise to which we can subscribe and get the answer asynchronously.

async function fetchGreeting() {
  const response = await fetch(GRAPHQL_URL, {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
    },
    body: JSON.stringify({
      query: `
        query {
          greeting
        }
      `,
    }),
  });

  const responseBody = await response.json();
  console.log(responseBody);
}

A detail to take into account is that the method of the request is POST, which can confuse us if we are used to working with RESTful because this same request made in RESTful, where we just want to read information from the server, would be usually done with the method GET.

The thing is that with GraphQL we always make POST requests where we pass the query in the payload (body).

Finally, we just call our method fetchGreeting();

const GRAPHQL_URL = 'http://localhost:9000/';

async function fetchGreeting() {
  const response = await fetch(GRAPHQL_URL, {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
    },
    body: JSON.stringify({
      query: `
        query {
          greeting
        }
      `,
    }),
  });

  const responseBody = await response.json();
  console.log(responseBody);
}

fetchGreeting();

If you open the file in your browser and see the console on the developer tools you can see that we actually got the greeting data from the query 🙌!

Displaying the data

Now we have successfully got the data from the server lets update the loading title. The first thing we will do is destructure the response and return just the data from it.

Just replace this part of the code

const responseBody = await response.json();
console.log(responseBody);

With this one

const { data } = await response.json();
return data;

Then we will update the title with the greeting returned inside the data from the response

fetchGreeting().then(({ greeting }) => {
  const title = document.querySelector('h1');
  title.textContent = greeting;
});

So our app.js file will end up having this look

const GRAPHQL_URL = 'http://localhost:9000/';

async function fetchGreeting() {
  const response = await fetch(GRAPHQL_URL, {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
    },
    body: JSON.stringify({
      query: `
        query {
          greeting
        }
      `,
    }),
  });

  const { data } = await response.json();
  return data;
}

fetchGreeting().then(({ greeting }) => {
  const title = document.querySelector('h1');
  title.textContent = greeting;
});

Our index.html will have the loading title updated with the data fetched from our server!🎉

Conclusion

I hope you enjoyed the post and that this project has fulfilled the objective of showing how graphql works under the hood, at least in a very simple way. I know there are a lot of things that I have not explained or that I could have gone deeper but like any hello world project, it is intended for people who are beginners with it, so I wanted to keep it as simple as possible.

I'm looking forward to learning more about GraphQL and using it in future projects. If you've got any questions, suggestions, or feedback in general, don't hesitate to reach out on any of the social networks from my site or by mail.

Useful resources

Here is a collection of links and resources which have been useful for me to improve and learn more about GraphQL

Docs 📚

Project source code - The Github repository with all the code from the project.
GraphQL main site - GraphQL main site.
Apollo documentation - The Apollo platform docs.

Learn 📝

How to GraphQL - Free and open-source tutorials to learn all around GraphQL to go from zero to production.
GraphQL by Example - Great course where you learn GraphQL by writing full-stack JavaScript applications with Node.js, Express, Apollo Server, React, Apollo Client.
Introduction to GraphQL - A series of articles to learn about GraphQL, how it works, and how to use it.

Tools 🔧

Apollo GraphQL - Main site of the Apollo GraphQL implementation.
GraphQL Playground - Repository of the GraphQL Playground IDE we used on the project.

IDEs 💻

JS GraphQL - WebStorm and other IntelliJ-based IDEs plugin to support GraphQL language including tagged template literals in JavaScript and TypeScript.
GraphQL - GraphQL extension for VSCode adds syntax highlighting, validation, and language features like go to definition, hover information, and autocompletion for GraphQL projects. This extension also works with queries annotated with gql tag.
GraphQL for VSCode - VSCode GraphQL syntax highlighting, linting, auto-complete, and more!

Extras 🍍

GraphQL APIs - A list of public GraphQL APIs to test your skills or to build something with them.
GraphQL: The Documentary - A 30 min video that explores the story of why and how GraphQL appeared, and the impact it's having on big tech companies worldwide, including Facebook, Twitter, Airbnb, and Github.

Thank you for reading! You can read it too on my site along with others!

DEV Community: reymon359

Managing the infrastructure of a reusable ecommerce platform with Terraform

What is Terraform

Modules

Resources

Terraform Plan

Terraform Apply

State

How we use it

Version Control System

Terraform Variables

Environments and CICD

Conclusions

7 lessons learned from The Pragmatic Programmer and The Clean Coder

Table of Contents

Taking Responsibility

Testing

Teamwork

Estimating

Tracer bullets

Handling pressure

Refactoring

Main differences

Conclusions

Resources

Using GitHub Actions and Renovate App to safely update dependencies automatically

Table of Contents

Getting started

Set up Github Actions Workflow

Add Renovate

Conclusion

Useful Resources

Getting started with GraphQL and Node.js

Table of Contents

Getting started

Define the Schema

Add Resolver function

Set up the Server

Set up the Client

Fetching data from the server

Displaying the data

Conclusion

Useful resources

Docs 📚

Learn 📝

Tools 🔧

IDEs 💻

Extras 🍍