DEV Community: Jochen Rui

Build a Full Stack Translation Web App Using React JS, ExpressJS, TypeScript (Part 1) | Free Video

Jochen Rui — Fri, 13 Jan 2023 16:31:23 +0000

In this video series, you will learn how to program a full-stack application consisting of a ReactJS frontend and an ExpressJS backend, as well as how to set up Docker and Docker Compose. We will be using the DeepL API that's freely available for this.

Part 1

This first part will be setting up the Dockerfile, Docker-Compose, and env variables, as well as getting the DeepL API Key we will be needing later on.

For who is this Video Series?

Feel free to follow along. This is specially targeted towards developers who are new to programming or people who wish to refresh their knowledge of the used technologies. Part of the series is explaining little things along the way that may prove quite useful.

Table of Content

The content for this part is as follows:

00:00 - Intro
00:50 - setup Dockerfile
02:16 - setup Docker-Compose
04:43 - setup ENV
05:00 - Get API Key

If you have any questions feel free to let me know here or inside the video

Automatically scan your Project Dependencies for Vulnerabilities using Docker, Jenkins, ... (Part 2/2)

Jochen Rui — Wed, 12 Jan 2022 12:32:42 +0000

Recap of Part 1

thanks for joining once again. In the last post I introduced the OWASP Dependency-Check tool briefly and discussed it's increasing helpfulness in today's Software Engineering world due to increasing integration of open source & third party code.

Summary

Today we will take a look at how we can easily integrate the Dependency-Check into our existing CI/CD Pipeline or how we can create a standalone Dockerfile to run the Dependency-Check on demand.

Tools / Requirements

As for knowledge it would be nice to have basic knowledge of how to run Dockerfiles but other than that there's not much required.

As for tools we will use:
Docker

Setup

For the Setup we'll start with the Dockerfile and then discuss how we can run it standalone or integrate it in an existing CI/CD pipeline.

Dockerfile

####################
# FRONTEND DEPENDENCIES
####################

# we install the frontend dependencies to create a node_modules directory
FROM node:14 as frontend_dependencies
COPY path/to/package.json /frontend
WORKDIR /frontend
RUN npm i && npm ci

####################
# INSTALL DEPENDENCY CHECK
####################

# we download the dependency-check zip & unzip it
FROM ubuntu:14.04 as dependency_check_install
RUN apt-get update \
    && apt-get install -y wget unzip \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

RUN wget https://github.com/jeremylong/DependencyCheck/releases/download/v6.5.2/dependency-check-6.5.2-release.zip \
    && unzip dependency-check-6.5.2-release.zip \
    && rm dependency-check-6.5.2-release.zip

####################
# RUN DEPENDENCY CHECK
####################

# we require a Java image here to run the dependency-check
FROM openjdk:18-alpine as dependency_check

# add yarn for dependency check to run without errors
RUN apk add yarn

# copy dependency-check from the previous stage
COPY --from=dependency_check_install /dependency-check /dependency-check

# copy frontend node modules from the previous stage
COPY --from=frontend_dependencies /frontend/node_modules /frontend/node_modules


# copy backend python files to be checked
COPY path/to/pythoncode/src /backend

# this line is optional. It's only necessary if you wish to run the Dependency-Check standalone
RUN /dependency-check/bin/dependency-check.sh --scan . --out ./report

Dockerfile explanation

As for JS code we need to initialize the node_modules from the package.json first

As for the Python Backend code we need to copy the .py files over into the Docker container and use the experimental mode for Python.

By separating these steps into multiple Docker Stages we minimize the final Docker image size by excluding things such as node, wget and unzip.

Run Dependency-Check

The last line in the Dockerfile that's commented as Optional runs the Dependency-Check. If we want to run the Dependency-Check on demand outside a Pipeline we have to include this line here. It will scan all files in the directory and generate a report as HTML file.

We can also specify another Output format such as JSON, CSV, ... This may be helpful if we have an automatic Analyser in place who can process JSON or CSV files.

Jenkinsfile

If we want to include this Scan in a CI/CD Pipeline we would do it as follows in a Jenkinsfile example:

stage('Vulnerability Check') {
          stages {
            stage('OWASP dependency-check') {
              agent {
                dockerfile {
                  filename 'Dockerfile'
                }
              }
              steps {
                sh '/dependency-check/bin/dependency-check.sh --out ./report --scan /app --format HTML --format JSON --prettyPrint --enableExperimental --disableAssembly'
              }
              post {
                always {
                  archiveArtifacts "report/dependency-check-report.json"
                  archiveArtifacts "report/dependency-check-report.html"
                }
              }
            }
          }
        }

Jenkinsfile explanation

We specify the Dockerfile to be run. Don't forget to remove the last optional line in the Dockerfile if you use this approach

Then we specify the command to be executed. The flags --format specify the outut format. The --enableExperimental flag enables python scanning.

Finally we archive the results, so that we can view them later on in Jenkins.

Output

the generated .html will list the vulnerabilities and general information about the scan

Thank you for reading so far! :)

That's it for the little guide. If you have questions or anything is unclear feel free to leave a comment.

Also if you like this article or find it interesting be sure to leave a like or share it with people who might be interested in the topic :) It would help me a lot.

Automatically scan your Project Dependencies for Vulnerabilities using Docker, Jenkins, ... (Part 1/2)

Jochen Rui — Tue, 11 Jan 2022 10:10:26 +0000

What's a Dependency Check & what's the purpose?

I think we can all agree that nowadays we heavily rely on open source and 3rd party libraries & frameworks. By doing so we leverage the knowledge of other professionals around the world and increase the time to market by not having to write everything from scratch by ourselves. In addition we don't have to maintain these supporting libraries ourselves which again saves us a lot of time and effort.

While it sounds amazing to use publicly available code and it certainly does bring a ton of benefits, we have to take it with a tiny grain of salt. Reason being there's an inherent problem in using publicly available code: existing vulnerabilities in that code will also be included in our code. By doing so we inject their vulnerabilities into our application as well. While usually open-source projects handle their vulnerabilities well and fix them in a timely manner, these vulnerabilities are most often made publicly available for everyone to read about before they are fixed. Meaning that exploiters have the knowledge about these vulnerabilities as well and can use them to penetrate our application's security. In most cases the vulnerabilities are neglectable, but if they are severe risks it becomes crucial to handle them quickly.

OWASP Dependency-Check

The Open Web Application Security Project (OWASP) is a non-profit organisation working on improving the security of software (https://owasp.org/about/).

The Dependency-Check is one of their projects. It's a Software Composition Analysis (SCA) tool that scans projects for publicly disclosed vulnerabilities.

What files / languages can be scanned for vulnerabilities?

Dependency-Check can scan a multitude of file types & languages:

Node.js, Python, Ruby Gemspec, Swift, OpenSSL, Nuspec, Nexus, Jar, CocoaPods, CMake, Central, Autoconf, Assembly and Archive

How does the Dependency-Check work?

It scans the project for "evidence" which may lead to the identification of Dependencies using the Common Platform Enumeration (CPE). It's basically acts as a mapping and helps to identify the dependencies in our project. In addition to this Dependency-Check uses third party services such as NPM Audit API, OSS Index, retireJS and Bundler Audit as well.

Once Dependency-Check found the name of a dependency it checks the Common Vulnerabilities and Exposures (CVE) catalog if there are any existing vulnerabilities. the CVE program identifies, defines & catalogs publicly disclosed cybersecurity vulnerabilities. Currently there are ~170.000 vulnerabilities listed in CVE. CVE also allows for manual search of vulnerabilities, simply by inputing the dependency name on their website (https://cve.mitre.org/cve/search_cve_list.html)

Thank you for reading so far! :)

In the next part I'll show you how to add a Dockerfile to run a Dependency-Check on your existing project and also how to add the Dependency-Check as an automated step to your Jenkins Pipeline

Also if you like this article or find it interesting be sure to leave a like or share it with people who might be interested in the topic :) It would help me a lot.

Here's the link to the second part :)

React E2E Testing made easy using Cypress and Jenkins

Jochen Rui — Wed, 22 Sep 2021 06:50:19 +0000

What is End-To-End(E2E) Testing?

The primary goal of E2E Testing is to test the Application from the user's perspective. Thus regarding the Application as a Black Box - ignoring the internal logic and only testing what the Users see.

Drawbacks of E2E Testing

An error in the E2E Test Suite indicates that the User can't use the Application as intended. The problem is that we can't pinpoint the exact Line Of Code(LOC) that causes the error. Thus E2E Testing helps in finding significant errors but can't help in debugging them.

On the famous Testing Pyramid, E2E Tests can be found on top of Component and Integration Tests. As such there should be Unit and Integration Tests first. These help in catching errors early and debugging, thus increasing the pace of development.

Benefits of E2E Testing

E2E Tests are written in a way that resembles the User's way of operating our Application. As such E2E Tests gives great confidence in our Application by confirming that the key functionalities are working as intended from the User's point of view.

In addition to this E2E Tests ideally don't rely on Implementation Details, as such they are more robust and written in a way where fixing or updating them is fast and easy.

Practical Example

Now to the fun part: Code!

First we have to install Cypress

npm install cypress --save-dev
or
yarn add cypress --dev

Then we can create a simple cypress.json configfile in the root directory

{
    // specify the baseUrl from which we 
    // serve our applications in the test environment
    "baseUrl": "http://localhost:3000",

    // depends on project: allows accessing shadow dom without calling .shadow()
    "includeShadowDom": true,

    // optional: only necessary cypress component testing
    // not needed if all we do is e2e testing 
    "component": {
        "testFiles": "**/*.spec.{js,ts,jsx,tsx}",
        "componentFolder": "src"
    },
}

if our project is written in typescript we might want to add a tsconfig in the cypress subdirectory that extends our main tsconfig

cypress/tsconfig.json

{
  "compilerOptions": { "types": ["cypress"] },
  "extends": "../tsconfig.json",
  "include": ["integration/*.ts", "support/*.ts", "../node_modules/cypress"]
}

Writing Tests

After we finished the basic setup and installation we can now start writing tests.

describe("Sample Test Suite", () => {
  beforeEach(() => {
    // intercept outgoing HTTP Requests by defining the endpoint and mocked response
    cy.intercept("GET", "/some_endpoint", {
      statusCode: 200,
      body: {"a":1},
    });
  });

  it("sample test", () => {
    // uses baseUrl defined in cypress.json configuration
    cy.visit("/landing-page");
    // access DOM Nodes via e.g. class, id, data-test-id
    // & interact with DOM
    cy.get('[data-test-id="add-button"]').click();
    cy.get(".some_class").should("exist");
  });
});

In the example above we intercept Http Requests our application makes to the /some_endpoint endpoint. Thus we mock the backend and can run our tests without starting up a backend instance.

Now we can run the tests and see if our application works as intended. For this we can choose to run it with a UI and open chrome instance for easier debugging OR we can run it headless, e.g. for a quick run in CLI or as integrated step in our CI Pipeline in e.g. Jenkins, Azure Pipeline,...

Run Cypress in Dev Environment

To execute Cypress with an UI and controlled Chrome instance we can add this script to package.json

"cy:open": "node_modules/.bin/cypress open",

adding this allows us to easily start the cypress UI in the terminal

npm run cy:open

Jenkins Integration

To integrate Cypress into our Jenkins Pipeline, we can add these scripts to package.json

"cy:run": "node_modules/.bin/cypress run",
"ci:e2e": "start-server-and-test start http://localhost:3000 cy:run"

In addition we need to install start-server-and-test for this solution

npm install --save-dev start-server-and-test

This will ensure that our server is started before we try running our E2E Tests.

Now that all the preparations are done, we can add a step to our Jenkinsfile.

sh script: 'cd frontend; npm run ci:e2e'

Now when a Jenkins Build is triggered we will see a new stage in our Pipeline that displays a report of our E2E Tests.

Additional Information and Troubleshooting:

Depending on the Docker Image used, we may need to install additional OS specific dependencies. For this we can add a DockerFile step

# Install cypress OS dependencies
RUN apt-get install -qy \
    libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4  \
    libnss3 libxss1 libasound2 libxtst6 xauth xvfb procps