DEV Community: Rohit Gohri

Jira CI/CD Integration

Rohit Gohri — Sun, 08 Aug 2021 05:00:00 +0000

Integrate your CI/CD pipeline's Build and Deployment information into the Jira Development Panel. Enable better

Only supports Jira Cloud. Does not support Jira Server (hosted)

Prerequisites

Generate Credentials

Generate new OAuth Credentials and copy

See: https://support.atlassian.com/jira-cloud-administration/docs/integrate-with-self-hosted-tools-using-oauth/

Use with Any CI/CD Provider

As long as your CI/CD provider supports running Docker images you may use this integration. Tested in Drone.io, Gitlab CI, and Google Cloud Run.

Docker Images are available from:

Gitlab Container Registry: registry.gitlab.com/rohit-gohri/jira-ci-cd-integration
Github Container Registry: ghcr.io/rohit-gohri/jira-ci-cd-integration
Docker Hub: boringdownload/jira-integration

Pick whatever you want and is convenient for you.

Set Env Vars

Configuration for the Docker image is through env vars. Read more in options.

Jira-Gitlab CI/CD

Add a CI/CD Variable to your project for JIRA_CLIENT_ID and JIRA_CLIENT_SECRET (remember to mask them) and then add these steps to your pipeline (we use .post stage so it runs last)

jira-integration-on-success:
  stage: .post
  when: on_success
  image: registry.gitlab.com/rohit-gohri/jira-ci-cd-integration:v0
  script: jira-integration
  variables:
    BUILD_STATE: successful
    BUILD_NAME: gitlab-pipeline
    JIRA_INSTANCE: companyname

jira-integration-on-failure:
  extends: jira-integration-on-success
  when: on_failure
  variables:
    BUILD_STATE: failure

Jira-Drone.io Example

Add secrets for JIRA_CLIENT_ID and JIRA_CLIENT_SECRET and then add this to your pipeline:

steps:
  - name: jira-integration
    image: boringdownload/jira-integration:v0
    environment:
      BUILD_NAME: drone-pipeline
      JIRA_INSTANCE: companyname
      JIRA_CLIENT_ID:
        from_secret: jira_client_id
      JIRA_CLIENT_SECRET:
        from_secret: jira_client_secret

Usage With Github Actions

You can also use this as a Github Action if you want to send release information to Jira.

Add OAuth Creds as secrets to Github

See: https://docs.github.com/en/actions/reference/encrypted-secrets

Add Client ID as JIRA_CLIENT_ID
Add Client Secret as JIRA_CLIENT_SECRET

Update Github Workflow

Use in Builds Pipeline

- name: Jira Integration
  if: ${{ always() }}
  uses: rohit-gohri/jira-ci-cd-integration@v0
  with:
    event_type: build
    state: ${{ job.status }}
    jira_instance: companyname # Subdomain for Jira Cloud
    client_id: ${{ secrets.JIRA_CLIENT_ID }}
    client_secret: ${{ secrets.JIRA_CLIENT_SECRET }}

Use in Deployment Pipeline

- name: Jira Integration
  if: ${{ always() }}
  uses: rohit-gohri/jira-ci-cd-integration@v0
  with:
    event_type: deployment
    state: ${{ job.status }}
    issue: JCI-3, JCI-6 # Comma separated list of issues being deployed/released. You are expected to generate this yourself in a previous step
    jira_instance: companyname # Subdomain for Jira Cloud
    client_id: ${{ secrets.JIRA_CLIENT_ID }}
    client_secret: ${{ secrets.JIRA_CLIENT_SECRET }}

OpenAPI for Docusaurus! 📘

Rohit Gohri — Tue, 19 Jan 2021 17:30:00 +0000

Introducing "Redocusaurus": Redoc for Docusaurus

Redocusaurus

What is it?

It is a preset that allows you to easily integrate OpenAPI documentation into your docs site made with docusaurus. It has 2 main components:

docusaurus-theme-redoc

Using Redoc as a React component, this provides 2 theme components, @theme/ApiDoc and @theme/Redoc. You can use them directly in js pages in Docusaurus.

docusaurus-plugin-redoc

This creates routes from your OpenAPI files or URLs and renders with the components provided by the theme.

How to set it up?

Install the preset



npm i --save redocusaurus

Add it to your docusaurus.config.js:



module.exports = {
 // ...
 presets: [
   // ... Other preset configs
   [
     'redocusaurus',
     {
       specs: [{
         route: '/api/',
         spec: 'https://redocly.github.io/redoc/openapi.yaml',
       }],
     }
   ],
 ],
 // ...
};

Rebuild your site and you will have your OpenAPI docs available at the /api/ route.

Extra features over directly using Redoc

The theme includes redoc settings and some custom css to make it match the @docusaurus/classic theme. Along with support for Dark Mode and Server Side Rendering.

Docs

Find the docs and examples at the website : https://redocusaurus.vercel.app/

Github Repo

Find the source and know more on Github.

The Hidden Cost of AWS ELBs

Rohit Gohri — Tue, 01 Sep 2020 13:00:00 +0000

This past month we had a really high AWS bill, and it was supposed to be high but not THIS high. The culprit - "Data Transfer Out". I'm sure I'm not the only one who has had to deal with this.

Some Context

We had a NodeJS service running on EC2 in an Auto Scaling Group (ASG) behind an Application Load Balancer (ALB) that was supposed to handle a huge volume of requests. We were expecting a consistent increase in traffic this month, so we had made estimates on how it would affect our costs. We had expected some high data transfer out of our Application Load Balancer (ALB) but this was a consistent 4-5x of our estimates. Time to investigate.

Analysing the Data

After filtering Cost Explorer data with tags, it was clear that the ALB was the source of these costs. I immediately went to CloudWatch and checked the "Processed Bytes" metric of the ALB.

Cloud Watch

Processed Bytes is supposed to be "the total number of bytes processed by the load balancer over IPv4 and IPv6 ... includes traffic to and from clients", so ideally Processed Bytes should be greater than the "Data Transfer Out". But it was not..?

Our stats showed Processed Bytes to be around what our estimates had predicted for this amount of traffic. But Cost Explorer was showing a number that was 4-5 times that. Time to contact support.

AWS Support

We raised a support request to AWS, but we didn't have high hopes because we were not on the premium support plan. But it was worth a shot.

ELB Access Logs

Next up, we enabled ALB Access Logs to an S3 bucket and let it generate some logs for some time. Upon checking the logs, we were certain that this wasn't a problem at the application level. The access logs showed response size consistent with what we had estimated, no inflation due to extra headers or something like that.

VPC Flow Logs

We wondered if there was some other service running that might be communicating over an Elastic IP or public IP instead of a private IP. To check that, we enabled VPC Flow logs to another S3 bucket and let it generate some logs for some time.

We didn't find any services that might be communicating over Elastic IP but we did find some services that could be moved to the same Availability Zone (AZ) and reduced some Regional Data Transfer costs.

Frantic Googling

After eliminating all possible sources of Data Transfer we still had no clue what was the source of the Cost other than that it was from the ALB. This led to frantic googling and sending/explaining the problem statement to various people.

Then over the weekend, trying to think of keywords that would not lead me down to more documentation, I finally found this StackOverflow answer:

Mar 20 '19 Comments: 5 Answers: 1

I'm curious to know how much total outbound bytes an ELB generates. As far as I can tell, this will be something like

(size of http responses) + (size of SSL handshake transactions)

I can calculate the former by looking at my web server logs. However, I'm having a hard…

Open Full Question

A. The handshake of a TLS connection is pretty much constant and do not depend on your application. So your function is really : number of connections * ( size of http responses + constant)

And it finally clicked, the missing piece of the puzzle. Since HTTPS was handled directly by the ALB, it wouldn't show in the Access Logs as the instance didn't have to deal with it.

Here is an article that estimates the payload to average ~6.5k (depending on your certificate size). Our response sizes were really small, and this could be a major part of the response size if the user was visiting the site for the first time. Since a lot of the traffic was new, it would be a significant chunk.

That led to even more Googling on how HTTPS might affect AWS ELB costs and we came upon this very good article that also suggested increasing the default timeout for idle connections to 10 minutes for the ALB along with changing the certificate.

Testing Different HTTPS Certs

We were currently using the default certificate generated by Amazon Certificate Manager (ACM) with the Load Balancer. We tested the size of the TLS handshake with Wormly Test SSL Tool and then replaced it with one generated with Let's Encrypt. To be thorough we also tested TLS handshake sizes of different websites with different Certificate Authorities.

Certificate Issuer	Certificate Type	SSL Handshake Size (Bytes)
ACM	Wildcard (RSA 2048 bits)	5971
Let's Encrypt	Single (RSA 2048 bits)	3753
Let's Encrypt	Wildcard (RSA 2048 bits)	3702
Let's Encrypt	Wildcard (ECC 256 bits)	3323
DigiCert	Single (ECC 256 bits)	3311
Sectigo	Wildcard (RSA 2048 bits)	6720

DigiCert was the best with respect to size, but Let's Encrypt was close enough (& free!). ECC certificates are not compatible with some older browsers, so we generated both the smaller ECC and RSA certificates through acme.sh and uploaded them through AWS CLI. The ALB will automatically select the best one depending on the client (read more here).

After adding them to the ALB listener and letting Cost Explorer catch up for a day, we saw a significant decrease in our Data Transfer Out Costs for the same number of requests.

It was still not equal to the estimates we had, but it will never be because we never considered the TLS handshake size in our estimates. AWS Costs really are a mystery until you actually get the bill.

CI/CD for your Resume!

Rohit Gohri — Fri, 28 Aug 2020 20:08:44 +0000

After ending up with countless PDFs of my resume and never being sure of which one is the final version. I decided to start versioning it with the tool I already knew, Git. After finding a good enough LaTeX template, a git init, and some commits later, I had a good enough resume template with a working Github Action that would publish the generated PDF as a release artifact on every tag. I also added generated PDFs as build artifacts for every commit to preview changes without having access to a working LaTeX env, so that one may be able to make edits from their phone.

My Workflow

I simply edit the tex file using the Web Editor and commit my changes. The "Preview" workflow will build and generate a pdf that I can preview using the build artifact.

Once I'm satisfied with the edits, I will create a new release, this will trigger the "Release" pipeline. It will build and publish the pdf as a release artifact.

The release is what allows me to always have access to the latest version of my resume at the /releases/latest/download/Preview.pdf path of my repository.

Download latest version

More than that, I can preview it directly in my browser using Google Docs by using https://docs.google.com/viewer?url=https://github.com/<username>/<repo>/releases/latest/download/Preview.pdf. I can create a short url of this that I can give to anyone knowing it'll always point to the latest resume without having to worry about updating it.

Preview latest version using Google Docs

Submission Category: Wacky Wildcards

When I first realized I could basically code my resume in LaTeX, it finally stopped becoming something I dreaded doing and became something I actively wanted to improve. Adding CI/CD to it was the next natural step.

Yaml File or Link to Code

This is a Github Template Project with a LaTeX resume template and Github Workflows. You may use this template or add your own LaTeX files to build those.

rohit-gohri / resume-template

1 Page LaTeX Resume template with CI/CD

Resume

Automatically built resume with LaTeX. The structure for sections is defined in resume.cls file. While the content is in resume.tex.

Based on the CV For Freshers template on Overleaf.

Preview latest version

Download latest version

My Resume

Resources

Blog Post: CI/CD for your Resume

Using as a template

Use the green "Use this template" button to create a new repository with this repository's files as a template. There are two files only:

resume.cls: Defines the structure and all the custom commands. No need to modify this.
resume.tex: This is the file where all your content for the resume goes. Your name, email, links, etc. are defined through commands. Feel free to move things around and change them.

If there…

View on GitHub

Additional Resources / Info

My Resume that is using a similar workflow, but with some added integrations to sync my blog with the latest version of my resume: resume

How to get ESlint and Prettier to play nice in VS Code?

Rohit Gohri — Tue, 28 Apr 2020 18:30:00 +0000

Do you have your project setup with both Prettier and Eslint? Do you want them to run in a specific order instead of using ESLint to run prettier? Here's how to set your own order in VS Code with this extension:

Visual Studio Code Market Place: Format Code Action

This extension will register a new "codeAction" which will run 'Format Document' using the default formatter (prettier in our case) in VS Code.

source.fixAll.format

Using this and the editor.codeActionsOnSave setting in VS Code you can specify the order in which to run the eslint codeAction, source.fixAll.eslint, and format document.

Install

Requires: VS Code 1.44+

Install through VS Code extensions:

Visual Studio Code Market Place: Format Code Action

Can also be installed in VS Code: Launch VS Code Quick Open (Ctrl+P), paste the following command, and press enter.

ext install rohit-gohri.format-code-action

Usage

You will need both the prettier-vscode and vscode-eslint extensions installed for this config to work.Disbale formatOnSave and use the source.fixAll.format codeAction in whatever order you want with VS Code settings:

  "editor.formatOnSave": false,
  "editor.defaultFormatter": "esbenp.prettier-vscode",
  "editor.codeActionsOnSave": [
    "source.fixAll.format",
    "source.fixAll.eslint"
  ]

This runs 'Format Document' with the default formatter (in this case prettier) and then eslint --fix for all document types.

Or for a specific lanuage only:

  "editor.formatOnSave": true,
  "editor.defaultFormatter": "esbenp.prettier-vscode",
  "[javascript]": {
    "editor.formatOnSave": false,
    "editor.codeActionsOnSave": [
        "source.fixAll.format",
        "source.fixAll.eslint"
    ]
  },

This would run prettier by default, but for javascript files would run prettier and then eslint. If you want to reverse the order then just reverse the array.

Motivation

I wanted to use prettier and eslint together, to override some of prettier's mpre opinionated style rules with a eslint config (particularly brace-style).

Instead of using workarounds like prettier-eslint which don't work for all eslint configurations as it hasn't been updated to work with eslint v6 properly. We can simply run prettier and eslint one by one like one would do on the command line. Here is how that looks like in my package.json:

  "scripts": {
    "format": "npm run prettier:fix && npm run lint:fix",
    "lint": "eslint src --ext=js",
    "lint:fix": "npm run lint -- --fix",
    "prettier": "prettier -c",
    "prettier:fix": "npm run prettier -- --write"
  }

The npm run format command allows me to fix some of the prettier formatting with eslint by running them both in an order. I wanted to replicate this same behavior with my editor's "Format on Save" functionality.

VS Code only allows setting one default formatter. So I could either run Prettier or run ESLint on save. That was until it introduced "codeActionsOnSave". This separated formatters and "Source" fixers like vscode-eslint. This is how one would enable both prettier and eslint in VS Code:

"editor.formatOnSave": true,
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.codeActionsOnSave": {
  "source.fixAll.eslint": true
}

But this would often lead to prettier being run after eslint, and eslint errors still being present as this defined no order for formatters and coder actions.

Since the March 2020 (v1.44) update, VS Code allows setting codeActionsOnSave as an array. This allows setting the order of the code actions.

This extension uses the CodeActionProvider api to implement a simple code action that runs 'Format Document' on the current file. This allows us to disable formatOnSave and use it as a codeAction instead in a specific order with other extensions.

Source

You can find the source on Github:

vscode-format-code-action

DangerJS Plugin - TODOs

Rohit Gohri — Mon, 16 Mar 2020 18:30:00 +0000

What is It?

A plugin for danger-js that collects all the "TODO" comments introduced in a pull request (PR) and posts them as a comment on the PR.

NPM JS - Danger Plugin Todos

It's a pretty simple plugin that just scans your diff for matching comments, it supports different language style comments including JS, C, C++, Ruby, Java, Elm, HTML, Bash, etc. (open an issue if something is missing). You can also specify custom keywords other than "TODO" or "FIXME".

How to set it up?

Set up danger-js and add it to your continuous integration (CI) pipeline
Add danger-plugin-todos to the "dangerfile" as mentioned in the README

Next time your CI pipeline runs for any PR this plugin will scan the diff to get all the To-Do comments and post them through danger as a comment on the PR. The comment will update on consecutive runs.

Possible Workflows

Scenario A:

Sometimes when you start work on a feature you see things along the way that need to be fixed and you end up adding a To-Do or Fix-Me comment for them. The feature branch then gets merged and the comments quickly gets forgotten if they were not related to a part of the code base that sees frequent updates.

To track such To-Dos this plugin can be used, as a list of them will be added as a comment to the PR. This list of To-Dos can then be moved to a new issue before the PR is merged and the issue added to the backlog.

Scenario B:

This plugin can also be used to create an actual auto-updating To-Do List.

Just add all your tasks for the PR in the code as To-Dos and open a Work In Progress (WIP) PR, this will create a list of To-Dos in the PR comments.

Now, as you work on them and remove them the comment will be updated and your list will become smaller. Once you are done with them the PR will be ready to merge.

More Features Incoming

There are some new ideas that are being worked on (check the issues here), like:

Generate a link to create a new issue with a list of To-Dos automatically : This would be pretty great for our Scenario A above. Link can be customized to support different issue trackers
Show completed To-Dos : PRs that complete and remove old To-Dos should be celebrated. This would add a success message on removing old To-Dos.

Anyways, I hope this is useful for you. If you have any ideas on how to make this better feel free to message me on Twitter or open an issue on the repo.

DEV Community: Rohit Gohri

Jira CI/CD Integration

Prerequisites

Generate Credentials

Use with Any CI/CD Provider

Set Env Vars

Jira-Gitlab CI/CD

Jira-Drone.io Example

Usage With Github Actions

Add OAuth Creds as secrets to Github

Update Github Workflow

Use in Builds Pipeline

Use in Deployment Pipeline

OpenAPI for Docusaurus! 📘

What is it?

How to set it up?

Extra features over directly using Redoc

Docs

Github Repo

The Hidden Cost of AWS ELBs

Some Context

Analysing the Data

Cloud Watch

AWS Support

ELB Access Logs

VPC Flow Logs

Frantic Googling

Testing Different HTTPS Certs

CI/CD for your Resume!

My Workflow

Submission Category: Wacky Wildcards

Yaml File or Link to Code

rohit-gohri / resume-template

1 Page LaTeX Resume template with CI/CD

Resume

Resources

Blog Post: CI/CD for your Resume

Links

Using as a template

Additional Resources / Info

How to get ESlint and Prettier to play nice in VS Code?

Install

Usage

Or for a specific lanuage only:

Motivation

Source

DangerJS Plugin - TODOs

What is It?

How to set it up?

Possible Workflows

Scenario A:

Scenario B:

More Features Incoming