DEV Community: Satej

Git-ting it Right : Best Practices for your Git Repository

Satej — Sat, 27 Jun 2020 17:31:23 +0000

Over the years, I've worked on multiple software development projects, consisting of anywhere from 2 to over 30+ active software developers working on the same repository. It takes very little time to go from wonderfully written code to an absolute wreckage - both in terms of your code base and your repository.

In this article, I dive into some git best practices that I've learned on the job and actively use while working. I expect this list to grow and I'll add those points as and when I can.

1. Use multiple branches

There are few very reasons to exclusively use the main branch. Let's see what they are:

You're the only developer working on a repository
You love chaos

The solution to avoiding absolute anarchy is pretty easy - use other branches.

Many organizations create the following branches apart from the ones mentioned in the Point #2:

Main

The Main branch typically contains your code that is safe to deploy to production. You would ideally never commit directly to this branch, and code is to be only added via pull-requests here.

Development or Dev

This branch is where code that is being actively developed and worked on would reside. It may have some bugs that are being fixed. If you're working on a feature, you'd ideally create a new feature branch(more on this below) from the development branch, and merge this back into the development branch.

Production and Pre-Production

Larger companies may have a separate Production and Pre-Production/Staging branch to release their code onto internal or public networks. These are typically hooked up to plugins that automatically deploy your code when a new commit is made.

Test

Similarly, the Test branch may be used for releases onto a test network or test devices.

2.Follow a Branch Naming Convention ( and stick to it )

As your product grows, you're going to want to add new features and in the process fix bugs in your code. Some of these bugs will have your team on their toes.

It's important to distinguish what kind of code you're merging into your working branch, primarily to allow you to track changes in the future. This is where branch naming conventions help.

I typically use these three the most, but I've also seen some teams use the test branch for any QA or review related purposes.

Feature Branches

Let's say you're working on a sparkling new feature to allow users to save a post they see on their feed. You can name your branch along the lines of :

feature/save-post-from-feed
feature/save-newsfeed-post
feature/what-does-the-feature-do

While there's nothing stopping you, I'd refrain from naming it along the lines of:

feature/allowtheusertosaveapostfromthenewseed
feature/ALLOWTHEUSERTOSAVEPOST

Using just verbs with hyphens helps easily read and understand the task being carried out.

BugFix Branches

Let's say you're fixing a bug that isn't extremely crtical. The feature you built to save a post has an issue with the button and you're working on a fix. With Bugfixes, I personally like to reference the issue number on the repository to see a detailed explanation of what is being fixed.

bugfix/button-change-for-save-post-#388
bugfix/save-post-#388
bugfix/what-does-it-fix

Don't forget to write in the pull-request, what your fix does.

HotFix Branches

Finally, for those fixes that are absolutely critical in terms of their severity and need to be fixed immediately, you would use the hotfix branches.

hotfix/allow-admin-facebook-login-#392
hotfix/what-does-it-fix

Using these distinctions allows for some traceability when working on a larger code base.

3. Write Commit Messages (or atleast try)

Okay, we've all been guilty of this. Especially when it is 3am, and you honestly don't care what the commit message says. Don't forget to add a commit message that covers the important information. Here's a sample

git commit -m "Moved chartData from Statistics.js to Redux Chart State"

You can also link issues to pull requests with your commit message. I'd highly recommend this link on linking a pull request to issues.

4. Use Commit Hooks

If you aren't familiar with what commit hooks are, that's not a problem. Think of them as scripts that run before, during or after you perform any function with git.

There are a couple of use-cases for running commit hooks

You want to automatically indent your code before a commit is made
You want to check for code quality before a pull request is made
You want to send an email after a pull request is made

I personally use pre-commit hooks the most to automatically format and indent my code with a lot of help from husky. If you'd like to learn more about commit hooks, I'd recommend checking out Git Hooks.

5. Automate your Deployment

Continuous Integration/Continous Delivery(CI/CD) is a topic in itself, but setting up a workflow that let's you write code and at the push of a button deploys to a server of your choice is an incredible feeling.

For my personal website, I use Netlify to automatically build my website the moment a new commit is pushed to my Main branch. All Netlify requires is access to your repository and a few more instructions regarding which build command to run and what directory to serve from. For larger projects, you definitely want to check out Docker and a tool like TravisCI or CirleCI.

And that's about it!

Did you like this article? Let me know on Twitter at @SatejSawant. If you spot something incorrect, please write to me at satejs93[at]gmail[dot]com. You can also find me on LinkedIn.

NPM Audit Fix: The Complete Guide to How It Works - Part 2

Satej — Tue, 09 Jun 2020 20:12:42 +0000

In my earlier post, I dived into how NPM Audit works. In this concluding post, I talk specifically about how the fix flag works in npm audit fix.

To do a quick re-cap if you haven't read the earlier article, NPM will use your package.json and package-lock.json files to create an object consisting of your dependencies. These dependencies are then sent over to your registry(typically NPM) to check for any reported vulnerabilities. Once it receives a response from the registry, NPM begins the process of fixing those vulnerabilities.

The response from NPM outlines the actions to be taken regarding vulnerabilities in your dependencies. The response body looks something like this:


"actions": [
        {
            "isMajor": false,
            "action": "install",
            "resolves": [
                {
                    "id": 1486,
                    "path": "http-proxy",
                    "dev": false,
                    "optional": false,
                    "bundled": false
                }
            ],
            "module": "http-proxy",
            "target": "1.18.1"
        }
    ],

There may (and probably will) be multiple objects in this actions object returned from the registry.

The first job for NPM is to segregate these individual objects based on their importance and type. It uses a reducer for this, and begins with the object below as the original value. If you aren't very familiar with reducers, check out this Mozilla Developer Network(MDN) link on how the reduce function works in Javascript.

{
    install: new Set(),
    installFixes: new Set(),
    update: new Set(),
    updateFixes: new Set(),
    major: new Set(),
    majorFixes: new Set(),
    review: new Set()
}

Alright, so how exactly does audit.js act on this actions array?

To begin with, the actions array seen in the previous snippet is run through a reducer.

In this reducer, each action is passed to a filter function called filterEnv. This function returns a new object with only the relevant dependencies. The relevant dependencies are identified basis the environment you're building in - essentially dev and production.

There are four buckets a fix may fall into

install
update
major
review

Using the isMajor field and the action field seen in each of the objects from the sample response body above, the sets seen earlier are populated. Depending on the set, information regarding the module, target, id and path of the vulnerability fix is added. You can see this in the npm audit.js file here.

Since the sample response is of action type install, the install Set would be populated with the module and target. It would look something like:

(http-proxy::1.18.1, ... )

Meanwhile, the installFixes Set would be populated with the id and path. It would look something like:

(1486::http-proxy, ... )

Similarly, the other sets are also filled depending on the type of vulnerability you're dealing with.

Finally, NPM will install these dependencies by passing the contents of these sets to an instance of the Auditor Class which extends the Installer Class. The run() method is called on this instance which actually runs the installation.

Does NPM Audit Fix handle every vulnerability?

Nope. If a particular object has the action key with value review, it is added to the review set. These items require manual review by the developer and are not updated automatically.

But, I heard about a --force flag?

Correct! But that's primarily for what are considered as "major changes".

You can identify if a change is a major one through the isMajor flag. These are essentially actions that that involve potentially breaking changes. If the --force flag is provided, NPM will go ahead and fix the vulnerability - albeit with the inherent risk that something may break.

For other cases where the flag is not specified, NPM will produce a log asking you to fix these manually or use npm audit fix --force.

I referred to a couple of resources while writing this article. Here they are:

Did you like this article? Let me know on Twitter at @SatejSawant. If you spot something incorrect, please write to me at satejs93[at]gmail[dot]com. You can also find me on LinkedIn.

NPM Audit: A Complete Guide to How It Works - Part 1

Satej — Fri, 29 May 2020 20:28:17 +0000

Recently, I began working with a research lab to help them with their front-end architecture. In the process of refactoring their build process, I came across this message almost half a dozen times.

found 611 vulnerabilities (600 low, 4 moderate, 7 high)
  run `npm audit fix` to fix them, or `npm audit` for details

That number of vulnerabilities was initially over 15,000 and after running npm audit and npm audit fix a couple of times, I had a sense of accomplishment.

But, what exactly did I do?

In this two part article, I dive into exactly that. This article helps you understand how npm audit works, while Part 2 wraps it up with what the --fix flag specifically does. So let's get started.

NPM AUDIT

If you have never heard of the command before, npm audit helps you find (and fix) security vulnerabilities in your project's dependency tree.

To begin with, npm audit, needs two files to be present - package.json and package-lock.json.

Without those, you'll run into either of the two:

npm ERR! code EAUDITNOPJSON
npm ERR! audit No package.json found: Cannot audit a project without a package.json

npm ERR! code EAUDITNOLOCK
npm ERR! audit Neither npm-shrinkwrap.json nor package-lock.json found: Cannot audit a project without a lockfile

If you don't know how to create a package-lock.json file, you can run this command :

npm i --package-lock-only

So why does it need these two files?

NPM fetches the dependencies and dev dependencies by reading both these files. In the absence of the package-lock.json file, it uses the npm-shrinkwrap.json file.It also uses the shrinkwrap file if both of the files are present. If you want to see exactly how this is done, here is a link to the audit.js file in the NPM repository. The dependencies from the package-json file and the contents of the package-lock file are passed to a function called audit.generate() in the form of parameters called require and sw.

Okay, what next?

In the generate function, a deep clone of the shrinkwrap(or package-lock) file is created. After passing the inputs through a series of functions to scrub and sanitize the data, an object is created containing two keys, namely requires and dependencies. You can checkout the code on how the data is scrubbed here. The require and dependencies keys enumerate the packages required in the project along with their version, integrity(the hash) and more information.

Once it has made this object, npm calls the audit.submitForFullReport(auditReport) function. This makes a POST request to https://registry.npmjs.org/-/npm/v1/security/audits with this JSON object as the body. NPM Audit uses the npm-registry-fetch package for this.

Here is a sample request body of the data sent across:

{
    "name":"package-name",
    "version":"1.0.0",
    "requires":{
        "http-proxy": "1.18.0"
    },
    "dependencies":{
        "http-proxy": {
          "version": "1.18.0"
        }
    }
}

The response of this API call returns detailed information on any reported warnings and vulnerabilities contained in your repository after checking them with your registry. In most cases, this registry is NPM.

Below, you will see a sample response for the request body from above. We see a sample vulnerability in http-proxy. Each such object contains information about who it was reported by, the relevant dates and more about the finding.

["1486": {
            "findings": [
                {
                    "version": "1.18.0",
                    "paths": [
                        "http-proxy"
                    ]
                }
            ],
            "id": 1486,
            "created": "2020-02-21T14:16:24.023Z",
            "updated": "2020-05-18T14:50:08.944Z",
            "deleted": null,
            "title": "Denial of Service",
            "found_by": {
                "link": "https://twitter.com/_awry",
                "name": "Grant Murphy",
                "email": ""
            },
            "reported_by": {
                "link": "https://twitter.com/_awry",
                "name": "Grant Murphy",
                "email": ""
            },
            "module_name": "http-proxy",
            "cves": [],
            "vulnerable_versions": "<1.18.1",
            "patched_versions": ">=1.18.1",
            "overview": "Versions of `http-proxy` prior to 1.18.1 are vulnerable to Denial of Service. An HTTP request with a long body triggers an `ERR_HTTP_HEADERS_SENT` unhandled exception that crashes the proxy server. This is only possible when the proxy server sets headers in the proxy request using the `proxyReq.setHeader` function.   \n\nFor a proxy server running on `http://localhost:3000`, the following curl request triggers the unhandled exception:  \n```

curl -XPOST http://localhost:3000 -d \"$(python -c 'print(\"x\"*1025)')\"

```",
            "recommendation": "Upgrade to version 1.18.1 or later",
            "references": "- [Patch PR](https://github.com/http-party/node-http-proxy/pull/1447/files)",
            "access": "public",
            "severity": "high",
            "cwe": "CWE-400",
            "metadata": {
                "module_type": "",
                "exploitability": 4,
                "affected_components": ""
            },
            "url": "https://npmjs.com/advisories/1486"
        }]

Finally, after collating all this information and grouping the issues based on their seriousness, NPM typically calls audit.printFullReport(auditResult) which prints the report as you see it on the screen. To do this, NPM uses a package called the npm audit report.

And that's how you see the outfit that you're familiar with.

┌───────────────┬──────────────────────────────────────────────────────────────┐
│ Low           │ Prototype Pollution                                          │
│ Low           │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ minimist                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=0.2.1 <1.0.0 || >=1.2.3                                    │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ react-scripts                                                │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ react-scripts > jest > jest-cli > jest-config > babel-jest > │
│               │ @jest/transform > jest-haste-map > fsevents > node-pre-gyp > │
│               │ rc > minimist                                                │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://npmjs.com/advisories/1179                            │
└───────────────┴──────────────────────────────────────────────────────────────┘

In Part 2 of this article, I dive into how the --fix flag works. Adding a link soon!

Did you like this article? Let me know on Twitter at @SatejSawant. If you spot something incorrect, please write to me at satejs93[at]gmail[dot]com.

I referred to a couple of resources while writing this article. Here they are:

For Part 2 of this article, refer to this link

Deno? Do you mean Node?

Satej — Fri, 15 May 2020 19:01:23 +0000

The last couple of days, I've come across a lot of my favourite tweeple talking about Deno.

Tyler McGinnis

@tylermcginnis

Is it pronounced Deno or Deno?

22:48 PM - 14 May 2020

22 252

Having just recently re-entered the JavaScript universe after almost a year of Java, I was like wait a second.

Are you guys talking about Node?

A few google searches later, I came to realize that that the JavaScript universe had given birth to it's newest creation - Deno.

So why is everyone on the internet suddenly talking about this cute little dinosaur in the rain?

Earlier this week, the team at Deno, dropped its first stable release, Deno v1.0. It's got a lot of cool features, and people are really excited to see if this will "kill" Node.

But there's nothing wrong with Node?

Well, that's partly what I thought too, until I saw Ryan Dahl's(he created Node) talk at JSConf EU from 2018 and came across a couple other articles such as this.

In his talk, Ryan speaks about 10 things that he regrets building into Node. I've elaborated more on what I consider the most important from those ten, but definitely check out the video below.

Security

While V8 is a secure sandbox, in some situations, there's no reason for certain applications to have access to the file system or the network. Access to these entities should be restricted and access-controlled. Ryan's example of a linter not needing access to the underlying system perfectly encapsulates this.

Promises

Promises were initially added in June 2009, but later removed in February 2010 from Node - in an effort to remain minimal and do away with the overhead they introduced of an extra object into every callback. While this move allowed the eco-system to develop Promises as we know them today, Dahl attributes the problem with the "current aging async APIs" to not sticking with promises initially.

Package.json

This file has become the heart of pretty much every node project. The original idea apparently wasn't really to have a directory of files that package.json has become. This is made worse by the fact that npm has become a private centralized source of these packages. Coupled along with the "unnecessary" information about the package name, license,etc - things could be better.

Node Modules

Dahl believes having node_modules massively complicates the module resolution algorithm. There were simpler ways to do this. I like this particular article that elaborates a bit on the topic of module resolution

As a disclaimer, this talk is from 2018, and I'm sure things have changed ever since. I do understand that fundamental issues regarding native TypeScript support and security remain.

While these issues exist, due to the large number of users that Node has, it is massively difficult to bring about sweeping changes in the current system.

Thus, Deno.

So, what is Deno?

Deno is a new runtime for executing JavaScript and TypeScript outside of the web browser.

And how is it different from Node again?

Security

To begin with, the code is executed in a secure sandbox, just like it would be on a browser. Your code cannot access the hard drive, open network connections without your permission. It always requires explicit permission for file, network and environment access. It has built in flags for this such as --allow-net.

First Class TypeScript Support

One of the most painful issues I've come across while working with JavaScript is the lack of type-checking built in. TypeScript supports that, and so does Deno. All of Deno's standard modules are written in TypeScript.

Promises

In Deno, the lowest level binding layer to the system, called "ops" are tied to promises. All callbacks in Deno thus, arise from promises.

Rust APIs

Deno internally is a collection of Rust modules that are integrated at different layers.

Package Management

Deno doesn't use NPM. Yeah. Wow. It directly references URLs or file paths. Oh, it also does not use package.json in its module resolution algorithm.

Caching

Code that is remote is cached on the first execution, and not updated till you explicitly ask for it to be reloaded.

TDLR?

Deno definitely seems promising. It's simplicity is definitely a step up from Node, but only its adoption over a longer time will really tell.

The one tweet that really stood out to me, is this one:

Bradley Farias

@bradleymeck

Most exciting thing about Deno? Node getting a competitor. Competition is a way to bring progress by showing innovation and allowing evolution beyond the initial shock.

23:01 PM - 13 May 2020

24 308

I'm really excited to see how Node steps up to the competition now.

Are you planning on trying out Deno? Let me know in the comments what you're building with it!