DEV Community: Mohammad Reza Ghasemi

Removing sensitive data from the git history

Mohammad Reza Ghasemi — Sun, 05 Dec 2021 09:24:03 +0000

Perhaps you saved sensitive data like a password or a token in your project. From a security point of view, it’s a bad practice to put them in your code directly and keep them in the repository. For example, the best approach in the Node.js app and other languages is to save sensitive data in the environment variables. So it is kept in a separate file called .env. Not to be tracked by Git, it is added to .gitignore. But if you check your previous commits, you will see that sensitive data exists in the git history. So what should be done now?!

You have two options: Git built-in command git filter-branch or BFG Repo-Cleaner tool. In this article, I want to show how to use the BFG Repo-Cleaner tool to rewrite your repository’s history.

The BFG Repo-Cleaner is a tool that’s built and maintained by the open-source community. It provides a faster, simpler alternative to git filter-branch for removing unwanted data. (GitHub)

mongoose.connect('mongodb+srv://user<password>@cluster0.fzrkd.mongodb.net/myFirstDatabase?retryWrites=true&w=majority',
{
  useNewUrlParser: true,
  useCreateIndex: true,
  useFindAndModify: false,
  useUnifiedTopology: true,
}).then(() => console.log('DB connection successful!'));

The above code is a part of a Node.js app that allows you to connect to the MongoDB Atlas via mongoose driver. For that, you need a connection string that has been bold in that code. That connection string is a sample of sensitive data.

At first, create a file .env.local to save your connection string in a variable called DATABASE.

DATABASE=mongodb+srv://user<password>@cluster0.fzrkd.mongodb.net/myFirstDatabase?retryWrites=true&w=majority

Then replace that connection string with the one in .env.local as process.env.DATABASE in your code like below:

mongoose.connect(process.env.DATABASE, {
  useNewUrlParser: true,
  useCreateIndex: true,
  useFindAndModify: false,
  useUnifiedTopology: true,
}).then(() => console.log('DB connection successful!'));

In Node.js, you can access environment variables through process.

You will commit the changes, although previous ones show the connection string right in the code. To replace that connection string with process.env.DATABASE the rest of the commits should be followed by these instructions:

If in a macOS, you can use Homebrew to install the BFG Repo-Cleaner tool: brew install bfg and if in Windows, use Chocolatey: choco install bfg-repo-cleaner or download its jar file from their site.
Replace the sensitive data with the one you want, then commit your changes. Don’t forget this step. In this case, replace mongodb+srv://user<password>@cluster0.fzrkd.mongodb.net/myFirstDatabase?retryWrites=true&w=majority with process.env.DATABASE.
Create a text file (e.x. replacements.txt) to the substitutions. According to Tyle answered in StackOverflow, in this case, the text file contains:

DATABASE=mongodb+srv://user<password>@cluster0.fzrkd.mongodb.net/myFirstDatabase?retryWrites=true&w=majority==>process.env.DATABASE

Then run the below code from your project folder. If you installed BFG through its jar file from their site, replace the bfg command with java -jar bfg.jar.

bfg --replace-text replacements.txt

After getting the “BFG run is complete!” message, run this code:

git reflog expire --expire=now --all && git gc --prune=now --aggressive

Now check your previous commits, and you will see the replacements overall in the repository.

What are Snippets in VSCode?

Mohammad Reza Ghasemi — Fri, 03 Dec 2021 12:40:22 +0000

Code snippets are templates that make it easier to enter repeating code patterns, such as loops or conditional-statements. (Visual Studio Code)

For example, in JavaScript, instead of inserting console.log(’Hello World…’) you can use VSCode built-in snippet log by inserting it, then press the tab button to show that entirely on the editor. There are many other built-in snippets on the VSCode that you can use:

Press shift+cmd+p in Mac or shift+ctrl+p in Windows to open the Command Palette.
Insert Insert Snippet command in the Command Palette to get a list of the snippets for the language of the current file, then press the enter button to see them.

Many extensions on the VS Code Marketplace include snippets. You can search for extensions that contain snippets in the Extensions view (shift+cmd+x in Mac or shift+ctrl+x in Windows) using the @category:"snippets" filter. (Visual Studio Code)

How to create your own snippets?

I’m working with React, and there is a popular snippet extension for React on the VS Code Marketplace, but most of its snippets are useless for me. So I prefer to create my own snippets instead.

I want to create a React TypeScript component snippet which takes its name from the file name:

Open the Command Palette by pressing shift+cmd+p in Mac or shift+ctrl+p in Windows.
Insert configure user snippets
Select the language for which the snippets
should appear.
For example, to create a snippet for a React
TypeScript file, choose typescriptreact then press the enter button.
VSCode will open a ***.json (typescriptreact.json for the previous selection).
Insert the below attribute on the ***.json (typescriptreact.json) object:

"React Arrow Function Component": {
  "prefix": "trafc",
  "body": [
    "type Props = {};\n",
    "const $TM_FILENAME_BASE = ({}: Props) => {",
    "  return <></>;",
    "};\n",
    "export default $TM_FILENAME_BASE;\n"
  ],
  "description": "Create TypeScript React arrow function component"
}

You can use trafc snippet for React TypeScript file.

Now I want to describe what the above code says:

“React Arrow Function Component” is the snippet name. It is displayed via IntelliSense if no description is provided.
prefix is the snippet that we can use on the VSCode to display the body.
body is an array of strings consisting of the code we want to be displayed via the prefix snippet. Every line of our codes should be placed in a string. If you need to use enter character type \n.
description as its name shows is the description of our snippet.

Another example is to create a ‍‍console.log with a description like console.log("status:", status). The snippet configuration is shown in the below section:

"console.log with description": {
  "prefix": "lg",
  "body": ["console.log('${1:desc}:', ${1:desc})"],
  "description": "Insert a console.log() with description as the           same as the variable."
}

The new thing is the ${1:desc}. The 1 means after you insert the snippet, the cursor will be placed on that position.
desc means these two parts should be the same in order to VSCode type them simultaneously. For more information check this link.