DEV Community: Aoibhe Wilson 🇨🇭

Hoarding Secrets in NodeJS

Aoibhe Wilson 🇨🇭 — Mon, 14 Dec 2020 23:30:29 +0000

Originally posted at Valtech Switzerland on Medium

When working with NodeJS it's likely you'll wind up with a few secrets; not secrets like "Popeyes has the best fries" but API keys, encryption keys, database passwords, and other things you wouldn't want anyone else to learn.

These secrets are values you'll need to use in your code, but you never want to directly include them in the actual code, especially when you code is distributed to users, or if you use public repositories for your code. So how do we run these values in our code? Let's learn a bit about Environment Variables.

If you're already comfortable with environment variables you can jump right to implementation.

The Execution Environment

When you run code on a computer, whether your physical machine, or a virtual machine in the cloud, the operating system and configured software on that machine are your code's execution environment. When running Node scripts, information about the execution environment is made available through special variables. We call these environment variables.

On a physical server you might set these values directly in the operating system. If you're deploying code to a cloud environment there will usually be a preferences pane where you can set your environment variables, some are even set for you. But what about local development? You could set them in your system settings, your dev computer is, after all, an Execution Environment in itself. Setting the variables directly like this is tedious, however and can quickly get out of hand when you work on multiple projects.

There's a better, more flexible way; lets learn about…

.Env Files

A .env file is a text file that lives on your machine where you define key-value pairs. This file is then loaded by the Node runtime and the values are made available as environment variables. You can have a file for each project so you don't have to worry about mixing secrets or changing system variables when you change project. When you need a new variable you add it to the file and rerun your script.

Getting started won't take much but, these are your secrets we're talking about. You can, very easily, accidentally share them with the world. There are plenty of stories in dev and security circles of sites with publicly published encryption keys, admin log-ons, etc. proper care should be taken from the start.

So with that in mind…

Let's Use Some Secrets

Create a new folder for your project as you normally would. For me this means running the following in my terminal:

If you want to follow along with my code, you can clone the repository from Github. You'll just need to add your own .env file. No promises, but I may even add more examples as they come up.

Next is a very important file for guarding our secrets. Whether you use Git or not it's a good habit to create this file before you add a single secret to your project. This helps to prevent you or someone else who works on your project from sharing your secrets in public and preserving them in your project's Git history. If you're not familiar with Git, I'm talking about the Git Ignore file. This file tells Git – if it's ever initialised in your project – to ignore specific files. In this case we want Git to ignore our .env files.

Create the file .gitignore in the root of your new project and add the following content:

With that file saved in our root we're now safeguarded against the very common accident of committing our secrets to a code repository, we've even covered several formats for the file (though it's recommended to only ever have the one .env file). With that peace of mind we can set up and start using our .env files.

Using the DotEnv Package

I mentioned before that the .env file needs to be loaded into the node runtime. To do that I use a package named DotEnv. DotEnv is a well-maintained module that will load our .env file environment variables and make them available to our node process.

Install DotEnv as a Dev dependency now using either npm (what I'm using if you're following me) or yarn.

Now we need to set up our Node package script so that DotEnv is run and parses our environment variables. Open your package.json file and modify your scripts to look like this:

What we're doing in our 'dev' script is pre-loading dotenv and running the config before node runs our entry script. I prefer this method because we don't need to add any code that will only be used on our dev environment. Remember, these environment variables will be set by other means on our deployment environments; we only need DotEnv for our local development.

Creating our .Env File

With our .gitignore insurance plan in place and DotEnv ready to use, let's create our .env file and share some secrets.

Create the file named .env in your project root and add some secrets like so:

You can use string values directly, single quotes, double quotes, etc. For more on how the file is parsed you should review the parsing rules. You can have multi-line values, whitespace, empty rules, and a lot more.

With our .env file created now we can use these super-secret keys in our code.

Retrieve our Secrets

Let's create our 'index.js' file and read our variables in. Create the file and add the following code passages. We'll explain each one as we go.

No secrets slipping out here in the code but run npm run devand the truth is revealed.

So what happened here? DotEnv ran before our script and loaded all of our key-value pairs into an object set to the env property of our Node process. In the first line we access our chosen key and assign it to a constant. We now have the value of our secret ready to write out (just for our demonstration) without having to reveal it in our code. It stays safely in our .env file.

Let's add some more. Since process.env is an object, we can use destructured assignment to access a couple of our environment variables at once like so:

Run npm run dev again and 'Ta-Da' we've aired more of our secrets out in the open.

One more example: what if we need a default value for one of our environment files? A common example is the PORT value. On cloud environments especially the PORT that your script runs on is decided by the environment and the PORT value is set for you but if it's not, you still want a port number for your process to listen on. Let's do that like so:

This is known as 'short-circuit evaluation' we're using the logical 'OR' operator to assign the second operand if and only if the first is null (or falsy). So if we set the PORT .env variable, which we have, our script will report that it is listening on that value, 8081 in our case. If we didn't set the value, it would listen on 3000 instead.

That's it for the basics.

So, When Would I Use This?

When you're working in Node there are a lot of places you'll encounter a need for secrets.

Databases

To connect to a Database you'll need to provide a user, password, and a database name at a minimum. No one needs to know what those values are and having them get accidentally published to your public repository or out there in plain text on your server could be a disaster.

API Keys

Many times when we want to use an API for our development we will have an account and keys from the API provider. The API may limit our rates or we may be charged by usage. Anyone with our keys can make their own requests as us! Publishing API keys is a really good way to rack up a huge bill with your API provider. Even worse - the API could have full access to sensitive information like your collection of heirloom dad-jokes.

Environment Configuration

When you deploy your code to the cloud certain environment variables are going to be provided for you. PORT is a common example. Your provider will usually document what variables they make available for you and will also give you space to set your own. If you can set a default for these you don't really need to add them to your local .env file (reduce the risk) but now you know how to set them if you need them.

Configuration Without a Release

Not everything in your .env has to be a secret. One neat use for your environment variables is being able to change a setting in your code without needing to deploy a new version to your server. For example you could add an on/off switch for the new feature you're about to release. The code is deployed but with a flag tied to an environment variable. When marketing sends out the email to users you don't need to start a deployment, you just set your switch in your server environment and your new feature comes to life.

There are a lot of things that can be done here (but beware of the fact that you're not able to provide for version control on these settings).

That's It

So, first, I'm going to say it again. Before adding a .env file and any secrets add a .gitignore file with .env. Even if you don't use git initially. Protect yourself from making the same mistake all the rest of us have.

I hope you've got an overview of how to use Environment Variables in Node. There's a lot more out there you can dig into, the DotEnv README is a good start, but this should be enough to get you started and give you some basic protection against broadcasting your secrets to the world.

Recapping the Steps

Add a .gitignore file to your project root with a rule to ignore '.env' at a minimum
Install DotEnv (or a similar module) as a dev dependency and add the preloader to your package script
Add your secrets as key-value pairs in your .env file

Your environment variables are waiting in process.env. Good luck, and happy developing!

Zap your Network: Leverage Automated Integrations for Better Reach

Aoibhe Wilson 🇨🇭 — Thu, 09 Apr 2020 22:11:22 +0000

When you're trying to get ahead in the tech sector building your network and raising your visibility are two things that can go a long way to improving your odds. I've not been particularly good at it but my network has managed to save my career several times over the years.

Writing blog posts is one of the many tools at your disposal. To get the most out of your writing efforts you'll want to get an audience. If you're reading this on Dev.to then you're already in a good place to start but where should you spread the word?

Sharing for Reach

Part of this involves knowing your audience, where will you reach your peers, recruiters, etc. For me, as an older frontend developer, I'm looking at Twitter with its active tech community (and the bulk of my network), Medium (paywalled, but added reach), and LinkedIn (with a lot of my past employers and coworkers, and a steady stream of recruiters). Those two, I feel will get me started on building some reach. To help me track though, I'll also want to link things to bit.ly so I can get some stats on who is reading and where they're finding it. The thing is, now I have to write this post, push the link to bit.ly, then share it to Twitter, and share it to LinkedIn. If finding time isn't your strong point (it definitely isn't mine) the idea of spending an extra ten minutes per post may not fly.

Enter Zapier

If you aren't familiar with it Zapier is an online tool to help you automate things using connections between services. They offer a list of integrations as triggers and actions. When something happens on your trigger, they set off the appropriate action.

From their homepage, once you're logged in, you can chose a service to integrate as your trigger, and one to integrate as your action. There are also a number of prebuilt integrations. We'll walk through the steps and, if everything goes well, this post will be my first network broadcast.

The Integration we're going to build here depends on multi-step Zaps and Formatters. I learned after writing that these are Professional features that will cost you $19/month after a free trial. The free plan exists but limits you to a single step and no Formatters.

Integrating with Dev

So, if you enter Dev into the first box... we get nothing. It turns out there's no integration with Dev yet but, no worries, Dev provides us with an RSS feed that we can use with Zapier's RSS Integration.

Next let's chose Bitly as our connecting app. Bitly will let us make short URLs for the post to share to our other services. Choose Bitly from the list and then we'll chose our triggers.

Zapier offers us a few triggers to use with our RSS feed. We want to take an action whenever there's a new item in our feed. For Bitly, we want to create a BitLink. Chose 'Make a Zap' and we'll configure the details of our actions.

First up we need our Dev details. The customisation for the RSS feed requires a feed URL for your posts and a trigger in the feed that signals a new item has been posted. Your feed URL is created automatically by Dev in the format https://dev.to/feed/username. Once you have a published a post you can view the URL in your browser and see the JSON structure generated. For the new item trigger let's stick with the recommended option and base our new item event off of a new GUID or URL showing u in our feed. The Dev feed doesn't require a username or password so we can leave those fields blank. Hit 'Continue' and you'll have the opportunity to test the connection. As long as you have a published post you should be good to go.

Next up is our Bitly configuration. The first step is to sign into Bitly and then give the Zapier integration permission to access your account. Once you're signed in we move on to the customisation for the links we'll create. Zapier provides us with four properties to manage the creation of our Bitly links: group, link, title, and domain. For my purposes, I'm currently only interested in link and title, the other two are a bit more advanced. So how are we automating things if we have to type in the URL we want to convert and the title from the post? Zapier takes care of this for us by storing the data from each step in the process. We can set the link and the title fields by clicking in the field and then choosing from the list of properties Zapier found in our RSS items. So let's chose our post link for link, and title for title. Now Zapier will pick up the data from each new post and send these properties to Bitly. Run the test to confirm your setup then select 'done editing'. If everything is good at this point you should be able to publish a Dev post and see a new BitLink in your Bitly dashboard.

A quick note on the tests, they will actually take the action you're setting up so you may need to go in and delete a new link or post after you finish the test.

Integrating LinkedIn

Now we have Bitly links, so what do we do with them next? You could manually share the link around but we're here to automate. One of the advantages of Zapier is that they allow you multi-step integrations. Take a look at the customisation screens we have so far and you'll see an add button between and after your steps.

We want the data from the RSS and Bitly so let's add our LinkedIn integration after the Bitly step.

Like with Bitly, LinkedIn will ask us to log in. If you've logged in before then you can chose your account from the list. Once you log in grant the integration the requested permissions. Now, just like with Bitly we can fill out the customisation data for LinkedIn. We'll be making a post that links back to our Dev post. For the comment field, chose something generic that will accompany each story your publish. I went with I've just published a new post on Dev! Give it a look!. Set the audience visibility to your liking. All of my personal data was stolen from my security clearance paperwork in a massive data breach a few years ago, so I'm choosing 'Anyone'. The remaining fields are for the post itself. Like before lets chose the data from the previous steps to fill out the title, description, and link. There's an option for an image too, but at the moment my RSS feed doesn't include a field for it. You could use something generic here if you wanted though. Hit continue and off to the test phase.

Managing your Data

Well, that's not ideal… The RSS feed here puts the content of your article into the description field. This isn't quite what we want for LinkedIn which is more of a preview of the article. We need to shorten things down. Fortunately Zapier gives us some built in tools for that. Let's add a step after our RSS feed. Choose 'Formatter' from the Helpers section. We'll choose 'Text' as our Action Event. This gives us a few options but, I'm going to use 'Truncate' for this task. The customisation options here take the text from out new item and a max length in characters. There's also an option to add an ellipsis to the end. With our options selected we can test, and then update our LinkedIn post to contain our truncated description now listed under Text in the inputs.

Good so far?

From this point, our remaining integrations are going to follow the same pattern. I'll cover the rest of my integrations here but if you feel comfortable enough to go off and try the rest yourself, give it a go. There are other helpers you can add including conditional steps so you could, for example only pass the posts to LinkedIn if the post met a certain condition. You could set up Zaps for different channels and audiences. The possibilities are endless.

Twitter Integration

Still here? You're the best! Let's move onto our next integration. Go ahead and click the add button after the LinkedIn integration. We'll chose 'Create Tweet' for our Action Event. Next we need to log into Twitter and connect our account. Just a head's up, Zapier is going to ask your permission to send you updates here. The customisation options here provide us a tweet message, an image, and an option to shorten links. Zapier suggests here you may want to use an intermediate step to transform your earlier data, for example, to shorten your post description. Note that you can mix and match text and data to craft your message. Test, test, and it's onto the next one.

Medium Integration

The Medium integration is our first integration where we'll be duplicating our whole post so there's some consideration we need to take to make sure we aren't tanking our SEO. Luckily the Medium Integration has us covered.

Let's go with 'Create Story' as our Action Event. Go ahead and link your Medium account next. In the customisation options, we can see we a few more customisation options than our other integrations. We can go ahead and plug in the title from our RSS feed. For format, we need to use HTML. In content we can insert our RSS feed's RAW description (the plain description strips out the HTML tags you need to format the Medium post). I'm going to make an addition to mine by adding a paragraph mentioning that the story was originally posted in Dev (pardon my slow typing, still getting used to a Swiss keyboard). The final important field to add here is the Canonical Url. We want this to be the original URL from Dev. This way the search engines will know where the original content resides and won't penalise us for duplicate content. There are some other options here for fine-tuning who sees your posts and how they're added. Once we're happy, test this step and we're all set.

Throw the Switch

With all our steps complete, we're ready to turn on the Zap! Toggle the switch at the bottom of the page and you're all set. The next post we publish should broadcast to our whole network. Happy posting!

Edited, 10 April: Earlier I used 'description' in my Medium integration which resulted in an unformatted wall of text at Medium. I reached out to Zapier support and Owari helped me out by pointing out the 'raw description' field in the RSS. Since I started this out with an empty RSS feed, I hadn't seen that field when I first set up my steps. Thanks Owari!