How do I set up a Local HashiCorp Vault Cluster in 4 command lines?

What I'd like to give you is a Vault environment you can get up and running with in less than 25 minutes, with 4 command lines!*

• Disclaimer: Those 4 command lines work only after installing the latest versions of Git, Virtualbox, and Vagrant.¹

How does that sound? A fully operational set of 3 Vault servers, with a back-end protected by ACLs?

Have you done this in just 4 command lines, before?

If not, then, well, let's go!

The diagram shows 3 Virtual Machines, on a "host-only" network. That means that, rather than being a part of the internet, they're on the equivalent of your local wireless network confined to your laptop. You know how there are Personal Pizzas? This is a Personal Network. And you don't have to worry about any of that stuff, because VirtualBox and Vagrant will do the work for you, as we'll see.

The names of the Virtual Machines, indicated in the diagram by the white rectangles, are instance5, instance6, and instance7. Each one has both Consul and Vault installed on it. The Consuls are configured to form a cluster, which acts as a storage and High Availability "backend" for the Vault servers.

Make sense so far? Good. If not, hey, there's always the comment section.

First, let's get some prerequisites out of the way.

If you get stuck with the prerequisites, tools to install, or downloading the code, please have a look at the resources on the internet, or invest a few minutes to vent in the comments.

Especially for Vagrant, the Getting Started guide takes about 30 minutes once you have Vagrant and Virtualbox installed: https://www.vagrantup.com/intro/getting-started/index.html

If you get an error with Vault working improperly, though, make a post ASAP on the Vault Discussion Group, because they can help tell you whether it's a bug or not.

OS-Specific Prerequisites

• MacOS: OSX 10.13 or later
• Windows: Windows must have Powershell 3.0 or later. If you're on Windows 7, I recommend Windows Management Framework 4.0, because it's easier to install

Install VirtualBox and Git

The first things to get are Virtualbox and Git, since they will let you run a "Virtual Machine," and allow you to get code for this walk-through, respectively.

Install Vagrant

The first tool is Vagrant, a program that helps with all the little details of development environments. Think of it like Infrastructure as Code, but for Virtual Machines, Containers, and automating the boring parts of setting up the programs you have to run on them to do development work.

Install Tools

1. Make sure you have Git installed
2. Install the latest version of Vagrant (NOTE: WINDOWS 7 AND WINDOWS 8 REQUIRE POWERSHELL >= 3)
3. Install the latest version of VMWare or Virtualbox

Reason for this Article

While you're waiting for all this stuff to install, here's a note about why I'm writing this.

If you search for how do I set up a hashicorp vault cluster?, you'll get some results that cover the concepts, some that discuss how to use Vault's API, and some that discuss how to enable Vault Enterprise features in general.

For instance, HashiCorp has an excellent Deployment Guide, here:

Vault Deployment Guide

And a solid Reference Architecture, here:

Vault Reference Architecture

But there are some pre-requisites for these. For instance, in the Deployment Guide, you would first need to know how to install a Vault Server, and know how to network containers or other infrastructure to run it on, before beginning the guide.

And the Reference Architecture is more of an overview of an example architecture, rather than getting into the nitty gritty of what kinds of commands to run that aren't directly related to Vault or Consul.

These are also, appropriately, not prescriptive or opinionated about what kind of networking stack or other infrastructure with which to set up your Vault cluster.

I wrote this guide for people who, I assume, already know a little about scripting, Git, configuring new SSH connections, installing software, and Virtual Machines, because these are hard to explain and have much better resources elsewhere.

Anyway, that's why I'm writing about an easy Vault cluster setup. Install a couple of cross-platform, Open-Source tools (Vagrant, VirtualBox, and Git), run a couple of universal commands, wait 10 minutes, and voila, your new Vault Cluster is ready to play.

Screencast Example

Here's a screencast example of the setup, if you'd like to open it on the side while running through this guide:

Download the Code for the Vault Cluster Setup

Related Vendor Documentation Link: https://help.github.com/articles/cloning-a-repository

git clone https://github.com/v6/super-duper-vault-train.git

Use this Code to Make a Vault Cluster

Related Vagrant Vendor Documentation Link: https://www.vagrantup.com/intro/index.html#why-vagrant-

1. cd super-duper-vault-train
2. vagrant up ## NOTE: You may have to wait a while for vagrant up to complete, and there will be some "connection retry" errors for a long time before a successful connection occurs, because the VM is booting. Make sure you have the latest version, and try the Vagrant getting started guide, too

NOTE: You've made it this far. Now is a great time to find your local Philz or Pete's or Starbucks to get a well deserved <insert favorite snack or beverage>, because the vagrant up process... well... it takes a while. Let Vagrant do the work while you grab a perk.

1. vagrant status
2. vagrant ssh instance5 After you ssh to that VM named instance5, you'll see your command prompt change to show vagrant@instance5. You can also vagrant ssh to other VMs listed in the output of vagrant status.
3. You can now use Vault or Consul from within the VM for which you ran vagrant ssh. For example, try the commands consul members or vault status from within any of your VMs.

Vault

Explore the Vault Cluster

ps -ef | grep vault ## Check the Vault process (run while inside a Vagrant-managed Instance)

ps -ef | grep consul ## Check the Consul process (run while inside a Vagrant-managed Instance)

vault version ## Output should be Vault v0.10.2 ('3ee0802ed08cb7f4046c2151ec4671a076b76166')

consul version ## Output should show Consul Agent version and Raft Protocol version

The Vagrant boxen have the following IP addresses:

192.168.13.35

192.168.13.36

192.168.13.37

Both Vault and Consul are running on each of them.

Vault is on port 8200.

Consul is on port 8500.

Open the Links in Tabs

http://192.168.13.35:8200 (Vault)

http://192.168.13.35:8500 (Consul)

http://192.168.13.36:8200 (Vault)

http://192.168.13.36:8500 (Consul)

http://192.168.13.37:8200 (Vault)

http://192.168.13.37:8500 (Consul)

Start Vault Data

Related Vendor Documentation Link: https://www.vaultproject.io/api/system/init.html

Start Vault.

Run this curl command on one of the Vagrant-managed VMs, or somewhere on your computer that has curl installed.

    curl -s --request PUT -d '{"secret_shares": 3,"secret_threshold": 2}' http://192.168.13.35:8200/v1/sys/init

Unseal Vault

Related Vendor Documentation Link: https://www.vaultproject.io/api/system/unseal.html

If successful, this process will unseal the Vault at 192.168.13.35:8200. You can use the same process for 192.168.13.36:8200 and 192.168.13.37:8200.

1. Use your unseal key to replace the value for key abcd1430890..., and run this on the Vagrant-managed VM.

    curl --request PUT --data '{"key":"abcd12345678..."}' http://192.168.13.35:8200/v1/sys/unseal

1. Run that curl command again. But use a different value for "key":. Replace efgh2541901... with a different key than you used in the previous step, from the keys you received when running the v1/sys/init endpoint.

    curl --request PUT --data '{"key":"efgh910111213..."}' http://192.168.13.35:8200/v1/sys/unseal

Non-Vagrant

Please refer to the file PRODUCTION_INSTALLATION.md in the repository we cloned earlier.

Codified Vault Policies and Configuration

To Provision Vault via its API, please refer to the
provision_vault folder in the repository we cloned earlier.

It has data and scripts.

The data folder's tree corresponds to the HashiCorp Vault API
endpoints, similar to the following:

https://www.hashicorp.com/blog/codifying-vault-policies-and-configuration#layout-and-design

You can use the Codified Vault
Policies and Configuration
with your initial Root token, after
initializing and unsealing Vault,
to configure Vault quickly via its API.

The .json files inside each folder
correspond to the payloads to send to Vault
via its API, but there may also be .hcl,
.sample, and .sh files for convenience's sake.

Further reading for manual setup:

https://medium.com/rigged-ops/building-a-local-hashicorp-vault-cluster-5575fe322a17

---

1. Slippery when wet, all rights reserved, even the contradictory ones. Blades are extremely sharp. This project may contain FOSS code. Unauthorized use or reproduction of material on this web site is strictly required and subject to congratulations. Do Not Eat. 🥽 Safety Goggles Recommended. Offer valid at participating locations, which in this case means all locations. Intended to be combined with other offers or somehow cleverly duplicated. Limit one Vagrant per visit. Cash value 1/100000000th of a Bitcoin, but don't hold me to it because who knows what that means these days. This is the fine print, why are you still reading this? Really, this is getting silly, go code. ↩

Comments

[Harvey Jones]

I was looking for an article on getting Vault installed on ubunto. Big new I didn't find any on the first page so I just clicked on yours. Although this article is for MacOs and windows but it kinda helped a little.

I then continued my search and on the 5th page of google I found this blog.carbonteq.com/vault-installat... and I was only able to understand it because I had read your article so ... passive thanks... I guess.

[🦄N B🛡]

This is designed assuming that you're on your own laptop. Because everything runs on Vagrant, and Vagrant will create VirtualBoxVMs for you, these instructions will work on any operating system on which you can install Vagrant and VirtualBox.

[Harvey Jones]

Thanks. Although my setup is now working but anything goes wrong next time I will get virtual machine and follow the instructions.

[Joe Hobot]

Thats bit more than 4 lines :) but ok .. I get it ;)

[🦄N B🛡]

// , Technically there are only 4 commands until the cluster comes up. But you're correct that just referring to those means being a little over-optimistic, because the installation of the prerequisites Git, VirtualBox, and Vagrant isn't necessarily a matter of CLI commands.

[Joe Hobot]

Just teasing :)
Good post tho ..

[rrrrnaveen]

Thanks a lot NB. It was going fine and my system crashed. This is my beginning to explore Hashi Vault and this is just awesome, the way you set this up.
I did a big mistake, forgot to copy keys & root token.
System rebooted and its in sealed state.

When I first this command, it displayed all keys & root key(This where I should have saved it)
curl -s --request PUT -d '{"secret_shares": 3,"secret_threshold": 2}' 192.168.13.35:8200/v1/sys/init

Now Iam stuck in sealed status. Please suggest me

[rrrrnaveen]

Should I destroy and recreate it, perhaps???

[🦄N B🛡]

// ,

And

"Every Network is a Personal Network if you try hard and believe in yourself." --Nathan Basanese

[Troy]

Nice write up. Link to the ASCII in your console?? 🦄