Using Puppet Bolt with REST APIs

#configurationmanagem #puppetbolt #devops

Puppet Bolt is an open source automation and orchestration tool. One of the common tasks in an orchestration workflow is interacting with an external system using a REST API. This could be something like creating a user or updating a record in a CMDB. A built-in task for making HTTP request calls was added in Bolt 2.30.0 and JSON output parsing was added in Bolt 2.32.0. This enables us to quickly add a step to a plan to integrate with a REST API endpoint. In this blog post we’ll walk through how to fetch data (GET request) and submit data (POST request) using the HashiCorp Vault API to show the interaction.

HashiCorp Vault Setup

HashiCorp Vault is bundled as a compiled binary that can be run easily by simply downloading the binary from the HashiCorp Vault download page. Unzip the download bundle, set the vault binary to be executable on linux or mac os x and finally run the following command to start Vault.

In this example the Vault root token is being manually set to “vaultsecret”.

vault server -dev -dev-root-token-id="vaultsecret"

Initialize the Bolt project

Ensure that the latest version of Puppet Bolt is installed before getting started.

Puppet Bolt utilizes Project directories as launching points for running Bolt operations. Create a directory for our Puppet Bolt project name restproject.

mkdir restproject

Change the working directory to restproject directory

cd restproject

Now that we have a directory for hosting our Bolt project, we need to initialize the project.

bolt project init

The command should generate output similar to that shown below if it ran successfully.

Create the Bolt YAML plan

In order to utilize plans in Bolt, we need to create a directory named plans.

mkdir plans

Now that we have our plans directory created we’ll plan out what we want to accomplish as part of our plan. We’ll keep this plan as simple as possible to show how easy it is to use Puppet Bolt to make API calls. We’ll accomplish the following tasks:

Write a static password to HashiCorp Vault (POST method)
Read the static password created from HashiCorp Vault (GET method)

Interacting with the Vault API

The API endpoint requires authentication which is why we are passing a X-Vault-Token header with the Vault root token.
The Vault API returns a JSON payload and the json_endpoint parameter for the read step has been set to true to properly parse the returned JSON.

We’ve got a plan of what we want to do and now we are ready to create the Bolt plan.

Create a file named api.yaml with the following content in the plans directory.

--------
parameters:
  targets:
    type: TargetSpec

steps:
  - name: write_password
    task: http_request
    targets: $targets
    parameters:
      method: post
      body: '{"data": {"password": "supersecurepassword"}}'
      base_url: '[http://localhost:8200/v1/secret/data/boltsecret'](http://localhost:8200/v1/secret/data/boltsecret')
      headers:
        Content-Type: application/json
        X-Vault-Token: 'vaultsecret'
  - name: read_password
    task: http_request
    targets: $targets
    parameters:
      method: get
      base_url: '[http://localhost:8200/v1/secret/data/boltsecret'](http://localhost:8200/v1/secret/data/boltsecret')
      json_endpoint: true
      headers:
        Content-Type: application/json
        X-Vault-Token: 'vaultsecret'

return: $read_password.first.value['body']['data']['data']['password']

The original JSON payload returned from HashiCorp Vault is displayed below for the purposes of understanding the JSON parsing used to return just the password value.

{
    "body": {
      "request_id": "e896d6c8-20c3-02ee-ccd7-b97778217acb",
      "lease_id": "",
      "renewable": false,
      "lease_duration": 0,
      "data": {
        "data": {
          "password": "supersecurepassword"
        },
        "metadata": {
          "created_time": "2020-10-31T17:06:54.890898Z",
          "deletion_time": "",
          "destroyed": false,
          "version": 1
        }
      },
      "wrap_info": null,
      "warnings": null,
      "auth": null
    },
    "status_code": 200
  }

Now that we’ve created our plan we can ensure that it’s recognized by Bolt by running the following command.

bolt plan show

If the plan registers properly the output should include a restproject::api entry.

aggregate::count
aggregate::nodes
aggregate::targets
canary
facts
facts::external
facts::info
puppet_agent::run
puppetdb_fact
reboot
restproject::api  
secure_env_vars
terraform::apply
terraform::destroy

With the plan registered we are now ready to run the plan by running the bolt plan run restproject::api command. The target for the plan is localhost as we want to run the API call from the machine we are running Bolt on.

bolt plan run restproject::api --target localhost

If the plan ran successfully it should have generated output similar to that displayed below.

Starting: plan restproject::api
Starting: task http_request on localhost
Finished: task http_request with 0 failures in 0.26 sec
Starting: task http_request on localhost
Finished: task http_request with 0 failures in 0.26 sec
Finished: plan restproject::api in 0.57 sec
"supersecurepassword"

If the plan completed successfully the value of the password created in the first step should be returned.

This example walked through how to interact with REST API endpoints using the http_request task that is included with Puppet Bolt.