Consul service discovery with Nomad and Fabio

#consul #nomad #hcl

the problem

When you run multiple containers of the same image, each gets a random port. Consul solves this - services register themselves and clients query by name instead of host:port.

run Consul

asdf plugin-add consul
asdf install consul 1.19.0
asdf local consul 1.19.0

Or through Nomad using raw_exec:

job "consul" {
  datacenters = ["dc1"]

  group "consul" {
    task "consul" {
      driver = "raw_exec"

      config {
        command = "consul"
        args    = ["agent", "-dev"]
      }

      artifact {
        source = "https://releases.hashicorp.com/consul/1.19.0/consul_1.19.0_linux_amd64.zip"
      }
    }
  }
}

UI at http://127.0.0.1:8500/ui

register a service

Add a service block to any Nomad task:

task "server" {
  driver = "docker"

  service {
    name = "http-echo"
    port = "http"
    tags = ["http-echo", "urlprefix-/echo"]

    check {
      name     = "alive"
      type     = "http"
      path     = "/"
      interval = "10s"
      timeout  = "2s"
    }
  }
}

internal load balancing with Fabio

Fabio reads urlprefix-/path tags from Consul and routes traffic automatically.

job "fabio" {
  datacenters = ["dc1"]

  group "fabio" {
    network {
      port "lb" { static = 9998 }
      port "ui" { static = 9999 }
    }

    task "fabio" {
      driver = "docker"

      config {
        network_mode = "host"
        image        = "fabiolb/fabio:1.6.3"
        args         = ["-proxy.strategy=rr"]
      }

      resources {
        cpu    = 200
        memory = 128
      }
    }
  }
}

Routes dashboard: http://localhost:9998/routes

Once Fabio is running and your service has urlprefix-/echo, requests to :9999/echo are load balanced across all healthy instances.

# watch round-robin in action
watch -n 0.2 "curl -s http://localhost:9999/echo"

Consul KV for environment variables

Store config in Consul KV and inject it into tasks via the template block:

template {
  data = <<EOH
DB_HOST="{{ key "config/db_host" }}"
DB_PASS="{{ key "config/db_pass" }}"
EOH

  destination = "secrets/file.env"
  env         = true
}

Set values at http://127.0.0.1:8500/ui/dc1/kv or via CLI:

consul kv put config/db_host "db01.example.com"
consul kv put config/db_pass "secret"

Originally published at https://bard.sh/posts/consul_service_discovery/

DEV Community