loading...
Cover image for Kubernetes Configuration Management with Ansible (Part 2)

Kubernetes Configuration Management with Ansible (Part 2)

baptistemm profile image Baptiste Mille-Mathias ・3 min read

Follow-up of this post. We explained the directory organization and the manifest files naming convention, now let's explore the ansible part.

Ansible was chosen because I knew it a bit since 2015 when I started automating some repetitive tasks to create a Kafka cluster prototype. Ansible is a good candidate because it can talk to HTTP API and it comes with a list of k8s related modules that make easy to query or manage Kubernetes objects (they were rewritten a couple of time before being now really good).

Overview of the Code

I don't plan to review all code, but have a look of key parts of the Ansible code.

  • The cluster accept a parameter which is the target cluster, then we load the common variables file then the cluster variable file from config directory
---
- name: "Deploy manifests on cluster {{ target_cluster }}"
  connection: local
  gather_facts: false
  vars:
    authorized_cluster: ["dev", "prod"]
  hosts: localhost

  pre_tasks:
    - name: Check if the target_cluster variable was properly set
      fail:
        msg: "You need to specify the variable target_cluster with one value ({{ authorized_cluster }})"
      when: target_cluster not in authorized_cluster

    - name: Load common variables
      include_vars:
        file: configs/common.yml

    - name: "Load cluster-scoped variables for {{ target_cluster }}"
      include_vars:
        file: "config/{{ target_cluster }}.yml"
  • for the given cluster, we create a list of all yaml files from the common and $cluster directories. We will also define some variables than will be used later
---
- name: "Retrieve manifests files from common and {{ cluster }} directories"
  find:
    paths: ["{{ yaml_dir }}/common", "{{ yaml_dir }}/{{ cluster }}"]
    patterns: ['*.yaml', '*.yml']
    file_type: file
    recurse: true
  register: manifests_list

- name: "Define variable from manifests filename"
  vars:
    filename: "{{ item.path | basename }}"
    kind: "{{ item.path | basename | regex_replace('.*_(.*)_.*_.*_.*\\.ya?ml', '\\1') }}"
    name: "{{ item.path | basename | regex_replace('.*_.*_(.*)_.*_.*\\.ya?ml', '\\1') }}"
    namespace: "{{ item.path | basename | regex_replace('.*_.*_.*_(.*)_.*\\.ya?ml', '\\1') }}"
    state: "{{ item.path | basename | regex_replace('.*_.*_.*_.*_(.*)\\.ya?ml', '\\1') }}"

  set_fact:
    manifests_information: "{{ manifests_information + [{ 'filepath': item.path, 'filename': filename, 'kind': kind, 'name': name, 'namespace': namespace, 'state': state }] }}"
  loop: "{{ manifests_list.files | sort(attribute='path') }}"
  loop_control:
    label: "{{ filename }}"
  • Now we check each file honors the filename convention else we throw an error.
- name: "Check files match the naming convention"
  fail:
    msg: >
      The file {{ item.filepath }} does not observe the manifest naming convention.
  when: item.filename == item.kind or
    item.filename == item.name or
    item.filename == item.namespace or
    item.state not in [ "present", "absent" ]
  with_items:
    - "{{ manifests_information }}"
  loop_control:
    label: "{{ item.filepath }}"
  • for each file, call the k8s module, the path of of the file is passed to argument using the lookup plugin template, so the manifest can have logic and variables inside.
---
- name: "Deploy {{ item.kind }} {{ item.name }} on namespace {{ item.namespace }} to {{ item.state }}"
  k8s:
    host: "https://xxxxxx:6443"
    api_key: "{{ token }}"
    state: "{{ item.state }}"
    definition: "{{ lookup ('template', item.filepath) }}"

For the code we're done, for a stripped down version it does not require much than that to work.

Coding your Manifests

One of the feature this small playbook permits is to have simple manifest yaml file like this

---
apiVersion: project.openshift.io/v1
kind: Project
metadata:
  name: "{{ cluster-admin-ns }}"
...

but you can also using feature like Ansible variables and Jinja templating together.
For instance, I had to put a config file into a secret, which require to have the content base64 encoded. It's convenient when you can do that automatically.

  • In this example, the content for the key fluent.conf come from a file fluent.conf.j2 loaded by the template lookup (so the variable are replaced) then passed to the filter b64encode.
---
apiVersion: v1
data:
  fluent.conf: "{{ lookup('template', 'fluent.conf.j2') | b64encode }}"
kind: secret
metadata:
  name: fluent-forwarder-secret
  namespace: logging
...

with the configuration file treated as a template

# fluent.conf.j2
# As this file is interpreted as a ansible template, you can add
# jinja code inside to put some logic.

<source>
  @type  forward
  port  24224
  tag logs.openshift
</source>

<match **>
  # Send all types to Splunk
  @type splunk_hec
  hec_host {{ splunk.host }}
  hec_port {{ splunk.port }}
  hec_token {{ splunk.token }}
  index {{ splunk.index }}
</match>

so it gives once loaded something like

apiVersion: v1
data:
  fluent.conf: IyBmbHVlbnQuY29uZi5qMgojIEFzIHRoaXMgZmlsZSBpcyBpbnRlcnByZXRlZCBhcyBhIGFuc2libGUgdGVtcGxhdGUsIHlvdSBjYW4gYWRkCiMgamluamEgY29kZSBpbnNpZGUgdG8gcHV0IHNvbWUgbG9naWMuCgo8c291cmNlPgogIEB0eXBlICBmb3J3YXJkCiAgcG9ydCAgMjQyMjQKICB0YWcgbG9ncy5vcGVuc2hpZnQKPC9zb3VyY2U+Cgo8bWF0Y2ggKio+CiAgIyBTZW5kIGFsbCB0eXBlcyB0byBTcGx1bmsKICBAdHlwZSBzcGx1bmtfaGVjCiAgaGVjX2hvc3Qgc3BsdW5rLmRvbS50bGQKICBoZWNfcG9ydCA4MDg5CiAgaGVjX3Rva2VuIDM1NDM0My0zNDUzMzMtMTUyMzMKICBpbmRleCBmYWtlLWluZGV4CjwvbWF0Y2g+
kind: Secret

... Just what expect the secret API.
So I think we're good for now, I just wanted to give you an idea of what it is possible to do. This is a skeleton that can be enhanced with more check and feature.

Perhaps I'll come back with new idea.

Posted on by:

baptistemm profile

Baptiste Mille-Mathias

@baptistemm

Looking for the perfect world, I'm a Linux and Openshift engineer with strong DevOps mindset and feets on critical and production services.

Discussion

pic
Editor guide
 

Good job!
Sometimes it might be difficult to code a manifest from an already existing resource in the cluster (for instance, patching one field of a cluster operator).
I'm wondering to what extent Kustomize can help and if it could be integrated with your solution.
Cheers