Integrate K6 with InfluxData

In this post we will describe the process of integrating k6.io with InfluxData (InfluxDB cloud). We recently integrated load tests with k6 in our development process in GWI and our journey just begun.

InfluxDB v1 is already supported from k6 out of the box to persist metrics but support on v2 is not yet fully covered. Let's start by reviewing how to deploy a local instance of v1 and integrate with k6 for the sake of introduction.

InfluxDB V1

First we need to create our grafana-datasource.yaml file to provision Grafana, with the following configuration

apiVersion: 1
datasources:
  - name: myinfluxdb
    type: influxdb
    access: proxy
    database: k6
    url: http://influxdb:8086
    isDefault: true

Let's create also the docker-compose file that will do the following:

• Setup InfluxDB v1
• Setup Grafana with predefined dashboards
• Run k6 tests and direct output to InfluxDB

networks:
  k6:
  grafana:
services:
  influxdb:
    image: influxdb:1.8 # Version 2.x introduces some breaking compatibility changes. K6 support for it comes via an extension
    networks:
      - k6
      - grafana
    ports:
      - "8086:8086"
    environment:
      - INFLUXDB_DB=k6

  grafana:
    image: grafana/grafana:latest
    networks:
      - grafana
    ports:
      - "3000:3000"
    environment:
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_BASIC_ENABLED=false
    volumes:
      - ./grafana-datasource.yaml:/etc/grafana/provisioning/datasources/datasource.yaml
  k6:
    image: grafana/k6:latest
    networks:
      - k6
    ports:
      - "6565:6565"
    environment:
      - K6_OUT=influxdb=http://influxdb:8086/k6
    volumes:
      - ./dist:/scripts

Running our tests we can easily review and analyze through the Grafana Dashboards

InfluxData

InfluxData is the cloud version of InfluxDB and it is based on V2. Main difference is that is organized in buckets and we can query data based on Flux Query.

The current Grafana dashboards are mainly developed to support V1 so how can we support backwards compatibility without creating our own dashboards?

First thing first lets review how the docker-compose file is changes to persist data to InfluxData:

networks:
  k6:
  grafana:
services:
  grafana:
    image: grafana/grafana:latest
    networks:
      - grafana
    ports:
      - "3000:3000"
    environment:
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_BASIC_ENABLED=false
    volumes:
      - ./grafana-datasource.yaml:/etc/grafana/provisioning/datasources/datasource.yaml
  k6:
    build: .
    networks:
      - k6
    ports:
      - "6565:6565"
    environment:
      - K6_OUT=xk6-influxdb=<influxdb_url>
      - K6_INFLUXDB_ORGANIZATION=<influxdb_org_id>
      - K6_INFLUXDB_BUCKET=Platform_Performance_Tests
      - K6_INFLUXDB_INSECURE=true
        # NOTE: This is an Admin token, it's not suggested to use this configuration in production.
        # Instead, use a Token with restricted privileges.
      - K6_INFLUXDB_TOKEN=<influxdb_token>
    volumes:
      - ./dist:/scripts

The change appararently is the output and here we use the xk6 binary xk6-influxdb
Provide proper environment variables retrieved from your InfluxData configuration.

Now we can actually persist our data to InfluxData lets see how to use Grafana Dashboards from V1 with backwards compatibility.

The InfluxDB 1.x data model includes databases and retention policies. InfluxDB Cloud replaces databases and retention policies with buckets. To support InfluxDB 1.x query and write patterns in InfluxDB Cloud, databases and retention policies are mapped to buckets using the database and retention policy (DBRP) mapping service.

Our dashboards are using InfluxQL (v1) to query data so create a datasource with InfluxQL

Now the only thing remain is to add custom HTTP header Authorization and provide as value the InfluxData token to make appropriate connection. So we are good to go and use the predefined dashboards from grafana site

Conclusion

If someone is experienced enough using Flux Query you do not need to worry about backwards compatibility from v2 to v1 API. But yet again i wanted to share my experience trying to move to InfluxData and how we managed to migrate our Grafana Dashboards without any extra handling!