Collect Logs from a Self-Managed Kubernetes Cluster into Tencent Cloud CLS

Self-managed Kubernetes clusters do not automatically inherit the console-driven log collection experience of managed TKE clusters. The source article explains how Tencent Cloud CLS can collect logs from a self-managed Kubernetes cluster by using a Kubernetes CRD named LogConfig and three components: Log-Provisioner, Log-Agent, and LogListener.

For TKE users, the source points to the TKE log-collection document and console path. This article focuses on the self-managed Kubernetes path.

Prerequisites

The source article lists four prerequisites:

• Kubernetes cluster version 1.10 or later;
• CLS enabled, with a logset and log topic already created;
• the CLS topic ID, topicId;
• the region endpoint for the log topic, CLS_HOST;
• Tencent Cloud API credentials required for CLS-side authentication: TmpSecretId and TmpSecretKey.

How the collection architecture works

The Kubernetes deployment includes one custom resource and three runtime components.

Component	Role from the source article
LogConfig	Defines where logs are collected from, how they are parsed, and which CLS log topic receives them.
Log-Provisioner	Synchronizes the log collection configuration defined in LogConfig to the CLS side.
Log-Agent	Watches LogConfig and container changes on nodes, then calculates the real host-machine path of container log files.
LogListener	Collects matching log files from the host path, parses them, and uploads them to CLS.

Deployment flow

The source article uses this sequence:

1. Define the LogConfig resource type with a CRD.
2. Define a LogConfig object.
3. Create the LogConfig object.
4. Configure the CLS authentication ConfigMap.
5. Deploy Log-Provisioner.
6. Deploy Log-Agent and LogListener.
7. Search the collected logs in the CLS console.

Step 1: define the LogConfig CRD

Using /usr/local/ on the master node as the example path:

wget https://mirrors.tencent.com/install/cls/k8s/CRD.yaml
kubectl create -f /usr/local/CRD.yaml

Step 2: define the LogConfig object

Download the sample declaration:

wget https://mirrors.tencent.com/install/cls/k8s/LogConfig.yaml

The source explains that LogConfig.yaml has two main parts:

Section	Purpose
clsDetail	Defines the log parsing format and target CLS topicId.
inputDetail	Defines the log source: where the logs are collected from.

Replace clsDetail.topicId with the real topic ID created in CLS.

Supported parsing formats

Single-line full text

Use this when one line is one complete log entry. CLS stores the line in __CONTENT__ and does not extract fields.

apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig
spec:
  clsDetail:
    topicId: xxxxxx-xx-xx-xx-xxxxxxxx
    logType: minimalist_log

Example collected output:

__CONTENT__: Tue Jan 22 12:08:15 CST 2019 Installed: libjpeg-turbo-static-1.2.90-6.el7.x86_64

Multi-line full text

Use this for logs such as Java stack traces. The source uses a beginning-line regex so a timestamped line starts a new log event, and later stack-trace lines are appended to the current event.

apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig
spec:
  clsDetail:
    topicId: xxxxxx-xx-xx-xx-xxxxxxxx
    logType: multiline_log
    extractRule:
      beginningRegex: '\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}\s.+'

Single-line full regex

Use this when a complete single-line log should be parsed into multiple key-value fields.

apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig
spec:
  clsDetail:
    topicId: xxxxxx-xx-xx-xx-xxxxxxxx
    logType: fullregex_log
    extractRule:
      logRegex: '(\S+)[^\[]+(\[[^:]+:\d+:\d+:\d+\s\S+)\s"(\w+)\s(\S+)\s([^"]+)"\s(\S+)\s(\d+)\s(\d+)\s(\d+)\s"([^"]+)"\s"([^"]+)"\s+(\S+)\s(\S+).*'
      beginningRegex: '(\S+)[^\[]+(\[[^:]+:\d+:\d+:\d+\s\S+)\s"(\w+)\s(\S+)\s([^"]+)"\s(\S+)\s(\d+)\s(\d+)\s(\d+)\s"([^"]+)"\s"([^"]+)"\s+(\S+)\s(\S+).*'
      keys:
        - remote_addr
        - time_local
        - request_method
        - request_url
        - http_protocol
        - http_host
        - status
        - request_length
        - body_bytes_sent
        - http_referer
        - http_user_agent
        - request_time
        - upstream_response_time

Multi-line full regex

Use this when one structured log event spans multiple lines and fields still need to be extracted.

apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig
spec:
  clsDetail:
    topicId: xxxxxx-xx-xx-xx-xxxxxxxx
    logType: multiline_fullregex_log
    extractRule:
      beginningRegex: '\[\d+-\d+-\w+:\d+:\d+,\d+\]\s\[\w+\]\s.*'
      logRegex: '\[(\d+-\d+-\w+:\d+:\d+,\d+)\]\s\[(\w+)\]\s(.*)'
      keys:
        - time
        - level
        - msg

JSON logs

For JSON logs, CLS extracts first-level keys as fields.

apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig
spec:
  clsDetail:
    topicId: xxxxxx-xx-xx-xx-xxxxxxxx
    logType: json_log

Delimiter logs

For delimiter logs, define the delimiter and the keys that map to each segment.

apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig
spec:
  clsDetail:
    topicId: xxxxxx-xx-xx-xx-xxxxxxxx
    logType: delimiter_log
    extractRule:
      delimiter: ':::'
      keys:
        - IP
        - time
        - request
        - host
        - status
        - length
        - bytes
        - referer

Supported Kubernetes log sources

The source article gives three source types.

Container stdout

Collect all container stdout logs in the default namespace:

apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig
spec:
  inputDetail:
    type: container_stdout
    containerStdout:
      namespace: default
      allContainers: true

Collect stdout from the ingress-gateway deployment in the production namespace:

apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig
spec:
  inputDetail:
    type: container_stdout
    containerStdout:
      allContainers: false
      workloads:
        - namespace: production
          name: ingress-gateway
          kind: deployment

Collect stdout from pods labeled k8s-app=nginx:

apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig
spec:
  inputDetail:
    type: container_stdout
    containerStdout:
      namespace: production
      allContainers: false
      includeLabels:
        k8s-app: nginx

Container files

Collect /data/nginx/log/access.log from the nginx container in the ingress-gateway deployment:

apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig
spec:
  topicId: xxxxxx-xx-xx-xx-xxxxxxxx
  inputDetail:
    type: container_file
    containerFile:
      namespace: production
      workload:
        name: ingress-gateway
        type: deployment
      container: nginx
      logPath: /data/nginx/log
      filePattern: access.log

Collect the same file path from pods with label k8s-app=ingress-gateway:

apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig
spec:
  inputDetail:
    type: container_file
    containerFile:
      namespace: production
      includeLabels:
        k8s-app: ingress-gateway
      container: nginx
      logPath: /data/nginx/log
      filePattern: access.log

Host files

Collect every .log file under /data on the host:

apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig
spec:
  inputDetail:
    type: host_file
    hostFile:
      logPath: /data
      filePattern: '*.log'

Step 3: create the LogConfig object

After editing LogConfig.yaml, create the object:

kubectl create -f /usr/local/LogConfig.yaml

Step 4: configure CLS authentication

Download the sample ConfigMap:

wget https://mirrors.tencent.com/install/cls/k8s/ConfigMap.yaml

Set TmpSecretId and TmpSecretKey to the API key ID and API key value used for CLS authentication. Then create it:

kubectl create -f /usr/local/ConfigMap.yaml

Step 5: deploy Log-Provisioner

Log-Provisioner discovers and watches the log topic ID, collection rule, and file path in LogConfig, then synchronizes that configuration to CLS.

wget https://mirrors.tencent.com/install/cls/k8s/Log-Provisioner.yaml
kubectl create -f /usr/local/Log-Provisioner.yaml

Before applying the file, set the CLS_HOST environment variable in Log-Provisioner.yaml to the endpoint of the target CLS topic region.

Step 6: deploy Log-Agent and LogListener

The source separates responsibilities:

• Log-Agent pulls log-source information from LogConfig and calculates the absolute host path for container logs.
• LogListener collects and parses files from that host path, then uploads them to CLS.

wget https://mirrors.tencent.com/install/cls/k8s/Log-Agent.yaml
kubectl create -f /usr/local/Log-Agent.yaml

If the host Docker root is not /var/lib/docker, update the Log-Agent.yaml volume mapping. The source screenshot shows /data/docker mounted into the container as an example.

In English, the highlighted YAML is saying: when Docker data lives under /data/docker on the host, mount that path into the Log-Agent container so the agent can map container log files back to their real host locations.

Step 7: verify logs in the CLS console

After CRD creation, LogConfig creation, authentication, Log-Provisioner, Log-Agent, and LogListener are all deployed, open the CLS log search page for the target topic.

Kubernetes logs have been collected and can be searched in the CLS console. The top area is the histogram over time; the lower area displays matching raw log events.

Deployment checklist

• Confirm Kubernetes version is 1.10 or later.
• Create a CLS logset and log topic, then record topicId.
• Find the right regional CLS_HOST.
• Prepare TmpSecretId and TmpSecretKey.
• Create the LogConfig CRD.
• Choose a parsing format: single-line text, multi-line text, full regex, multi-line full regex, JSON, or delimiter.
• Choose the log source: container stdout, container file, or host file.
• Apply LogConfig, ConfigMap, Log-Provisioner, and Log-Agent.
• If Docker is not rooted at /var/lib/docker, mount the actual Docker root path into the agent.
• Verify collected logs in CLS search.