KafkaUser in another namespace

We are integrating fluentbit into Kafka via Kubernetes deployed using strimzi.io and we hit our first issue.

When creating a KafkaUser it will not create the secret needed for tls in our fluent namespace

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  name: fluent
  namespace: fluent
  labels:
    strimzi.io/cluster: debezium-cluster
spec:
  authentication:
    type: tls
  authorization:
    type: simple
    acls:
      - resource:
          name: '*'
          patternType: literal
          type: topic
        operation: All
      - resource:
          name: '*'
          patternType: literal
          type: group
        operation: All
      - resource:
          type: cluster
        operation: All

it simply sits there like

kubectl get kafkauser                         
NAME     CLUSTER            AUTHENTICATION   AUTHORIZATION   READY
fluent   debezium-cluster   tls              simple

Reading up it seems that this is a long running issue and although there is a fix for java applications, it would appear you need to deploy something else to mirror the secret generated into the fluent namespace.

In one of the comments it lead us to https://config-syncer.com/docs/v0.14.7/setup/install/ which had a comment about another tool emberstack/kubernetes-reflector.

Installation is pretty straight forward.

helm repo add emberstack https://emberstack.github.io/helm-charts
helm repo update
helm upgrade --install reflector -n emberstack --create-namespace  emberstack/reflector

Then you need to annotate the KafkaUser yaml and apply it

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  name: fluent
  namespace: kakfa
  labels:
    strimzi.io/cluster: kakfa-cluster
spec:
  authentication:
    type: tls
  template:
    secret:
      metadata:
        annotations:
          reflector.v1.k8s.emberstack.com/reflection-allowed: "true"
          reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: "fluent"
  authorization:
    type: simple
    acls:
      - resource:
          name: '*'
          patternType: literal
          type: topic
        operation: All
      - resource:
          name: '*'
          patternType: literal
          type: group
        operation: All
      - resource:
          type: cluster
        operation: All

and then finally create a empty secret in the fluent namespace and annotate it to mirror the secret created previously.

apiVersion: v1                                                                                  
kind: Secret
metadata:
  name: fluent                                                                             
  namespace: fluent
  annotations:
    reflector.v1.k8s.emberstack.com/reflects: "kafka/fluent"                                                                          
type: Opaque

When completed the secret is mirrored (and maintained)

kubectl get secret fluent -n kafka
NAME     TYPE     DATA   AGE 
fluent   Opaque   5      26m

kubectl get secret fluent -n fluent
NAME     TYPE     DATA   AGE                                                                    
fluent   Opaque   5      19m

You can now reference the secret in your config.

Read how to sync the Kafka Cluster CA certificate into your namespace to enable the sync of the Kafka Cluster CA Certificate.