DEV Community

JJ Dev
JJ Dev

Posted on

How do I troubleshoot issues about Amazon MSK cluster?

1. Create Topic use SASL/SCRAM

When I ran this command to create a topic. These are some errors below.

bin/kafka-topics.sh --create --bootstrap-server <endpoint> --topic <topic_name> --command-config client.properties
Enter fullscreen mode Exit fullscreen mode

[java.lang.OutOfMemoryError Java heap space]
[ERROR org.apache.kafka.common.errors.TimeoutException: The AdminClient thread has exited. Call: createTopics]

Image description

Soulation:
The error OutOfMemoryError occurred, which is a common issue when connecting to Kafka using SASL_SSL. This is not actually related to memory, but rather because the client was not configured to connect using SSL.
You should configure your admin client with --command-config <ssl.conf>


2. Public Access by SASL/SCRAM

If you are using the SASL/SCRAM or mTLS access-control methods, and want to enable public access msk. You will show the error.
Image description

Soulation:
After you set the Apache Kafka ACLs for your cluster, update the cluster's configuration.
Add Cluster configuration allow.everyone.if.no.acl.found=false
Image description

3. Consumers ACL rules

When you config allow.everyone.if.no.acl.found=false, must be set for specific consumer groups, and --group '*' cannot be used.

[org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: console-consumer-1736]
Image description

Soulation:
Now, explicitly specify that user:user_name can read the topic:demo-topic and use group:test.

bin/kafka-acls.sh --command-config client_sasl.properties --bootstrap-server <endpoint> --add --allow-principal "User:user_name" --operation Read --group=test --topic demo-topic
Enter fullscreen mode Exit fullscreen mode

Top comments (0)