Running Our AWS Deployments From Slack With Amazon Q (and the Things Nobody Tells You)

For the last couple of years, most of the day-to-day deployment work on one of the platforms I look after has happened inside a single private Slack channel. Nobody opens the CodePipeline console to kick off a staging build. Nobody SSHs anywhere to restart a container. A developer types something like this and gets on with their day:

@Amazon Q run start-pipeline-dev2
@Amazon Q run switch-branch-dev1 feature/my-branch
@Amazon Q run cont-restart api-cluster api-stg5

The service behind this is Amazon Q Developer in chat applications, which most of us still call by its older name, AWS Chatbot. (AWS has renamed it more than once. The console currently lists it under "ChatOps for AWS". I'll mostly say Chatbot here because that's what everyone on the team says.)

I recently wrote an internal manual on how our setup actually works, because too much of it lived in my head and in one colleague's Slack history. This post is a cleaned-up version of that document with names and identifiers replaced. If you're thinking about ChatOps on AWS, or you inherited a Chatbot setup and want to understand it, this should save you some digging.

What it actually is

Chatbot is a thin, authenticated proxy between a Slack channel and the AWS SDK. That's the whole mental model. It takes a message, resolves it to an API call, checks whether the call is allowed, makes the call under an IAM role, and posts results back. There is no agent running in your account, no secrets stored in Slack, no database of commands.

Two independent flows are worth keeping separate in your head:

Commands are synchronous. Slack → Chatbot → AWS service. You type it, it runs.

Notifications are asynchronous. AWS service → SNS topic → Chatbot → Slack. Pipeline stage changes, CloudWatch alarms and so on get published to an SNS topic that the channel configuration is subscribed to. Chatbot formats the payload and drops it in the channel.

Once you internalise that the second path is just an SNS subscription, a lot of troubleshooting gets easier. When notifications stop arriving, it's almost always the subscription or the topic, not Chatbot itself.

The message pipeline, step by step

When someone mentions the bot in the channel, this happens:

• Slack forwards the message event to the Chatbot endpoint via the registered Slack App.
• Chatbot looks up the alias. start-pipeline-prod expands to something like codepipeline start-pipeline-execution --name myapp-prod --region us-east-1.
• Chatbot evaluates the action against both the IAM role it runs under and the guardrail policies attached to the channel. Fail either one and you get an access denied message in Slack.
• The SDK call is made directly against the service. For standard commands there's no Lambda in the middle.
• State changes flow back through SNS as described above. One thing that surprised a few people on the team: Chatbot doesn't keep command history. If you need to know who stopped the staging pipeline on Tuesday, the answer is in CloudTrail, filtered by the Chatbot role. Slack's own history helps too, but CloudTrail is the record.

Two permission layers, and why you want both

This is the part I'd read twice if you're setting this up.

The IAM role is the execution boundary. Chatbot assumes it for every API call. It defines the ceiling of everything that could possibly happen through any channel using that role. Ours was created from the "policies from a template" option in the console and then trimmed.

The guardrail policies are the channel boundary. They're attached to a specific channel configuration, and they only ever restrict. They never grant. The effective permission set is the intersection of the two.

Why bother with the second layer? Because you can share one IAM role across several channels and give each channel a different effective scope. Our production channel and a wider engineering channel can point at the same role while only one of them is allowed to touch prod pipelines. It also means a Slack user who gets added to the wrong channel by accident can't do anything the guardrail doesn't already allow for that channel.

The cost is that "access denied" now has two possible causes. When it happens, go to CloudTrail, find the denied action, and check which layer is missing it. Don't guess.

Aliases are what make it usable

Chatbot accepts raw SDK commands:

@Amazon Q codepipeline start-pipeline-execution --name myapp-prod --region eu-west-1

Nobody wants to type that. Aliases map a short name to the full command, and you can pass parameters through at runtime. @Amazon Q alias list shows everything registered, with the expansions.

When a single SDK call isn't enough: Lambda aliases

Some things can't be expressed as one API call. Branch switching is the obvious example. To repoint a pipeline's source stage at a different branch you have to GetPipeline, edit the source stage in the returned definition, and UpdatePipeline with the modified document. That's a small Lambda function.

Chatbot handles this fine. The alias just becomes lambda invoke --function-name branch-switch-fn, and the function does the multi-step work and returns a message.

The detect-changes family works the same way. It flips PollForSourceChanges on the source stage so a pipeline stops auto-triggering on push. We use it when a staging environment needs to be frozen for QA while people keep merging into the branch. Set it to false, test, set it back to true.

Wiring an existing Lambda into Chatbot

This bit trips people up, so here's the order that works.

1. Fix the Lambda's resource-based policy: The function has to trust the Chatbot service principal explicitly. Having lambda:InvokeFunction on the IAM role is not enough.

aws lambda get-policy --function-name FUNCTION_NAME --region eu-west-1

If chatbot.amazonaws.com isn't in there:

aws lambda add-permission \
  --function-name FUNCTION_NAME \
  --statement-id AllowChatbot \
  --action lambda:InvokeFunction \
  --principal chatbot.amazonaws.com \
  --region eu-west-1

2. Update the guardrail: In the channel configuration, make sure the guardrail policy includes lambda:InvokeFunction scoped to the function ARN. Without this, the role can permit it all day and Chatbot will still refuse.

*3. Register the alias in Slack:
*

@Amazon Q alias add switch-branch-dev1 lambda invoke --function-name branch-switch-fn --region eu-west-1

4. Test it and check the CloudWatch log group: for the function to confirm the invocation actually arrived. If the command fails silently, it's step 1 nine times out of ten.

Things that have bitten us

I'll finish with the troubleshooting table from the manual, because it's the part I look at most.

• Bot doesn't respond at all: The Slack App authorisation expired or the OAuth token was revoked. Reconnect the Slack client in the Chatbot console.
• Access denied: Check CloudTrail for the exact denied action, then figure out which of the two permission layers is missing it.
• Pipeline notifications stopped: Someone deleted the SNS subscription or recreated the topic. Re-subscribe the channel configuration.
• get-logs gives an expired link: The pre-signed S3 URL lives for ten minutes. Our failure notifications include a pre-filled get-logs command for exactly this reason. Run it as soon as you see the red message.
• Lambda alias fails silently: Resource-based policy is missing the Chatbot principal. See above.
• Branch switch does nothing: Usually a typo in the branch name, occasionally an expired GitHub token inside the function. The Lambda logs will tell you.
• detect-changes false didn't stop auto-triggering: Confirm with GetPipeline that the flag actually changed. Run it again and check with pipeline-status.

Was it worth it?

Yes, with one caveat. The setup took an afternoon. The value came from being disciplined about aliases and guardrails, and from writing down where everything lives, since the Chatbot console, the IAM role, the SNS topic, and the Slack channel ID are spread across four different places and none of them link to each other.

If you run a small team where developers regularly need to deploy to shared environments, this removes a surprising amount of friction and a surprising number of interruptions. Just decide early that CloudTrail is your audit log and the guardrail is your safety net, and you'll be fine.

Happy to answer questions in the comments if you're setting something similar up.