Automation rules do exactly what you tell them. AI-generated text and queries can be close but not quite right. Put them together and a small error can repeat across hundreds of issues.
Short answer: let AI draft, let a person approve, and only then let automation act.
Pattern 1: Draft, then review
If a rule posts an AI-written summary to a ticket, have it post as an internal comment first. A person reads it and decides whether to share it with the customer.
Pattern 2: Test on a narrow scope
When a rule uses a generated query, run it on one project, or add a label condition, before widening it. Check the audit log after the first few runs.
Pattern 3: Don't automate irreversible actions
Bulk transitions, deletions and customer-facing emails should wait for a person's sign-off. Comments and labels are easy to undo, so they are safer starting points.
Pattern 4: Write down the assumption
Add a note in the rule description: what the AI step is expected to produce and who reviews it. Six months later, someone will need to know.
A quick checklist
- Is the output visible only to staff first?
- Is the rule limited to a test scope?
- Is the action reversible?
- Is there a named reviewer?
If you answer "no" to any of these, add a manual step.
What's the first automation rule you would trust with an AI step?
Disclosure: I work with Empyra, an Atlassian Platinum Solution Partner in Irving, Texas and Bangalore. More at atlassian intelligence services.
Top comments (1)
To answer the closing question, a rule that only adds a label, and I'd pick that for the first AI step too since a wrong label is one bulk edit away from gone.
One thing I'd add to the narrow-scope pattern: the first few runs mostly tell you about the cases where the rule fired. A generated JQL condition that is slightly off usually doesn't misfire loudly, it just matches fewer issues than you meant, and in the audit log that looks the same as a rule that is working. So on the test project I'd also seed an issue that should not match and one that should, and check both outcomes, not just the hits.