I built an IAM policy analyzer. The most common first question wasn't about features. It was "what happens to my policies after I paste them?"
Fair question. You're handing over your infrastructure's permission model to a text box. If the vendor can't answer that question clearly, you shouldn't trust them with anything else.
But there's a second problem most security tools create: they gate the answer behind a demo call.
The Demo Call Problem
"Request a demo" means:
- You fill out a form
- Someone emails you a Calendly link
- You spend 30 minutes on a screen share with a sales engineer
- They show you a sandboxed environment with synthetic data
- Your actual IAM policy never touches the product
You walk away knowing how the tool looks, not how it works on your infrastructure.
Real demo mode â the kind where you paste an actual policy and get real findings â answers three questions immediately:
- Does this tool find issues I actually have? Not curated examples. Your policies.
- What does the output look like? Is it actionable or a wall of text?
- Is the analysis fast enough to fit into my workflow? Or do I need to context-switch and come back?
A sales demo answers none of these.
What Real Demo Mode Requires
Building a tool anyone can use without signing up means solving a few problems:
Input privacy. Your policies can't be logged. Shieldly uses a one-way SHA-256 hash of your input as a cache key. The plaintext never hits a log, never lands in a training set, never sits in an analytics database. The hash lets us return cached results for identical policies without storing the policies themselves.
Rate limiting that doesn't break the experience. You need enough free analyses to evaluate the tool (20 analysis units per day on the free tier) without opening a vector for abuse. Server-side caps, not localStorage counters that anyone can clear.
Real analysis, not a reduced model. Some tools run a lighter model for demo traffic. Shieldly runs the same analysis pipeline whether you're signed in or not â Standard AI for free users, Advanced and Enterprise for paid tiers. The demo isn't a downgraded experience. It's the real thing, just gated by a daily cap.
Evaluating a Security Tool Without a Demo Call
If a tool has real demo mode, you can test it in 5 minutes:
- Paste a policy you already reviewed manually. You know what the issues are. Does the tool catch them? Does it find things you missed?
-
Paste a policy you know is clean. Does it produce false positives? A tool that flags
s3:ListBucketas a finding on a read-only reporting role isn't useful. - Check the findings format. Are findings grouped by severity? Does each one explain why it's a problem and how to fix it? If the output is just "this policy has 7 issues" with no detail, you'll need a second tool to actually remediate.
Try It With Your Own Policy
Shieldly's IAM analyzer is free at shieldly.io/app/iam. No signup, no credit card. Paste any IAM policy â resource policy, identity policy, SCP â and get real findings in seconds.
The free tier gives you 20 analysis units per day. Builder ($19/mo) adds API access and Cost Advisor. Pro ($49/mo) adds a compliance panel. Team ($99/mo) adds auto-scanning and Slack alerts for up to 10 engineers.
If you paste a policy and the findings aren't useful, that's honest feedback I want to hear. If they are useful, you just evaluated a security tool in less time than a demo call would have taken.
Top comments (0)