Report-only is the safest way to find out what a Conditional Access policy would do. That only helps if you read the results correctly. This post covers what each result means, where to find it, and what to check before you move the toggle from Report-only to On. It's all read-only, and nothing here changes a policy.
TL;DR
- Report-only policies are evaluated at sign-in but not enforced. Users aren't prompted for MFA or blocked.
- You'll see four report-only results: Success, Failure, User action required and Not applied.
- Read them in the sign-in logs (Conditional Access and Report-only tabs), in Policy impact (preview), or in the Conditional Access insights and reporting workbook.
- Exception: report-only policies that require a compliant device can still prompt macOS, iOS and Android users for a device certificate.
What's the difference between report-only and On?
| Report-only | On | |
|---|---|---|
| Evaluated at sign-in | Yes | Yes |
| Grant and session controls enforced | No | Yes |
| Users prompted for MFA or terms of use | No | Yes |
| Can block a sign-in | No | Yes |
Most policies can run in report-only. The exception is anything in the User actions scope.
Watch the device-compliance edge case. Microsoft warns that report-only policies requiring a compliant device can prompt users on macOS, iOS and Android to select a device certificate during evaluation, even though compliance isn't enforced. The prompts can repeat until the device is compliant. Microsoft's suggested fix is to exclude those platforms from report-only policies that check device compliance.
What do the report-only results mean?
| Result | Meaning | Example |
|---|---|---|
| Report-only: Success | All conditions matched, and the required non-interactive grant controls and session controls were satisfied | An MFA claim already in the token satisfies an MFA requirement |
| Report-only: Failure | All conditions matched, but not all required non-interactive grant controls or session controls were satisfied | A block control applies to the user, or the device fails a compliant-device requirement |
| Report-only: User action required | All conditions matched, but the user would have to do something, like MFA or terms of use. Report-only doesn't prompt them | |
| Report-only: Not applied | Not all conditions matched | The user is excluded, or the policy only applies to certain named locations |
Failure is the result to study before enforcing. Under On, those sign-ins would be blocked.
User action required isn't a failure. In the insights workbook, Microsoft notes that report-only can't determine success or failure for interactive controls because they aren't enforced. These users would get a prompt under On. Plan your comms for them.
Where do I find report-only results in the sign-in logs?
- Sign in to the Microsoft Entra admin center as at least a Reports Reader.
- Browse to Entra ID > Monitoring & health > Sign-in logs.
- Filter. A Correlation ID narrows to one event. The Conditional Access filter shows policy success and failure.
- Open a sign-in and look at the Conditional Access and Report-only tabs.
Each event shows which policies were enabled, in report-only, applied or not applied. Select a policy on the Conditional Access tab to see Audience under Resource. That lists every resource requested in the sign-in, which explains why a policy aimed at one app shows up on a sign-in to another.
Exporting? Download the sign-in logs as JSON. That's the format Microsoft names for including report-only result data.
How do I see the impact across all users?
There are two views above single events:
- Policy impact (preview). Needs at least Security Reader. It's a snapshot of a policy's potential or existing impact on interactive sign-ins over the past 24 hours, 7 days or 1 month, with links to sample sign-in events.
- Conditional Access insights and reporting workbook. Needs Microsoft Entra ID P1, a Log Analytics workspace receiving sign-in logs, and at least Security Reader plus Log Analytics workspace Contributor. You can select report-only and enabled policies, set a time range from 4 hours up to 90 days, and break results down by condition.
The workbook can also combine policies. In the summary, Failure counts sign-ins where at least one selected policy returned Failure or Report-only: Failure. That's how you check the combined effect of a new policy with the ones already enforced.
How do I test a change to a policy that's already On?
Microsoft's guidance: before modifying an enforced policy, create a copy in report-only with your proposed changes. Compare the copy's results against the enforced original. When you're happy, update the enforced policy and remove the copy. Report-only also lets you test a brand-new policy alongside your enforced ones without affecting access.
For a single "would this user be affected?" question, the What If tool is the other Microsoft-documented troubleshooting route.
When is it safe to switch to On?
Microsoft's wording is simple. After confirming your settings with policy impact or report-only mode, move Enable policy from Report-only to On. In practice, before you flip it, confirm:
- You've reviewed every Report-only: Failure sign-in and each one is intended.
- User action required users know a prompt is coming.
- Not applied matches your intended exclusions, especially break-glass accounts.
- The mobile and macOS device-certificate prompts above aren't hitting users.
Our free checklist turns this into a one-page pre-flight list.
FAQ
Does report-only mode block users or prompt for MFA?
No. It evaluates the policy without enforcing grant or session controls. The documented exception is the device-certificate prompt on macOS, iOS and Android for compliant-device policies.
What does "Report-only: User action required" mean?
The policy would apply, but the user would need to complete a control like MFA. Report-only doesn't prompt them, so success or failure can't be known.
Why don't report-only results show in my CSV export?
Microsoft says to download sign-in logs as JSON to include report-only result data.
Can every policy run in report-only?
Most can. Policies that use the User actions scope are the exception.
Related posts
- Conditional Access without the Monday lockout: report-only to enforce
- Conditional Access baseline: naming, pilot groups, break-glass
- Compliance before Conditional Access
Free checklist and the full pack
- Free: Conditional Access pre-enforcement checklist: https://cashflow4375.gumroad.com/l/ujkqxb?utm_source=devto
- Full pack: Conditional Access rollout pack with baseline design, rollout waves and review templates: https://cashflow4375.gumroad.com/l/nhuyrc?utm_source=devto
Admin Pack Studio; not affiliated with Microsoft.
Top comments (0)