DEV Community

Runbook Foundry
Runbook Foundry

Posted on

A Conditional Access Baseline for Small Microsoft Entra Tenants (Report-Only First)

Conditional Access is the most useful security control in a Microsoft 365 tenant and the easiest one to lock yourself out with. One badly scoped "block" policy can stop every admin from signing in, including you.

The answer isn't to avoid it. It's to build a small, boring baseline, test each policy in report-only mode, and only then switch it on. This guide covers which policies to start with in a small tenant (roughly 5 to 300 users), how to read the report-only results, and a rollout order that doesn't wreck anyone's Monday.

Menu paths and recommendations were checked against Microsoft Learn in October 2026. Microsoft renames things often, so treat the docs as the source of truth if a label has moved.

Prerequisites

Licensing. Conditional Access needs Microsoft Entra ID P1, which is included in Microsoft 365 Business Premium and E3/E5. Risk-based policies need P2. If you only have the free tier, use security defaults instead. They give you MFA registration, admin MFA, legacy authentication blocking and device code flow blocking with no configuration.

Security defaults vs Conditional Access. You can't run both. Microsoft's guidance is that organizations replacing security defaults with Conditional Access must turn security defaults off (Entra ID > Overview > Properties > Manage security defaults), and should then turn on Conditional Access policies straight away so there's no gap. Plan that switch for a quiet time and have your policies ready first.

Check for Microsoft-managed policies. Open Entra ID > Conditional Access > Policies and look at the Created by column. Microsoft creates some policies in your tenant in report-only mode, for example MFA for all users, MFA for admins accessing admin portals, blocking legacy authentication and blocking device code flow. According to Microsoft Learn, they're switched on no less than 30 days after they're introduced unless you turn them off, and you'll get email and Message center notice two weeks before. Know what's there so nothing surprises you, and add your emergency access exclusions to them too.

Role. You need at least the Conditional Access Administrator role to create policies.

Step 0: Emergency access accounts

Do this before anything else. Microsoft recommends two or more emergency access ("break-glass") accounts:

  • Cloud-only accounts on your *.onmicrosoft.com domain, not synced or federated
  • Permanently assigned Global Administrator
  • Protected with a passkey (FIDO2) or certificate-based authentication, using a different method from your normal admin accounts
  • Credentials stored securely, in separate locations, and not tied to one person's phone
  • Sign-in alerts set up, so any use is noticed
  • Tested at least every 90 days

Put both accounts in a security group such as CA-Exclude-EmergencyAccess. Exclude that group from every policy that blocks or restricts sign-in. Microsoft notes that report-only policies don't strictly need the exclusion, but add it from day one anyway, so it's already there when you switch the policy on.

Also make a list of service accounts and anything that signs in non-interactively, such as the Entra Connect sync account if you use hybrid identity, or a scanner that sends email. These need deliberate handling, not blanket exclusions.

The starter baseline: six policies

Use a clear naming standard so the list is readable at a glance, for example CA01-AllUsers-Block-LegacyAuth. Every policy below starts with Enable policy = Report-only.

CA01: Block legacy authentication

  • Users: All users. Exclude the emergency access group.
  • Target resources: All resources.
  • Conditions > Client apps: Configure = Yes. Check only Exchange ActiveSync clients and Other clients.
  • Grant: Block access.

Legacy protocols can't do MFA, and Microsoft's analysis attributes the large majority of password spray attacks to them. Before enforcing, check who still uses them: Entra ID > Monitoring & health > Sign-in logs, add the Client app column, filter to the legacy protocols, and check the non-interactive sign-ins tab too. The usual culprits are old multifunction printers, line-of-business apps and ancient mail clients.

CA02: Require MFA for all users

  • Users: All users. Exclude the emergency access group (and the directory synchronization accounts role if you use Entra Connect).
  • Target resources: All resources, with no app exclusions.
  • Grant: Grant access > Require authentication strength > the built-in Multifactor authentication strength.

Microsoft's tip is that every organization should have this baseline: all users, all resources, MFA. Resist adding "trusted office IP" exclusions unless you really need them. Attackers don't sit in your office, but stolen sessions and compromised devices can look like they do.

CA03: Require phishing-resistant MFA for admins

  • Users: Directory roles. Include at least Global Administrator, Privileged Role Administrator, Security Administrator, Conditional Access Administrator, Exchange Administrator, SharePoint Administrator, User Administrator, Helpdesk Administrator, Authentication Administrator, Privileged Authentication Administrator, Application Administrator, Cloud Application Administrator, Password Administrator and Billing Administrator. Exclude the emergency access group.
  • Target resources: All resources.
  • Grant: Require authentication strength > Phishing-resistant MFA.

Warning: make sure every admin has registered a passkey (FIDO2), Windows Hello for Business or certificate-based authentication first, using a Temporary Access Pass if needed. Turning this on before they're registered is the classic lockout. Also note that Conditional Access only targets built-in roles. Custom and administrative-unit-scoped roles aren't covered.

CA04: Require MFA for Microsoft admin portals

  • Users: The same admin roles as CA03. Exclude the emergency access group.
  • Target resources: Select resources > Microsoft Admin Portals.
  • Grant: Require authentication strength > Multifactor authentication.

This overlaps with CA02 and CA03 on purpose. If someone later edits or scopes down one policy, this one still protects the admin portals.

CA05: Secure security info registration

  • Users: All users. Exclude all guest and external users, and the emergency access group.
  • Target resources: User actions > Register security information.
  • Conditions > Locations: Include Any location. Exclude All trusted locations.
  • Grant: Require authentication strength.

This stops an attacker who has just phished a password from registering their own MFA method. Issue new starters a Temporary Access Pass so they can register from anywhere. One caveat: report-only mode doesn't evaluate policies scoped to User actions, so you won't get report-only data for this one. Test it on a small test group instead. Microsoft also notes that from July 6, 2026, these policies apply during Windows Hello for Business and macOS Platform SSO registration, so test those flows if you use them.

CA06: Block device code flow

  • Users: All users. Exclude the emergency access group and any documented exceptions.
  • Target resources: All resources.
  • Conditions > Authentication flows: Configure = Yes, select Device code flow.
  • Grant: Block access.

Device code flow is rarely needed in small businesses but often abused in phishing. If you have Teams Rooms or similar shared devices that rely on it, scope a narrow exception rather than leaving the flow open to everyone.

Next-wave policies (after the baseline is on)

  • MFA for guests. Include guest and external users, target all resources, require MFA (if CA02 doesn't already cover them).
  • Risk-based policies (P2). Require MFA for medium-or-higher sign-in risk and a secure password change for high user risk.
  • Block countries you don't operate in, using a named location. Treat it as noise reduction, not a strong control, because VPNs get around it.
  • Device-based access once Intune enrollment is in good shape, plus app protection on mobile.
  • Shorter admin sessions: sign-in frequency and no persistent browser for admin roles.

Running report-only properly

Report-only means the policy is evaluated at every sign-in and logged, but nothing is enforced. Users aren't prompted and nobody is blocked. Give each policy at least 7 days, and 14 if your business has weekly or month-end cycles.

Three places to look:

  1. Policy impact. Open the policy and select the Policy impact view. It shows what the policy would have done to interactive sign-ins over the past 24 hours, 7 days or month. It needs at least the Security Reader role.
  2. Sign-in logs. Entra ID > Monitoring & health > Sign-in logs, open a sign-in and check the Report-only tab. Results are:
    • Report-only: Success: would have been allowed
    • Report-only: Failure: would have been blocked. Investigate every one.
    • Report-only: User action required: the user would have been prompted (for example, for MFA)
    • Report-only: Not applied: conditions didn't match, for example an excluded user
  3. Conditional Access insights workbook, if you send sign-in logs to a Log Analytics workspace. It's best for seeing several policies together.

For each Failure, decide: is it an attacker (good, the policy works), a legacy device that needs replacing, or a legitimate scenario that needs a narrow, documented exclusion?

A report-only quirk: policies that require a managed device can still pop a certificate prompt on macOS, iOS and Android during evaluation. Microsoft's advice is to exclude those platforms from report-only device policies.

Switching policies on

Turn on one policy at a time, in this order:

  1. CA01 (block legacy auth), once the sign-in logs show no legitimate legacy use
  2. CA04, then CA03 (admins first; it's a small group you can help in person)
  3. CA02 (all users), after a user communication with the date and how to register
  4. CA05 and CA06

For each one:

  • [ ] Report-only for 7 to 14 days with no unexplained failures
  • [ ] Exclusions documented with an owner and a review date
  • [ ] Help desk told what to expect and how to issue a Temporary Access Pass
  • [ ] Emergency access account sign-in tested
  • [ ] Switch Enable policy from Report-only to On
  • [ ] Watch sign-in logs closely for the first 48 hours
  • [ ] Rollback plan ready: switch the policy back to Report-only, rather than deleting it

Keeping it healthy

  • Review exclusions every 90 days. Exclusions are where baselines quietly die.
  • Export your policies (JSON via Microsoft Graph) after every change, so you have a backup and a history.
  • Test the emergency access accounts quarterly.
  • Review any new Microsoft-managed policies when Message center announces them.

A small tenant with these six policies on, real emergency access accounts and documented exclusions is ahead of most organizations its size, and you got there without locking anyone out.


Resources: Free M365 + Google Workspace hardening checklist ยท Full pack with 16 report-only policy JSONs, a safe PowerShell deployer and a rollout runbook: Conditional Access Policies Baseline ($59; code LAUNCH30 takes 30% off paid kits through Oct 31, 2026). I make these templates.

Drafted with AI assistance and reviewed for accuracy.

Not affiliated with Microsoft or Google.

Top comments (0)