DEV Community

Cover image for Case study: the VIP list nobody updated
CD Security Lab
CD Security Lab

Posted on Originally published at gitlab.com

Case study: the VIP list nobody updated

#azure #security #kql #cybersecurity

Source: CD's readiness notes Sentinel-Readiness/Watchlist.md (VIP travellers allowlist, restricted-employee blocklist, SearchKey), rebuilt as a lab story and cross-checked against Microsoft Learn on 2026-10-28. Lab names only; screens are drawings.

In January, Contoso's SOC did the sensible thing. Executives travel, "impossible travel" alerts on them were noise, so an analyst built a VIP Users watchlist from an HR spreadsheet and told the analytics rule to skip those twelve accounts. Everyone got their evenings back.

In October a former regional director's account, never disabled properly, signs in from two continents in one hour. No incident. The account was still on the VIP list. Meanwhile the new CFO, hired in June, generates four "impossible travel" incidents a week because nobody added her. The list was right for exactly one month.

It happened for real: Colonial Pipeline, 2021

The lab story above is invented. The pattern is not. In May 2021 the largest fuel pipeline on the US East Coast stopped for five days because of one account nobody was using any more.

What happened

Date (2021) Event
29 April Attackers log in to Colonial Pipeline's network through a VPN account. According to Mandiant, as reported by Bloomberg, the account was no longer in use and its password had appeared in a leak.
7 May DarkSide ransomware hits the business network. Colonial shuts down its 5,500-mile pipeline, which carries about 45% of the fuel used on the East Coast.
7 May Colonial pays a ransom of 75 bitcoin, about 4.4 million USD.
12 May The pipeline starts to restart. Gas stations across the South-East run dry in the days around it.
7 June The US Justice Department recovers 63.7 bitcoin, about 2.3 million USD at the time.
8 June CEO Joseph Blount tells the US Senate the VPN was a legacy profile without multi-factor authentication, and that the password was a complicated one.

Why it worked

  • Door 1, an account that should not exist. Nobody used the VPN account, so nobody disabled it. It is the ex-director from the lab story: still on a trusted list, still able to sign in.
  • Door 2, no second factor. A leaked password was enough. A complicated password does not help once it is in a leak.
  • Door 3, nobody watching the quiet accounts. A sign-in from an account that has been silent for months is one of the strongest signals you can get, and nothing was looking for it.

What it cost

Who What they lost
Colonial Pipeline 4.4 million USD ransom (about half recovered later), five days of shut-down, a Senate hearing
Drivers and airlines on the East Coast Fuel shortages, panic buying, emergency rules for fuel transport in 17 states and Washington, D.C.
Everyone else The attack that turned ransomware into a national-security topic in the US

The same doors in a Microsoft tenant

This post's fix closes Door 1 and Door 3. Add Door 2 and you have the full set:

  • Door 1, the leaver still trusted: steps 1 to 3 in section 5 below. Feed the lists from Entra groups, move leavers to TerminatedEmployees, and find inactive accounts with the signInActivity report in Entra ID (or an access review for inactive users) and disable them.
  • Door 2, no MFA: a Conditional Access policy that requires MFA for all users and all resources, including the VPN, which can use Entra ID for sign-in.
  • Door 3, the quiet account that wakes up: the query below, as a scheduled analytics rule, every hour. 
// Accounts that signed in today but had no successful sign-in in the previous 60 days
let today = SigninLogs
    | where TimeGenerated > ago(1d)
    | where ResultType == "0"
    | distinct UserPrincipalName;
let before = SigninLogs
    | where TimeGenerated between (ago(60d) .. ago(1d))
    | where ResultType == "0"
    | distinct UserPrincipalName;
today
| join kind=leftanti before on UserPrincipalName
Enter fullscreen mode Exit fullscreen mode

Your workspace must keep SigninLogs for at least 60 days for this to work. New joiners also show up once: that is expected, and it is exactly the list HR should confirm.

TL;DR

  • Problem: a watchlist is a snapshot of the business. The business moves; the snapshot does not. A stale allowlist suppresses the exact alerts you most need; a stale blocklist misses the people it was built for.
  • Who it hurts: the SOC (blind to compromised ex-VIP accounts, flooded by new-VIP noise), HR and legal (a leaver's access is "monitored" on paper only), the exec whose account is abused, and the budget that pays analysts to close false positives.
  • Fix: feed the watchlist from a source of truth (an Entra group HR already maintains) on a schedule, use the right template and SearchKey, add a Terminated Employees watchlist that raises severity instead of lowering it, and alert when the list is older than 14 days.
  • Cost: the watchlist itself is close to free. The stale one cost about 10 analyst hours a month in noise, and one unseen compromise.
  • Prove it: _GetWatchlist('VIPUsers') matches the Entra group exactly, and the test sign-in from a leaver's account opens a High incident.

Table of contents

1. The problem

A watchlist is a guest list at the door. Security checks every name against it. Two kinds exist, from my notes:

  • Allowlist: "these people may do unusual things, do not stop them." VIP travellers.
  • Blocklist: "if any of these people show up, call me immediately." Restricted or terminated employees.

A guest list printed in January is wrong by February. Someone left, someone joined, someone changed name. The bouncer does not know; the paper looks the same. In Sentinel the paper is the Watchlist table, the bouncer is your analytics rule, and _GetWatchlist('alias') is how the rule reads the list.

A guest list from January checked at the door in October

2. Who gets hurt

Who What they lose
SOC analysts The suppressed sign-in from the ex-director is the one incident they needed. The CFO's 16 false positives a month are the ones they did not.
HR / legal "Leavers are monitored" is a control on paper. The leaver's account is on the trusted list.
The executive Her account is either over-watched (noise, slower response) or invisible (no response)
Budget Analyst time on noise, and the cost of a breach found late
The product Watchlists get blamed for being "useless", when the process around them was missing

3. Cost and value

What a watchlist costs: almost nothing. Items land in the Watchlist table (tiny rows), limits are generous (10 million active items per workspace, 3.8 MB per local upload, 500 MB via Azure Storage). The Logic App that refreshes it weekly runs 4 times a month; at consumption pricing that is cents. Checked 2026-10-28.

What the stale list cost in the lab story:

Item Lab estimate
False positives on the new CFO about 16 incidents a month × 15 min triage = 4 hours
Other joiners/movers not on the list (3 people) another 6 hours a month
Suppressed sign-ins on 3 leavers still on the list 0 incidents, which is the problem
Total about 10 analyst hours a month, plus one missed compromise

Value of the fix: the same rule, the same watchlist feature, but fed from the Entra group HR already keeps. Zero extra licence. The 10 hours come back, and leavers move from the allowlist to a blocklist that raises severity.

How other tools do it: every SIEM has the same concept (lookup tables, reference sets, asset lists) and the same failure mode. The fix is never a feature; it is a feed and an expiry date.

4. What went wrong, in the lab

Workspace law-contoso-sec. Watchlist VIPUsers, created in January from the VIP Users template (mandatory column User Principal Name, SearchKey set to that column). The analytics rule:

let vip = _GetWatchlist('VIPUsers') | project UserPrincipalName = tolower(SearchKey);
SigninLogs
| where ResultType == "0"
| where tolower(UserPrincipalName) !in (vip)      // skip VIPs
| ... impossible-travel logic ...
Enter fullscreen mode Exit fullscreen mode

Watchlist VIPUsers: 12 rows from January, 3 leavers still listed, the new CFO missing

Three things nobody set up:

  1. No feed. The list came from a spreadsheet. HR changed the Entra group sg-exec-leadership all year; the watchlist never heard about it.
  2. No expiry. Nothing told the SOC the list was 280 days old. (Watchlist rows show a TimeGenerated that Sentinel refreshes every 12 days, so even that date lies about freshness.)
  3. Wrong direction for leavers. A leaver should move from "trust more" to "watch more". There was no Terminated Employees list at all.

5. The fix

Step 1: pick the source of truth. The Entra group sg-exec-leadership, owned by HR. If a name is not in the group, it is not a VIP.

Step 2: refresh the watchlist on a schedule. A Logic App, weekly:

  1. Trigger: Recurrence, every 7 days.
  2. Action: Microsoft Graph, list members of sg-exec-leadership (userPrincipalName).
  3. Action: Microsoft Sentinel connector, Watchlists - Add a new watchlist item for each member (or build the CSV and call the Watchlist API).
  4. Removals: bulk upload appends and never deletes. For a list this small, the honest approach is delete-and-recreate the watchlist once a week from the group, or delete departed items explicitly from the diff.

Logic App: weekly refresh of VIPUsers from the Entra group

Step 3: add the blocklist. Create TerminatedEmployees from the template (mandatory: User Principal Name, UserState, Termination date), fed from HR's leaver process. Rule: any successful sign-in by a listed account → High incident.

let leavers = _GetWatchlist('TerminatedEmployees') | project UserPrincipalName = tolower(SearchKey);
SigninLogs
| where ResultType == "0"
| where tolower(UserPrincipalName) in (leavers)
| project TimeGenerated, UserPrincipalName, IPAddress, Location, AppDisplayName
Enter fullscreen mode Exit fullscreen mode

Step 4: alert on staleness. A scheduled rule that compares the watchlist with the group is ideal; a simpler one fires if the Logic App run history shows no success in 14 days.

6. Prove it

  1. _GetWatchlist('VIPUsers') | count equals the member count of sg-exec-leadership. The three leavers are gone, the CFO is there.
  2. Test sign-in with a lab leaver account (lab.leaver01@contoso.com, on TerminatedEmployees): a High incident within the rule's 5-minute schedule.
  3. The CFO travels to the Singapore office: zero impossible-travel incidents.
  4. The Logic App run history shows a green run every Monday.

Before and after: 16 false positives and 0 leaver alerts, then 0 and 1

Gotchas

  • Bulk update appends. Re-uploading a smaller CSV does not remove anyone. Delete items, or delete and recreate the list.
  • SearchKey decides performance and joins. Set it to the column you filter on (UPN here). Changing it later means recreating the list.
  • Case. tolower() both sides; UPNs arrive in mixed case.
  • Narrow time ranges return empty watchlists. _GetWatchlist() on a 5-minute query window can come back empty or partial; widen the scope when testing.
  • 28 days is not an expiry. The Watchlist table keeps 28 days and Sentinel refreshes rows every 12 days, so the list never disappears on its own. Freshness is your job.
  • Daily cap. A new watchlist with zero rows right after upload can be the workspace daily cap, not a bug.
  • Templates are preview. Columns can still change; pin yours in the Logic App.

Checklist

  • [ ] Each watchlist has an owner and a source of truth (group, HR export)
  • [ ] Template + SearchKey chosen on purpose (UPN)
  • [ ] Scheduled refresh (Logic App or API), removals handled explicitly
  • [ ] Allowlist for VIPs and blocklist for leavers
  • [ ] Staleness alert (no refresh in 14 days)
  • [ ] Count matches the source, test leaver sign-in raises High
  • [ ] Reviewed quarterly with HR

What changed recently

Checked against Microsoft Learn on 2026-10-28:

  • My notes describe allowlist/blocklist and SearchKey; Learn now adds hard limits (10 million items, 3.8 MB local upload, 500 MB via Azure Storage), the 12-day refresh and the 28-day table retention (watchlists page updated 2026-06-23).
  • Watchlist templates (VIP Users, Terminated Employees, High Value Assets, Network Addresses, Identity Correlation, Service Accounts) exist, still in preview (schemas page updated 2026-06-28).
  • The manage page (updated 2026-07-01) states that bulk update appends and de-duplicates exact rows; it does not delete. That rule shaped step 2.
  • Sentinel in the Azure portal retires after 31 March 2027; watchlists live under Microsoft Sentinel → Configuration → Watchlist in the Defender portal.

Sources

News sources

Cheat sheet: a watchlist needs a feed and an expiry

Thanks for reading! I'm Cuong, and on CD Security Lab I rebuild real Microsoft security cases in a safe lab and explain them simply. Follow along:

Got a question about your own tenant? Drop it in the comments.

Top comments (0)