Azure AD - Cleanup old Service Principals / App-Registrations

Background

An Azure AD can extensively grow in numbers of service principals due to lifetime of applications and service connections. Some of them are created for one-time use and have been forgotten or an application is just not in use anymore.

After 6 years of usage, our development Active Directory counts over 9000 service principals and no one clearly knows whether these applications are still in use. Therefore, we had to think about a process on how we can cleanup the directory from obsolete applications.

The Process

The cleanup process consists off of multiple steps to determine whether an application is still in use or not.

1. Select all applications and iterate over them by
2. checking if the creation date is prior to 90 days
3. checking for valid credentials and certificates
4. checking if no sign-in activities happened in the last 90 days

If all criteria are fulfilled the applications is tagged for deletion.

Prerequisites

To perform some of the following configuration steps, an account with Global Admin privileges is needed.

Step #1, #2 and #3 can easily be accomplished through the Micosoft-Graph API endpoint

GET /applications

We get all applications inside the tenant and additionally can check for the creation date as well as for credentials and certificates. To query applications, a service principal with proper authorization for Mircosoft Graph with

Application.ReadWrite.All

must be granted.

Step #4 requires some preparation.

By Default, Azure persists Sing-In Logs only for 30 days. To increase the retention period of the logs to the desired 90 days, we have to set a custom destination in the Azure-AD diagnostic settings.

To get all sign-in activities, we have to select the following categories:

• AADManagedIdentitySignInLogs
• AADNonInteractiveUserSignInLogs
• AADServicePrincipalSignInLogs
• SigninLogs

It seems the category ManagedIdentitySignInLogs only contain logs of authorization flows between Azure resources configured with managed identities and will therefore be ignored in the following step.

To reduce the amount of storage, we create data slices on a daily bases with the minimum amount of information needed and store them in a custom table (ApplicationSignInLogs_CL). The retention time is set to 90 days. The script itself runs inside an automation account with a scheduled job and utilizes the Azure logs ingestion api to store the data inside the log analytics workspace.

The following Kusto query selects and aggregates all relevant logs:

// select & aggregate user sign-in logs
let UserLogs = view() {
    SigninLogs
    | project TimeGenerated, AppId, AppDisplayName
    | summarize NumOfLogs = count() by bin(TimeGenerated, 1d), AppId, DisplayName = AppDisplayName
    | where TimeGenerated >= datetime_add('day', -2, now()) and TimeGenerated < bin(now(), 1d)
};

// select & aggregate user non-interactive sign-in logs
let UserNonInteractiveLogs = view() {
    AADNonInteractiveUserSignInLogs
    | project TimeGenerated, AppId, AppDisplayName
    | summarize NumOfLogs = count() by bin(TimeGenerated, 1d), AppId, DisplayName = AppDisplayName
    | where TimeGenerated >= datetime_add('day', -2, now()) and TimeGenerated < bin(now(), 1d)
};

// select & aggregate service principal sign-in logs
let PrincipalsLogs = view() { 
    AADServicePrincipalSignInLogs 
    | project TimeGenerated, AppId, DisplayName = ServicePrincipalName 
    | summarize NumOfLogs = count() by bin(TimeGenerated, 1d), AppId, DisplayName 
    | where TimeGenerated >= datetime_add('day', -2, now()) and TimeGenerated < bin(now(), 1d)
};

// union all logs and create one entry per principal per day
union UserLogs, UserNonInteractiveLogs, PrincipalsLogs
| summarize NumOfLogs = sum(NumOfLogs) by TimeGenerated, AppId, DisplayName

We end up having one log entry per application per day.

Application selection

A Python script queries all applications and checks for the defined criteria. If an application is tagged for deletion, an entry for further processing is created in a storage account.

apps = []

# loop trough all applications and check for creation date & expired certificates / credentials
done = False
iterations = 0
while done is False:
    if(iterations == 0):
        url = 'https://graph.microsoft.com/v1.0/applications/'
        applications = requests.get(url, headers=authHeaderGraph).json()
    else:
        url = applications['@odata.nextLink']
        applications = requests.get(url, headers=authHeaderGraph).json()

    for app in applications['value']:

        # check for creation date prior 90 days
        if(app['createdDateTime'] != None):
            creationDate = datetime.fromisoformat(app['createdDateTime'][:19])
            if(creationDate > (datetime.now() - relativedelta(months=3))):
                continue

        #check for valid credentials
        hasValidCred = False
        for entry in app['passwordCredentials']:
            expDate = datetime.fromisoformat(entry['endDateTime'][:19])
            if(expDate > datetime.now()):
                hasValidCred = True
                break

        #check for valid certificates
        hasValidCert = False
        for entry in app['keyCredentials']:
            expDate = datetime.fromisoformat(entry['endDateTime'][:19])
            if(expDate > datetime.now()):
                hasValidCert = True
                break 

        # at least one valid certificate or credentials
        if(hasValidCred is False and hasValidCert is False):
            apps.append(app)

    if('@odata.nextLink' not in applications):
        done = True

    iterations = iterations + 1

# get aggregated sign-in logs and aggregate to latest login date per application (in the last 90 days)
query = {
    "query": "ApplicationSignInLogs_CL "
                "| project TimeGenerated, AppId "
                "| summarize LatestLogin = max(TimeGenerated) by AppId "
                "| where LatestLogin > ago(90d) "
                "| project AppId"
}
principalLogs = requests.post(
    "https://api.loganalytics.azure.com/v1/workspaces/{workspace-id}/query", 
    json=query, headers=authHeaderLogs ).json()

# helper function to check for log entries
def signInLogExists(appId):
    for entry in principalLogs['tables'][0]['rows']:
        if(entry[0] == appId):
            return True
    return False

# loop trough all applications and check for login in the last 90 days. If no login occurred, create entry in storage account for further processing
for i, app in enumerate(apps):
    appId = app['appId']
    blob = container_client.get_blob_client(appId)

    if(signInLogExists(appId) == False):
        blob.upload_blob(datetime.now().strftime("%Y-%m-%dT%H:%M:%S"))

Conclusion

The above process describes a possible way on how to cleanup the Azure Active Directory from obsolete service principals.

Learnings:

• We falsely assumed that all activities related to service principals are logged in the service principal sign-in logs. However, principals as part of an identity provider log their activities in the user sign-in logs as users authenticate trough the principal to the Azure Active Directory.
• Aggregation of the the sign-in logs save a lot of storage and time when analysing the login activities.
• A manual check before deleting tagged principals is recommended. However, delete principals can be restored within 30 days of deletion.