Solved: Rest API Explained Part 2 – Advanced Topics with PowerShell on Azure/Graph

#devops #programming #tutorial #cloud

🚀 Executive Summary

TL;DR: PowerShell’s Invoke-RestMethod often drops Azure AD/Graph API results because it doesn’t automatically follow pagination links like ‘@odata.nextLink’. This problem is solved by implementing custom do-while loops to fetch all pages, encapsulating this logic in reusable functions, or by leveraging the Microsoft.Graph PowerShell SDK which handles pagination automatically.

🎯 Key Takeaways

API pagination, indicated by ‘@odata.nextLink’, is a performance feature, not a bug, designed to return data in manageable chunks (e.g., 100-999 items) rather than a single large response.
Invoke-RestMethod does not inherently follow ‘@odata.nextLink’, requiring explicit implementation of a ‘while’ or ‘do-while’ loop to iterate through subsequent pages and aggregate the complete dataset.
The Microsoft.Graph PowerShell SDK abstracts pagination complexity for Graph API interactions, allowing cmdlets like ‘Get-MgUser -All’ to automatically retrieve all results without manual loop management.

Tired of PowerShell’s Invoke-RestMethod dropping Azure AD/Graph API results? Learn why pagination (@odata.nextLink) breaks your scripts and how to fix it for good with three practical solutions from a senior engineer.

Wrestling with PowerShell and Graph API: Why You’re Losing Data and How to Win

I remember my first real run-in with this problem. It was 2 AM, and I was on a call for a critical security audit. We needed a full list of all guest users in our Azure AD tenant, and my “brilliant” PowerShell script was proudly reporting back 999 users. The security lead was looking at the Azure portal, which showed over 5,000. For a solid 15 minutes, my heart was in my stomach, thinking our directory sync was catastrophically broken. It wasn’t. It was just me, getting schooled by API pagination. It’s a rite of passage, a frustrating one, and something Invoke-RestMethod doesn’t warn you about.

So, What’s Actually Happening Here? The “Why” Behind Missing Data

Let’s get this straight: this isn’t a bug. It’s a feature. APIs like Microsoft Graph are built for performance and stability. They will not, under any circumstances, dump 50,000 user objects on you in a single web request. That would be slow, resource-intensive, and prone to timeouts.

Instead, they “paginate” the results. They give you one “page” of data—usually 100 or up to 999 items—and if there’s more data to be had, the response includes a special property, usually called @odata.nextLink. This property is a URL that points directly to the next page of results.

The problem is that PowerShell’s Invoke-RestMethod is a simple tool. It makes a request, gets the response, and its job is done. It sees the @odata.nextLink property as just another piece of data. It has no built-in logic to automatically follow that link and fetch the next page. So, your script runs, gets the first page, and happily finishes, leaving you with a fraction of the data you actually needed.

The Fixes: From Duct Tape to a Proper Engine

We’ve all been there. You need a fix, and you need it now. But you also want to do it right next time. Here are three ways to handle API pagination, from the quick-and-dirty to the professional standard.

Solution 1: The Quick Fix (The `Do-While` Loop)

This is the workhorse. It’s not the prettiest, but it’s explicit, easy to understand, and it gets the job done for a one-off script. The logic is simple: keep fetching pages as long as the API tells you there’s a next page to fetch.

You make your initial request, then you start a loop that checks for the nextLink. If it exists, you use that URL for the next request and add the new results to your collection. You repeat until the nextLink is gone.

# NOTE: Assumes you have a valid $token for authentication
$headers = @{
    "Authorization" = "Bearer $token"
    "Content-Type"  = "application/json"
}

# Initial API call
$uri = "https://graph.microsoft.com/v1.0/users?`$select=id,displayName,userPrincipalName"
$response = Invoke-RestMethod -Method Get -Uri $uri -Headers $headers

# Create an array to hold ALL the results
$allUsers = @($response.value)

# Loop as long as a 'nextLink' is present in the response
while ($response.'@odata.nextLink') {
    Write-Host "Fetching next page..."
    $response = Invoke-RestMethod -Method Get -Uri $response.'@odata.nextLink' -Headers $headers
    $allUsers += $response.value
}

Write-Host "Total users found: $($allUsers.Count)"

Pro Tip: Notice the property is '@odata.nextLink' (in quotes). The “@” symbol is a special character in PowerShell (used for splatting), so you need to quote the property name to access it correctly. I’ve seen juniors burn hours on this simple syntax error.

Solution 2: The “Smarter” Fix (The Reusable Function)

Writing that while loop every single time is tedious and error-prone. As engineers, we hate repeating ourselves. The next logical step is to wrap that logic in a reusable function. This is what separates a quick script from a real tool in your toolkit.

This function takes the initial URI and your authentication headers as parameters, handles the entire pagination loop internally, and just hands you back the complete, consolidated data set. Clean, simple, and reusable.

function Get-AllApiResults {
    param(
        [Parameter(Mandatory=$true)]
        [string]$InitialUri,

        [Parameter(Mandatory=$true)]
        [hashtable]$Headers
    )

    $allResults = [System.Collections.Generic.List[object]]::new()
    $uri = $InitialUri

    do {
        $response = Invoke-RestMethod -Method Get -Uri $uri -Headers $Headers

        if ($response.value) {
            $allResults.AddRange($response.value)
        } else {
            # Handle cases where the response itself is the collection
            $allResults.AddRange($response)
        }

        $uri = $response.'@odata.nextLink'
        if ($uri) {
            Write-Verbose "Found nextLink, fetching more data from $uri" -Verbose
        }

    } while ($uri)

    return $allResults
}

# --- How you'd use it: ---
# $token and $headers are the same as before
$initialUrl = "https://graph.microsoft.com/v1.0/groups"
$allGroups = Get-AllApiResults -InitialUri $initialUrl -Headers $headers

Write-Host "Found $($allGroups.Count) total groups."

Solution 3: The “Modern” Fix (Use the Official SDK)

If you’re working heavily with the Microsoft Graph API, stop reinventing the wheel. Microsoft has invested a ton of effort into building the Microsoft.Graph PowerShell SDK. The engineers who built the API also built the tools to talk to it.

The SDK abstracts away all this complexity. You don’t need to worry about nextLink or manual loops. The cmdlets are designed to handle it for you with a simple switch.

Here’s a comparison of getting all users:


The Old Way (Invoke-RestMethod) You have to manage the token, headers, and the pagination loop yourself.

# You need to implement the full # while-loop logic shown in # Solution 1 to get all users. # A single call will fail. $uri = "https://graph.microsoft.com/v1.0/users" $response = Invoke-RestMethod ... # ...and so on

| The SDK Way (Get-MgUser) The SDK handles authentication and pagination for you.

# First, install it: # Install-Module Microsoft.Graph # Connect with the right permissions Connect-MgGraph -Scopes "User.Read.All" # Get ALL users. The -All switch # handles all pagination automatically. $allUsers = Get-MgUser -All # That's it. You're done. Write-Host "Found $($allUsers.Count) users."

|

Warning: The SDK is the best option for Microsoft Graph. However, you will absolutely encounter other APIs (from other vendors, or even older Azure APIs) that don’t have a convenient SDK. Knowing how to build the manual while loop (Solution 1) is a fundamental skill that will never go out of style.

So next time your script returns a suspiciously round number of results, don’t panic. Take a breath, check your API response for a nextLink, and remember you have the tools to wrestle it into submission.