DEV Community

Kinga
Kinga

Posted on • Updated on

Azure DevOps Pipeline: no secrets, no certificates

If you are using Azure DevOps pipelines with Service Connections, you may have noticed a mysterious blue dot next to your service connection name

blue dot

There's also a notification on top of the Service connection page:

Convert your existing Azure Resource Manager service connections which use secrets to authenticate to leverage Workload identity federation instead, for improved security and simplified maintenance.

which opens a Create an Azure Resource Manager service connection using workload identity federation documentation.

Why is it important

Azure Resource Manager Service Connections that use Service Principal are using Service Principal Id and either key (a secret) or certificate to authenticate.
Managed Identity authentication, on the other hand, is for self hosted agents.

I never understood why can't we use Managed Identity for Microsoft hosted agents, since Azure DevOps runs on Azure anyway. 🤔 It seems that someone at Microsoft was of the same opinion.

So here we are, it seems we can finally (mind it's still in preview) get rid of these secrets and certificates =)

New Azure Service Connection

I went ahead and created a new service connection using Workload Identity federation (automatic)

New service connection

Details page looks very familiar, there's a link to Manage service connection roles and to Manage Service Principal

Details page

Manage Service Principal

Let's see what happens here... Manage Service Principal redirects me to Azure Portal. It's an App Registration page

App Registration page

After clicking on Managed application in local directory link, I'm redirected to the Enterprise Application page

Enterprise Application page

Grant API Permissions

Perfect, granting access to the identity used by the pipeline is therefore exactly the same, as to any other service using Managed Identity.

Let's say that my pipeline will create some items in a SPO list. I'm using PnP.PowerShell, so both: Graph and SharePoint API permissions are required. Obviously, I'm scoping access using Sites.Selected.

What you need to note down here, is the information displayed on the Enterprise Application page. The Application Id is the same in both cases, but you need the correct Object Id.



$appId ="3fxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx6d"
$objId = "bdxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx34"


Enter fullscreen mode Exit fullscreen mode

Grant API Permissions

Use MsGraph PowerShell to grant Sites.Selected API permissions



$tenantID = "{tenant-id}"
$objId = "bdxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx34" #from Enterprise Application page
$appId = "3fxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx6d"

Connect-MgGraph -TenantId $tenantID -Scopes "AppRoleAssignment.ReadWrite.All", "Application.Read.All"
# Get Service Principal
$sp = Get-MgServicePrincipal -ServicePrincipalId  $objId

### STEP 1: GRANT API PERMISSIONS TO MANAGED IDENTITY

#Retrieve the Azure AD Service Principal instance for the Microsoft Graph (00000003-0000-0000-c000-000000000000) or SharePoint Online (00000003-0000-0ff1-ce00-000000000000).
$servicePrincipal_Graph = Get-MgServicePrincipal -Filter "DisplayName eq 'Microsoft Graph'"
$servicePrincipal_SPO = Get-MgServicePrincipal -Filter "DisplayName eq 'Office 365 SharePoint Online'"

#Get AppRole Id for Sites.Selected
$appRole_GraphId = ($servicePrincipal_Graph.AppRoles | Where-Object { $_.AllowedMemberTypes -eq "Application" -and $_.Value -eq "Sites.Selected" }).Id
$appRole_SPOId = ($servicePrincipal_SPO.AppRoles | Where-Object { $_.AllowedMemberTypes -eq "Application" -and $_.Value -eq "Sites.Selected" }).Id

# Grant API Permissions
$graphParams = @{
    principalId = $sp.Id
    resourceId  = $servicePrincipal_Graph.Id
    appRoleId   = $appRole_GraphId
}
$spoParams=@{
    principalId = $sp.Id
    resourceId  = $servicePrincipal_SPO.Id
    appRoleId   = $appRole_SPOId
}
New-MgServicePrincipalAppRoleAssignment -ServicePrincipalId $sp.Id -BodyParameter $graphParams
New-MgServicePrincipalAppRoleAssignment  -ServicePrincipalId $sp.Id -BodyParameter $spoParams

#Quick check if everything went well
Get-MgServicePrincipalAppRoleAssignment -ServicePrincipalId $sp.Id


Enter fullscreen mode Exit fullscreen mode

Let's double check. Both, the App Registration page and the Enterprise App page show that API permissions have been granted:

API permissions on App Registrations pageon

API permissions on Enterprise Application page

Grant permissions to a SPO site

Next, grant permissions to the SPO site. This script is the continuation of the example above and is using $sp object to retrieve the Service Principal Display Name



### STEP 2: GRANT SPO ACCESS TO MANAGED IDENTITY
$tenantName = "{tenant-name}.sharepoint.com"

$spoSiteId = $tenantName + ":/sites/" + $siteName + ":"
$appRole = "write"
$application = @{
    id          = $appId
    displayName = $sp.DisplayName
}

Connect-MgGraph -Scope Sites.FullControl.All
New-MgSitePermission -SiteId $spoSiteId -Roles $appRole -GrantedToIdentities @{ Application = $application }



Enter fullscreen mode Exit fullscreen mode

The pipeline

So far everything looks really familiar. Let's look into the pipeline now.

I want to connect to the SP site using PnP.PowerShell. This time however, I won't be using service principal id or certificate. I want to use the Managed Identity from the pipeline's context.

This requires an additional step. First, I need to retrieve the token from the Azure Context, and then I'm using the token to connect to SPO.



$azAccessToken = Get-AzAccessToken -ResourceUrl $url
$conn = Connect-PnPOnline -Url "$url/sites/$(siteName)" -AccessToken $azAccessToken.Token -ReturnConnection
$web = Get-PnPWeb -Connection $conn


Enter fullscreen mode Exit fullscreen mode

This example pipeline has some extra try/catch. You wouldn't do it productively, but it will help you to see where any potential errors occur =)



variables:
- name: tenantName
  value: "contoso"
- name: siteName
  value: "siteName"

steps:
- task: AzurePowerShell@5
  name: ConnectPnpOnline
  inputs:
    azureSubscription: DEV_Connection
    azurePowerShellVersion: latestVersion
    ScriptType: InlineScript
    Inline: |
      $url = "https://$(tenantName).sharepoint.com"
      Write-Host "##[debug]Connecting to $url/sites/$(siteName)"

      Write-Host "##[group]Install/Import  PS modules"
      Install-Module PnP.PowerShell -Scope "CurrentUser" -Verbose -AllowClobber -Force
      Write-Host "##[endgroup]"

      try {
        $azAccessToken = Get-AzAccessToken -ResourceUrl $url
        $conn = Connect-PnPOnline -Url "$url/sites/$(siteName)" -AccessToken $azAccessToken.Token -ReturnConnection
        Write-Host "##[debug]Get-PnPConnection"
        Write-Host $conn.Url
      }
      catch {
          Write-Host "##[error] 1 (Connect-PnPOnline -AccessToken): $($_.Exception.Message)"
      }

      try {
        Write-Host "##[debug]Get-PnPWeb"
        $web = Get-PnPWeb -Connection $conn
        Write-Host  $web.Title
      }
      catch {
          Write-Host "##[error] 2 (Get-PnPWeb): $($_.Exception.Message)"
      }
  displayName: A couple of tests



Enter fullscreen mode Exit fullscreen mode

The script above grants Managed Identity write permissions, so let's try to create an item. Make sure you have a list named "Test"



variables:
- name: tenantName
  value: "contoso"
- name: siteName
  value: "siteName"

steps:
- task: AzurePowerShell@5
  name: DeploySPFx
  inputs:
    azureSubscription: DEV_Connection
    azurePowerShellVersion: latestVersion
    ScriptType: InlineScript
    Inline: |
      # Write-Host "##[group]Install/Import  PS modules"
      # Install-Module PnP.PowerShell -Scope "CurrentUser" -Verbose -AllowClobber -Force
      # Write-Host "##[endgroup]"

      $url = "https://$(tenantName).sharepoint.com"

      try {
        $azAccessToken = Get-AzAccessToken -ResourceUrl $url
        $conn = Connect-PnPOnline -Url "$url/sites/$(siteName)" -AccessToken $azAccessToken.Token -ReturnConnection
        Add-PnPListItem -List "test" -Values @{"Title"="$(Build.BuildId)"} -Connection $conn
      }
      catch {
          Write-Host "##[error]$($_.Exception.Message)"
      }
  displayName: Add item


Enter fullscreen mode Exit fullscreen mode

It works! =)

Item created

So cool! 🤩 Now we can get rid of the secrets and certificates

Why not Connect-PnPOnline -ManagedIdentity?

Currently, the Connect-PnPOnline -ManagedIdentity doesn't seem to support the Workload Identity federation. Would be great if this changed, so we don't have to retrieve the token separately

How do I know which service principal is really used?

You may use the following script to retrieve the object ID of the service principal:



$x = (Get-AzContext).Account.Id
$y = Get-AzADServicePrincipal -ApplicationId $x
Write-Host "##[debug] $($y.Id)"


Enter fullscreen mode Exit fullscreen mode

Execute it from the AzurePowerShell@5 task with azureSubscription set to the name of the service connection using the Workload Identity federation

UPDATE 18.09.2024: It seems like the DefaultAzureCredential will support the Azure Pipeline's workload identity federation soon-ish: Support for Azure DevOps Workload Identity but "Unfortunately, it didn't make the cut for the current quarter. They're reconsidering the work for next quarter". Can't wait =)

Top comments (4)

Collapse
 
eduardpauldev profile image
Eduard Paul • Edited

Before trying to use this to deploy spfx read: github.com/pnp/powershell/issues/3...

and be aware that officially only certificate auth is supported: learn.microsoft.com/en-us/sharepoi...

in any case great post and thanks for sharing. Hope Microsoft will take a look on this.

Collapse
 
kkazala profile image
Kinga • Edited

This is the ticket I opened myself, and I see you are linking to your own reply. I'm not done with this ticket yet, and I hope the PnP team is working in it.
I don't find replies form Jake very helpful, as they're just "mudding the waters", but... he admits himself that refreshing the token works with his old app registrations.

What I'm describing is not authenticating, but rather using of already authenticated identity. I'm running my code in a pipeline, in a context of already authenticated service principal, and I'm refreshing a token that already exists to exchange Azure/Graph token for a SPO token. The idea itself is not new and you can use it with service principals that you created yourself.

This token refresh works and I'd like to have this "rewrite" supported by Connect-PnPOnline natively. This is what this ticket is about.

regarding your problem...
The key here is to make sure that the Service Principal has correct API permissions to execute. I see you write "I could connect and read web props using the access token but, as soon as i try to install spfx solution I get the "The remote server returned an error: (401) Unauthorized."."
Please note that authentication and authorization are two different things. You get "unauthorized". It looks like you managed to refresh the token correctly, otherwise you wouldn't be able to read the web properties in the first place and you would be getting another error message.
I haven't seen your code and I don't know which libraries you are using. What I know, however, is that PnP is using different endpoints to execute different actions, and you need to make sure that your service principal has API Permissions for both, Graph and SharePoint APIs.

I am using the above approach to deploy my spfx solutions to app catalog, and it works just fine. The above example shows creating items for simplicity, but in my environment, it's app deployment.

I'm therefore 100% sure it works

If you want to check your code gradually, i'd suggest the following:

  • create your own app registration, and generate certificate
  • add all required API permissions (SPO and Graph) and if using Sites.Selected, grant access to the SPO site
  • authenticate in the pipeline using clientID and certificate and deploy your spfx app.
  • now change the service connection to the one using Workload Identity federation (or update existing one)
  • in Azure, make sure that the service principal used by the Workload Identity federation has the same permissions as your own app registration
  • test again. Assuming that both app registrations have permissions configured the same way, you will have the same results
Collapse
 
eduardpauldev profile image
Eduard Paul

I wanted to address a missing piece of information that might help anyone encountering a similar issue 😊 First off, I want to express my gratitude for your post as it led me to discover what I believe is the most effective method for connecting pipelines to the Microsoft 365 ecosystem. I realize I should've expressed my gratitude in my previous comment rather than in the GitHub issue (but it was really late in day 💤).

In my case, I utilized the Sites.FullControl.All permissions for both Graph and SharePoint APIs, along with AppCatalog.ReadWrite.All for testing purposes (global app catalog). And Get-PnPWeb functions smoothly with your approach and my configuration. However, I consistently encountered a "(401) Unauthorized" error as soon as the code reached the Add-PnPApp section (the only different line from your example). Interestingly, a previous app registration with identical permissions (but using a pfx+password) was functioning correctly. This led me to suspect that the issue lies with the access token generated from WIF versus certificate authentication or that some Microsoft endpoints does not support this kind of auth (very feasible for me - and could be different across tenants), though I'm not entirely certain.

Thread Thread
 
kkazala profile image
Kinga

A friend of mine once told me that things we write sound "harder" compared to if we said them, and I think she is right. I really hope you didn't feel scolded by my answer. Was not my intention.

It bothers me a lot that the deployment doesn't work for you. I just updated my script to deploy an app to the tenant-level app catalog and documented each step: Deploy SPFx app using pipeline's Workload Identity federation. And.. it works 🤷‍♀️One think that comes to my mind is that maybe you have to wait? (see the last section of the post referenced).