DEV Community: Shpend Kelmendi

How to deploy across multiple Entra Id Tenants with Multitenant App Registrations

Shpend Kelmendi — Mon, 08 Dec 2025 00:00:00 +0000

You have to deploy your workload.

You follow best practice and provision your workload per IaC and CI/CD pipelines. Now that sounds easy. But how do you do this when you need to deploy your workload to multiple Entra ID tenants?

In this blog post you will learn how this works and how you can do it by yourself step by step.

Problem

Many organizations work with multiple Azure tenants - perhaps you have separate tenants for different customers, business units or environments. When you need to deploy the same workload across these tenants using Azure DevOps pipelines, you face a challenge:

service connections are tied to a single tenant.

For my open source project (Secret Rotation) I was challenged to solve this problem.

So there are one home tenant (Entra ID) and two Entra External IDs as CIAMs. One for non productive environment and one for productive environment.

PS: I can really recommend to keep them apart.

Solution (I used)

The solution is to use a multitenant app registration that can authenticate across multiple Entra ID tenants. Here's how it works:

Create a multitenant app registration in your primary tenant (home tenant)
Create service principals in each target tenant (Non-Prod, Prod)
Assign appropriate permissions to each service principal
Use the same app registration credentials to deploy to all tenants

This approach centralizes authentication while distributing authorization, giving you the flexibility to deploy anywhere while maintaining security.

Prerequisites

Access and permissions to
- Entra ID (e.g. Contributer) and
- Entra External IDs (Application Developer role or higher)
Azure CLI installed
Access to Azure DevOps

1. Create and configure multitenant app registration

First, create a service connection in Azure DevOps:

In Azure DevOps:

Navigate to: Project → Project Settings → Service Connections
Create a new Azure Resource Manager service connection
Recommendation: Use Workload Identity Federation for better security
Click on "Manage App Registration" to open the Azure Portal

Configure multitenant support:

Option A - Using Azure Portal UI:

In the app registration, go to: Manage → Authentication (Preview)
Under Settings → Supported account types
Select: "Accounts in any organizational directory (Any Microsoft Entra ID tenant - Multitenant)"
Click "Save"

Option B - Using Azure CLI:

# Login and select your subscription linked to your home tenant
az login

# Get your app registration details from the Azure Portal Overview page
$clientIdFromHomeTenant = "{YOUR_APP_CLIENT_ID_IN_HOME_TENANT}"

# Update the app registration to multi-tenant
az ad app update --id $clientIdFromHomeTenant --sign-in-audience AzureADMultipleOrgs

2. Create service principal in target tenant(s)

For each tenant where you want to deploy, you need to create a service principal that represents your multitenant app:

# Define your target tenant and the app client ID from Tenant X
$tenantXId = "{YOUR_APP_REG_IN_TENANT_X}"

# Login to the target tenant (--allow-no-subscripts is needed if no subscription is linked, e.g. Entra External Id)
az login -t $tenantXId --allow-no-subscriptions

# Create the service principal (enterprise application)
az ad sp create --id $clientIdFromHomeTenant

Important: This creates an enterprise application in Tenant B, not a full app registration. The app registration always lives in Tenant A. The enterprise application in Tenant B is essentially a "pointer" to the app in Tenant A.

The result will look similar to the following extract:

{
  "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#servicePrincipals/$entity",
  ...
  "appDisplayName": "{YOUR_APP_REG_IN_HOME_TENANT}",
  "appId": "{APP_ID_FROM_HOME_TENANT}",
  "appOwnerOrganizationId": "{HOME_TENANT_ID}",
  ...
  "displayName": "{YOUR_APP_REG_IN_HOME_TENANT}",
  ...
  "servicePrincipalNames": [
    "{APP_ID_FROM_HOME_TENANT}"
  ],
  "servicePrincipalType": "Application",
  "signInAudience": "AzureADMultipleOrgs",
  ...
}

You can see that the information are taken from the App Registration in home tenant (Entra ID).

3. Assign Entra ID role to service principal

If your deployment needs to create or manage Entra ID resources (like app registrations), you need to assign appropriate directory roles:

Note: Some directory roles need to be activated before they can be assigned. Here's how to check and activate if needed:


# Check if the role is already activated
$activatedRole = az rest --method GET `
  --url "https://graph.microsoft.com/v1.0/directoryRoles?`$filter=roleTemplateId eq '$roleId'" `
  | ConvertFrom-Json

if ($activatedRole.value.Count -eq 0) {
  Write-Host "Activating role..."

  $activateBody = @"
    { "roleTemplateId": "$roleId" }
"@

  # Activate role because not active yet
  $activatedRole = az rest --method POST `
    --headers "Content-Type=application/json" `
    --url "https://graph.microsoft.com/v1.0/directoryRoles" `
    --body $activateBody | ConvertFrom-Json
}

Now you can assign the entra role to your service principal:

# Login to the target tenant
$tenantXId = '{YOUR_APP_REG_IN_TENANT_X}'
az login -t $tenantXId --allow-no-subscriptions

# Get the service principal object ID
$appName = "{YOUR_APP_NAME}"
$spObjectId = $(az ad sp list --display-name $appName --query "[0].id" -o tsv)

# Get the role definition ID (example: Application Administrator)
$roleId = $(az rest --method GET --url 'https://graph.microsoft.com/v1.0/directoryRoleTemplates' --query "value[?displayName=='Application Administrator'].id" -o tsv)

# Prepare the role assignment request with tenant scope
$bodyContent = @{
  "@odata.type" = "#microsoft.graph.unifiedRoleAssignment"
  "roleDefinitionId" = $roleId
  "principalId" = $spObjectId
  "directoryScopeId" = "/"
} | ConvertTo-Json

# Direct passing the body content failed. If you know why, let me know :). 
# Current workaround with json file
Set-Content -Path body.json -Value $bodyContent -Encoding UTF8

# Assign the role
az rest --method POST `
  --headers "Content-Type=application/json" `
  --url "https://graph.microsoft.com/v1.0/roleManagement/directory/roleAssignments" `
  --body '@body.json'

# Cleanup
Remove-Item body.json

4. Assign Azure RBAC roles (optional)

If you need to deploy Azure resources, assign appropriate Azure roles at the subscription or resource group level:

# Login and set the subscription context
az login
$subscriptionId="{YOUR_SUBSCRIPTION_ID}"
az account set --subscription $subscriptionId

# Assign Contributor role at subscription level
az role assignment create `
  --assignee $clientIdFromHomeTenant `
  --role "Contributor" `
  --scope "/subscriptions/$subscriptionId"

5. Use in Azure DevOps pipelines

Now you can use the same service connection to deploy to multiple tenants. In my secret rotation project I used DefaultAzureCredential. It uses the tenant id passed in bicepparam file.

Secret rotation is implemented based on local-deploy (Experimental). This allows me to run the secret-rotator on a monthly basis and create or rotate the secrets.

schedules:
  - cron: "0 20 1 * *" 
    displayName: Monthly Secret Rotation
    branches:
      include:
        - master

jobs:
  - job: RotateSecrets
    displayName: Rotate Azure Secrets
    pool:
      vmImage: ubuntu-latest
    steps:
      - task: AzureCLI@2
        displayName: rotate
        inputs:
          azureSubscription: "{YOUR_SERVICE_CONNECTION}"
          scriptType: "pscore"
          scriptLocation: inlineScript
          inlineScript: |
            $env:BICEP_TRACING_ENABLED = "true"
            bicep local-deploy ./bicep/secret-rotation/main.bicepparam

Conclusion

Deploying to multiple Entra ID tenants doesn't have to be complicated. By leveraging multitenant app registrations, you can:

Maintain a single app registration in your primary tenant
Deploy to as many target tenants as needed
Simplify your CI/CD pipeline

The key is understanding that the app registration lives in one place (home tenant), while service principals in each target tenant provide the necessary permissions to deploy resources.

Key takeaways:

Convert your app registration to multitenant
Create service principals in each target tenant
Assign appropriate Entra ID and Azure roles to your service principal per tenant
Use the same credentials with different tenant IDs in your pipelines

This approach scales well and makes managing deployments across multiple tenants much more maintainable.

Do you have a better approach? Any suggestion how to improve? Let me know :)

Quick Azure Cost Management: Budget and Notifications with Bicep and alternatives

Shpend Kelmendi — Fri, 21 Nov 2025 00:00:00 +0000

Managing costs in the cloud is crucial to avoid unexpected expenses.

How many times have I read "Azure just charged me 1k dollars because of a misconfiguration".

Did it happen to you as well? Me too. And I know how stupid you feel after that. Sometimes it is not even our fault. (cough* strange defaults from Azure *cough)

So to avoid that, we can setup budgets and get notified when we reach certain thresholds.

It's important to know which scope you want to set the budget on. It can be at management group level, subscription level or resource group level. Choose the scope you are responsible for and want to monitor costs for.

What are the options? Let's explore them.

Using Bicep

You can create a simple Bicep template to create a budget at subscription level with email notification. Here is a sample Bicep code:

targetScope = 'subscription'

@description('Name of the budget')
param budgetName string = 'MonthlyBudget'

@description('Monthly budget amount')
param amount int

@description('Email recipients for alerts')
param emailRecipients array

@description('Start date for the budget (YYYY-MM-DD)')
param startDate string = '${utcNow('yyyy-MM')}-01'

resource subscriptionBudget 'Microsoft.Consumption/budgets@2023-05-01' = {
  name: budgetName
  properties: {
    amount: amount
    category: 'Cost'
    timeGrain: 'Monthly'
    timePeriod: {
      startDate: startDate
    }
    notifications: {
      Forecasted_GreaterThan_90_Percent: {
        enabled: true
        operator: 'GreaterThan'
        threshold: 80
        contactEmails: emailRecipients
        thresholdType: 'Forecasted'
      }
      Actual_GreaterThan_80_Percent: {
        enabled: true
        operator: 'GreaterThan'
        threshold: 80
        contactEmails: emailRecipients
        thresholdType: 'Actual'
      }
      Actual_GreaterThan_100_Percent: {
        enabled: true
        operator: 'GreaterThan'
        threshold: 100
        contactEmails: emailRecipients
        thresholdType: 'Actual'
      }
    }
  }
}

This bicep template creates a monthly budget with specified amount and email notifications when the actual or forecasted costs exceed 80% and 100% of the budget. You will be notified via email to the specified recipients until end of date (because no end date is specified, it will run for 10 years after last deployment).

You can deploy this Bicep template using the Azure CLI with the following command:

az deployment sub create -l switzerlandnorth -f budget.bicep -p amount=100 emailRecipients="['YOUR-EMAIL@ADDRESS.COM']"

After successful deployment, you should see the budget in the Azure portal like this:

Using AZ CLI

Alternatively, you can create a budget using the Azure CLI. Here is an example command:

# Sign into Azure CLI with your account
az login

# Select a subscription to monitor with a budget
az account set --subscription "Your Subscription"

# Create an action group email receiver and corresponding action group
email1=$(az monitor action-group receiver email create --email-address test@test.com --name EmailReceiver1 --resource-group YourResourceGroup --query id -o tsv)
ActionGroupId=$(az monitor action-group create --resource-group YourResourceGroup --name TestAG --short-name TestAG --receiver $email1 --query id -o tsv)

# Create a monthly budget that sends an email and triggers an Action Group to send a second email.
# Make sure the StartDate for your monthly budget is set to the first day of the current month.
# Note that Action Groups can also be used to trigger automation such as Azure Functions or Webhooks.

az consumption budget create-with-rg --amount 100 --budget-name TestCLIBudget -g $rg --category Cost --time-grain Monthly --time-period '{"start-date":"2024-06-01","end-date":"2025-12-31"}' --notifications "{\"Key1\":{\"enabled\":\"true\", \"operator\":\"GreaterThanOrEqualTo\", \"contact-emails\":[],  \"threshold\":80.0, \"contact-groups\":[\"$ActionGroupId\"]}}"

Source of code

But be careful: it's in preview and might change in the future. Another challenge with this approach is readability and maintainability of the code.

Using Azure Portal

You can also create a budget directly from the Azure Portal. Follow these steps:

Navigate to Subscription or Resource group depending on your responsibility.
Click on "Budgets" under the "Cost Management" section.
Click on "+ Add" to create a new budget.
Enter the budget details such as name, amount, time period.
Set alerts by setting first condition to 80% and second to 100% with email notifications.
Add all email addresses that should receive the notifications.
Hit "Create" to finalize the budget setup.

Conclusion

I highly recommend setting up budgets for your Azure resources to avoid unexpected costs.

This should be done as early as possible in your cloud journey.

It's one of the best practices in the Azure Well-Architected Framework under Cost Management pillar. By applying budgets, you can monitor and control your cloud spending effectively and achieve the fullfil the following design principle:

"Continuously right-size investment as your workload evolves with the ecosystem."

Using Bicep makes it easy to automate the setup and ensures consistency across your environments.

Another benefit for using Bicep is readability and maintainability of your infrastructure as code.

Other benefits of IaC include version control, collaboration, and repeatability. But that is a topic for another blog post.

You can customize the budget amount and notification thresholds as per your requirements.

Go always for the automated way if possible!

PS: If you still get surprised by unexpected costs, consider contacting Azure Support for a refund. They are usually quite helpful in such cases.

Happy budgeting!

Practical guide to organizing Azure resources

Shpend Kelmendi — Mon, 17 Nov 2025 00:00:00 +0000

Organizing Azure resources is one of those rare tasks that sounds straightforward until you actually start doing it.

Suddenly you’re trying to balance cost allocation, security boundaries, governance rules, domain ownership, deployment pipelines, and team autonomy. All while the number of services grows faster than your architecture diagram can keep up.

If designing cloud boundaries feels like organizing a city while it’s still being built, that’s because it is.

And like any good city, Azure needs structure: districts, buildings, addresses, zoning rules, and a way to understand who owns what. Why? Because moving buildings (or Azure resources) is not always easy.

This guide takes a practical look at how to design resource boundaries in Azure—using concepts from Domain-Driven Design (DDD), Team Topologies, CAF, Well-Architected Framework.

Why this is hard: it's about people and the domain, not just resources

Maybe you have seen the one video:

Well it's funny, but it is also how some people think about organizing Azure resources. Most architecture decisions aren’t purely technical.

They reflect how teams collaborate, how business domains interact, and what outcomes the organization values.

When teams ask how they should group Azure resources, they are often really asking:

How do we ensure ownership is clear? Who is responsible for this workload?
How do we track costs reliably?
How do we deploy our workload? What do we deploy together?
How do we enforce consistent governance without slowing down?
How do we react to changes in requirements and evolution of the architecture?

Understanding these drivers makes resource organization far more meaningful than simply creating a subscription or a resource group "because that’s what the documentation says."

Approaches for our problem

I see some approaches to solve this problem:

Bounded context from Domain Driven Design (DDD)
Team Topology
Azure Resource Grouping Mechanism
- Mgmt Group
- Subscription
- Resource Group
- Tags
- Service Groups

So let's go through those and how those can help us.

DDD and team topologies: let your domain shape your cloud solution

One of the biggest challenges in cloud architecture is aligning resource boundaries with business domains and teams.

Domain-driven design (DDD)

DDD gives you with bounded context a powerful tool to define clear boundaries around different parts of your system. Its part of the strategic design patterns in DDD. With bounded contexts, you can encapsulate specific business logic, data models, and workflows within well-defined limits. Owned and managed by dedicated teams, bounded contexts reduce complexity and improve maintainability. It helps to answer the question:

"What belongs together?"
"What can evolve independently?"

A bounded context doesn't mean you have to deploy it separately. It can be a module within a larger application. But when it comes to cloud resources, aligning bounded contexts with Azure boundaries can bring significant benefits:

Each bounded context can map to its own resource group.
For bigger organizations, a dedicated subscription may be required. IMO this is not always necessary.
Independent life-cycles, data ownership, and pipelines align naturally with Azure boundaries.

Team topologies

Team topologies is another useful framework to consider when organizing Azure resources. It was introduced by Matthew Skelton and Manuel Pais in their book "Team Topologies". It emphasizes the importance of team structure and interactions in shaping software architecture. The four fundamental team types are:

Stream-aligned teams: Own their domain end-to-end -> subscription or resource group.
Platform teams: Provide shared services (networking, identity, monitoring, ...) and reduces cognitive load for teams using it. -> subscription or/and management group.
Enabling teams: Support other teams -> no dedicated boundaries needed but governance needs to be considered.
Complicated subsystem teams: Specialized workloads that require deep expertise -> isolated subscription.

Maybe you are in a small organization where one team wears multiple hats. That's fine. You should think about how your solution will scale as the organization grows. Aligning Azure structure with team responsibilities reduces friction and clarifies ownership.

Azure grouping mechanism

Now that we have an understanding of DDD and Team Topologies, let's look at Azure grouping mechanisms and their purpose. Azure provides multiple mechanisms to organize resources, each optimized for different scenarios.

Overview of Azure grouping mechanisms

Let's have a look at what is available:

Source: https://learn.microsoft.com/en-us/azure/governance/service-groups/overview

Mechanism	Description
Management groups	Logical containers that organize management groups and subscriptions into a hierarchy for unified policy and access management. Policies and RBAC permissions applied at this level inherit down to all subscriptions and resources below.
Subscriptions	Logical container linked to a credit card. Each subscription has its own limits, quotas, and provides isolation between different parts of your organization.
Resource groups	Logical containers that hold related Azure resources. Goal is to group resources the same lifecycle: deploy, update, and delete together.
Azure Service Groups	Logical container kept in tenant. Allows flexible grouping of resources across subscriptions, resource groups and resources. Currently in public preview.
Tags	Key-value pairs of metadata that can be applied to resources, resource groups and subscriptions. Tags do not inherit automatically. You have to enforce this by policy or tagged individually at each level.

Comparison of Azure grouping mechanisms

Scenario	Resource Group	Subscription	Management Group	Service Group	Tags
Can it contain and organize resources in a hierarchy?	Yes (contains resources)	Yes (contains resource groups)	Yes (contains subscriptions, management groups)	Yes (contains resources, resource groups, or subscriptions but only within Azure Service Group)	No
Can resources from different subscriptions be grouped together?	No	No	Yes	Yes	Yes**
Can the same resource belong to multiple groups of this type?	No	No	No	Yes	Yes
Can policies be enforced on this grouping?	Yes	Yes	Yes	No	Yes***
Are policies inherited downward in the hierarchy?	Yes (to resources)	Yes (to resource groups and below)	Yes (to subscriptions and below)	No	No
Can role-based access control (RBAC) be applied?	Yes	Yes	Yes	Yes (Service Group only)	No
Are RBAC permissions inherited downward in the hierarchy?	Yes (to resources)	Yes (to resource groups and resources)	Yes (to subscriptions and below)	Partial (only to child Service Groups, not to member resources)	No
Can it be used for cost management and billing?	Yes	Yes	Yes	No	Yes ****
Can it be used for different environments (prod, test, dev)?	Yes	Yes	Yes	No	Yes
Can it be used for ownership tracking?	Yes	Yes	Yes	No	Yes ****

* Applying a policy will enforce it to all members within the scope. For example, on a Resource Group it only applies to the resources under it.

** Tags can be applied across scopes and are added to resources individually. Azure Policy has built-in policies that can help manage tags.

*** Azure tags can be used as criteria within Azure Policy to apply policies to certain resources. Azure tags are subject to limitations.

**** To avoid inconsistencies, it's recommended to apply tags from the beginning and enforce them via policies. If you are already in this situation, you have different options:

Notify subscription/resource group owners to apply tags.
Notify users with access to subscriptions/resource groups to apply tags.
Stop/disable resources without required tags (not recommended for production workloads and only after clarified with management).

Conclusion

Resource organization in Azure is not a one-size-fits-all task. It requires a thoughtful approach that considers business domains, team structures, governance needs, and operational practices. By leveraging principles from DDD and Team Topologies, and utilizing Azure's grouping mechanisms effectively, you can create a resource organization strategy that aligns with your organization's goals and workflows.

Fix expired secrets in Azure DevOps Service Connections

Shpend Kelmendi — Wed, 12 Nov 2025 00:00:00 +0000

Workload Identity Federation is the recommended approach for Service Connection. So why would I use the old approach with an app registration where I need to rotate the secret by myself? Well, there are some pipeline tasks which doesn't support Workload Identity Federation.

And in this case, you have to use the old approach.

And today was the day of rotation.

And a team member tried to renew the secret but didn't know how to do it, because the problem is still not fixed correctly in the UI.

Let's check the steps to fix it.

1. Problem

You have created a service connection with app registration (automatic).

Your secret expired.

You don't know how to fix it.

2. Solutions

2.1 Renew secret

Go to ADO -> Project Settings
Click on Service Connection, which makes trouble
Click on "Manage App registration"
In Azure Portal -> Manage -> Certificate & secrets
Delete your expired secret
Create a new secret
Go back to ADO
Click on the Edit button
Change description
Click the Save button
Click on Edit again
Click on Verify
It should show Verification succeeded

You may need to repeat steps 8-13 if verification does not succeed the first time.

2.2 Use Workload Identity Federation

Verify if there is a newer version of the task that supports Workload Identity Federation.

In our example, it was the task AzureFileCopy which supports with from version 6 workload identity federation.

2.3 Implement yourself

A lot of the tasks can be done by yourself with pure scripts. Like the task mentioned above AzureFileCopy:az storage blob upload -f /path/to/file -c mycontainer -n MyBlob

The benefit would be that you can use the Ubuntu agent instead of the Windows agent.

3. Final thoughts

Recommendations are to use Workload Identity Federation where possible, because it's better not to handle secrets in any place and to forget the rotation. If this is not possible, try to implement it yourself if you feel confident.

If not, you can still go for the hacky way.

When the cloud has a cold: Lessons in resilience

Shpend Kelmendi — Thu, 30 Oct 2025 00:00:00 +0000

October 9th - Azure Front Door takes a nap. Traffic reroutes, dashboards go red, and half the internet suddenly learns what "global edge dependency" really means.

October 20th - an AWS region (US-EAST-1) goes down. Not the apocalypse, just enough to make dashboards bleed red, engineers reach for coffee, and LinkedIn light up with "That's why we use multi-cloud" posts.

October 29th - Azure Front Door stumbles again. Same story, different day. And somewhere in between, a few other providers quietly joined the chaos with their own "we're experiencing elevated error rates" moments.

It's been one of those months when you realize: even the clouds catch colds. The uptime gods don't play favorites - not AWS, not Azure, not anyone.

And that's when Barry O'Reilly's Residuality Theory came to mind again - the idea that what really matters isn't the outage itself, but what's left behind.
The tangled complexity.
The assumptions that didn't hold.
The edge cases you dismissed with a confident, "That'll never happen."

Spoiler: it just did and will again.

1. The shared responsibility model and why it matters

You're probably familiar with the Fallacies of Distributed Computing. These were observations made by L. Peter Deutsch and colleagues at Sun Microsystems about common misconceptions in the development of distributed systems.

Here's the list again as a refresher:

The network is reliable;
Latency is zero;
Bandwidth is infinite;
The network is secure;
Topology doesn't change;
There is one administrator;
Transport cost is zero;
The network is homogeneous;

But there's one more that should be added:

"The cloud provider is responsible for everything".

Let's get one thing straight:
Just because your app runs in the cloud doesn't mean it's automatically secure, scalable, performant or resilient.

Cloud providers like Azure and AWS take care of the infrastructure, but resilience is a shared responsibility.
They'll keep the lights on - but how your app reacts when the lights flicker? That's on you.

As you can see in the image above, Microsoft takes responsibility for the core platform and additionally provides resilience capabilities when the right components are selected (e.g. Zone-Redundancy, Automatic Backup, ...).

By components, I mean the Azure services. Depending on the category (SaaS, FaaS, PaaS, IaaS), Microsoft takes on more or fewer responsibilities and provides varying levels of support to help you achieve reliability.

For each component, there are also Reliability Guides available to help you with implementation.

At the end of the day, you're the captain of your workloads. You decide what "reliable" means, and that decision drives how you actually build and wire everything together. Microsoft can hand you the tools, but you still have to steer the ship.

2. What can we do about it?

You can't stop outages or incidents. It will happen. But you can design for failure.
Resilience isn't luck, it's the result of intentional, repeated practice.
Here are some options how I would build into a system:

2.1 Risk Storming

Risk Storming is a collaborative workshop technique designed to help teams identify, visualize, and discuss software risks before they become real problems and promotes shared ownership of quality. It's used mostly as pre-mortem to identify risks before they happen.

How to apply it

Prepare
- Gather your team (architect, ops, and devs, ...) in a (virtual) room. Virtual room: Use a shared whiteboard tool like Miro or Mural. In person: Use a whiteboard and post-its
- Prepare post-its:
  - 🔴 high impact/high likelihood → address immediately
  - 🟡 high impact/low likelihood → create mitigation
  - 🟢 low impact → document and monitor
- Define a scale you want to use (e.g. low/medium/high or numeric value 1-5)
- Prepare examples for better understanding
Draw your architecture (at different levels)
- Virtual room: use your drawings from your architecture documentation
- In person: use post-its (different color)
- This way you can put the identified risks close to the component.
- It can be current architecture or architecture which you plan to build or change
Identify risks (individually)
- Timebox (e.g. 10 min)
- everybody asks them self three questions in silence (no collaboration):
  1. What can go wrong? (Description)
  2. How likely is it? (Likelihood)
  3. What is the negative impact if the risk does occur? (Impact)
- Benefit of this approach is that each person can bring risks from their perspective
- Keep in mind that to build a solution you need people, process and technology. So consider those in your risks. For example:
  - People: We don't have the knowledge.
  - Process: Manual works slows down delivery.
  - Technology: Selected technology is not fulfilling the quality requirements.
Converge the risks
- Everybody put the identified risks on the diagram close to the area where the risk has been identified
- Big benefit of this approach is the visual approach. By doing this it gives you at a glance the hotspots of risks where you-> Focus on hotspot
Review risks and take actions
- Gather and summarize risks
- Don't ignore risks identified by single person or risks with a lot of disagreement

2.2 Residuality Theory

Most risk management frameworks deal with the knowns, predictable failures you can plan for. Barry O'Reilly's Residuality Theory takes a different route: it focuses on how to survive the unknowns.

You start by identifying stressors, unexpected forces that might hit your system. Not just technical ones like "database outage" but also business and human ones: "marketing goes viral", "supplier vanishes", or "new law kills a feature overnight". No idea is too crazy, that's the whole point.

Then, you apply those stressors and watch what happens. Which parts of your system survive - the residues - and which collapse? You evolve the architecture by adding or adapting components that can handle that stress next time. And then you run the experiment again.

Residuality Theory doesn't just build resilient systems; it builds organizations that can absorb chaos and come out smarter.

Resilience isn't about predicting the storm - it's about sailing through it.

If you want to dive deeper into this topic, here are a few gems worth your time:

2.3 Designing for failure

Let's talk frameworks.
Azure's Well-Architected Framework is basically the cloud version of life advice your grandparents would give you:

keep it simple,
plan for the unexpected,
and don't spend all your money in one place.

The Reliability pillar boils down to this:

"Assume things will fail, and make sure your system doesn't fail with them."

That means designing with principles like:

Prioritize your critical workloads - not every app needs five-nines availability.
Keep your architecture simple enough to understand under pressure.
Use automation and self-healing mechanisms.
Test your recovery, not just your uptime.
Document trade-offs - because every design choice costs something.

And that last one - trade-offs - is the sneaky one.
Resilience sounds amazing, until your CFO sees the cloud bill.
Or your security team starts asking why you've got redundant data flows across three regions.
That's the point where architecture becomes negotiation.

The trade-offs

Here's the ugly truth:
Every resilience decision is also a complexity decision.

Implement multi-cloud failover?
Congratulations, you now have two sets of APIs, IAM policies, monitoring tools, and probably twice as many headaches.
Multi-cloud can make sense - especially for regulatory reasons or when you truly need provider diversity.
But if you're doing it to "avoid downtime," you might just be swapping one type of risk for another.

In the Azure Well-Architected Framework, this is where trade-offs between pillars come into play:

More Resilience might mean less Cost Optimization.
Higher Security can sometimes conflict with Performance Efficiency.
Greater Scalability may reduce Operational Simplicity.

The trick isn't to pick one - it's to be conscious of what you're giving up, and why.
Because architecture isn't about perfection.
It's about balance.

3. Lessons Learned

Here's what all of this boils down to, after years of watching clouds both shine and storm:

Everything fails eventually.
Design for failure.
You own your architecture's reaction to chaos.
The provider gives you tools; you choose how to use them.
Complexity is the enemy of resilience.
Every extra moving part needs love, documentation, and monitoring.
Culture matters as much as code.
Teams that share responsibility, learn openly, and test failure often build systems that survive it.

4. Closing Thoughts

Cloud architecture isn't about chasing perfection.
It's about designing systems - and teams - that can bend without breaking.

Outages like the ones from AWS and Azure are humbling reminders that resilience isn't a checkbox.
It's a mindset.
It's what happens when you accept that things will go wrong - and prepare anyway.

So next time the cloud sneezes, don't roll your eyes.
Take notes.
Because in those few chaotic hours, you'll find more lessons about resilience than in any documentation page.

And if you’re curious to dig deeper into the Azure Front Door outage, join the Azure Incident Retrospective — sign up here.
It takes place today, October 30 (5:15 PM – 6:00 PM UTC+01:00) and October 31 (5:15 PM – 6:00 PM UTC+01:00).

Remember: You can't avoid failure. But you can architect for recovery.

DEV Community: Shpend Kelmendi

How to deploy across multiple Entra Id Tenants with Multitenant App Registrations

Problem​

Solution (I used)​

Prerequisites​

1. Create and configure multitenant app registration​

2. Create service principal in target tenant(s)​

3. Assign Entra ID role to service principal​

4. Assign Azure RBAC roles (optional)​

5. Use in Azure DevOps pipelines​

Conclusion​

Further readings​

Quick Azure Cost Management: Budget and Notifications with Bicep and alternatives

Using Bicep​

Using AZ CLI​

Using Azure Portal​

Conclusion​

Further readings​

Practical guide to organizing Azure resources

Why this is hard: it's about people and the domain, not just resources​

Approaches for our problem​

DDD and team topologies: let your domain shape your cloud solution​

Domain-driven design (DDD)​

Team topologies​

Azure grouping mechanism​

Overview of Azure grouping mechanisms​

Comparison of Azure grouping mechanisms​

Conclusion​

Further readings​

Fix expired secrets in Azure DevOps Service Connections

1. Problem​

2. Solutions​

2.1 Renew secret​

2.2 Use Workload Identity Federation​

2.3 Implement yourself​

3. Final thoughts​

When the cloud has a cold: Lessons in resilience

1. The shared responsibility model and why it matters​

2. What can we do about it?​

2.1 Risk Storming​

How to apply it​

2.2 Residuality Theory​

2.3 Designing for failure​

The trade-offs​

3. Lessons Learned​

4. Closing Thoughts​

Problem

Solution (I used)

Prerequisites

1. Create and configure multitenant app registration

2. Create service principal in target tenant(s)

3. Assign Entra ID role to service principal

4. Assign Azure RBAC roles (optional)

5. Use in Azure DevOps pipelines

Conclusion

Further readings

Using Bicep

Using AZ CLI

Using Azure Portal

Conclusion

Further readings

Why this is hard: it's about people and the domain, not just resources

Approaches for our problem

DDD and team topologies: let your domain shape your cloud solution

Domain-driven design (DDD)

Team topologies

Azure grouping mechanism

Overview of Azure grouping mechanisms

Comparison of Azure grouping mechanisms

Conclusion

Further readings

1. Problem

2. Solutions

2.1 Renew secret

2.2 Use Workload Identity Federation

2.3 Implement yourself

3. Final thoughts

1. The shared responsibility model and why it matters

2. What can we do about it?

2.1 Risk Storming

How to apply it

2.2 Residuality Theory

2.3 Designing for failure

The trade-offs

3. Lessons Learned

4. Closing Thoughts