[Az] Day 03: Secret Management and Centralized Log Monitoring on Azure.

Introduction

In this article, we will discuss the process of establishing a secure and automated system for secret management for the environment.

Additionally, we will cover how to implement centralized log monitoring enhancing observability and operational efficiency.

1. Secret Management:
   
   Utilizes Azure Key Vault to securely manage and store sensitive information such as credentials, certificates, and secrets. This guarantees consistent and secure access across all applications and services.

2. Centralized Application Log Management:
   
   Implements a centralized logging system using Azure Log Analytics and Application Insights to collect and analyze logs from all applications. This setup allows for effective performance monitoring, issue troubleshooting, and operational insight maintenance across the environment.

3. CI/CD flow:
   

   Illustration of the software CI/CD flow

• The purpose of this CI/CD flow is to ensure that environment secrets, especially for production, are automatically generated by Pulumi and stored securely in Azure Key Vault.
• When developers need to deploy an application, they only need to specify the secret name in the deployment spec.
• During the deployment process, the CI/CD pipeline will automatically replace the secret name with the actual secret from Key Vault.
• This approach guarantees that sensitive production secrets remain secure and are not exposed to the team members.

Table of Contents

Configuration

Before we start coding, it's important to define our configuration settings. This involves specifying resource names and subnet address spaces that we'll use throughout the project.

Resource Groups

We categorize our resources into different Azure resource groups for better organization and management:

Resource Group	Description
Shared Resource Group	Where our Key Vault and logging components reside.
Hub VNet Resource Group	Contains our main VNet hub.
AKS VNet Resource Group	Contains resources specific to the AKS cluster.
CloudPC VNet Resource Group	For resources related to virtual desktops or cloud PCs.

Allocated Subnets

Again, this is the subnet Ip address spaces that we have defined in the previous post.

VNet Name	Subnet Name	Address Prefix	Total	Usable
1. Hub VNet	1.1 Firewall Subnet	192.168.30.0/26	64	59
	1.2 Firewall Management Subnet	192.168.30.64/26	54	59
	1.3 General Subnet	192.168.30.128/27	32	27
2. AKS VNet	2.1 AKS Subnet	192.168.31.0/24	256	251
3. CloudPC VNet	3.1 CloudPC Subnet	192.168.32.0/25	128	123
	3.2 DevOps Subnet	192.168.32.128/27	32	27

The configuration file

Here is the config.ts file will be used for all pulumi our projects:

View code:

inline

Note: Adding a number as a prefix to the Azure resource group names helps keep them sorted in sequence, making them easier to find and navigate.

The Common Project

To promote code reusability and maintainability, we create a common project named az-commons.
This library contains utilities and helper functions that we'll use across all our Pulumi projects.

The azEnv Module

This module provides functions to retrieve Azure environment configurations:

• Tenant ID: Identifies the Azure Active Directory (EntraID) tenant.
• Subscription ID: Identifies the Azure subscription where resources will be deployed.
• Current Principal: The object ID of the user or service principal executing the scripts.
• Region Code: The Azure region.

View code:

inline

The naming Module

This module helps generate resource names with a consistent prefix based on the Pulumi stack name:

• getGroupName: Prepends the stack name to a resource group name.
• getName: uses to format the name with convention {stack}-nameWithoutNumber-{suffix} and remove the numbers from the name.

View code:

inline

The stackEnv Module

This module provides functions to retrieve Pulumi stack environment configurations:

• isDryRun: Indicates whether the current execution is a dry run (preview) or an actual deployment.
• Organization: The Pulumi organization name.
• Project Name: The name of the Pulumi project.
• Stack: The name of the Pulumi stack.
• StackReference: This helper function ensures that a project correctly references stacks within the same organization and environment. For example, the dev stack of project az-02-hub-vnet will reference the dev stack of project az-01-shared. This mechanism prevents cross-environment resource referencing, ensuring that resources from different environments (e.g., dev and prod) are kept isolated and properly aligned within the intended environment.

View code:

inline

The Shared Project

Following the instructions from Day 01, we create a new project named az-01-shared.
This project will include the following components:

The Vault Module

Creating a Azure Key Vault is a secure storage solution for managing secrets, keys, and certificates. It helps safeguard cryptographic keys and secrets used by cloud applications and services.

• Vault Options:

  • enablePurgeProtection: This option enables purge protection for the Key Vault. When enabled, it prevents the permanent deletion of the vault and its contents for a specified retention period, even if a delete operation is performed. This is crucial for compliance and recovery scenarios.
  • enabledForDiskEncryption: This setting allows the Key Vault to be used for Azure Disk Encryption. It is necessary for encrypting virtual machine disks, ensuring that data at rest is protected.
  • softDeleteRetentionInDays: This specifies the number of days that deleted vault items (like keys, secrets, and certificates) are retained in a "soft deleted" state. During this period, they can be recovered. The minimum value is 7 days, and the maximum is 90 days.
  • enableRbacAuthorization: This enables Role-Based Access Control (RBAC) for managing access to the Key Vault. It requires authentication through EntraID, allowing for more granular and secure access management.

• Vault Roles Management: To implement the principle of least privilege, we create two EntraID groups:

  • AZ ROL DEV-SHARED-VLT READONLY: For read-only access to the Key Vault.
  • AZ ROL DEV-SHARED-VLT WRITE: For write access to the Key Vault.

View code:

inline

The Log Module

This module provisions a Log Analytics Workspace, which is used for collecting and analyzing telemetry data from various sources, providing insights into resource utilization and performance.

• Workspace Options:
  • immediatePurgeDataOn30Days: which allows data to be purged immediately after 30 days.
  • workspaceCapping: Sets a daily data ingestion quota to control costs and manage data volume.
  • sku: Defines the pricing tier for the workspace, which affects cost and features.

View code:

inline

The AppInsight module

This module provisions an Application Insights component for monitoring web applications, linking it to a Log Analytics Workspace for data ingestion.

• AppInsights Options:
  • kind and applicationType: Define the type of application being monitored, in this case, a web application.
  • retentionInDays: Sets the data retention period to 30 days.
  • immediatePurgeDataOn30Days: Allows data to be purged immediately after 30 days.
  • ingestionMode: Specifies that data ingestion is done through Log Analytics.

View code:

inline

The main index.ts module:

This is a main module for the shared project, and a similar structure is maintained across all related projects. This file is tasked with:

• Establishing Resource Groups.
• Deploying all the Azure Resources above.
• Exporting essential resource details for future use by other projects.

View code:

inline

Deployment and Cleanup

Deploying the Stack

To deploy the stack, execute the pnpm run up command. This provisions the necessary Azure resources. We can verify the deployment as follows:

• EntraID groups configured for Key Vault access control:
  

• Successfully deployed Azure resources:
  

Cleaning Up the Stack

To remove the stack and clean up all associated Azure resources, run the pnpm run destroy command. This ensures that any resources no longer needed are properly deleted.

Conclusion

By following this guide, we have successfully automated the deployment of secure secret management and centralized log management using Azure Key Vault, Log Analytics, and Application Insights with Pulumi.

This setup ensures that sensitive data is securely stored and that we have real-time monitoring of application performance across our environment.

Implementing RBAC and the principle of least privilege enhances the security posture of our infrastructure. Centralized logging enables us to efficiently troubleshoot issues and gain operational insights.

References

• az-01-shared.
• az-commons.

Next

Day 04: Develops a Virtual Network Hub for Private AKS on Azure

In the next article, We'll walk through the process of developing the first Hub VNet for a private AKS environment using Pulumi.
We will demonstrate how to seamlessly integrate a VNet with an Azure Firewall, along with configuring outbound public IP addresses.

Thank You

Thank you for taking the time to read this guide! I hope it has been helpful, feel free to explore further, and happy coding! 🌟✨

Steven | GitHub