Ganiyat Olagoke Adebayo

Posted on Feb 14

End-to-End Monitoring in Azure: From Logs to Alerts

#aws #devops #cloud #ai

Monitoring is essential for maintaining application reliability, performance, and security in the cloud.
With Azure Monitor, organizations can collect telemetry from applications, virtual machines, and networks, analyze it centrally, and trigger alerts when issues arise.

This walkthrough follows a practical implementation that moves from creating a Log Analytics workspace to configuring full monitoring and alerting. However, before delving into the project proper, we need to prepare the environment.

Environment Preparation for Azure Monitor Project (Step-by-Step)

Before configuring monitoring, we must deploy the infrastructure that will generate telemetry.
This setup includes resource groups, security identities, virtual machines, and web applications.

All resources in this guide are deployed using the Azure portal.

Step 1 – Create the Resource Group

In the portal search bar, type Resource groups.

Click Create.

Select your Subscription.
Enter rg-alpha as the name.
Choose East US as the region.

Click Review + Create → Create.

Result

A container is available to hold all lab resources.

Step 2 – Create the App Log Examiners Security Group

This group will later be granted permissions to review monitoring data.

Search for Microsoft Entra ID (Azure Active Directory).

Select Groups.

Click New group.

Configure:
- Group type: Security
- Group name: App Log Examiners
- Description: App Log Examiners
Click Create.

Result

An identity group exists for access delegation.

Step 3 – Deploy and Configure Windows VM (WS-VM1)

This VM will host IIS and generate logs for monitoring.

Create the VM

Search Virtual Machines.
Click Create → Azure virtual machine.

Basics configuration:

Resource group: rg-alpha
VM name: WS-VM1
Region: East US

Availability: No infrastructure redundancy
Security type: Standard
Image: Windows Server 2022 Datacenter: Azure Edition

Size: Standard_D4s_v3
Username: prime
Password: your secure password

Inbound port: RDP (3389)

Click Review + Create → Create.

Restrict RDP access

After deployment:

Open the VM → Networking.
Select the RDP rule.

Change Source to My IP address.

Save.

Add HTTP inbound rule

From Networking → Add inbound port rule.

Configure:
- Source: Any
- Service: HTTP
- Action: Allow
- Priority: 310
- Name: AllowAnyHTTPInbound
Click Add.

Connect to the VM

Open the VM → Connect.
Under Native RDP, choose Download RDP file.

Open it.
Username: .\prime
Enter your password.

Install IIS

Inside the VM:

Right-click Start → Windows PowerShell (Admin).
Run as Admin:

Install-WindowsFeature Web-Server -IncludeAllSubFeature -IncludeManagementTools

Deploy sample website

cd c:\inetpub\wwwroot\
wget https://raw.githubusercontent.com/Azure-Samples/html-docs-hello-world/master/index.html -OutFile index.html

Result

A Windows web server is now producing logs.

Step 4 – Deploy Linux VM (Linux-VM2)

This machine will be used later for network and monitoring configuration.

Create the VM

Go to Virtual Machines → Create.

Basics:

Resource group: rg-alpha
Name: Linux-VM2
Region: East US
Image: Ubuntu Server 20.04

Size: Standard_D2s_v3
Authentication: Password
Username: Prime

Public inbound ports: None
Review and create.

Install Network Watcher Agent

Open the VM.
Select Extensions + applications.
Click Add.

Choose Network Watcher Agent for Linux.

Create.

Result

Linux VM ready for monitoring integration.

Step 5 – Deploy Web App with SQL Database

This deployment provides application + database telemetry.
Ensure that you’re signed into the Azure Portal.

In your browser, open a new browser tab and navigate to https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.web/web-app-sql-database

Open the GitHub quickstart.
Click Deploy to Azure.

If prompted, sign in.

Edit the template

Select Edit template.

Delete lines 158–174 and the comma on line 157.

Click Save.

Provide parameters

Resource group: rg-alpha
Region: East US
SKU: F1
SQL admin: prime
Password: your secure password

Click Create.

Result

A web application backed by SQL is deployed.

Step 6 – Deploy Linux Web App

Open the quickstart link.
Click Deploy to Azure.

Provide configuration

Resource group: rg-alpha
Region: East US
Web app name: AzureLinuxAppXXXX
SKU: S1
Runtime: PHP 7.4

Click Create.

Environment Setup Complete

At this point, your lab includes:

Resource group
Identity security group
Windows IIS VM
Linux VM
Web app with SQL
Linux web app

These components will generate the telemetry required for Log Analytics, Application Insights, data collection rules, and alerts.

Conclusion

The environment is now fully prepared to support centralized monitoring.
The next phases will connect these resources to Azure Monitor, enable diagnostics, and implement alerting mechanisms.

This preparation stage ensures realistic, production-style visibility across both PaaS and IaaS services.

Project Exercises

Deploy Log Analytics
Monitor web applications
Configure monitoring for compute services
Configure alerts

Each exercise builds on the previous one.

Step 1 – Deploy Log Analytics Workspace

Create the workspace

Sign in to the Azure portal.
In the search bar, type Log Analytics workspaces.

Select Create.

Choose your Subscription and Resource Group.
Enter a Workspace name.
Select a Region.

Click Review + create → Create.

Step 2 – Install and Configure the Azure Monitor Agent on Linux-VM Using a Data Collection Rule (DCR)

Once the Linux VM is deployed, the next step is to enable monitoring by installing the Azure Monitor Agent (AMA) and configuring a Data Collection Rule (DCR) to send telemetry to the Log Analytics workspace.

This ensures that performance data from the Linux VM is automatically collected and available for analysis.

Step 2.1 – Create a Data Collection Rule

In the Azure Portal search bar, type Data Collection Rules.

Click + Create to start a new DCR.

Basics

Rule name: DCR-Linux-VM
Subscription: Your subscription
Resource Group: rg-alpha
Region: East US (or the region matching your VM/workspace)
Platform type: Linux

Click Next: Resources >.

Resources

Click + Add resources.

Search for Linux-VM2 and select it.

Click Apply.

Click Next: Collect and deliver >.

Collect and Deliver

Click + Add data source.

Data source type: Performance Counters
Click Next: Destination >.

Destination

Click + Add destination.

Destination type: Azure Monitor Logs
Destination Details: Select LogAnalytics1 (rg-alpha)
Click Add data source.

Click Next: Review + create.

Review and Deploy

Review all settings.
Click Create.

Note: Assigning the DCR automatically deploys the Azure Monitor Agent to the VM. You do not need to manually add the agent under Extensions.

Step 2.2 – Verify Agent Installation and Data Ingestion

Search for Log Analytics workspaces in the portal.

Select LogAnalytics1 from the workspace list.

In the left-hand menu, select Logs.

Switch the mode from Simple to KQL (Query Language mode).

In the query window, enter:

Heartbeat
| where Computer contains "Linux-VM2"
| sort by TimeGenerated desc

Click Run or press Shift + Enter.

Result:
If results are returned, the Azure Monitor Agent is installed correctly, and performance data is flowing into the Log Analytics workspace.

Outcome

The Linux VM is now fully monitored:

Performance data is automatically collected.
Logs are centralized in LogAnalytics1.
The VM is ready for inclusion in further monitoring exercises, alerts, and analysis.

Step 3 – Configure Log Analytics Data Retention and Daily Cap

After creating the Log Analytics workspace, it is important to configure how long data is kept and how much data can be ingested daily. This helps control costs and ensures compliance with organizational policies.

Step 3.1 – Set Data Retention

In the Azure Portal search bar, type Log Analytics and select Log Analytics workspaces.
Click on LogAnalytics1 from the workspace list.
In the left-hand menu, under Settings, select Usage and estimated costs.
Click Data Retention.
Set the retention slider to 60 days.

Click OK.

Step 3.2 – Configure Daily Data Cap

On the LogAnalytics1 workspace page, under Settings → Usage and estimated costs, select Daily cap.
Switch the setting to On.
Set the daily cap to 10 GB.

Click OK.

Outcome

Data will be retained for 60 days in the workspace.
Daily ingestion will be limited to 10 GB, preventing unexpected costs.
The workspace is now ready for all subsequent monitoring tasks.

Step 4 – Enable Access to the Log Analytics Workspace

After creating and configuring the workspace, you need to assign permissions to users or groups who will be analyzing the data. In this lab, the App Log Examiners security group will be granted read access.

Step 4.1 – Open the Workspace

In the Azure Portal search bar, type Log Analytics and select Log Analytics workspaces.
Click LogAnalytics1 from the list of workspaces.
In the workspace menu, select Access control (IAM).
Click Add → Add role assignment.

From the list of roles, select Log Analytics Reader.

Click Next.
On the Members page, click Select members.
Search for and select App Log Examiners security group.

Click Select.
Click Review + Assign to complete the assignment.

Outcome

The App Log Examiners group now has read access to the Log Analytics workspace.
Members of this group can view and query telemetry data without having write permissions.
The workspace is ready for monitoring and analytics tasks.

Step 5 – Configure Monitoring for the Web App and SQL Database

After preparing the infrastructure and the Log Analytics workspace, the next objective is to ensure that the application layer sends telemetry for analysis.

In this section, you will:

Enable Application Insights
Disable unnecessary debugging features
Route HTTP logs to Log Analytics
Send SQL Insights data to the workspace

Step 5.1 – Enable Application Insights

In the Azure Portal search bar, type rg-alpha and open the resource group.

From the resource list, select the App Service associated with the Web App and SQL Database deployment.

Under Monitoring, choose Application Insights.

Click Turn on Application Insights.

Ensure Create new resource is selected.
Confirm the Log Analytics workspace is set to LogAnalytics1.

Click Apply.
In the confirmation dialog, select Yes.

Outcome

The web application can now collect request rates, failures, dependencies, and performance metrics.

Step 5.2 – Disable .NET Core Snapshot Debugger

In lab environments, this feature is not required and may generate unnecessary overhead.

Stay within the App Service.
Under Monitoring, open Application Insights.
In Instrument your application, select .NET Core.

Set Snapshot Debugger to Off.
Click Apply.
Confirm by choosing Yes.

Outcome

Diagnostic capture is optimized for essential telemetry only.

Step 5.3 – Send Web App HTTP Logs to Log Analytics

From the App Service menu, select Diagnostic settings.
Click + Add diagnostic setting.

Configure the following:

Property	Value
Diagnostic setting name	httplogs
Categories	HTTP logs
Destination	Send to Log Analytics workspace
Subscription	Your subscription
Workspace	LogAnalytics1

Click Save.

Outcome

Web traffic logs are now centralized in the workspace.

Step 5.4 – Send SQL Insights to Log Analytics

Return to the rg-alpha resource group.
Select the SQL database created during deployment.

Under Monitoring, choose Diagnostic settings.

Click Add diagnostic setting.

Provide:

Property	Value
Diagnostic setting name	InsightLogAnalytics
Categories	SQL Insights
Destination	Send to Log Analytics workspace
Subscription	Your subscription
Workspace	LogAnalytics1

Click Save.

Outcome

Database performance and behavior can now be correlated with application activity.

Result of Step 5

At this point, you have achieved:

Application performance monitoring
Centralized HTTP request logging
SQL telemetry integration
Unified visibility across app and database

Your Log Analytics workspace is becoming the single source of truth for operational data.

6. Configure Monitoring for Compute Services

step 6.1 – Create a Data Collection Endpoint

In the Azure portal search bar, type Monitor and open Azure Monitor.

Under Settings, select Data Collection Endpoints.

Select Create.

Basics tab

Property	Value
Endpoint name	IaaSVMCollectionEndpoint
Subscription	Your subscription
Resource Group	rg-alpha
Region	East US

Select Review + Create.
Select Create.

6.2 – Create a Data Collection Rule

In the Azure portal search bar, type Monitor and open it.
Under Settings, select Data Collection Rules.
Select Create.

Basics tab

Property	Value
Rule name	WinVMDCR
Subscription	Your subscription
Resource Group	rg-alpha
Region	East US
Platform type	Windows
Data collection endpoint	IaaSVMCollectionEndpoint

Select Next.

Resources

Select Add resources.
On Select a scope, check WS-VM1.

Select Apply.
Select Next.

Collect and deliver

Select Add data source.
Choose Windows Event Logs.

Configure:

Application → Critical, Error
Security → Audit Failure
System → Critical, Error

Select Next.

Destination

Property	Value
Destination type	Azure Monitor Logs
Subscription	Your subscription
Account or namespace	LogAnalytics1

Select Add data source.

Select Review + Create.

Select Create.

The rule will now begin sending VM logs to the workspace.

Step 6.3 – Add IIS Log Collection to the Existing Rule

In the Azure portal search bar, type Monitor and open it.
Select Data Collection Rules.
Choose WinVMDCR in resource group rg-alpha.

Add source

Under Configuration, select Data sources.

Select Add.

Choose IIS Logs.
Select Next.

Destination

Property	Value
Destination type	Azure Monitor Logs
Subscription	Your subscription
Account or namespace	LogAnalytics1

Select Add data source.

IIS telemetry is now included in the rule.

Step 7 – Configure Connection Monitor for the Linux VM

This validates network connectivity between machines and stores results in LogAnalytics1.

In the Azure portal search bar, type Network Watcher and open Azure Network Watcher.

Under Monitoring, select Connection Monitor.

Select Create.

Basics

Property	Value
Connection Monitor name	LinuxVMPubIP
Subscription	Your subscription
Region	East US 2
Workspace	LogAnalytics1

Select Next.

Test group

Enter name LinuxIPTest.
Select Add sources.

Azure Endpoints
Type → Virtual machines
Select Subnet
Check Linux-VM
Select Add endpoints.

Test configuration

Select Add Test Configuration.

Name → DefaultHTTP.
Select Add Test Configuration.

Destinations

Select Add destinations.

Azure Endpoints
Type → Virtual machines
Select Subnet
Check WS-VM1
Select Add endpoints.

Select Add Test Group.

Finish

Select Review + Create.
Select Create.

You have now:

Connected Windows event logs
Connected IIS logs
Configured VM telemetry
Enabled network performance monitoring

Your compute environment is fully integrated with Log Analytics.

8 Configure Alerts and Notifications

Step 8.1 – Create an Action Group to Send Email

In the Azure portal search bar, type Monitor and open Azure Monitor.

From the left menu, select Alerts.

Choose Action groups.

Select Create.

Basics

Property	Value
Subscription	Your subscription
Resource Group	rg-alpha
Region	Global
Action group name	NotifyCPU
Display name	NotifyCPU

Select Next: Notifications.

Notifications

Notification type → Email/SMS message/Push/Voice
Name → NotificationEmail
Select the pencil (edit) icon.

Inside the pane:

✔ Email
Enter → prime@fabrikam.com

Select OK.

Select Review + Create.
Select Create.

**Your email notification channel is ready.

Step 8.2 – Create an Alert for VM CPU Utilization

We will trigger an email when CPU usage is high.

In the Azure portal search bar, type rg-alpha and open the resource group.
From the resources list, select Linux-VM2.

Under Monitoring, select Alerts.

Choose Create → Alert rule.

Condition

Signal name → Percentage CPU.

Keep default threshold values.
Select Next.

Actions

Select Select action group.
Choose NotifyCPU.

Select Select.

Details

Property	Value
Alert rule name	HighCPU

Select Review + Create.

Select Create.

** Now, whenever CPU crosses the threshold, an email goes to prime@fabrikam.com.

Step 9 – Clean Up the Subscription

Important for labs and cost control.

Delete the Resource Group

In the Azure portal search bar, type Resource groups and open it.
Select rg-alpha.
Select Delete resource group.
Type rg-alpha to confirm.
Select Delete.

Delete the Security Group

Search for Microsoft Entra ID and open Microsoft Entra ID.
Select Groups.
Open App Log Examiners.
Select Delete → Yes.

Executive Summary – Azure Monitoring & Observability Implementation

This project delivered an end-to-end monitoring and observability framework across application, database, and infrastructure resources using Azure Monitor in Microsoft Azure.

The engagement established centralized telemetry collection, standardized data ingestion, controlled access to operational data, and proactive alerting. The result is improved service reliability, faster incident response, and stronger cost governance.

Objectives

Centralize logs and metrics into a single analytics platform
Enable performance visibility for web and database workloads
Collect security and system events from virtual machines
Monitor network connectivity between resources
Implement automated alerts for critical conditions
Apply role-based access and data retention policies

Solution Delivered

Centralized Log Platform

A Log Analytics workspace was configured as the unified destination for platform logs, application diagnostics, performance counters, and network telemetry.

Retention policies and a daily ingestion cap were implemented to balance operational insight with financial control.

Application & Database Observability

Application performance monitoring was enabled through Azure Application Insights, providing visibility into requests, dependencies, and failures.

Diagnostic pipelines were configured so that:

Web server HTTP logs
SQL Insights

flow into the central workspace for correlation and investigation.

Infrastructure Monitoring

Using Data Collection Rules and endpoints, virtual machines were onboarded to send:

Windows event logs (critical, error, audit failures)
IIS operational data
Linux performance telemetry

Agent deployment was automated through rule association, ensuring consistency and reducing administrative overhead.

Network Visibility

Connectivity monitoring between Linux and Windows workloads was implemented via Azure Network Watcher.

This provides continuous validation of reachability and supports rapid isolation of communication failures.

Proactive Alerting

An action group was created to deliver email notifications when CPU utilization breaches defined thresholds.
This converts raw telemetry into actionable operations signals.

Governance & Security

Access to monitoring data followed least-privilege principles by assigning read-only rights to a designated security group.
Temporary lab resources were removed at the end of the project to eliminate unnecessary spend.

Business Impact

The implementation enables:

Faster troubleshooting through correlated data
Reduced downtime via early detection
Improved accountability with auditable logs
Predictable monitoring costs
Scalable onboarding of future workloads

The environment now reflects a production-ready monitoring baseline aligned with cloud operational best practices.