DEV Community: Vimal

The Math Behind Machine Learning: A Beginner’s Map

Vimal — Fri, 30 Oct 2026 07:41:31 +0000

Machine learning can seem like a collection of mysterious algorithms. But many of its core ideas are built from a handful of mathematical tools.

Think of these tools as parts of a workflow: represent data, compare examples, describe uncertainty, and improve predictions.

1. Linear algebra: giving data a shape

A machine-learning model needs data in a form it can work with. Linear algebra provides that form.You may already know the equation of a straight line:

y = ax + b

Here, x is the input, y is the output, a is commonly known as slope, and b commonly known as intercept tells us where the line crosses the y-axis. For example, if y = 2x + 1, then every time x increases by 1, y increases by 2.

 y
 5 |             *
 4 |          *
 3 |       *
 2 |    *
 1 | *
 0 +---------------- x
    0  1  2  3  4

Each * is a point (x, y) on the line. A graph makes the relationship between inputs and outputs visible.

In terms of machine-learning model given y = ax + b, the model uses input x to predict output y. During training, it adjusts a and b to make its predictions better.

A line on a graph is a simple model; vectors and matrices give us a compact way to represent and calculate its predictions.

Imagine describing a house using three numbers:

[ size, bedrooms, age ]
[ 1200, 3,         20 ]

That list of numbers is a vector: one object represented by several values. A collection of objects can be arranged into a matrix, like a spreadsheet:

              size   bedrooms   age
House A       1200      3        20
House B       1800      4         5
House C        900      2        30

In short: vectors describe individual examples; matrices organize many examples.

2. Analytic geometry: deciding what is similar

Once data is represented as vectors, we can compare examples mathematically.

For instance, two houses with similar size, bedroom count, and age may be considered similar. A model can use this idea to make predictions: if two inputs are alike, their predicted outputs like price of the house may also be alike.

Geometry gives us ways to describe similarity and distance between vectors.

In short: geometry helps answer, “Which examples are close to this one?”

3. Matrix decomposition: uncovering useful patterns

A large matrix can contain patterns that are hard to spot by looking at every number. Matrix decomposition breaks a matrix into simpler pieces that can reveal important structure.

This can help with tasks such as finding patterns in data or making some learning operations more efficient.

In short: decomposition helps simplify data without losing the patterns a model needs.

4. Probability: handling noise and uncertainty

Real-world data is rarely perfect. Measurements may be inaccurate, labels may be mistaken, and future outcomes may be unpredictable.

Probability theory gives us a language for describing this uncertainty. It helps us reason about questions such as:

How much variation might be noise?
How likely is a particular outcome?
How confident should we be in a prediction?

In short: probability helps a model represent what it knows—and what it doesn’t know for sure.

5. Vector calculus: understanding how changes affect a model

A model’s predictions depend on its parameters—the adjustable values it learns from data. Changing those parameters can improve or worsen its performance.

Vector calculus helps describe how a function changes as its inputs change. One especially useful idea is the gradient: it indicates the direction in which a function increases most quickly.

For learning, we often use the gradient to figure out how to adjust parameters to reduce errors.

In short: gradients tell us how parameter changes affect performance.

6. Optimization: improving the model

Training often means finding parameter values that make a model perform well. Optimization is the process of searching for those values.

A simple way to picture it is walking downhill on a landscape:

The landscape represents the model’s error.
Your position represents the current parameter values.
The gradient points toward a direction of change.
The goal is to reach a low point, where the error is smaller.

The process repeats: calculate performance, adjust parameters, and check again.

In short: optimization uses mathematical search to improve a model.

How the pieces work together

Here’s the full story:

Linear algebra turns examples into vectors and matrices.
Geometry helps compare those examples.
Matrix decomposition can reveal patterns in the data.
The model makes predictions, while probability helps describe uncertainty.
Vector calculus calculates how changes to parameters affect performance.
Optimization uses that information to improve the model.

You don’t need to master all of these ideas before beginning machine learning. But knowing what each one is for makes the subject easier to navigate: linear algebra represents the data, geometry compares it, probability describes uncertainty, and calculus and optimization help the model learn and improve predictions.

AI as an Experience Layer

Vimal — Tue, 20 Oct 2026 08:07:15 +0000

Overview

Modern digital platforms are no longer built around a single user interface or a single application runtime. As software systems mature, they increasingly expose the same business data through multiple channels: transactional systems, APIs, data warehouses, analytics surfaces, reporting layers, and AI-powered agents.

This architectural shift matters because AI is not just another feature bolted onto an app. It is becoming a new surfacing layer over the platform itself. In the same way that web apps, mobile apps, and dashboards expose business capabilities to users, AI agents and copilots can expose operational and analytical capabilities to intelligent systems.

The key idea is simple: the underlying platform remains the source of truth, while different layers consume and present that truth in different ways.

High-level architecture

Why this architecture matters

The traditional architecture model assumes one main interface to the system: the web app, the mobile app, or the internal API layer. But modern platforms are multi-surface systems.

A user might interact through a web dashboard, a BI report, an internal operations console, or an AI assistant. Each path reads or writes the same business domain in a different way, but each path depends on the same core data foundation.

This is where AI becomes strategically important: it is no longer just a user interaction pattern, but a platform capability that can be surfaced as a controlled access layer over operational and analytical data.

Layer responsibilities

1. Data platform layer

This layer contains the authoritative operational data and the analytical data stores. The operational database handles transactional workflows and the system-of-record behavior. The warehouse or lakehouse is optimized for analytics, data modeling, and broader downstream consumption.

2. Application layer

This layer exposes business behavior through APIs and internal services. It enforces validation, domain rules, orchestration, and access control. The underlying technology stack is intentionally abstracted here; the architecture is concerned with capabilities, not a specific framework.

3. ETL and integration layer

This layer moves data from operational systems into curated analytical stores. ETL and streaming pipelines clean, transform, and unify data so reporting and AI workloads consume trusted, structured information rather than brittle transactional data directly.

4. Reporting and analytics layer

This layer turns curated data into business insight. Dashboards, semantic models, scorecards, and metrics provide a human-readable layer on top of the data platform so stakeholders can monitor performance and make decisions.

5. AI layer: Agent host, LLM, MCP, and Skills

The AI layer combines familiar components with two platform-specific elements:

Agent host (for example, VS Code or Copilot) provides the user experience and coordinates the conversation.
LLM interprets requests and produces responses or tool-call decisions.
MCP gives the host and model a standard way to discover and invoke platform capabilities.
Skills provide domain terminology and workflow guidance for using those capabilities appropriately.

MCP and Skills solve different problems and work best together:

	MCP	Skills
Primary role	Connects an agent to tools and other exposed capabilities	Explains the domain and how work should be done
Provides	Discoverable tool definitions, input schemas, and results	Terminology, workflow steps, constraints, and examples
Answers	“What can the agent call, and how?”	“Which approach makes sense in this domain?”
Without it	The agent may lack a consistent way to access platform capabilities	The agent may use tools without enough business context

MCP is an integration contract, not an authorization system; access controls must still be enforced by the host, MCP server, and underlying services. Skills guide model behavior but do not replace validation or business rules in those services.

Design principles

Separation of concerns

Operational, analytical, reporting, and AI workloads should be separated by purpose. Each layer should be optimized for its specific responsibility instead of trying to do everything in one runtime.

Source-of-truth discipline

The transactional system remains the authoritative source of record, while the warehouse provides curated analytical data. Reporting and AI systems should consume trusted data rather than duplicate operational logic or bypass governance controls.

Governed tool exposure

MCP matters because it gives the AI a bounded interface instead of full direct access to everything. This is the difference between an AI system that can act safely and one that can only talk generically about the platform.

Domain grounding

Skills matter because they connect the AI to the actual business language and workflow patterns of the organization. This is what turns a general-purpose model into a useful domain-aware collaborator.

Multi-surface product design

A platform should not assume one user channel. Instead, it should support multiple surfaces:

human users through web and mobile experiences,
analysts through BI and reporting,
agents through MCP or similar tool surfaces,
and operational workflows through APIs and internal services.

Framework independence

The architectural pattern should not be tied to a single framework or language. APIs can be built in .NET, Node.js, Python, Java, or any other stack. The architecture is about capabilities, boundaries, and responsibilities—not vendor lock-in.

Why AI is becoming a surfacing layer

AI is increasingly becoming a new access surface over the platform, much like an API or dashboard is. It is not merely a chatbot sitting on top of the UI; it is an interface that can interpret, reason, and act over enterprise data when properly connected to the right tools and context.

That makes AI a strategic platform concern rather than just an application feature. When done well, it provides:

natural-language interaction,
operational assistance,
workflow guidance,
and faster access to business insight through structured tool exposure.

But this only works when the platform has a solid foundation: trusted data, curated integrations, and clear access boundaries.

The real differentiator is not simply “AI in the product.” The differentiator is how the platform exposes capabilities to the model and how much domain context the model receives. That is where MCP and Skills become foundational.

MCP gives the model a formal way to discover and invoke capabilities. Skills give the model the operational understanding needed to act correctly. Together, they turn AI from a clever interface into a grounded, enterprise-aware operating layer.

Strategic implication

The modern platform architecture is becoming a layered ecosystem:

transactional systems handle live operations,
APIs expose business capabilities,
ETL pipelines bring data into analytical stores,
BI surfaces provide business visibility,
and AI acts as a new interaction layer over the same platform capabilities.

AI does not replace these systems. It adds another interface for interacting with them, one that requires governance, semantic context, and well-defined tooling to be useful and safe.

This is where MCP and Skills become strategic differentiators. A team that adds a chatbot without MCP and domain-specific skills is merely adding a surface that can talk about the system. A team that adds MCP and Skills creates a system that can truly operate within the platform, grounded in domain context and tool boundaries.

Summary

AI is evolving from a feature to a platform surface. In the same way that the web, mobile, and reporting experiences expose the business to users, AI agents can expose business capabilities and insights to intelligent systems.

The winning architecture is not a monolith but a layered platform built around a trusted data foundation, well-defined application services, managed integration pipelines, reporting surfaces, and an AI layer that is governed, contextualized, and tool-driven.

This is the emerging reality of modern software architecture: a system designed for many interfaces, not just one.

Track Page Views using Application Insights

Vimal — Mon, 01 Jun 2026 13:39:59 +0000

To track page views in an Angular application using Azure Application Insights, you typically integrate the Application Insights JavaScript SDK and hook it into Angular routing events.
Below is a clean, production-friendly approach tailored for Angular apps.

Step-by-Step Implementation

1️⃣ Install Application Insights SDK

npm install @microsoft/applicationinsights-web

2️⃣Create an App Insights Service

Create a reusable service to initialize telemetry.

// app-insights.service.ts
import { Injectable } from '@angular/core';
import { ApplicationInsights } from '@microsoft/applicationinsights-web';

@Injectable({
  providedIn: 'root'
})
export class AppInsightsService {
  appInsights: ApplicationInsights;

  constructor() {
    this.appInsights = new ApplicationInsights({
      config: {
        instrumentationKey: 'YOUR_INSTRUMENTATION_KEY',
        enableAutoRouteTracking: false // we'll manually track
      }
    });

    this.appInsights.loadAppInsights();
  }

  logPageView(name?: string, uri?: string) {
    this.appInsights.trackPageView({
      name,
      uri
    });
  }
}

3️⃣Hook into Angular Router Events

Angular SPAs don’t reload pages, so you must manually track navigation changes using the Router.

TypeScript// app.component.ts
// app.component.ts
import { Component } from '@angular/core';
import { Router, NavigationEnd } from '@angular/router';
import { filter } from 'rxjs/operators';
import { AppInsightsService } from './app-insights.service';

@Component({
  selector: 'app-root',
  template: '<router-outlet></router-outlet>'
})
export class AppComponent {

  constructor(
    private router: Router,
    private appInsightsService: AppInsightsService
  ) {
    this.monitorRouting();
  }

  monitorRouting() {
    this.router.events
      .pipe(filter(event => event instanceof NavigationEnd))
      .subscribe((event: NavigationEnd) => {

        this.appInsightsService.logPageView(
          event.urlAfterRedirects,
          window.location.href
        );
      });
  }
}

✅ Result

Each route change (e.g., /home → /dashboard) will:

Trigger NavigationEnd
Send a pageView telemetry event to Application Insights

🚀 Alternative (Automatic Tracking)

You can enable auto tracking:

config: 
{  
 instrumentationKey: 'YOUR_KEY', 
 enableAutoRouteTracking: true
}

✅ Pros:

Minimal setup

⚠️ Cons:

Less control over naming and metadata
Harder to enrich telemetry

🔍 Enhancements (Highly Recommended)

✅ Add Custom Properties

this.appInsights.trackPageView({
  name: event.urlAfterRedirects,
  uri: window.location.href,
  properties: {
    userRole: 'Admin',
    feature: 'SearchModule'
  }
});

✅ Track Logged-in User
TypeScriptthis.appInsights.setAuthenticatedUserContext('user-id');
✅ Track API calls, errors, and events

this.appInsights.trackEvent({ name: 'ButtonClicked' });

this.appInsights.trackException({ exception: error });

👉 Best practice:

Use App Insights Web SDK in Angular
Use Application Insights SDK in .NET API
Correlate telemetry via operationId

🧾 Summary

Install App Insights SDK
Create telemetry service
Subscribe to NavigationEnd
Call trackPageView()
(Optional) Add custom telemetry

✅ Final Takeaway

Angular does not trigger full page reloads, so you must manually track page views using Router events to get accurate telemetry in Application Insights.

Build a Streaming UI without over-complicating it

Vimal — Sat, 23 May 2026 18:53:40 +0000

If you are building a UI that needs to show progress, logs, or live updates, you do not need to jump straight to WebSockets. In many cases, SSE is the simpler and more appropriate choice. It is a browser-friendly way for the server to push events to the client over HTTP. Think of it as a one-way live stream: the server sends updates, and the browser listens.

SSE is a great option when you need server → client streaming and want to keep the implementation simple.

What is SSE?

SSE stands for Server-Sent Events.

It is a browser-native way for the server to push updates to the client over HTTP.

When to use SSE

Use SSE when	Avoid SSE when
The communication is one-way	You need bidirectional chat
You want to avoid polling	You need low-latency command/control flows
You want a lightweight HTTP-based stream	You need binary transport
You want the browser to handle the connection natively

Examples:

progress updates
live logs
status feeds
task completion notifications
AI-generated content token by token

Why use SSE instead of polling?

Polling is one of the most common used techniques for implementing such scenarios. Polling means the browser repeatedly asks the server for new data. That creates extra traffic and awkward timing.

SSE is better because:

the browser opens a single connection
the server sends events as they happen
the client gets updates immediately
you avoid repeated polling requests

For a small streaming UI, SSE is often the simplest option.

Technical Flow and Architecture

The implementation of an SSE stream relies on a simple front-to-back flow:

Backend Endpoint: The backend (e.g., ASP.NET) exposes an HTTP endpoint and writes data chunks as SSE events.
Frontend Consumer: The client-side framework (e.g., Angular) initializes a native browser EventSource object to listen to the endpoint.
UI Rendering: The interface listens for incoming chunks and renders the updates in real time as they arrive.
Stream End: Once complete, the application treats the closed stream as a normal finish state.

This is especially useful for progress UIs, logs, and status updates.

Reference

Full code is available here:

https://github.com/vimalamaldas/Streaming-Sample

What this sample does

Backend

An ASP.NET endpoint sends streaming chunks as SSE.

Frontend

An Angular service creates an EventSource and exposes the stream to a component.

Integrate a Copilot Studio Agent into an Existing .NET App Using Agents SDK

Vimal — Sun, 19 Apr 2026 17:16:17 +0000

Introduction

If your application already has an API, the fastest way to add AI is to integrate an agent created using Microsoft Copilot studio.
Microsoft Copilot studio is an agent building platform that offers integration of agents into your applications using Microsoft Agents SDK.

This walkthrough shows how to wire a Copilot Studio agent into an existing .NET application using the Microsoft Agents SDK approach.

Pre-requisites

Agent created using Microsoft Copilot studio
Agent is attached to a knowledge source (SharePoint, Web, Azure Search, 3rd party APIs)
.NET API with an app registration having API permissions Copilot Studio.Copilots.Invoke

TL;DR

Add CopilotStudioDocumentRetrievalService to your existing interface.
Retrieve Agent settings from Copilot Studio
Support both user passthrough and OBO token flow.
Stream agent output through SSE for chat-like UX.

Architecture: Existing App + Copilot Studio Agent

The architecture consists of three main components

.NET API with SSE Endpoint
Create an endpoint that provides a response as and when the API receives a response from Copilot agent. This is very important to provide notifications to the client as when response is available.
Retrieval Layer using Agents SDK (This is the Copilot client)
This layer invokes the Copilot agent using the Agents SDK leveraging the configuration settings for the agent.
A Copilot Agent created using Microsoft Copilot Studio attached to a knowledge source that acts as the grounding data.

Step 1: Add the SDK dependencies

Microsoft.Agents.CopilotStudio.Client Microsoft 365 Agents SDK for invoking Copilot agent
Microsoft.Identity.Client for authentication

<ItemGroup>
  <PackageReference Include="Microsoft.Agents.CopilotStudio.Client" Version="1.3.176" />
  <PackageReference Include="Microsoft.Identity.Client" Version="4.78.0" />
</ItemGroup>

Step 2: Retrieve Agent settings from Copilot studio

Navigate to the Copilot agent, Select Settings > Advanced. Under Metadata, you need the Environment ID, Tenant ID, and Schema name of the agent. Copy these metadata values and add it to app settings.

{
  "Retrieval": {
    "Provider": "CopilotStudio"
  },
  "CopilotStudio": {
    "EnvironmentId": "<env-id>",
    "SchemaName": "<agent-schema-name>",
    "AuthMode": "User",
    "TenantId": "<tenant-id>",
    "AppClientId": "<app-client-id>",
    "AppClientSecret": ""
  }
}

Retrieve and add AppClientId (client Id) and AppClientSecret (secret) of the App registration that has API permissions to invoke Copilot agent. App registration requires the following API permissions:

Power Platform API > CopilotStudio.Copilots.Invoke

Add strongly typed Copilot settings for configuration using Options pattern

public class CopilotStudioOptions
{
    public string EnvironmentId { get; set; } = string.Empty;
    public string SchemaName { get; set; } = string.Empty;
    public string AuthMode { get; set; } = "App";
    public string TenantId { get; set; } = string.Empty;
    public string AppClientId { get; set; } = string.Empty;
    public string AppClientSecret { get; set; } = string.Empty;
}

Step 3: Register DI for CopilotStudio

builder.Services.Configure<CopilotStudioOptions>(
    builder.Configuration.GetSection("CopilotStudio"));

builder.Services.AddHttpClient();
builder.Services.AddHttpContextAccessor();

builder.Services.AddKeyedScoped<IDocumentStreamingService, CopilotStudioDocumentRetrievalService>("Copilot");

Step 4: Initialize Copilot client

Initialize the Copilot client using the configuration values that was originally copied from the Copilot agent metadata.

connectionSettings = new ConnectionSettings
{
    EnvironmentId = options.EnvironmentId,
    SchemaName = options.AgentIdentifier,
    Cloud = PowerPlatformCloud.Prod,
    CopilotAgentType = AgentType.Published
};

scope = CopilotClient.ScopeFromSettings(connectionSettings);

Use AskQuestionAsync(...) to stream activity text and map it into your existing result model.

Step 5: Intialize conversation with Copilot

Initialize a conversation session with Copilot before asking a question:

This is a setup phase before actually asking a question. Copilot needs to establish a conversation context (session ID, connection state, etc.) before handling queries. The code waits for this initialization to complete, then proceeds to AskQuestionAsync(...) to get actual results.


await foreach (var startupActivity in copilotClient.StartConversationAsync(emitStartConversationEvent: true, cancellationToken: cancellationToken))

StartConversationAsync(...) to initialize a new conversation session with Copilot
emitStartConversationEvent: true tells Copilot to emit initialization events (e.g., handshake messages, acknowledgments)
await foreach iterates over the stream of activities Copilot sends during startup
cancellationToken allows the operation to be cancelled if needed

Once the session is initialized use AskQuestionAsync(...) that takes the user query query as input parameter to get actual results, use await to asynchronously iterate over activities returned from Copilot as they arrive

await foreach (var activity in copilotClient.AskQuestionAsync(query, conversationId: null, ct: cancellationToken))

Step 6: Expose SSE endpoint for near real-time responses

Add SSE specific code in the controller to process and send real-time data to client

Response.ContentType = "text/event-stream"Tells the client this HTTP response is an SSE stream, so data can arrive continuously instead of all at once.

Response.Headers.Append("Cache-Control", "no-cache")Prevents caching so the browser/proxy always receives fresh streamed events.

await foreach (var activity in retrievalService.StreamActivitiesAsync(...))Reads activities one-by-one asynchronously from the service as they become available.

JsonSerializer.Serialize(activity)Converts each activity object into JSON text.

await Response.WriteAsync($"data: {json}\n\n", ...)Writes each item in SSE format.
Each SSE message must start with data: and end with a blank line (\n\n), which signals one complete event to the client.

Overall, this is a streaming endpoint that pushes incremental activity updates to the frontend in real time.

Response.ContentType = "text/event-stream";
Response.Headers.Append("Cache-Control", "no-cache");

await foreach (var activity in retrievalService.StreamActivitiesAsync(request.Query, top, cancellationToken))
{
    var json = JsonSerializer.Serialize(activity);
    await Response.WriteAsync($"data: {json}\n\n", cancellationToken);
}

Quick test commands

curl -X POST "https://localhost:5001/api/query" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <user-token>" \
  -d '{"query":"Summarize the onboarding flow","top":5}'

curl -N "https://localhost:5001/api/query/stream?query=What%20changed%20in%20the%20last%20release%3F&agentType=Copilot"

Final thought

Copilot Studio integration is the easiest to introduce AI capabilities to existing application. Use Copilot studio to create agents and integrate the agent using Microsoft Agents SDK.

Modern Logging and Telemetry in .Net

Vimal — Fri, 20 Feb 2026 15:29:53 +0000

Introduction

Logging and telemetry are essential pillars of operational excellence in modern software systems. They provide real-time visibility into application behavior, system performance, and user interactions, enabling DevOps teams to detect and troubleshoot issues quickly before they impact users. Beyond just technical insights, this data informs product management decisions and enhances customer engagement by enabling proactive improvements and a deeper understanding of how users interact with the product.

🎯 Article Objective

The objective of this article is to introduce you to the fundamentals of OpenTelemetry and its integration with Azure Application Insights in a .NET application. It aims to highlight the importance of logging and telemetry in achieving operational excellence, particularly in modern cloud-native systems. By the end of this article, readers will understand how to:

Set up OpenTelemetry for distributed tracing and metrics collection in a .NET Web API.
Enable real-time visibility into application performance using Application Insights.
Enhance troubleshooting, DevOps workflows, and customer experience through effective telemetry.
Implement correlation IDs for traceability across services.
Collect critical performance metrics such as CPU usage, memory availability, and request duration.

This hands-on guide empowers developers to build more reliable, observable, and user-centric applications.

🔍 What is OpenTelemetry?

OpenTelemetry is an open-source standard for collecting telemetry data — logs, metrics, and traces — from your application.

It helps answer:

How long does a request take?
What services were involved in processing it?
Where are the bottlenecks?

📘 Why Not Just Use Logging?

Traditional logging is useful, but lacks context. Here’s how OpenTelemetry compares:

Feature	Logging	OpenTelemetry
Simple log messages	✅	✅
Tracing across services	❌	✅
Built-in metrics (CPU, memory)	❌	✅
Visualization in tools like Application Insights	❌	✅
Correlation across services	❌	✅

🎯 Why Use OpenTelemetry with Application Insights?

Application Insights is a powerful observability platform in Azure.
OpenTelemetry allows your app to send standardized, portable telemetry data.
Gain end-to-end visibility into distributed systems, including APIs, microservices, and external dependencies.

🛠️ Step-by-Step: Use OpenTelemetry in .NET 7

We’ll build a simple Web API that uses OpenTelemetry to trace requests and collect system metrics.

📦 Step 1: Add Required Packages

Run this in your terminal:

dotnet add package Microsoft.ApplicationInsights.AspNetCore
dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Instrumentation.Http
dotnet add package OpenTelemetry.Exporter.ApplicationInsights

🧠 Step 2: Configure Program.cs

using Microsoft.ApplicationInsights.Extensibility;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using OpenTelemetry.Metrics;

var builder = WebApplication.CreateBuilder(args);

// Enable Application Insights
builder.Services.AddApplicationInsightsTelemetry();

// OpenTelemetry setup
builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource.AddService("MyWebApi"))
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSource("MyWebApi")
        .SetSampler(new AlwaysOnSampler())
        .AddApplicationInsightsExporter(options =>
        {
            options.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
        }))
    .WithMetrics(metrics => metrics
        .AddAspNetCoreInstrumentation()
        .AddRuntimeInstrumentation()
        .AddProcessInstrumentation()
        .AddApplicationInsightsMetricsExporter(options =>
        {
            options.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
        }));

var app = builder.Build();

// Middleware to add correlation ID to response
app.Use(async (context, next) =>
{
    var traceId = System.Diagnostics.Activity.Current?.TraceId.ToString();
    if (!string.IsNullOrEmpty(traceId))
    {
        context.Response.Headers.Add("X-Correlation-ID", traceId);
    }
    await next();
});

app.MapControllers();
app.Run();

📂 Step 3: Create a Controller (TraceDemoController.cs)

using Microsoft.AspNetCore.Mvc;
using System.Diagnostics;

namespace MyWebApi.Controllers
{
    [ApiController]
    [Route("[controller]")]
    public class TraceDemoController : ControllerBase
    {
        private static readonly ActivitySource ActivitySource = new("MyWebApi");

        [HttpGet("hello")]
        public IActionResult Get()
        {
            using var activity = ActivitySource.StartActivity("SayHello");

            var traceId = Activity.Current?.TraceId.ToString();
            return Ok(new
            {
                Message = "Hello, world!",
                TraceId = traceId
            });
        }
    }
}

⚙️ Step 4: Add Configuration (appsettings.json)

Add your Application Insights connection string:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information"
    }
  },
  "APPLICATIONINSIGHTS_CONNECTION_STRING": "InstrumentationKey=YOUR-KEY-HERE"
}

📊 Metrics You Get Automatically

OpenTelemetry + Application Insights gives you:

Metric	Source
Request duration	`AddAspNetCoreInstrumentation`
CPU usage	`AddRuntimeInstrumentation`
Memory available	`AddProcessInstrumentation`
Request count	`AddAspNetCoreInstrumentation`

All these show up in Azure Application Insights under Metrics.

🧪 Testing the API

Run the app:

dotnet run

Call the endpoint:

GET http://localhost:5000/TraceDemo/hello

You’ll see the X-Correlation-ID in the response headers — that’s your trace ID!

📈 Viewing in Application Insights

Go to Azure Portal > Application Insights.

Use:

Application Map for service relationships.
Metrics to view CPU, memory, and requests.
Traces to search with X-Correlation-ID.
Performance to analyze slow endpoints.

✅ Summary

With OpenTelemetry + Application Insights, you now have:

✔ Distributed tracing

✔ Built-in performance metrics

✔ Context-rich diagnostics

✔ Cloud-native observability

This is a big step up from traditional logging and a must-have for any production-ready .NET app.

🧭 What’s Next?

Connect logs (ILogger) to trace context.

Export telemetry to other tools (Jaeger, Prometheus, etc.).

Set up alerts based on CPU, memory, or request duration.

🧠 Got Questions?

Feel free to drop issues or suggestions!

Managing NPM Registries in restricted environments

Vimal — Wed, 23 Jul 2025 05:26:27 +0000

Problem statement:

Recently we faced a challenge where our organization does not allow us to connect to the public NPM registry to develop locally. However the CI/CD pipelines of the client organization allowed connection to the public registry. As a team, we were pondering on different approaches to handle this unique situation. We could probably use two copies of package.json one for local having reference link to private registry and then another one for CI/CD pipeline. Is this a good approach or is there a better approach? Let us explore this in the article.

Introduction

This is a common scenario in some enterprise environments, developers work behind strict firewalls or disconnected networks that block access to the public NPM registry. Yet, the CI/CD pipelines (like Azure DevOps) may be fully open to the internet. This mismatch creates a development headache—how do you keep your dependencies synced and builds working smoothly?

A common (but bad) idea is to maintain two separate package.json files—one for local dev and one for CI/CD. But this introduces more problems than it solves.

The better, cleaner approach? Use .npmrc files smartly.

In this article let us explore how to:

Use a private registry for local development
Let Azure DevOps pipelines use the public registry
Securely manage credentials
Avoid sync issues between environments

❌ Why Two `package.json` Files Is a Bad Idea

Some teams try to maintain one package.json for local (private registry) and another for CI/CD (public registry).

Here’s why that’s problematic:

Dependency drift: Easy to forget syncing both files
Lockfile conflicts: package-lock.json mismatches can cause broken builds
Hard to audit: Tools like npm audit, dependabot, etc. lose context
Increased complexity: More files, more chances of human error

✅ The Right Way: Use `.npmrc`

Local Development Setup

In your Angular or Node project, create a .npmrc file only for local use:

# .npmrc (DO NOT commit to Git)
registry=https://your-private-registry.company.com/
always-auth=true

Or for scoped packages:

@your-org:registry=https://your-private-registry.company.com/

`.gitignore` It!

To avoid leaking sensitive registry info, add .npmrc to .gitignore:

.npmrc

Now you can safely run:

npm install

Azure DevOps CI/CD Setup

✅ Option 1: Use Public NPM Registry (Recommended for Pipelines)

Azure DevOps build agents can access the public registry by default. So just omit .npmrc in your repo and let it install dependencies directly from https://registry.npmjs.org.

- script: npm ci
  displayName: 'Install Dependencies'

Option 2: Use Private Registry in CI/CD (If Required)

If your pipeline must use a private registry, create a secret token in Azure DevOps (npm_token) and inject .npmrc at build time:

variables:
  NPM_TOKEN: $(npm_token)

steps:
  - script: |
      echo "registry=https://your-private-registry.company.com/" > .npmrc
      echo "//your-private-registry.company.com/:_authToken=$(NPM_TOKEN)" >> .npmrc
    displayName: 'Configure Private Registry'

  - script: npm ci
    displayName: 'Install Dependencies'

Full Azure DevOps Pipeline for Angular

trigger:
- main

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: NodeTool@0
  inputs:
    versionSpec: '18.x'
  displayName: 'Install Node.js'

- script: |
    echo "registry=https://registry.npmjs.org/" > .npmrc
  displayName: 'Use Public NPM Registry'

- script: npm ci
  displayName: 'Install Dependencies'

- script: npm run build
  displayName: 'Build Angular App'

- task: PublishBuildArtifacts@1
  inputs:
    pathToPublish: 'dist'
    artifactName: 'angular-app'

Local vs CI/CD Setup

Local vs CI/CD

Key Takeaways

Don’t manage two package.json files
Use .npmrc to cleanly separate dev and CI registry logic
Keep .npmrc out of version control
Use secure tokens in Azure DevOps

Bonus Tips

Use `npmrc` CLI for Switching

If you switch between multiple registries, use npmrc:

npmrc private
npmrc public

Scoped Packages

To avoid overriding global registry settings, scope packages and use this pattern:

@your-org:registry=https://private.registry.com/

Wrapping Up

Using .npmrc properly lets you handle both restricted dev environments and open CI/CD pipelines with ease. It’s clean, secure, and scalable—everything a good DevOps setup should be.

Let me know in the comments your feedback, if you want a complete deploy-to-Azure setup or if you're facing similar restrictions in your organization!

Designing for Dependency Injection — Think Like a Developer, Not a Container

Vimal — Sun, 20 Jul 2025 16:21:47 +0000

Before reaching for .AddScoped or new, take a step back and ask what your class needs to do — not what it needs to create. This should model a functionality from the business domain.

Design Thinking First, DI Later

A well-designed class doesn't scream for injection. It simply:

Has a clear purpose.
Exposes what it needs to perform that purpose.
Delegates responsibilities it shouldn't own.

Let’s unpack that with a practical approach.

Step 1: Identify the Class's Responsibility

Ask: What behavior is this class responsible for? . Let us analyse an order processing functionality. How would you model the behavior and construct the class?

Example:

public class OrderProcessor
{
    public void ProcessOrder()
    {
        // Validate order
        // Charge payment
        // Send confirmation email
    }
}

Looking at the above class, you’ve already spotted the problem: This class does too much. This is linked to the 'S' in SOLID principles, the class should handle only a single-responsibility.

Step 2: Delegate Responsibilities via Abstractions

Break it down. Let other services handle each concern.

public interface IOrderValidator
{
    bool Validate(Order order);
}

public interface IPaymentService
{
    void Charge(Order order);
}

public interface INotificationService
{
    void SendConfirmation(Order order);
}

Now rewrite OrderProcessor as a coordinator, not a worker:

public class OrderProcessor
{
    private readonly IOrderValidator _validator;
    private readonly IPaymentService _payment;
    private readonly INotificationService _notifier;

    public OrderProcessor(
        IOrderValidator validator,
        IPaymentService payment,
        INotificationService notifier)
    {
        _validator = validator;
        _payment = payment;
        _notifier = notifier;
    }

    public void ProcessOrder(Order order)
    {
        if (!_validator.Validate(order)) return;

        _payment.Charge(order);
        _notifier.SendConfirmation(order);
    }
}

You’ve just designed for Dependency Injection — without thinking about the container.

Step 3: Make It Testable and Flexible

This class can now be tested using mocks:

var mockValidator = new Mock<IOrderValidator>();
mockValidator.Setup(v => v.Validate(It.IsAny<Order>())).Returns(true);

var mockPayment = new Mock<IPaymentService>();
var mockNotifier = new Mock<INotificationService>();

var processor = new OrderProcessor(mockValidator.Object, mockPayment.Object, mockNotifier.Object);
processor.ProcessOrder(new Order());

✅ No real implementations required. ✅ You can test logic in isolation.

Step 4: Let the Container Wire It Up

Only now do you register it in .NET Core:

builder.Services.AddScoped<IOrderValidator, OrderValidator>();
builder.Services.AddScoped<IPaymentService, StripePaymentService>();
builder.Services.AddScoped<INotificationService, EmailNotifier>();
builder.Services.AddScoped<OrderProcessor>();

Summary: The Design-First DI Checklist

Step	Ask Yourself…	Outcome
1	What does this class need to do?	Define the class’s responsibility
2	What can be delegated to collaborators?	Extract abstractions
3	How can this class be tested in isolation?	Refactor for injectability
4	Where do I register this in DI container?	Configure DI at the edges

📌 Remember

DI isn’t about using a container.
It’s about designing classes that ask for what they need, instead of building it themselves.

Start from behavior. Inject contracts. Let the container worry about construction.

Dependency Injection in .NET Core

Vimal — Sun, 20 Jul 2025 15:56:40 +0000

Overview

Dependency Injection (DI) is a software design pattern and technique that deals with how components acquire their dependencies. Rather than creating dependencies internally, a class receives them from the outside — improving modularity, testability, and flexibility.

In .NET, Dependency Injection is a first-class citizen, built directly into the ASP.NET Core framework.

Purpose

Dependency Injection aims to separate the creation of a component’s dependencies from the component’s behavior, promoting Inversion of Control (IoC). The class should focus only on what it needs to do, not on how to create the things it needs.

"Don't call me, I’ll call you." — This summarizes Inversion of Control, where the framework injects required services into your class.

Some of the common dependencies needed in application are:

Services (ILogger, DbContext, etc.)
Repositories (ICustomerRepository)
Configurations (IOptions)
External integrations (IEmailService, IPaymentGateway)

Types of Dependency Injection

There are three common forms of DI:

Constructor Injection (most common)
Setter Injection (property-based)
Interface Injection (less common in .NET)

1. Constructor Injection

The dependency is provided through the class constructor.

public interface IMessageService
{
    string GetMessage();
}

public class EmailMessageService : IMessageService
{
    public string GetMessage() => "Message via Email";
}

public class NotificationManager
{
    private readonly IMessageService _messageService;

    public NotificationManager(IMessageService messageService)
    {
        _messageService = messageService;
    }

    public void Notify()
    {
        Console.WriteLine(_messageService.GetMessage());
    }
}

Registration in .NET Core

builder.Services.AddTransient<IMessageService, EmailMessageService>();
builder.Services.AddTransient<NotificationManager>();

2. Setter (Property) Injection

Dependencies are set via public properties. Not natively supported by the built-in container, but possible manually.

public class LoggerService
{
    public IMessageService MessageService { get; set; }

    public void Log()
    {
        Console.WriteLine(MessageService?.GetMessage());
    }
}

In this case, MessageService would need to be set manually after object construction.

3. Interface Injection

The dependency provides an interface to inject itself into the consumer. Rare in .NET:

public interface IInjectable
{
    void Inject(IMessageService messageService);
}

Consumers then implement Inject to receive the dependency.

Lifetimes in .NET DI

When registering services, you specify how long a service lives:

Transient – new instance each time
Scoped – one instance per request
Singleton – one instance for the app lifetime


builder.Services.AddTransient<IMyService, MyService>(); // new each time
builder.Services.AddScoped<IMyService, MyService>();    // one per web request
builder.Services.AddSingleton<IMyService, MyService>(); // shared instance

Dependency Graph

.NET Core builds a dependency graph during application startup. Circular dependencies (A → B → A) are automatically detected and cause an error.

Benefits of DI

Decoupling: Easier to replace or extend components
Testability: Easier to write unit tests using mocks
Reusability: Services can be reused across classes
Maintainability: Easier to manage and evolve large applications

Real-world Example

public interface IPaymentProcessor
{
    void Process(decimal amount);
}

public class StripePaymentProcessor : IPaymentProcessor
{
    public void Process(decimal amount)
    {
        Console.WriteLine($"Processed ${amount} via Stripe.");
    }
}

public class CheckoutService
{
    private readonly IPaymentProcessor _paymentProcessor;

    public CheckoutService(IPaymentProcessor paymentProcessor)
    {
        _paymentProcessor = paymentProcessor;
    }

    public void Checkout()
    {
        _paymentProcessor.Process(100);
    }
}

Registration:

builder.Services.AddScoped<IPaymentProcessor, StripePaymentProcessor>();
builder.Services.AddScoped<CheckoutService>();

Usage:

var checkout = app.Services.GetRequiredService<CheckoutService>();
checkout.Checkout(); // Outputs: Processed $100 via Stripe.

Anti-Patterns to Avoid

Service Locator Pattern: Using IServiceProvider.GetService() everywhere breaks DI principles.
Too Many Dependencies: If a class has too many injected services, it's likely violating Single Responsibility Principle.
Mixing lifetimes improperly: Injecting Scoped into Singleton can cause unexpected behavior.

Conclusion

Dependency Injection in .NET Core encourages clean architecture by separating behavior from object creation. It's built-in, powerful, and essential for test-driven and scalable .NET applications.

By focusing on what your classes need to do, and not how they get their dependencies, your application becomes more maintainable, testable, and adaptable to change.

Distribute iOS mobile applications(without App Store)

Vimal — Wed, 16 Jul 2025 00:59:02 +0000

Ever built an app just for yourself and wondered, "How do I get this running on my iPhone without publishing it to the App Store?" Whether you're testing a side project, creating a tool for personal productivity, or experimenting with Flutter or SwiftUI, deploying a private app to your own device is totally doable—no paid Apple Developer account required.

In this post, I’ll walk you through how to:

Build and install a personal iOS app on your device using Xcode
Run the app without using TestFlight or the App Store
Fix that annoying issue where the app stops working once you unplug the USB
Decide if a free Apple ID is enough for your needs—or if it's time to go Pro

Let’s dive into how you can run your own iOS app —no distribution headaches, no gatekeepers, just your code on your phone.

1. Free (“Personal”) Provisioning via Xcode

Apple lets you install debug builds on your own device using just a free Apple ID—but with some limitations (e.g. 1 app per bundle‑ID, the provisioning profile expires weekly, and you can’t share to other people).

Install Xcode
– On your Mac, download Xcode from the App Store.
– Make sure you have at least macOS Sonoma (14.0) or later.
Open Your Project
– If you’re using Flutter, open your project root and execute.
```
    flutter build ios

    open ios/Runner.xcworkspace 
```
– If you’re native Swift/Obj‑C: open your .xcodeproj or .xcworkspace.
Add Your Apple ID
1. In Xcode, go to Xcode ▶️ Settings ▶️ Accounts, click “+”, and sign in with your Apple ID.
2. Select your Apple ID, then “Manage Certificates” → click “+” → “iOS Development”.
Set Bundle Identifier
1. In the Project Navigator select your main target (e.g. Runner).
2. Under Signing & Capabilities → Team, choose your personal Apple ID.
3. Change the Bundle Identifier to something unique (e.g. com.yourname.myapp).
Connect Your iPhone

– Plug your iPhone into the Mac via USB (or enable wireless debugging).
– In Xcode’s toolbar, select your device from the run‑destination menu.
Build & Run

– Click ▶️ (Run). Xcode will automatically generate a provisioning profile, sign the app, install it on your device, and launch it.
– If you see a “Untrusted Developer” alert on your phone, go to Settings ▶️ General ▶️ VPN & Device Management and trust your Apple ID profile.
Re‑sign Weekly

Free provisioning profiles last 7 days. After that, you need to re‑build & run from Xcode to renew.

⚡Tip:
If you’re expecting to be able to tap the app icon and launch it without ever reconnecting,enable wireless debugging so Xcode can install & re‑sign over Wi‑Fi, without a cable each time:

With your device plugged in, open Window → Devices and Simulators in Xcode.

Select your iPhone or device in the left list, then check Connect via network.

Now you can unplug the USB cable—and as long as both Mac and iPhone are on the same Wi‑Fi, you can build & Run wirelessly.

2. Paid Apple Developer Program (₹8,400/year)

If you want more flexibility— Ad‑Hoc distribution, TestFlight, multiple devices, push notifications, or to share with up to 100 testers—you’ll need a paid account.

What is Ad-Hoc Distribution?
Ad-Hoc distribution lets you install an app on up to 100 physical iOS devices per year using an IPA file (the iOS app package), without going through the App Store or TestFlight.

Key Features:

App is signed with a special “Ad-Hoc” provisioning profile
You manually specify the UDIDs (device IDs) of the devices you want to allow
You export an IPA from Xcode or CI
You can install the app using:
- Apple Configurator (USB)
- OTA (Over-the-Air) tools like Diawi, Firebase App Distribution, etc.

Enroll
– Go to https://developer.apple.com and sign up for the Apple Developer Program.
– Pay the annual fee (₹8,400 in India).
Register Devices (for Ad‑Hoc)
– In App Store Connect → Your App → Devices, add each device’s UDID.
Create an App ID & Provisioning Profile
– In Certificates, IDs & Profiles, register your App ID (bundle‑ID), and generate an Ad‑Hoc or Development provisioning profile that includes your registered devices.
Build & Distribute
- In Xcode, select your paid “Team” under Signing.
- Archive your build (Product ▶️ Archive), then export an Ad‑Hoc IPA.
- Distribute via:
  - TestFlight (easiest for multiple testers)
  - Direct install (using tools like Apple Configurator or a simple web‑hosted OTA link)
App Updates

You can push updates through TestFlight or simply rebuild & redistribute the IPA.

Choosing Between the Two

Feature	Free Provisioning	Paid Developer Program
Cost	Free	₹8,400/year
Device count	1 (per bundle‑ID)	Up to 100 (Ad‑Hoc), unlimited TestFlight
Profile expiration	7 days	1 year
App distribution to others	No	Yes (Ad‑Hoc, TestFlight)
Push Notifications, App Store	No	Yes

✅ When Should You Use Ad-Hoc?
Use Ad-Hoc if:

You have a private app you want to install on your team’s or family’s devices
You want to avoid App Store delays
You want more freedom than TestFlight’s 90-day limit

If it’s just for your own testing and you don’t mind re‑signing weekly, the free path is simplest. If you want reliable longer provisioning, wider distribution, or App Store/TestFlight support, go with the paid Developer Program.

Leave me a message if you’d like more detail on any of these steps!

Securely Managing Secrets in Configuration in .NET Applications

Vimal — Fri, 04 Jul 2025 10:18:37 +0000

The Problem

Many developers make the mistake of adding connection strings to appsettings file of .Net application. This makes its way in to the repository and becomes a target of database attacks.Storing connection strings as plain text in configuration files and checking them into a repository can lead to security vulnerabilities, as sensitive information like passwords can be easily accessed by unauthorised users.

Possible Solution

.Net provides a number of configuration providers for storing and reading configuration. Storing sensitive configuration items like connection strings in an environment variable is the safest way to avoid connection string or password exposure. Configuration is read from the process that the application is running in.
This code works locally as well on Azure after deployment.

In the case of Azure App Service, just need to add the configuration item and value under Settings > Environment Variables on the App Service in Azure portal.

To add a connection string to environment variables locally, follow these steps:

1. Set Environment Variable in Your Operating System

On Linux or macOS:

You can set the environment variable in your terminal session:

export DBConnectionString="Server=tcp:your-server.database.windows.net,1433;Database=your-database;User ID=your-username;Password=your-password;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;"

To make it persistent across sessions, add the above line to your shell configuration file (e.g., ~/.bashrc, ~/.zshrc, or ~/.bash_profile), and then reload the file:

source ~/.bashrc

On Windows:

Open the Start Menu and search for "Environment Variables."
Click on Edit the system environment variables.
In the System Properties window, click Environment Variables.
Under User variables or System variables, click New.
Add the following:
- Variable name: DBConnectionString
- Variable value: Server=tcp:your-server.database.windows.net,1433;Database=your-database;User ID=your-username;Password=your-password;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;
Click OK to save.

2. Verify the Environment Variable

To ensure the environment variable is set correctly, you can print it in your terminal:

On Linux/macOS:

echo $DBConnectionString

On Windows (Command Prompt):

echo %DBConnectionString%

On Windows (PowerShell):

$env:DBConnectionString

3. Access the Environment Variable in Your Application

In your application, you can access the environment variable using Environment.GetEnvironmentVariable:

var connectionString = Environment.GetEnvironmentVariable("DBConnectionString");

4. Test the Configuration

Run your application and ensure the connection string is being picked up correctly. You can log the connection string for debugging purposes:

Console.WriteLine($"SQL Connectionstring: {connectionString}");

Summary

Set the environment variable using your operating system's tools.
Access the variable in your application using Environment.GetEnvironmentVariable.

Let me know if you need further assistance!

Practical Caching in .NET: Strategies, Eviction, Keys, and Metrics

Vimal — Sat, 21 Jun 2025 13:42:51 +0000

Key Objectives

Understand the different caching options available in .NET.
Learn about cache eviction strategies and when to use them.
Discover best practices for choosing cache keys.
Explore how to measure cache efficiency using instrumentation and logging.
See practical code samples for real-world caching scenarios.

Why Caching Matters

Caching is a fundamental technique for improving application performance and scalability. By storing frequently accessed data in a fast-access layer, you reduce database load, speed up response times, and can even improve reliability during transient back-end failures.

Fetching data from cache saves networks calls to database, query execution time which speeds up the overall response time.

Caching Options in .NET

.NET provides several built-in caching mechanisms:

In-Memory Cache (IMemoryCache)
Stores data in the memory of the application process. Fast, but data is lost on restart and not shared across servers.
Distributed Cache (IDistributedCache)
Supports external cache stores like Redis or SQL Server. Data is shared across multiple app instances and survives restarts.
Response Caching
Caches HTTP responses at the middleware level for web APIs.
Output Caching (ASP.NET Core 7+)
Caches the output of controllers or Razor pages.

Cache Eviction Strategies

Eviction determines when items are removed from the cache. Common strategies in .NET:

Absolute Expiration The item is removed after a fixed time, regardless of usage.

  _memoryCache.Set(key, value, TimeSpan.FromMinutes(10));

Sliding Expiration The item stays in the cache as long as it is accessed within a time window.

_memoryCache.Set(key, value, new MemoryCacheEntryOptions
{
    SlidingExpiration = TimeSpan.FromMinutes(5)
});

Manual Eviction Remove items explicitly when data changes.

_memoryCache.Remove(key);

Size-based Eviction Set a size limit for the cache; least recently used items are evicted when the limit is reached.

Choosing Cache Keys

Cache keys should be:

Unique: Represent the data being cached (e.g., "user_123" for user with ID 123).
Consistent: Use a predictable pattern for easy invalidation.
Scoped: Include context if needed (e.g., "product_{id}_details").
Example:

string cacheKey = $"employee_{employeeId}";

Measuring Cache Efficiency

To know if your cache is effective, instrument your code to log:

Cache Hits: Data was found in the cache.
Cache Misses: Data was not found and had to be loaded.
Evictions: When items are removed due to expiration or manual removal.
Retrieval Times: Time taken to fetch from cache vs. database.
Sample Instrumentation:

var stopwatch = Stopwatch.StartNew();
if (_memoryCache.TryGetValue(key, out var value))
{
    _logger.LogInformation("Cache hit for {Key} at {Time}", key, DateTime.UtcNow);
}
else
{
    _logger.LogInformation("Cache miss for {Key} at {Time}", key, DateTime.UtcNow);
    value = await LoadFromDatabaseAsync();
    _memoryCache.Set(key, value, TimeSpan.FromMinutes(10));
}
stopwatch.Stop();
_logger.LogInformation("Retrieval for {Key} took {Elapsed} ms", key, stopwatch.ElapsedMilliseconds);

Advanced:

For distributed caches like Redis, you can use Redis server metrics or middleware like App Metrics for more detailed analysis.

Example: In-Memory Cache with Eviction and Instrumentation

public class EmployeeCacheService
{
    private readonly IMemoryCache _memoryCache;
    private readonly ILogger<EmployeeCacheService> _logger;

    public EmployeeCacheService(IMemoryCache memoryCache, ILogger<EmployeeCacheService> logger)
    {
        _memoryCache = memoryCache;
        _logger = logger;
    }

    public async Task<Employee> GetEmployeeAsync(int employeeId)
    {
        string key = $"employee_{employeeId}";
        var stopwatch = Stopwatch.StartNew();

        if (_memoryCache.TryGetValue(key, out Employee employee))
        {
            _logger.LogInformation("Cache hit for {Key} at {Time}", key, DateTime.UtcNow);
        }
        else
        {
            _logger.LogInformation("Cache miss for {Key} at {Time}", key, DateTime.UtcNow);
            employee = await LoadEmployeeFromDb(employeeId);
            _memoryCache.Set(key, employee, new MemoryCacheEntryOptions
            {
                AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(10)
            });
            _logger.LogInformation("Cache set for {Key} at {Time}", key, DateTime.UtcNow);
        }

        stopwatch.Stop();
        _logger.LogInformation("Retrieval for {Key} took {Elapsed} ms", key, stopwatch.ElapsedMilliseconds);
        return employee;
    }

    public void EvictEmployee(int employeeId)
    {
        string key = $"employee_{employeeId}";
        _memoryCache.Remove(key);
        _logger.LogInformation("Cache evicted for {Key} at {Time}", key, DateTime.UtcNow);
    }
}

Summary

.NET offers several caching options: in-memory, distributed (Redis, SQL), and response/output caching.
Choose the right eviction strategy for your scenario: absolute, sliding, manual, or size-based.
Use clear, unique cache keys for easy management and invalidation.
Instrument your cache usage to measure hits, misses, evictions, and performance.
Caching is a powerful tool—use it wisely to boost your app’s speed and scalability!

What caching strategies have worked best for your .NET projects? Share your experience in the comments!