DEV Community: GOWTHAM THEJASWI

Understanding Indexing and Partitioning in Azure Cosmos DB for Social Media Analytics

GOWTHAM THEJASWI — Mon, 03 Mar 2025 18:15:27 +0000

Azure Cosmos DB is a globally distributed, multi-model database service that provides high availability, low latency, and elastic scalability. Two key features that significantly impact the performance and efficiency of Cosmos DB are Indexing and Partitioning. This blog will explore these concepts in the context of a social media application that tracks user interactions, posts, comments, likes, followers, and images.

Indexing in Cosmos DB

Indexing in Cosmos DB ensures that queries run efficiently without requiring full document scans. By default, Cosmos DB automatically indexes all properties of every document without requiring explicit schema definitions. This allows for faster query performance but may also result in increased storage and write costs.

Consider a user profile document where you frequently query by username and email

{
  "userId": "123",
  "username": "deva_juan",
  "email": "deva.juan@example.com",
  "bio": "Tech enthusiast",
  "followers": 150,
  "images": [
    {
      "imageId": "img001",
      "url": "http://example.com/image1.jpg",
      "description": "Profile picture"
    },
    {
      "imageId": "img002",
      "url": "http://example.com/image2.jpg",
      "description": "Vacation photo"
    }
  ]
}

Indexing Modes

Cosmos DB offers two indexing modes:

Consistent Indexing: Updates indexes synchronously with writes, ensuring strong consistency.
None: Used when you only need basic key-value access and do not require querying based on specific properties.

Types of Indexing Policies

Cosmos DB supports different indexing policies to cater to various query and performance needs:

Automatic vs. Manual Indexing

Automatic Indexing: Cosmos DB automatically indexes all fields unless explicitly excluded.

Manual Indexing: Developers can choose which properties to index, reducing storage and improving write performance.

Example:
If manual indexing is enabled, and only username and email need to be indexed:

{
  "indexingPolicy": {
    "includedPaths": [
      {
        "path": "/username/?"
      },
      {
        "path": "/email/?"
      }
    ],
    "excludedPaths": [
      {
        "path": "/*"
      }
    ]
  }
}

Range Indexing: Ideal for queries involving range comparisons (e.g., >, <).
Spatial Indexing: Used for geospatial queries, such as finding locations within a specified radius.

For geospatial data:

{
  "location": {
    "type": "Point",
    "coordinates": [-122.12, 47.67]
  }
}

Composite Index

Composite indexes optimize queries that filter or sort by multiple properties.

Composite Index Example
If you frequently query posts by userId and creationDate, define a composite index:

{
  "indexingPolicy": {
    "compositeIndexes": [
      [
        { "path": "/userId", "order": "ascending" },
        { "path": "/creationDate", "order": "descending" }
      ]
    ]
  }
}

Full index policy example

Below is basic social media index policy example.


{
  "indexingMode": "consistent",
  "automatic": true,
  "includedPaths": [
    {
      "path": "/*"
    },
    {
      "path": "/username/?"
    },
    {
      "path": "/email/?"
    },
    {
      "path": "/bio/?"
    },
    {
      "path": "/followers/?"
    },
    {
      "path": "/posts/?"
    },
    {
      "path": "/posts/*"
    },
    {
      "path": "/posts/comments/?"
    },
    {
      "path": "/posts/comments/*"
    },
    {
      "path": "/images/?"
    },
    {
      "path": "/location/?"
    }
  ],
  "excludedPaths": [
    {
      "path": "/_etag/?"
    }
  ],
  "compositeIndexes": [
    [
      { "path": "/username", "order": "ascending" },
      { "path": "/email", "order": "ascending" }
    ],
    [
      { "path": "/userId", "order": "ascending" },
      { "path": "/creationDate", "order": "descending" }
    ],
    [
      { "path": "/postId", "order": "ascending" },
      { "path": "/likes", "order": "descending" }
    ],
    [
      { "path": "/postId", "order": "ascending" },
      { "path": "/commentsCount", "order": "descending" }
    ]
  ],
  "spatialIndexes": [
    {
      "path": "/location/?",
      "types": [
        "Point",
        "Polygon",
        "MultiPolygon",
        "LineString"
      ]
    }
  ]
}

Partitioning in Cosmos DB

Cosmos DB uses partitioning to scale individual containers in a database to meet the performance needs of your application.

Why is Partitioning Important?

Partitioning is crucial for horizontally scaling data across multiple partitions, allowing for efficient data management and retrieval.

Types of Partitions

Logical vs. Physical Partitions

Logical Partition: A subset of data grouped by a partition key (e.g., customerId).

Physical Partition: The actual storage resources that host multiple logical partitions.

Consider a post system where posts are partitioned by userId:

{
  "postId": "post456",
  "userId": "user123",
  "content": "Hello World!",
  "creationDate": "2025-02-18T10:00:00Z"
}

This allows efficient retrieval of posts by user.

Ways to achieve Proper partition

Choosing the Right Partition Key

Selecting the right partition key is crucial for performance and scalability:

High Cardinality: Choose a key with many distinct values to distribute data evenly across partitions.
Even Data Distribution: Avoid “hot partitions” by ensuring data is spread across multiple partitions.
Query Efficiency: Queries that filter on the partition key are more efficient.

Handling Cross-Partition Queries

Cross-partition queries can be expensive since they require scanning multiple partitions. To optimize:

Always filter queries using the partition key.
Use cross-partition queries sparingly.
Leverage bulk operations to improve efficiency.

Social media example data

Indexing Strategy

User Profiles Indexing:

Index by username and email: This allows for fast lookups when searching for users.

Example Index Policy:

{
  "indexingPolicy": {
    "includedPaths": [
      {
        "path": "/username/?"
      },
      {
        "path": "/email/?"
      }
    ],
    "excludedPaths": [
      {
        "path": "*"
      }
    ]
  }
}

Post Interactions Indexing:

Composite Index on postId, creationDate, and likes: This optimizes queries that retrieve interactions related to specific posts.

Example Index Policy:

{
  "indexingPolicy": {
    "compositeIndexes": [
      [
        { 
          "path": "/postId", 
          "order": "ascending" 
        },
        { 
          "path": "/creationDate", 
          "order": "descending" 
        },
        { 
          “path”: “/likes”, 
          “order”: “descending” 
        }
      ]
    ]
  }
}

Lazy Indexing During Peak Times:

Use lazy indexing during peak write times to reduce latency while ensuring consistency during off-peak hours.

Partitioning Strategy

Partition by userId: This groups all interactions related to a specific user together, enhancing read performance when accessing a user's timeline.

Below is example data for User with posts

{
  "userId": "user123",
  "username": "ron_ray",
  "bio": "Tech enthusiast",
  "followers": ["follower1", "follower2", "follower3"],
  "images": [
    {
      "imageId": "img001",
      "url": "http://example.com/image1.jpg",
      "description": "Profile picture"
    },
    {
      "imageId": "img002",
      "url": "http://example.com/image2.jpg",
      "description": "Vacation photo"
    }
  ],
  "posts": [
    {
      "postId": "post456",
      "content": "Hello World!",
      "creationDate": "2025-02-18T10:00:00Z",
      "likes": 20,
      "commentsCount": 5,
      "comments": [
        {
          "commentId": "comment001",
          "userId": "user456",
          "content": "Great post!",
          "timestamp": "2025-02-18T10:01:00Z"
        },
        {
          "commentId": "comment002",
          "userId": "user789",
          "content": "@john_doe I agree!",
          "timestamp": "2025-02-18T10:02:00Z"
        }
      ]
    },
    {
      "postId": "post789",
      "content": "Learning Azure Cosmos DB!",
      "creationDate": "2025-02-18T10:05:00Z",
      "likes": 15,
      "commentsCount": 2,
      "comments": []
    },
    {
      "postId": "post101",
      "content": "",
      "creationDate": "2025-02-18T10:10:00Z",
      "likes": 30,
      "commentsCount": 1,
      "comments": [
        {
          "commentId": "comment003",
          "userId": "user234",
          "content": “Awesome picture!”,
          “timestamp”: “2025-02-18T10:11:00Z”
        }
      ],
      “images”: [
        {
          “imageId”: “img003”,
          “url”: “http://example.com/image3.jpg”,
          “description”: “Sunset at the beach”
        }
      ]
    }
  ]
}

Example Queries

Retrieve User Profile by Username:

SELECT * FROM Users u WHERE u.username = 'ron_ray'

This query fetches the user profile based on the username.

Get Latest Posts from a User:

SELECT * FROM Posts p WHERE p.userId = 'user123' ORDER BY p.creationDate DESC OFFSET 0 LIMIT 10

This retrieves the latest 10 posts from a specific user.

Fetch Comments on a Specific Post:

SELECT * FROM Comments c WHERE c.postId = 'post456' ORDER BY c.creationDate ASC

This query lists all comments on a specified post in chronological order.

Count Interactions (Likes/Comments) on a Post:

SELECT COUNT(1) FROM Interactions i WHERE i.postId = 'post456'

This counts the total interactions (likes or comments) for a specific post.

Search Posts by Content Keywords:

SELECT * FROM Posts p WHERE CONTAINS(p.content, 'Azure') ORDER BY p.creationDate DESC

This retrieves posts containing the keyword “Azure,” ordered by creation date.

Best Practices for Indexing and Partitioning in Cosmos DB

Monitor RU (Request Unit) Consumption: High RU costs can indicate inefficient indexing or partitioning.
Use Composite Indexes for Complex Queries: This can significantly improve query performance.
Regularly Review Index Policies: Adjust based on actual query patterns.
Choose the Right Partition Key: Ensure even data distribution to prevent performance bottlenecks.
Leverage Multi-Region Replication: For globally distributed applications, partitioning and indexing strategies should account for regional replication.

Conclusion

Efficient indexing and partitioning are critical for optimizing Azure Cosmos DB performance in social media applications. By understanding how indexing works and carefully selecting partition keys, you can ensure scalability, cost efficiency, and low-latency queries while managing large volumes of user interactions seamlessly. The social media analytics use case demonstrates how Azure Cosmos DB's indexing and partitioning strategies can be tailored to enhance performance and efficiency. Regular monitoring and fine-tuning will help keep your Cosmos DB instance performing at its best.

For more insights into managing indexing policies effectively, refer to Microsoft’s official documentation on Cosmos DB.

Simplifying Dynamic Object Creation in Angular 18 with the Factory Pattern

GOWTHAM THEJASWI — Mon, 27 Jan 2025 19:24:37 +0000

In this blog, we'll explore how to implement a generic factory design pattern in Angular 18, using a transportation service booking system as an example with a styled flow diagram with Mermaid.
The Factory Design Pattern is one of the most popular and widely used design patterns in software development. It provides a way to create objects without specifying their concrete classes.

What Is the Factory Design Pattern?

The Factory Design Pattern defines an interface for creating objects, but lets the subclasses alter the type of objects that will be created. This approach increases code flexibility and reduces dependencies on specific implementations.

Use Case: Transportation Service Booking

Imagine an application where users can book transportation services via different modes such as car, bike, bus, or train. Instead of directly instantiating the transportation services in components, we can use a factory design pattern to dynamically create and manage these service objects.

Implementation in Angular 18

1. Define an Interface for Transportation Services

This interface will serve as the blueprint for all transportation modes.

export interface TransportationService {
  book(from: string, to: string): string;
  calculateCost(from: string, to: string, rate: number): number;
  getDistance(from: string, to: string): number;
}

2. Create Specific Transportation Services

Create standalone components for each transportation mode and implement the TransportationService interface.

Car Service Component

import { Injectable } from '@angular/core';
import { TransportationService } from './transportation.service';

@Injectable({ providedIn: 'root' })
export class CarService implements TransportationService {
  rates =10;
  book(from: string, to: string): string {
    const cost = this.calculateCost(from, to);
    return `Car booked successfully! From: ${from}, To: ${to}, Cost: $${cost}`;
  }

  calculateCost(from: string, to: string): number {
    const distance = this.getDistance(from, to);
    return distance * this.rate;
  }

  getDistance(from: string, to: string): number {
    // Mock distance calculation
    return  Math.floor(Math.random() * 201);
  }
}

Bike Service Component

import { Injectable } from '@angular/core';
import { TransportationService } from './transportation.service';

@Injectable({ providedIn: 'root' })
export class BikeService implements TransportationService {
  rates =10;
  book(from: string, to: string): string {
    const cost = this.calculateCost(from, to);
    return `Bike booked successfully! From: ${from}, To: ${to}, Cost: $${cost}`;
  }

  calculateCost(from: string, to: string): number {
    const distance = this.getDistance(from, to);
    return distance * this.rate;
  }

  getDistance(from: string, to: string): number {
    // Mock distance calculation
    return  Math.floor(Math.random() * 201);
  }
}

Bus Service Component

import { Injectable } from '@angular/core';
import { TransportationService } from './transportation.service';

@Injectable({ providedIn: 'root' })
export class BusService implements TransportationService {
  rates =10;
  book(from: string, to: string): string {
    const cost = this.calculateCost(from, to);
    return `Bus booked successfully! From: ${from}, To: ${to}, Cost: $${cost}`;
  }

  calculateCost(from: string, to: string): number {
    const distance = this.getDistance(from, to);
    return distance * this.rate;
  }

  getDistance(from: string, to: string): number {
    // Mock distance calculation
    return  Math.floor(Math.random() * 201);
  }
}

Train Service Component

import { Injectable } from '@angular/core';
import { TransportationService } from './transportation.service';

@Injectable({ providedIn: 'root' })
export class TrainService implements TransportationService {
  rates =10;
  book(from: string, to: string): string {
    const cost = this.calculateCost(from, to);
    return `Train booked successfully! From: ${from}, To: ${to}, Cost: $${cost}`;
  }

  calculateCost(from: string, to: string): number {
    const distance = this.getDistance(from, to);
    return distance * this.rate;
  }

  getDistance(from: string, to: string): number {
    // Mock distance calculation
    return  Math.floor(Math.random() * 201);
  }
}

3. Create a Generic Factory Service

The factory service dynamically resolves the transportation service based on the type, with added support for generics.

import { Injectable, inject, Injector } from '@angular/core';
import { TransportationService } from './transportation.service';
import { CarService } from './car.service';
import { BikeService } from './bike.service';
import { BusService } from './bus.service';
import { TrainService } from './train.service';

@Injectable({ providedIn: 'root' })
export class TransportationFactory {
  private injector = inject(Injector);

  public getTransportationService(mode: string): TransportationService {
    const serviceMapping: Record<string, { new (...args: any[]): TransportationService }> = {
      car: CarService,
      bike: BikeService,
      bus: BusService,
      train: TrainService,
    };

    const serviceClass = serviceMapping[mode];
    if (!serviceClass) {
      throw new Error(`Invalid transportation mode: ${mode}`);
    }

    return this.getInstance(serviceClass);
  }

  private getInstance<T extends TransportationService>(serviceClass: { new (...args: any[]): T }): T {
    return this.injector.get(serviceClass);
  }
}

4. Integrate the Factory in a Component

Let's create a component where the user selects the transportation mode, and the appropriate service is booked dynamically.

Transportation Booking Component

import { Component, inject } from '@angular/core';
import { TransportationFactory } from './transportation-factory.service';
import { FormsModule } from '@angular/forms';

@Component({
  standalone: true,
  selector: 'app-transportation-booking',
  imports: [FormsModule],
  template: `
    <div class="container">
      <h1>Book Your Transportation</h1>
      <div>
        <label for="from">From:</label>
        <input id="from" [(ngModel)]="from" placeholder="Enter starting location" />
      </div>
      <div>
        <label for="to">To:</label>
        <input id="to" [(ngModel)]="to" placeholder="Enter destination" />
      </div>
      <div>
        <label for="mode">Mode:</label>
        <select id="mode" [(ngModel)]="selectedMode">
          <option value="car">Car</option>
          <option value="bike">Bike</option>
          <option value="bus">Bus</option>
          <option value="train">Train</option>
        </select>
      </div>
      <button (click)="bookTransportation()">Book</button>
      <p>{{ bookingMessage }}</p>
    </div>
  `,
  styles: [
    `
      .container {
        text-align: center;
        margin: 20px;
      }
      input,
      select {
        margin: 10px;
        padding: 5px;
      }
      button {
        margin-top: 15px;
      }
    `,
  ],
  providers: [
    CarService,
    BikeService,
    BusService,
    TrainService,
    TransportationFactory,
  ],
})
export class TransportationBookingComponent {
  from: string = '';
  to: string = '';
  selectedMode: string = 'car';
  bookingMessage: string = '';

  private transportationFactory = inject(TransportationFactory);

  bookTransportation(): void {
    if (!this.from || !this.to) {
      this.bookingMessage = 'Please enter both From and To locations';
      return;
    }

    const service = this.transportationFactory.getTransportationService(this.selectedMode);
    this.bookingMessage = service.book(this.from, this.to);
  }
}

Benefits:

Extensibility: Adding new transportation modes is easy; just create a new service and register it with the factory.
Maintainability: Changes to a specific mode's logic are isolated.
Testability: Easier to mock and test individual services and the factory.
Decoupling: The booking component doesn't depend on concrete service implementations.

Conclusion

The Factory Design Pattern is a powerful way to create objects dynamically in Angular applications. By implementing it in our transportation service booking example, we've demonstrated how this pattern promotes flexibility, maintainability, and scalability.
This implementation also highlights the use of Angular 18 features, such as standalone components, making it a modern and efficient solution. With a structured approach and proper separation of concerns, this design pattern can be applied to a wide range of use cases.