DEV Community: Javid Gahramanov

Unit Testing Project references & CI/CD Pipelines?

Javid Gahramanov — Tue, 11 Feb 2025 07:14:52 +0000

Unconventional approach, isn't it? Who has the time to unit test pipelines? Well, if you've ever had a deployment go sideways because someone forgot to update a pipeline file, you'll understand why I did this. Nobody wants to be the person who breaks the build right before the weekend.

Ensuring the integrity of a CI/CD pipeline is crucial, especially in a monorepo where multiple projects depend on each other. A simple update to one package can have a ripple effect, and before you know it, half the services are failing. Wouldn't it be nice to catch those issues before they land in production? That’s exactly what I set out to accomplish. Why Unit Test Pipelines?

- Catch Errors Early: Detect issues before they reach production.
- Check Dependencies: Ensure changes in one project don’t disrupt others.
- Early Feedback: Identify and fix problems quickly to reduce stress.
- Follow Best Practices: Avoid shortcuts to maintain quality.
- Prevent Costly Failures: Resolving issues early saves time and resources.

How I Did It
I used YamlDotNet for parsing YAML pipeline files and MSBuild API to scan project dependencies. By comparing what's in the pipeline with what projects actually need, I ensured everything stays aligned and prevented nasty surprises.

Parsing YAML with YamlDotNet
YamlDotNet is a fantastic .NET library for handling YAML, which is great because most CI/CD pipelines rely on YAML configurations. I used it to read and analyze pipeline definitions to extract relevant build steps, dependencies, and triggers.

public static IEnumerable<string> GetYamlPaths(string yamlFilePath)
{
    var yamlContent = File.ReadAllText(yamlFilePath);
    var deserializer = new DeserializerBuilder()
        .WithNamingConvention(CamelCaseNamingConvention.Instance)
        .IgnoreUnmatchedProperties()
        .Build();

    var config = deserializer.Deserialize<Yaml>(yamlContent);

    return (config.Trigger.Paths.Include ?? [])
        .Where(path => path.Contains("sdk", StringComparison.OrdinalIgnoreCase));
}

Necessary classes represent Yaml structure.

public class Yaml
{
    public Trigger Trigger { get; set; } = null!;
}

public class Trigger
{
    public Paths Paths { get; set; } = null!;
}

public class Paths
{
    public List<string>? Include { get; set; }
}

Scanning Dependencies with MSBuild API
The below code shows scanning for our needs. It is limited only by your imagination! You can check external packages as well, or even prevent specific packages from being updated if necessary. The possibilities are endless, and it all depends on how you choose to apply it.

The MSBuild API helped me dynamically analyze project dependencies. This was especially useful in a Monorepo where projects reference each other and changes can cascade.

public static List<string> GetProjectDependencies(string path)
{
    var solution = SolutionFile.Parse(path);
    var list = new List<string>();

    foreach (var projectInSolution in solution.ProjectsInOrder)
    {
        if (projectInSolution.ProjectType == SolutionProjectType.KnownToBeMSBuildFormat)
        {
            var projectPath = projectInSolution.AbsolutePath;
            var project = new Project(projectPath);

            var dependencies = project.Items
                .Where(item => item.ItemType == "ProjectReference" &&
                    Path.GetFileNameWithoutExtension(item.EvaluatedInclude)
                        .EndsWith("_SDK", StringComparison.OrdinalIgnoreCase))
                .Select(item => Path.GetFileNameWithoutExtension(item.EvaluatedInclude))
                .ToList();

            list.AddRange(dependencies);

            ProjectCollection.GlobalProjectCollection.UnloadProject(project);
        }
    }

    return list.Distinct().ToList();
}

What does the following test do?
Load the pipeline YAML file.
Get project dependencies.
Compare both lists and verify they match.

Sample Test Case

[Fact]
public void VerifyPipelineTriggers_MatchProjectDependencies()
{
    MSBuildLocator.RegisterDefaults();

    var baseDirectory = AppContext.BaseDirectory;
    var testsFolder = Directory.GetParent(baseDirectory)?.Parent?.Parent?.Parent?.FullName!;
    var rootDirectory = Directory.GetParent(testsFolder)?.FullName!;
    var yamlFilePath = Path.Combine(rootDirectory, "azure-pipelines.yml");
    var sln = Path.Combine(rootDirectory, "Service.Users.sln");

    var yamlPaths = PipelineTestHelper.GetYamlPaths(yamlFilePath)
        .Select(c => c.Split("services")[1])
        .Select(c => c.Replace('/', '.').TrimStart('.'))
        .ToList();

    var projectDependencies = PipelineTestHelper.GetProjectDependencies(sln);

    Assert.True(yamlPaths.SequenceEqual(projectDependencies, StringComparer.OrdinalIgnoreCase));
}

The test checks only paths in the YAML file and compares them with the suffix .Sdk to ensure that the SDK projects are included in the pipeline. You can extend it further based on your needs—your imagination is the only limit! For instance, you could enhance it to prevent certain vulnerable packages from being introduced into the project. This is especially beneficial when working with a large team of 40+ developers, ensuring that the project remains secure and aligned with best practices.

Conclusion
A well-tested pipeline leads to fewer disruptions, reduced manual intervention, and a more predictable release process. Investing in pipeline validation ultimately results in long-term software stability and operational efficiency.

Next Steps:
Planning Test app.settings Files: Many of our microservices follow the same structure, making it crucial to ensure that we do not overlook any common configurations. Implementing validation checks across these settings can prevent misconfigurations and inconsistencies across environments.

[Boost]

Javid Gahramanov — Wed, 11 Dec 2024 11:11:34 +0000

Polymorphic Serialization in .NET

Javid Gahramanov ・ Nov 29 '24

#dotnet #csharp #aspnet

[Boost]

Javid Gahramanov — Sat, 30 Nov 2024 20:57:11 +0000

Polymorphic Serialization in .NET

Javid Gahramanov ・ Nov 29 '24

#dotnet #csharp #aspnet

Polymorphic Serialization in .NET

Javid Gahramanov — Fri, 29 Nov 2024 18:58:12 +0000

In many applications, we often work with data that looks different but shares some common features. For example, a system might have different types of records or transactions that have similar properties but also their own unique details. Handling this kind of data can be tricky when saving or loading it as JSON.

We already know polymorphism lets us treat different types of objects as a single base type. Polymorphic serialization in .NET 7 makes it easy to work with different types of data that share common properties. With JsonPolymorphic and JsonDerivedType in System.Text.Json, you can save and load objects with a base class while keeping the unique details of each type.

In this article, I will talk about how JSON polymorphic serialization works and how we applied it to manage patient records in a Cardiology Practice Management System.

Let's look at the code:

Base type for patient records

[JsonPolymorphic(TypeDiscriminatorPropertyName = "recordType")]
[JsonDerivedType(typeof(ECGRecord), typeDiscriminator: "ECG")]
[JsonDerivedType(typeof(EchoCardiogramRecord), typeDiscriminator: "EchoCardiogram")]
[JsonDerivedType(typeof(CardiacEventRecord), typeDiscriminator: "CardiacEvent")]
[JsonDerivedType(typeof(ConsultationRecord), typeDiscriminator: "Consultation")]
public abstract class PatientRecordBase;

Other derived types
Each type of patient record inherits from PatientRecordBase and adds its own unique fields.

public class ECGRecord : PatientRecordBase
{
    public int HeartRate { get; set; }

    public RhytmType RhythmType { get; set; }
}

public class EchoCardiogramRecord : PatientRecordBase
{
    public double EjectionFraction { get; set; }

    public string Observations { get; set; }
    .....
}

public class CardiacEventRecord : PatientRecordBase
{
    public string EventDescription { get; set; }

    public string Trigger { get; set; }

    public string Outcome { get; set; }
    .....
}

public class ConsultationRecord : PatientRecordBase
{
    public string DoctorNotes { get; set; }

    //we use same approach for prescriptions as well.
    //Doctor may prescribe different things,right?
    // such as : pharmaceutical drugs,Lifestylechanges,advices,tests
    public List<PrescriptionBase> Prescriptions { get; set; }
   .....
}

Patient Records API


public class PatientRecordCreateRequest
{
    public string PatientId { get; set; }

    public DateTimeOffset DateTime { get; set; }

    public RecordType RecordType { get; set; }

    public PatientRecordBase Record { get; set; }
}

[Route("api/{patients}")]
[Authorize]
public class PatientRecordsController(IPatientService service, IMapper mapper) : ControllerBase
{
    [HttpPost]
    public async Task<IActionResult> CreateAsync([FromBody] PatientRecordCreateRequest request)
    {
        // PatientRecord structure is same as PatientRecordCreateRequest 
        var patient = await service.CreateAsync(mapper.Map<PatientRecord>(request));
        return Ok(patient);
    }
}

Let's see in action

Request payload

{
  "patientId": "p-7415879",
  "dateTime": "2024-11-30T06:07:44.390Z",
  "records": [
  {
    "recordType": "ECG",
    "heartRate": 72,
    "rhythmType": "Sinus"
  },
  {
    "recordType": "EchoCardiogram",
    "ejectionFraction": 55.6,
    "observations": "Normal function"
  },
  {
    "recordType": "Consultation",
    "doctorNotes": "Tachycardia during workout... "
  }
 ]
}

Thanks for reading!

Real-World .NET and EF Solutions: Simplifying Multitenancy and Soft Delete with Global Query Filters

Javid Gahramanov — Wed, 27 Nov 2024 09:48:01 +0000

Global query filters in Entity Framework are a powerful feature introduced in EF Core 2.0. They allow you to define filters at the model level that apply automatically to every query involving a particular entity. This eliminates the need to repeatedly include specific filter conditions in your queries. In this article, we'll explore two common use cases for global query filters in ASP.NET Core applications: Multitenancy and Soft Delete.

Why Use Global Query Filters?

Imagine you're working on a multitenant application where every entity is scoped by a TenantId. You'd typically filter queries to include only entities belonging to the current tenant. Without global query filters, you'd need to include the TenantId condition in every query, which can become repetitive and error-prone.

Another scenario is implementing Soft Delete, where instead of physically deleting records, you mark them as deleted using a flag (e.g., IsDeleted). Without a global filter, you’d need to ensure that every query excludes entities where IsDeleted = true.

Setting Up Global Query Filters in EF Core

Tenant and Softdelete Filtering
Let’s say you have an application with the following BaseEntity:

public class BaseEntity
{
    public bool IsDeleted { get; set; }

    public Guid TenantId { get; set; };
}

Define your DbContext with a global query filter for TenantId:

public class ApplicationContext(DbContextOptions<ApplicationContext> options, IRequestContext context) : DbContext(options)
{
    public DbSet<Product> Products { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        base.OnModelCreating(modelBuilder);
        modelBuilder.ApplyGlobalFilter<BaseEntity>(e => !e.IsDeleted && e.TenantId == context.TenantId);
    }
}

Tenant specific entity - Product

public class Product : BaseEntity
{
    public string Name { get; set; }  = null!;

    public decimal Price { get; set; } 
}

Extension to Register query filters with Convention-Based technique.

public static class ModelBuilderFilterExtensions
{
    public static void ApplyGlobalFilter<TEntity>(this ModelBuilder modelBuilder, Expression<Func<TEntity, bool>> expression)
    {
        var type = typeof(TEntity);
        var entities = modelBuilder.Model.GetEntityTypes()
            .Where(e => type.IsAssignableFrom(e.ClrType))
            .Select(e => e.ClrType);

        foreach (var entity in entities)
        {
            var param = Expression.Parameter(entity);
            var body = ReplacingExpressionVisitor.Replace(expression.Parameters.Single(), param, expression.Body);
            modelBuilder.Entity(entity).HasQueryFilter(Expression.Lambda(body, param));
        }
    }
}

How It Works

The ApplyGlobalFilter method in ModelBuilderFilterExtensions is responsible for dynamically adding filters to all entities that inherit from a specific type (BaseEntity in this case).

When you inject the ApplicationDbContext into your services or controllers, queries for entities inheriting BaseEntity will automatically include the TenantId filter.

When GetProductsAsync is called in the ProductService, the following occurs:

The ApplicationDbContext applies the global query filters (!IsDeleted && TenantId == context.TenantId) to the Products DbSet.
Only products belonging to the current tenant and not marked as deleted are included in the query results.
This streamlined approach ensures cleaner code and enforces consistent filtering across all queries without additional effort.

public class ProductService(ApplicationDbContext dbContext)
{
    public async Task<List<Product>> GetProductsAsync()
    {
        return await dbContext.Products.ToListAsync();
        // Automatically filters by the current TenantId
    }
}

Conclusion
EF's global query filter is a fantastic tool for simplifying repetitive filtering logic in applications. Whether you're managing multitenancy with TenantId or implementing soft delete with IsDeleted, they make your code cleaner and more maintainable.

Official doc : https://learn.microsoft.com/en-us/ef/core/querying/filters

Happy coding! 🚀