DEV Community: Vidisha Parab

Working with JsonObject, JsonNode, JsonValue and JsonArray [System.Text.Json API]

Vidisha Parab — Mon, 20 May 2024 08:58:19 +0000

Most of the articles which I found on the web about the System.Text.Json API dealt with serialization and de-serialization with little being explored about creating a JSON entity on the fly.
A recent use-case required us to build a data entity dynamically while running through a set of business logic (emphasis on the fact that the output here could not be modelled as a POCO since it was going to take form during the run-time). This entity would be serialized and sent over to a downstream service. While one could opt for the Dynamic Objects, we decided to work our way with JSON objects.

When working with dynamic data through the System.Text.Json API, everything revolves around the JsonNode class. It is an abstract class and is derived by the JsonArray, the JsonObject and the JsonValue. These can be thought of as the building blocks - we work with them more in the coming examples.

Before we actually dive into the System.Text.Json API, let's take a step back and re-visit the JSON standard.

The json.org quotes the following

JSON is built on two structures:
A collection of name/value pairs. In various languages, this is realized as an object, record, struct, dictionary, hash table, keyed list, or associative array.
An ordered list of values. In most languages, this is realized as an array, vector, list, or sequence.

It then elaborates how in JSON, this could take two forms - either an object and an array.

An object and an array in JSON can have a JSON value
A value in JSON can also in turn be an array, a number a object , etc

We kind of get an idea that a json value, json object and the json array are all intertwined.

Going back to System.Text.Json API, we have the JsonObject which can be used to realise an object in JSON, JsonArrayas an array and JsonValuefor a value in JSON.

We start with JsonValue and build our way up. The JsonValueis an abstract class (you cannot instantiate an abstract class). The next obvious thought is - how are we then going to realise a C# type here without being able to instantiate the type ? Our C# counterpart could be anything string, double, int, etc.

The JsonValue class contains static methods (Create) for the C# value types and based on in-built implicit conversion, they are going to give you a type of JsonNode for the C# value you provided.

var firstName = JsonValue.Create("Doofus");
var lastName = JsonValue.Create("Flintstones");
var addressPostcode = JsonValue.Create(1234);
var phoneNumber= JsonValue.Create(1234567890);
var knowsCSharp= JsonValue.Create(true);

You can always be explicit about the types too,

var identifier = JsonValue.Create<Guid>(Guid.NewGuid());
var yearAtDev = JsonValue.Create<int>(5);

You might wonder what's the advantage here ? Also we said we want a JsonValue but we are getting back a JsonNode - it will make sense in a while but for now what we can assert here is the type definition.

As of .NET 8, through System.Text.Json API we also get JsonValueKind

public enum JsonValueKind : byte
{
 Undefined = 0,
 Object = 1,
 Array = 2,
 String = 3,
 Number = 4,
 True = 5,
 False = 6,
 Null = 7
}

While working with serialization and deserialization of data, the widely adopted standard is JSON.

JsonValue.Create<int>(5) would create a JSON value of kind number and
JsonValue.Create<string>("5") would create a JSON value of kind string. This would matter when this data would be deserialized in any downstream system

{
 "age" : 5,
 "age2" : "5"

}

would be deserialized differently, age would have to be deserialized as an integer and age2 as a string. Hence it is good to assert type definitions while building up the json data at the source and through the different factory method overloads (can also be achieved with the implicit conversions) , System.Text.Json API provides a cleaner way

We can also use the GetValueKind method to get the JsonValueKind

   var guid = JsonValue.Create<Guid>(Guid.NewGuid());
   var jsonValueKind =  guid.GetValueKind();

Now that we have a fair understanding of the JsonValue, we try to use that to realise a JsonArray.

A JsonArray is but a collection of json values or json objects.

To be completely honest, I haven't explored much with JsonArrayexcept for a deserialization use-case. However I skimmed through the documentation and here is one sample to create a JSON array
JsonArray is a sealed class, you cannot inherit it but you can instantiate it.

   var jsonArray = new JsonArray();
   jsonArray.Add(JsonValue.Create(23));
   jsonArray.Add(JsonValue.Create("23"));

The Add method on the JsonArray accepts a parameter of type JsonNode and if you recall the definition from the json.org , a Json array could consist of a JSON object or a JSON value - this blends quiet well here, that a JsonNodein System.Text.Json could be very well be substituted by JsonValue or JsonObject i.e. you can add either of the types through the Addmethod on the JsonArray

But please feel free to correct me here and add any relevant use case with JsonArrayif you know of :)

Moving on to the JsonObject,

JsonObject is basically a collection KeyValuePair of string and JsonNode ! It derives from the JsonNode and well holds a JsonNode too.

In the beginning, the JsonNode had got me all confused but if you take a moment and appreciate the tree data structure and think of JSON as one, then a tree is just a collection of nodes. With JSON, the different entities are a value, object and an array so a node could be any one of these ? and may be then it makes sense that the JsonValue, JsonArray and JsonObject derive from the JsonNode . Anyway, I like to look at it that way.

With JsonObject then its just putting together the json values, arrays or even nested objects

  var jsonObject = new System.Text.Json.Nodes.JsonObject()
  {

   ["firstName"] = JsonValue.Create("Doofus"),
   ["lastName "] = JsonValue.Create("Flintstones"),
   ["address"] = new System.Text.Json.Nodes.JsonObject
   {
    ["line1"]= "Stockholm",
    ["line2"]= "Sweden"
   },
   ["knowsCSharp"] = JsonValue.Create(true),
   ["yearsAtDev "] = JsonValue.Create<int>(5),
   ["hobbies"] = new JsonArray() { "music", "dance"}

  };

Also now might be a good time to look at how we could work with deserialization here

Usually when we get data over the wire, we get a JSON. In our consumer when we get a JSON response (say through a REST call) , we deserialize it in a POCO

{
  "FirstName": "Doofus",
  "LastName": "Flintstones",
  "Address": {
    "street": {
      "line1": "Stockholm",
      "line2": "Sweden",
      "line3": "Nordic"
    },
    "postcode": 1234
  }
}

the POCO could be modelled as


class Person
{
      public string FirstName { get; set; } = string.Empty;
      public string LastName { get; set; } = string.Empty;
      public Address? Address { get; set; }


}

public class Address
{
      public Street? Street { get; set; }
      public int Postcode { get; set; }
}

public class Street
{
      public string Line1 { get; set; } = string.Empty;
      public string Line2 { get; set; } = string.Empty
      public string Line3 { get; set; } = string.Empty;
}

and we get the deserialized data by doing -

var person = System.Text.Json.JsonSerializer.Deserialize<Person>(jsonResponseAsString);

But what if we instead deserialize into a JsonObject and not a POCO ?

var personAsJsonObject = System.Text.Json.JsonSerializer.Deserialize<JsonObject>(jsonResponseAsString);

If you have used Newtonsoft before, you might be familiar with the JObject and we used an indexer to traverse it.

A JsonObject if you recall is a key value pair, the value being of type JsonNode

If we want to access the property FirstName then we could either use the indexer

personAsJsonObject["firstName"] or personAsJsonObject.TryGetPropertyValue("firstName", out var value)

Both are going to give you a JsonNode, but a JsonNode could be either a JsonValue,a JsonArray or JsonObject how would you know ?

On JsonNode, we can call AsValue(), AsArray() and AsObject() to get the corresponding type. Next, we could also assert the kind through GetValueKind and call any one of the factory methods we saw earlier to retrieve our value.

I did it very old-school way, something on the lines of this , you can have one each for JsonObject and JsonArray as well

public static JsonValue ToJsonValue(JsonNode node)
{
    try
    {
        return node.AsValue();
    }
    catch (InvalidOperationException)
    {
      //handle it at the caller level by letting it fall-back to 
      next in chain , perhaps JsonArray ?
    }
}

Following is an extract from the .NET source code, and as you see an InvalidOperationExceptionis going to get thrown if you try to call AsValue() - say for example on a JsonObject( i.e. JsonNodewhich is of type JsonObject) . AsValue() would only yield a JsonValueon being called on a JsonNodewhich holds a JsonValue, for others we get an exception

    public JsonValue AsValue()
    {
        JsonValue? jValue = this as JsonValue;

        if (jValue is null)
        {
            ThrowHelper.ThrowInvalidOperationException_NodeWrongType(nameof(JsonValue));
        }

        return jValue;
    }

Next after you have retrieved a JsonValue, you can call the nice helper methods to retrieve a string, boolean, int, etc.

jsonValue.TryGetValue(out string? value)

While there is a lot of ground to cover here and also to learn, even for me ..I hope this serves as a starting point/ basics when you in future would want to explore this path

Any suggestions/improvements welcome.

== problem with generics

Vidisha Parab — Fri, 05 Jan 2024 19:37:14 +0000

Evaluating equality for a type would mean to define the conditions any two instances of it, must meet in order for them to be considered equal.

.NET by default, gives you reference equality out of the box - two different instances of a type would never evaluate to be equal because they would essentially point to different memory locations.

Having said that, .NET also gives us the possibility to override the reference equality provided by default and define our definition of when two instances of a type be considered equal. These are made available through the System.Object contract -

virtual bool Equals(object? obj)

static bool Equals (object? objA, object? objB)

static bool ReferenceEquals (object? objA, object? objB)

Few things to remember here is ReferenceEquals is internally called by the default implementation of Equals(object? obj) to give us the reference equality and also the static bool Equals (object? objA, object? objB) internally calls the default virtual Equals. The static implementation is given as an additional sanity check if we ever try to evaluate a null instance for equality.

So if we want to override the default equality, we override the above methods in our type.

Also if we look closely, the argument for the Equals method is object and all the types in .NET derive from object class, this would definitely come with an overhead of casting our type (for which we are defining equality). To make things efficient, .NET also gives us the IEquatable contract which ensures type safety.

The last thing to remember before we move on to the actual purpose of the post, is the == operator we use to check if two instances are equal. Well this is not from .NET and it is a C# operator.

If we define our own equality for one of our types by overriding the previously mentioned three methods and the IEquatable contract and not override the == operator then we are bound to get ambiguous results. The C# operator == by default for a reference type will always evaluate the reference equality

Consider the Person type here, lets assume that for two person instances to be equal we will just look at the Name and not the City value. Two Person instances with same Name value are equal. We define the equality here as follows

public class Person : IEquatable<Person>
{
 public string Name { get; set; } = string.Empty;
 public string City { get; set; } = string.Empty;

 public static bool operator ==(Person left, Person right) => left.Equals(right);

 public static bool operator !=(Person left, Person right) => !left.Equals(right);
 public bool Equals(Person? other) => other?.Name == Name;


 public override bool Equals(object? obj)
 {
  if (obj is Person)
  {
   return Equals((Person)obj);
  }
  return false;
 }

 public override int GetHashCode() => Name.GetHashCode();

}

Now consider the below snippet, if we were to write a generic method which checks if the two person instances are equal we probably would do something like this

var person1 = new Person { Name = "Doofus", City = "London" };

var person2 = new Person { Name = "Doofus", City = "Madrid" };

Console.WriteLine( $"Are they equal through Equals ? {IsThePersonSameCheckWithEquals(person1, person2)}");

Console.WriteLine($"Are they equal through Operator? {IsThePersonSameCheckWithOperator(person1, person2)}");

static bool IsThePersonSameCheckWithEquals<T>(T person1, T person2)
 where T:class
{
 return person1.Equals( person2);
}

static bool IsThePersonSameCheckWithOperator<T>(T person1, T person2)
 where T : class
{
 return person1 == (person2);
}

If we observe the outputs, the equality evaluates to false with == operator in spite of us overriding it in our type Person .

For Generics, the compiler assumes that there is no overload for the == operator (and also disregards the one we wrote ourselves) and checks for reference equality instead
We should instead use the Equals to compare instances in a generic code implementation

Basics of accessing the Configuration in .NET Web App

Vidisha Parab — Tue, 19 Sep 2023 15:41:35 +0000

In the previous post we talked about Configuration Sources and Configuration Provider - two different concepts but intertwined together.

We also spoke in brief about how this

var builder = WebApplication.CreateBuilder(args);

wires up the defaults for our apps (in terms of logging, dependency injections, configuration, etc. )

We also established that appsettings.json and appsettings.{environment}.json are amongst the default configuration sources and are loaded by default through the json configuration provider. In addition to the defaults, we can also add our own custom sources and of course they would need their own custom configuration providers defined (i.e. you need to write your own).

Let's look at how you can access the configuration within your application from the builder

Say you have an appsettings.json file

{
    "ConnectionStrings": {
        "SqliteConnection": "Filename=:memory",
        "PostgresConnection": "Post gres connection string goes here"
    },
    "Logging": {
        "LogLevel": {
            "Default": "Information",
            "Microsoft.AspNetCore": "Warning"
        }
    },
    "AllowedHosts": "*",
    "Message": "Roofus Doofus !!"
    "Title" : {
        "HomePage" : "Stockholm i mitt hjärta"
       }
}

And you want to access the Message and the ConnectionStrings in your application. Since appsettings.json is shipped by the .NET framework and loaded by default, we don't have to worry about that part.

We can access the Message config from the file ,in its most basic form through an indexer on the Configuration collection provided by the builder object.

var message = builder.Configuration["Message"];

The config which we want to access becomes the key here

The connection string for Sqlite can be retrieved as :

var sqlConnectionString = builder.Configuration.GetConnectionString("SqliteConnection");

If you closely observe SqliteConnection is a level below (or nested within) ConnectionStrings . Usually, if we have to access a nested config then we do it through the ":" notation.
But the GetConnectionString is an extension method provided by the framework and it does the following behind the scenes

   public static string GetConnectionString(this IConfiguration configuration, string name)
        {
            return configuration?.GetSection("ConnectionStrings")?[name];
        }

But since the DI is one of the key highlights of .NET framework, we can inject the configuration as a dependency which will be resolved by .NET framework itself

The Configuration on the builder is of type ConfigurationManager which is a concrete implementation of IConfigurationBuilder, IConfigurationRoot

IConfigurationRoot extends IConfiguration and as msdn highlights here , IConfiguration represents a set of key/value application configuration properties and that is what we are interested in.

So we inject a dependency of IConfiguration in our services if we want to access any configuration within it

Following is a snippet from a RouteEndpoint (an endpoint handler) from the minimal api scaffold

app.MapGet("/", ([FromServices] IConfiguration configuration) =>
{
   var sb = new StringBuilder();

   //By default the appsettings.json is loaded by the json configuration provider when the host is built

   //if we have to access the configuration, we can do it via the IConfiguration interface

   //The hierarchical structure in the json file is flattened like this. you access the children through ":" notation

   var title= configuration.GetValue<string>("Title:HomePage");
   sb.Append($"Homepage Title : {title} \n");

//If you try to access a non-existent key, it will set the default value

   var nonExistentKeyInConfiguration = configuration.GetValue<bool>("Title:Shopping");

   sb.Append($"Trying to access a non-existent key from the configuration  will set the default value of T on GetValue<T>: {nonExistentKeyInConfiguration} \n");


//And if you try to cast the config to an invalid cast, e.g. our Title config is given as a string and we try to cast it to int then we will get an exception

   try
   {
      var inconsistentTypeCasting = configuration.GetValue<int>("Title:HomePage");

   }
   catch (Exception e)
   {
      sb.Append($"Inconsistent type casting of a config value while retrieving it will throw an exception of {e.InnerException?.Source}\n");

   }

   return sb.ToString();
});

Lets look at a more complex config and how to map it to a csharp complex type
You have the following in your appsettings.json

    "Pet": {
        "Name": "Roofus",
        "Description": "I climb through people's house fences and scare them'",
        "IsFriendly": false,
        "Age": 5
    }

For this you would create a complex type say Pet :
(OBS - The type name can be different than the config name, it does not have to match)

 public class Pet
 {
  public string Name { get; set; } = string.Empty;
  public string Description { get; set; } = string.Empty;
  public bool IsFriendly { get; set; }
  public int Age { get; set; }
 }

Going back to our json config,

    "Pet": {
        "Name": "Roofus",
        "Description": "I climb through people's house fences and scare them'",
        "IsFriendly": false,
        "Age": 5
    }

"Pet" is referred as a section and we bind the "section" calling the Bind on the IConfiguration injected in our service


 //we create the pet object where we want to bind the configuration
   var petUsingBind = new Pet();

   //Call the bind with the section name and pass the object created.
   configuration.Bind("Pet", petUsingBind);

To wrap up ,
In the previous post we saw that the default configurations are loaded in a particular order and the order in which the configuration sources are loaded determines if a config overrides the other i.e. the order matters !

Well, the good news is we can override the default behaviour and define our own through the ConfigureAppConfigurationavailable on the hostexposed through builder

builder.Host.ConfigureAppConfiguration((ctx, configBuilder) =>
{
 //we clear all the config sources previously read in the default order 

 configBuilder.Sources.Clear();

 var env = ctx.HostingEnvironment;

 configBuilder.AddEnvironmentVariables("ASPNETCORE_");
 //we are only reading the appsettings.json file here. explicitly skipping the environment specific
 configBuilder.AddJsonFile("appsettings.json",optional:true,reloadOnChange:true);

//We skip the user secrets
//We skip the command line arguments as well

});

We will look at the options pattern in the next post

References

The msdn documentation on fundamentals of configuration is really good and has all you need to know information. Please check it out to explore more in this area

Customization through ConfigureAppConfiguration

Further reading on ConfigurationManager (type of builder.Configuration)

Configuration Source vs Configuration Provider

Vidisha Parab — Thu, 14 Sep 2023 09:31:53 +0000

This is part 1 of the series I intend to write about getting onboarded with configuration in ASPNETCORE. I will try to post the follow up articles soon

Before we actually talk about configuration in ASPNETCORE, it is important to address the Configuration Source and the Configuration Provider. We try to do that here

Configuration Source

Configuration Source is the one which will act as the "input" to configure our application. It could be a json file, command line argument, an environment variable, the dotnet user secrets or it could even be an external config store - AWS Parameter Store or even a SQL database.

We all are familiar with the appsettings.json which gets shipped by default when we ask for a ASPNET Core web application scaffold. This then is a configuration source, where you would define the configuration to be used within your application as key value pairs.

The only important thing to stress here is that the configuration source is anything where we specify any "settings/configurations" with their "values" which we can feed in as a "configuration" to our application

Configuration Provider

Having gotten the configuration source out of our way now, its quiet easy to understand the concept of a configuration provider.

var builder = WebApplication.CreateBuilder(args);

CreateBuilder is kind of responsible to configure logging, set up the dependency injection container, configure a default host and also make available the default configuration sources to our application. The CreateBuilderis what makes us available the default "host". Using the default host object, we can also add more configuration sources to it using the extensions provided by the ASPNETCORE framework.

Well, the first thing you would do is to define a configuration source (lets say it is a json file - you have all your configurations and their values in it) but then you must make the ASPNETCORE know of it in order for it to be available in your application.

Comes in the Configuration Provider . It will "load" a specific configuration source and then make it available in your application, in its most basic form through builder.Configuration (more on it later)

Some of the default configuration providers by ASPNETCORE are Command-Line Configuration Provider, Environment variables configuration provider, JSON configuration provider. When we say default, it means ASPNETCORE will automatically handle them for us we need not do anything.

Following is a snippet from here . This is what is happening behind the scenes when ASPNETCORE is wiring up a default builder for you . Also notice, the order in which it loads the default configurations sources through their respective providers . So in the most vanilla form, any configuration provided by the command line arguments would override the ones before i.e. if you have a config key isKanelbulle set as false in your appsetting.json and while running the application, you pass in the same as an command line argument isKanelbulle set to true, the true overrides the false value set in the json file

  appConfigBuilder.AddJsonFile("appsettings.json", optional: true, reloadOnChange: reloadOnChange)
                    .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true, reloadOnChange: reloadOnChange);

            if (env.IsDevelopment() && env.ApplicationName is { Length: > 0 })
            {
                try
                {
                    var appAssembly = Assembly.Load(new AssemblyName(env.ApplicationName));
                    appConfigBuilder.AddUserSecrets(appAssembly, optional: true, reloadOnChange: reloadOnChange);
                }
                catch (FileNotFoundException)
                {
                    // The assembly cannot be found, so just skip it.
                }
            }

            appConfigBuilder.AddEnvironmentVariables();

            AddCommandLineConfig(appConfigBuilder, args);

Putting it together now,

Say you have a json file as the configuration source - lets call it MyDefaultConfigurations.json . Then to make it available in the application, you would instruct the default json configuration provider of ASPNETCORE to "load" it.

builder.Configuration.AddJsonFile("MyDefaultConfigurations.json", optional: true,reloadOnChange: true);

OBS - appsettings.json and appsettings.{Environment}.json are loaded by default, you don't have to register them explicitly as done above

Also an important thing to consider here is - in addition to the default configuration providers, you can build your own custom configuration providers as well.

A use case of this would be - say you have an external API which is responsible to give you some configurations after running a complex logic.

This then would become your configuration source. ASPNETCORE would obviously be unaware of this and none of the default providers would know how to handle it. This is then a good candidate for you to write your custom config provider which say would make a REST call to this service of yours to get the configuration and make it available in your application. There is already a good documentation on how to implement your custom configuration provider, link below.

Please feel free to correct :)

References

default-application-configuration-sources

Custom Configuration Provider Example

AutoFixture - Specimen Builders

Vidisha Parab — Tue, 12 Sep 2023 21:44:00 +0000

I will try to keep this one short by not going into details of what Autofixture is , why we need it... I am assuming you would be familiar with it

What is a specimen ?

In programming we work with types(i.e. int, string, decimal, etc. ) - a specimen is an example value of a particular type.

E.g. "Kanelbulle" is a specimen (an example value) for the type string or 0.23m , 0.50m are specimens for the type decimal.

What is a specimen builder in context Auto fixture ?

Having known what a specimen is now, we can safely assume that the logic ( basically set of instructions) to build a specific specimen is a specimen builder.

In terms of AutoFixture, it is a concrete implementation of the ISpecimenBuilder

Given a complex type like this

class Employee
{
 public string Name { get; set; }
 public int Code{ get; set; }
}

when we ask the fixture (fixture.Create<Employee>()) to give us anonymous data of type Employee, behind the scenes a set of specimen builders (either default or custom) are going to run and generate us a specimen for a string and int (which are basically the two properties of the Employee class here).

For e.g. the default specimen builder for a string spits out a guid as the value for us. For a complex type , like Employee in this case, a string gets a specimen prepended with the property name (as it is the case for all the properties of a complex type) something like this Namead6fb2f0-d23a-4bb0-a083-8ffca9a139aa

Before moving to writing a custom (concrete) implementation of the ISpecimenBuilder, I want to put across the way I have understood the AutoFixture pipeline (feel free to correct me here)

AutoFixture is responsible for creating Anonymous data for a particular type of T (** anonymous data is anything which is not going to affect our SUT /system under test in anyway i.e. we don't care for the values but the instance or type is needed by our SUT regardless of its value)
T could be a primitive type or any custom defined type
When creating an instance (anonymous data) of T, the first step is the customization phase where the AutoFixture library looks for a collection of the ISpecimenBuilders
AutoFixture has its own set of default Specimen Builders and then if we write any custom ones,they will override those default specimen builders if registered in this phase 5.The customization phase is taken care by calling the Customizations or the Customize on the fixture (we will see in a bit) instance
The next step in the pipeline is then to "run" these concrete specimen builders to generate the specimen (example values) for a given type
Last is the fall back step, where the Autofixture library applies fallbacks/defaults (tries real hard to generate a specimen for us !) and if still no specimen was able to be generated for a particular type(could be because of some validations in the constructor in case of a complex type), then eventually throw an error .

In nut shell , a request for fixture.Create() would go through a series of specimen builders to finally give us an instance of T

When would you want to create a custom specimen builder ?

If you want to influence the logic of building one in any specific way !

For e.g.

var fixture = new Fixture();

var date1 = fixture.Create<DateTime>();

The default specimen builder for DateTimeshipped with Autofixture is going to return you a specimen (value) of a random date time value per your system's culture but say you want to override this and instead want the UTC date to be returned as a specimen

In this case you can have your own specimen builder implementation. Refer this and you can build your UTC one in a similar way

 public class CurrentUtcDateTimeGenerator : ISpecimenBuilder
    {

        public object Create(object request, ISpecimenContext context)
        {
            if (!typeof(DateTime).Equals(request))
            {
                return new NoSpecimen();
            }

            return DateTime.UtcNow;
        }
    }

and then register this specimen builder (remember the Customize stage which we spoke about earlier ?)

fixture.Customizations.Add(new CurrentUtcDateTimeGenerator());

next whenever we ask for an instance of type DateTime from this fixture it is going to give us the current UTC date

A lot more things can be achieved through writing your own specimen builders.
For e.g. say you have a class

public class Employee { public string Name { get; set; } public string EmployeeCode { get; set; } }

You want to influence how EmployeeCode is generated , then you could override the default guid specimen for a string here.

 public class EmployeeCodeGenerator : ISpecimenBuilder
 {

  private readonly char[] _allowedChars = { 'V', 'I', 'D', 'S', 'H', 'A' };

  public object Create(object request, ISpecimenContext context)
  {

   //we use reflection here and pattern matching to see if the specimen we are trying to build is a property
   //we are interested in the EmployeeCode property of the Employee class

   if (request is PropertyInfo propertyInfo)
   {

    //next we match the name to influence the specimen building logic for the EmployeeCode property

    if (propertyInfo.Name.Contains("employeecode", StringComparison.InvariantCultureIgnoreCase))
    {
     //when we get hold of it we return our custom 3 digit code instead

     return new string(_allowedChars, 0, 3);

    }

   }

   return new NoSpecimen();
  }
 }

and then use the above like this

var fixture = new Fixture();
fixture.Customizations.Add(new EmployeeCodeGenerator());

var employee = fixture.Create<Employee>();
Assert.Equal(3, employee.EmployeeCode.Length);

Hope this was useful

Reference

Convention-based Customizations with AutoFixture

Athena - Question regarding 'Create a table' options

Vidisha Parab — Sun, 20 Jun 2021 10:25:00 +0000

I have just started learning AWS Athena through an online course and have been following it up with the official aws documentation. I understand that Athena uses the Hive as an underlying mechanism to issue ddl queries, so it is basically the Hive syntax which we use.

From their documentation, I have trouble understanding these things :

[COMMENT table_comment] Creates the comment table property and populates it with the table_comment you specify.

What is the comment table property and how does it helps us ? do we have any specific usage of this

[STORED AS file_format]

Specifies the file format for table data . By specifying this we would instruct how the data obtained from querying a resource should be saved in the output s3 file

But for this option, specifically
INPUTFORMAT input_format_classname OUTPUTFORMAT output_format_classname

from where would we get the input_format_classname and output_format_classname ?

reference

Any leads to an alternative reference or documentation would be appreciated :)

things I learnt today - delete remote branch through cli

Vidisha Parab — Thu, 03 Sep 2020 12:31:07 +0000

list only the remote branches
git branch -r

delete a remote branch
git push -d origin <branch_name>

remove stale/obsolete remote branch references from local
git fetch --all --prune

Can someone #explainlikeimfive the concept of virtual private cloud ?

Vidisha Parab — Wed, 12 Aug 2020 20:46:13 +0000

Hello !
I have just started with my cloud journey and so far have a fair understanding of what public,private and hybrid clouds are.However finding it a bit difficult to wrap my head around VPC. Any references to post/links would be appreciated as well !

Thanks !

c# basics -encapsulation , properties- manual,auto-implemented, readonly ,static

Vidisha Parab — Thu, 06 Aug 2020 14:46:14 +0000

Object oriented programming (OOP) focuses on modelling a business problem through objects. These objects are instances of a "Class". A class thus serves as a blueprint to create objects from. A class apart from being a piece of code, carries data and behavior the object must posses.The data fields are referred to as properties and the behavior are called as methods , together they are referred as class members. An important point to focus on here is one of the pillars of the OOP -Encapsulation. To encapsulate means to confine or encase something. In OOP we encapsulate a class by hiding its members. Here it is obvious to think, if we want to keep members hidden then why create the class in the first place ? However, when we create a class we have the liberty to expose the members. We decide which members should be accessible to the outside world , in view of this a class is also called as a "Contract". We are familiar with the access modifiers like private ,protected and public, which we decorate our class methods with. What about the data fields or the class properties ? Well, we expose them through getters and setters.Consider a class Employee, if we were to define a data field which holds an employee's name then the most basic way to do it manually as follows

In here we are exposing the field _employeeName through the public property EmployeeName in Employee (class) contract . We see two pieces of information,

get { }; set { };

. And mind you these are the most powerful tools we could use for our benefit. Since we are not directly exposing our data field _employeeName , we have a great level of control to validate the value we set for this at a field level ! For e.g. we could write much complex logic in the set { } block. We will get to it soon.
Another way to implement properties is to use the "autoimplemented" version provided by visual studio (when you use the snippet- prop. type prop and press tab).We use auto-implemented properties when we don't have to write any logic for getting or setting the data field - no validations or logic to get in the way- just plain simple accessing and setting the property. However an important thing to note here, behind the scenes the compiler creates a backing field for them as a default. Same can be seen for the

public int TimeHours { get; set; }

. Below is the IL (Intermediate Language) code for TimeHours

We could also make our properties read-only , the only thing we need to do is to have a private set;
Say you have to calculate the pay for an employee. This salary depends on the hours the employee puts in and their designation. We don't want any external agent to set this value and our class is solely responsible to do this internally . We can achieve that through the following

 public class Employee
    {
        private string _employeeName;

        public string EmployeeName
        {
            get
            {
                return _employeeName;
            }
            set
            {
                _employeeName = value;
                ;
            }
        }

        public int TimeHours { get; set; }
        public Designation EmployeeDesignation { get; set; }
        public double EmployeePay
        {
            get => CalculateEmployeePay();
            private set { }
        }

        private double CalculateEmployeePay()
        {
            switch (this.EmployeeDesignation)
            {
                case Designation.JuniorDeveloper: return TimeHours * 100;
                case Designation.MidLevelDeveloper: return TimeHours * 200;
                case Designation.SeniorDeveloper: return TimeHours * 300;
                default: return 0;
            }

        }
    }

    public enum Designation
    {
        JuniorDeveloper,
        MidLevelDeveloper,
        SeniorDeveloper
    }

Note that we have made the "setter" of property EmployeePay private thus making it read-only. We also introduced a logic in here by calling the method CalculateEmployeePay in the "getter" to return the employee's salary.

So far what we have discussed are "instance properties". They are specific to the object instance and are not shared between instances. If you recall, we said the class is a blueprint to create objects.These are also known as instances.

            Employee emp1 = new Employee();
            emp1.EmployeeName = "Saania";
            emp1.EmployeeDesignation = Designation.MidLevelDeveloper;
            emp1.TimeHours = 10;

            Employee emp2 = new Employee();
            emp2.EmployeeName = "Hormaz";
            emp2.EmployeeDesignation = Designation.JuniorDeveloper;
            emp2.TimeHours = 2;

In the above snippet emp1 and emp2 are two separate instances of the Employee class with each having their own set of class members (properties+methods). If I make change to the EmployeeName property of the instance emp1 then that is local to emp1 and would not reflect in emp2. But what if there is a requirement that the Employee class holds a count for the total employees. Now this data would be global to an organization, we cannot have local copies for it. Say emp1 is our first employee and after instantiating it we do something like

emp1.TotalEmployeeCount += 1

then this change will be only confined to emp1 .When we create emp2 a fresh instance would be created setting all its data-members to their defaults (effectively if we had a property TotalEmployeeCount then for emp2 its value would be 0 at the beginning) i.e. the effect of emp1 is totally lost. In such scenarios we opt for static members
Static properties are common between instances. Any change made to them is global. Also one thing to note is , the static properties cannot be accessed through the instances unlike the rest where we use the "." (dot) operator e.g

emp2.EmployeeName = "Hormaz";

. They are accessed through the class name. Let us create a static property which will help us maintain a count for all the employees.

public static int TotalEmployees { get; set; }

. If we have to access them then we will do that as

Employee.TotalEmployees += 1;

It is also important to note that static data is not thread safe. Say this Employee class is a part of a larger distributed systems network. There are systems which modify the TotalEmployees count.It may happen so that two different systems try to modify the data at the same time. An obvious question would come to our mind that it is an atomic operation, we are just incrementing its count by one. We will go back to the autoimplemented properties concept. We saw that it is just a syntactic sugar for the manual property with a backing field. The compiler does in fact creates a backing field for these properties. If we were to modify a property in essence it would be

read the value through "getter"
increment the value by 1
set the value through "setter"

It no longer is an atomic operation. There is a vulnerability that another thread (another process) gets access to its value before the previous one could increment and set the updated value. This leads to garbage data.

Format Enums as a Custom Name-Value Object List

Vidisha Parab — Sun, 26 Jul 2020 16:30:41 +0000

I recently came across a scenario where I had to expose an enum property as a list of drop-down items to choose from. Typically when a user would select something from the drop-down and the form posted back to the server, the selection would be bound as a legit enum value.However the enum which we had defined was very system specific and not user savy.I could not just let the user select a particular enum value without giving out a description of the purpose it served.

There is little we can do when defining enums as they have to follow certain conventions and sometimes we are required to give them system specific values which might not be very descriptive to the end user. E.g. SYSTEM_ERROR_102 which might be some error only known to the developer.We cannot show this directly to the user and have to give some description about it.

In the example ahead, let us assume we have a simple form with two inputs - a textbox and a drop-down. The user has to enter a product name and assign it with a product type (select from the drop-down). At the back-end we are maintaining a Product enum as

 public enum Product
 {
  Board,
  Table,
  Desk,
  Carpet
 }

For demo purpose let us assume the form is in Swedish and we need to show the product types in Swedish but at the same time when user selects a particular product type and the form is posted back it must be bound to one of the Product enum values . (Note the following is just one way to achieve it, there can be different ways to do it. Also your use case could be different, I have assumed this example for simplicity but the technique remains the same)

We start by defining our Product Enum as

public enum Product
 {
 [Description("Bord")]
  Board,
 [Description("Stol")]
  Table,
 [Description("SkrivBord")]
  Desk,
 [Description("Matta")]
  Carpet
 }

Here we have made use of the DescriptionAttribute from the System.ComponentModel namespace.You can also go for your own custom attribute but since this is readily available we are good to go here. You can read about the Description Attribute here.
This Description Attribute will have whatever friendly name you want your Enum value to be associated with.

My back-end was an API and I wanted to return a list of anonymous object of type Name and Value. The name would be what shown as a selection in the list and the value would be bound to the selection. In our example here, the Name would be the Swedish counterpart and the value would be the actual enum value (Table,Desk,etc).

 private dynamic FormatEnum(Enum value)
 {

 /*Since any valid (derived) Enum type could be passed to this function
 we need the exact Enum type passed to this func
 which we would require to process further.Here it would be "Product".
 */
 //type=Product;
 Type type = value.GetType();
  /*
Enums hold an integer value with the index starting from 0.
If we were to assign Product.Table and try to serialize it 
or save it in a db it would get stored as 1 
(Board=0, Table=1,Desk=2,Carpet=3).
Since we are working with dropdowns and want an user friendly interface, we are interested in the Name here.
Hence Enum.GetName(typeof(Product),Product.Table).
You could also do it with its index => Enum.GetName(typeof(Product),1)
 */
 //name = Table
 var name = Enum.GetName(type,value);
  if(name !=null)
 {//Now we are interested in the Description Attribute.
 /*The FieldInfo class belongs to the System.Reflection namespace and as
defined in MSDN, "Discovers the attributes of a field and provides access to field metadata."*/
    FieldInfo info = type.GetField(name);
    if(info !=null)
    {
    //Out of all the properties listed by the FieldInfo object we are interested in the  CustomAttributes which would give us access to the DescriptionAttribute
     DescriptionAttribute descrp = (DescriptionAttribute) Attribute.GetCustomAttribute(info,typeof(DescriptionAttribute));
     Console.WriteLine(descrp);
    if(descrp !=null)
    {
       return new { Name =  descrp.Description , Value = name };
    }
    }
 }
 return null;
 }

The FieldInfo object would look like this and as shown above we are interested in the CustomAttributes property

When we retrieve the DescriptionAttribute on an enum value we are interested in its "Description" property as that is what we set to begin with.

And then you could use this method like

void Main()
{
    var products = Enum.GetValues(typeof(Product)).Cast<Enum>()
        .Select(e=> FormatEnum(e)).ToList();
    Console.WriteLine(products);

}

Output :

Abstract or Virtual ?

Vidisha Parab — Thu, 23 Apr 2020 15:58:25 +0000

We have come across abstract and virtual methods while programming and I often get startled when I see these two keywords. Whenever I try to look up for them a lot of "tabular comparisons" are being thrown at me, distinguishing between the two without highlighting why they are so ?

I try to put together my understanding below.

Abstract and virtual methods are a part of a concept called as polymorphism from object oriented programming. Polymorphism is further derived from the idea of Inheritance.
Why do we need Inheritance ? Inheritance is supposed to be the best feature of OOP. We can extend and reuse the capabilities of a class further as per the business needs i.e. writing less code ! How does polymorphism fits into this ? Say we have a class which outlines how a pizza should be . It also has a method called Prepare, data properties for the pizza name and price. In its most basic form, this method would output a plain pizza.

 public class Pizza
    {
        public string Name { get; set; }
        public int Price { get; set; }
        public void Prepare()
        {
            Console.WriteLine("Here is a pizza dough base with some tomato sauce spread on it !");
        }
    }

Someone walks up to us and tells us that now we also have to prepare Hawaiian and Pepperoni pizza. One option would be to make separate implementations for them.But we already have a Pizza class, which has a method Prepare that outputs pizza in the basic form and also it has data properties which will help us to store the name and price, why not inherit from it (remember -lesser code)? But won't the method Prepare give us a basic pizza ? Override it !

We can override the method Prepare to have two different behaviors - one which would give us the Hawaiian pizza and the other which would give us the Pepperoni pizza. This right here is polymorphism - The Prepare method behaving differently in different contexts.

 public class HawaiianPizza : Pizza
    {
        /*the field like name and price need not be written here as we inherit 
          them impiclitly from the parent Pizza class*/

       //overridden implementation of Prepare method to give us a hawaiian pizza
    }

 public class PepperoniPizza : Pizza
    {
       /*the field like name and price need not be written here as we inherit 
          them impiclitly from the parent Pizza class*/

       //overridden implementation of Prepare method to give us a Pepperoni  pizza
    }

Now that we established what polymorphism and inheritance is, it is equally important to know how to achieve those. Inheritance is when a class derives from another class or an interface.The class which other classes derive from is called as a base class. Now this base class can be either an abstract class or a concrete class. Whenever a class is incomplete with at least one member not implemented then it is called as an abstract class. For e.g. the class Pizza below

 public abstract class Pizza
    {
        public string Name { get; set; }
        public int Price { get; set; }
        public abstract void Prepare();
    }

Here you can see that the method Prepare has no body or implementation (don't fret over the keyword abstract in that method, we will come to it soon !) . Given that an abstract class is incomplete , there is no point in creating any instance of it . What if an instance is created and someone calls the Prepare method ?
Bam ! It would be a disaster since we lack its implementation. And there we have it - Abstract classes can never be instantiated (the very first point of distinction the search results provide us). Also some search result tell us that "abstract classes" should be defined by the abstract keyword. well thank you !

Secondly, a base class could also be a concrete class. A concrete class is complete with all its implementations in place- just like a normal class and hence it can be instantiated !

Coming to our abstract and virtual methods.

By default any method be it of an abstract class or a concrete class cannot be overridden even if the class is inherited by some other class. So if we want a child class to have its own implementation of a particular base class method then we do so by marking the base class method as abstract or virtual -meaning allowing it to be overridden.

Depending on the pizza which we order,say the price would vary , there might be certain discounts on a particular type of pizza, or some logic involved in calculating the price of the pizza. In the base class - Pizza we don't want to handle that , we just want our Pizza class to give us out a pizza base with a tomato sauce layered on it. In this case we will declare a method CalculatePrice as abstract

public abstract int CalculatePrice();

.

Whenever we want the child classes to deal with implementation of a particular method we mark the method as an abstract.

When doing so, we will only define the signature of the method in the parent class

Now the implications of this -

Like we established that the abstract classes are incomplete , abstract methods are also incomplete ! Abstract methods do not have any default method implementation.
Given this any class inheriting from Pizza must override this method ! Obviously it wouldn't make sense to have a method which is just a signature, without any job to do ! (Think about interfaces , how it is required for us to implement them) It is mandatory to override abstract methods !
Also since it is an incomplete method(without any implementation) it is pointless to have it in a concrete class (because concrete classes are complete and we cannot introduce this kind of ambiguity in them !). Hence the reason Abstract methods can only be used in an abstract class.

What about virtual methods ? No matter what, we want to have a basic pizza - a pizza dough base and tomato sauce on it (a l'il bit of cheese doesn't hurt as well!). Here I am sure that under any case I need to have that and not depend on the child classes for the implementation. I will go ahead and mark the method as virtual

Whenever we want to have a default behavior or implementation of a method in a parent class we mark this method as virtual

When we mark it as a virtual method ,in addition to giving a default implementation we also give liberty to the child classes to override it and have their own implementation defined if the default one doesn't meet their needs.

 public abstract class Pizza
    {
        public string Name { get; set; }
        public int Price { get; set; }
        public abstract int CalculatePrice();
        public virtual void Prepare()
        {
            Console.WriteLine("Here is a pizza dough base with some tomato sauce spread on it !");
        }
    }

Implications

Though a default implementation is given in the parent class, the child classes can still have their implementations thus overriding the one given by the parent class.
Since a virtual method is complete with a default implementation it is not mandatory for the child classes to override it (have their own implementation). Virtual methods don't necessarily have to be overridden . If you don't override the Prepare method in the Pepperoni class you are not going to get Pepperoni,You get a basic Pizza !
Also since they have a fully functional implementation , they can be a part of both abstract and concrete class !

Here we have inherited class PepperoniPizza from the class Pizza .You also see the inherited members of the Pizza class

Hope you find this useful.Any suggestions or fixes are always welcome ! :)

Break the LINQ - Dissecting the "Where" of LINQ

Vidisha Parab — Fri, 27 Mar 2020 19:25:41 +0000

I have been a .net developer for quite some time now and there are things which I have just blindly implemented by referring to code snippets without caring about their underlying intricacies, one of them being LINQ. I’m sure we all know what it stands for and why it is used so I won’t be going there. What I really want to discuss here is the LINQ fluent functions which we use on a family of IEnumerables.
I always felt lazy to write these queries on my own and would always turn to online examples. I never bothered to look at the overloads and understand them. No more.Here is my very first attempt at writing a post.
To write a LINQ query efficiently there are three things you must be aware of

Delegates
Lambda Expressions
variety of extension methods offered on the Enumerable class

I would like to limit this post to dissect the most commonly used "Where" extension method. The documentation for it as below

The first parameter preceded by "this" keyword tells us right there itself that it is an extension method. Also notice that it returns an IEnumerable<TSource>. So Where is an extension method on a collection which is of type IEnumerable<TSource> which also returns an IEnumerable<TSource>. The collection referred here could be any collection which implements the IEnumerable interface e.g. List, IList.

We move to the second parameter of the method Func<TSource,bool> predicate. Func is a built-in delegate in .NET.

Delegate is a type to represent a method signature. E.g. if we have a method

public string FormatNumber(int number)

then we would write its “delegate” type as

public delegate string FormatNumberDelegate(int number)

Here the FormatNumberDelegate is a type that would represent the method FormatNumber or any method which accepts an integer and returns a string. If we were to represent this as Func then the resultant delegate would be
Func<int,string> i.e. "Func" is any method which takes in an integer parameter and returns a string.

Going back to our Where extension, Func<TSource,bool> represents any method which takes in a parameter of TSource and returns a boolean. The boolean returned is essentially the condition which we would like to evaluate. If from a list of Employees, we want to filter the female employees, then we would write a condition which evaluates to true (a boolean) when the employee is a female.

Lets say we have a class Employee as follows

public class Employee
    {
        public string FirstName { get; set; }
        public string LastName { get; set; }
        public string Designation { get; set; }
        public DateTime StartDate { get; set; }
        public DateTime EndDate { get; set; }
        public Gender Gender { get; set; }
    }

 public enum Gender
    {
        F,
        M
    }

so our condition would be
if(employee.Gender == Gender.F) return true;

The simplest way to write this is

bool FilterFemaleEmployees(Employee employee)
        {
            if (employee.Gender == Gender.F)
                return true;
            return false;
        }

and then we can use it with our Where extension as var femaleEmployees = employees.Where(FilterFemaleEmployees);

Comparing this with the documentation for the Where extension,
TSource = Employee
IEnumerable<TSource> = var femaleEmployees is IEnumerable<Employee> which is returned by the Where method
Func<TSource,bool> = bool FilterFemaleEmployees(Employee employee)

Lets take it to one level up by making use of anonymous delegate.

The FilterFemaleEmployees is a named delegate which is good but while using LINQ the main focus is on compact queries , writing efficient code with as few lines as possible.
With an anonymous delegate, the query would look like

var femaleEmployees = employees.Where(delegate (Employee employee)
            {
                if (employee.Gender == Gender.F)
                    return true;
                return false;
            });

whenever we are evaluating a condition, its result is going to return a boolean. i.e if we just return employee.Gender == Gender.F , it is going to return a true or a false based on the value of employee.Gender. So reducing the code further,

var femaleEmployees = employees.Where(delegate (Employee employee)
            {
                return employee.Gender == Gender.F;
            });

Now comes the lambda ! Lambdas are a great way to represent an anonymous function. There are three parts to a lambda expression :
the parameters, the => operator and the method body (statement or expressions).
We remove the delegate keyword and replace it with the following

 var femaleEmployees = employees.Where( (Employee employee)=>
            {
                return employee.Gender == Gender.F;
            });

Another good feature of a lambda is the parameter inference. In the above, we are supplying a parameter of type Employee to the anonymous function. But here we can make good use of the parameter inference and let the compiler figure out its type (another step towards compact code)

 var femaleEmployees = employees.Where( (employee)=>
            {
                return employee.Gender == Gender.F;
            });

If you hover over employee in visual studio, you would see the compiler has rightly interpreted its type

Another cool feature of lambda expressions is if we have a single parameter we don't need to enclose it in brackets, we can get rid of those

 var femaleEmployees = employees.Where( employee=>
            {
                return employee.Gender == Gender.F;
            });

More to compact coding !!

If we are having just a single statement and returning a single value we can well get rid of the curlys and the return statement too !
var femaleEmployees = employees.Where( employee=> employee.Gender == Gender.F);
And since we are inclined to have as much less written code as possible, a convention followed while writing lambda expressions is to name the parameters with single letters further reducing our code to
var femaleEmployees = employees.Where( e=> e.Gender == Gender.F);

And that's it ! That's how we reach the where of "Where" came from ! If you understand the Func delegate and the lambdas ,it is going to be easy to decipher and use those LINQ extension methods. You could evaluate even more complex conditions through the Where method.

A similar understanding can be extended to write the following extension methods as well

Any
public static bool Any<TSource> (this System.Collections.Generic.IEnumerable<TSource> source, Func<TSource,bool> predicate);

FirstOrDefault
public static TSource FirstOrDefault<TSource> (this System.Collections.Generic.IEnumerable<TSource> source, Func<TSource,bool> predicate);

SingleOrDefault
public static TSource SingleOrDefault<TSource> (this System.Collections.Generic.IEnumerable<TSource> source, Func<TSource,bool> predicate);

I would have loved to discuss about Join and the GroupBy but I will save it to a later post.