DEV Community: Corey Lasley

Client Side Load Balancing External REST Calls with Java and Spring Boot

Corey Lasley — Fri, 14 May 2021 00:51:42 +0000

Targets

Java 8+, Spring Boot 5, Any Client Side Framework such as Angular/React/Vue/etc.

Introduction

As uncommon of a need as it may be, there may come a time where you are required to write an application that makes various calls to an external REST API / microservice that exists in multiple locations for the purpose of high availability. Because there are multiple instances of the service running, it just doesn't seem right to configure your application with a proxy that only hits one of the multiple endpoints, thus making calls to the service in a load balanced manner might be your ultimate goal.

There are likely better ways of load balancing service calls in the enterprise, such as through load balancing hardware, or an implementation of a GSLB (Global Server Load Balancer) product such as NGINX. However with that being said, the need may arise where your application is required to perform this task without the extra expenses and/or additional potential points of failure.

If you are not familiar with setting up a Spring Boot application, and/or want more background into where I found my inspiration, I strongly encourage you to go through the Client-Side Load-Balancing with Spring Cloud LoadBalancer guide, before continuing on.

When a Proxy Isn't Enough

As a secondary need, when it comes to consuming an external API from a browser based application, is the Cross-Origin Resource Sharing (CORS) issue which occurs when a browser detects that a call is being made to an API that exists on a different domain. This can be solved by including Access-Control-Allow-Origin in the response header of the service; but, for the sake of this exercise, we are going to suppose that we either cannot, or do not want, to make this change within the service code.

With Spring Boot, one can use the Netflix Zuul Proxy to get around this issue, which is extraordinarily easy to configure. Unfortunately, you are going to be limited to consuming a single configured endpoint. If you do some more research, you will undoubtedly come across the Netflix Ribbon, which has much more promise in getting us to where we need to be, but is substantially more complex to setup than a Zuul Proxy, and even more disappointing, you will come to find out that Ribbon has officially been deprecated.

Spring Cloud LoadBalancer

Enter Spring Cloud LoadBalancer, which picks up where the Netflix Ribbon left off, and in fact it actually uses Ribbon under the hood. The official Spring site has some good information on how one can use LoadBalancer in the aforementioned guide: Client-Side Load-Balancing with Spring Cloud LoadBalancer, from which I base our solution.

Spring's guide comes pretty close, but doesn't really address our exact need. That is, what we ultimately need to be able to do: Intercept all API calls being made from our client application running in Spring Boot (gets, posts, puts, and deletes), and redirect them under the hood, in a load balanced manner, to several different external endpoints, without having to define each specific API call.

Our Scenario

We have a client application (lets say Angular) with a service that communicates with a RESTful service that exists at multiple endpoints for high-availability. We do not want our application to simply be configured to hit just one of these endpoints, so we are going to use Spring Boot to take advantage of the Spring Cloud LoadBalancer. Lets say that we have 3 external endpoints which host the API(s) that we want to be able to consume in our application:

https://myservice.host1.com, https://myservice.host2.com, https://myservice.host3.com

We want our client application to call the APIs via the "/api" path, as if these endpoints existed within our app. Therefore when calls are made, such as "/api/getthis" or "/api/savethat" we will have our Java Spring Boot application, that lives in the same space as our client application, intercept and redirect those calls to one of the 3 external endpoints. The following demonstrates one way that you can set this up.

Setting up the Java Spring Boot Load Balancer

After having setup your Spring Boot application (which is easy to do here), go to your main application class, which for my example, I have named LBExampleApplication, and make sure it contains the following:

package com.lb.example;

import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.loadbalancer.reactive.ReactorLoadBalancerExchangeFilterFunction;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.CoreSubscriber;
import reactor.core.publisher.Mono;
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.util.Enumeration;
import java.util.stream.Collectors;

@Slf4j
@SpringBootApplication
@RestController
public class LBExampleApplication {
   private final WebClient.Builder loadBalancedWebClientBuilder;
   private final ReactorLoadBalancerExchangeFilterFunction lbFunction;

   public LBExampleApplication(WebClient.Builder webClientBuilder,
                           ReactorLoadBalancerExchangeFilterFunction lbFunction) {
      this.loadBalancedWebClientBuilder = webClientBuilder;
      this.lbFunction = lbFunction;
   }

   public static void main(String[] args) {
      SpringApplication.run(LBExampleApplication.class, args);
   }

   @RequestMapping(value = "/api/*")
   public Mono<String> balanceIt(HttpServletRequest request) throws IOException {
      Boolean logDebugInfo = true;
      String uri = request.getRequestURI();
      String type = request.getMethod();
      Enumeration<String> headerNames = request.getHeaderNames();
      String url = "http://load-balancer" + uri;
      String body =  request.getReader().lines().collect(Collectors.joining(System.lineSeparator()));

      if (logDebugInfo) {
         if (headerNames != null) {
            while (headerNames.hasMoreElements()) {
               log.info(" --- Request Header: " + request.getHeader(headerNames.nextElement()));
            }
         }
         log.info(" --- Request Body: " + body);
         log.info(" --- " + type + " Reroute to Load Balancer: " + url);
      }

      switch (type) {
         case "GET":
            return loadBalancedWebClientBuilder.build().get().uri(url)
                  .retrieve().bodyToMono(String.class);
         case "POST":
                return loadBalancedWebClientBuilder.build().post().uri(url)
                  .header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
                  .bodyValue(body)
                  .retrieve().bodyToMono(String.class);
         case "PUT":
            return loadBalancedWebClientBuilder.build().put().uri(url)
                  .header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
                  .bodyValue(body)
                  .retrieve().bodyToMono(String.class);
         case "DELETE":
            return loadBalancedWebClientBuilder.build().delete().uri(url)
                  .retrieve().bodyToMono(String.class);
      }

      return new Mono<String>() {
         @Override
         public void subscribe(CoreSubscriber<? super String> actual) {
         }
      };
   }
}

NOTE: Logging has been added so that you can see what is going on in the debug console when running locally.

Next, create a class called LBExampleConfiguration

package com.lb.example;

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.client.DefaultServiceInstance;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.loadbalancer.core.ServiceInstanceListSupplier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import reactor.core.publisher.Flux;
import javax.annotation.PostConstruct;
import java.util.ArrayList;
import java.util.List;

@Configuration
public class LBExampleConfiguration {
    @Bean
    @Primary
    ServiceInstanceListSupplier serviceInstanceListSupplier() {
        return new IMServiceInstanceListSuppler("load-balancer");
    }
}

@Slf4j
class IMServiceInstanceListSuppler implements ServiceInstanceListSupplier {
    private final String serviceId;

    @Value("#{'${api.endpoints}'.split(',')}")
    private List<String> endpoints;

    private List<ServiceInstance> instances;

    IMServiceInstanceListSuppler(String serviceId) {
        this.serviceId = serviceId;
    }

    @PostConstruct
    void buildServiceInstance(){
        instances = new ArrayList<>();
        Integer sid = 0;
        for (String host : endpoints) {
            sid++;
            instances.add(new DefaultServiceInstance(serviceId + sid, serviceId, host, 443, true));
            log.info(" --- Added Host: " + host);
        }
    }

    @Override
    public String getServiceId() {
        return serviceId;
    }

    @Override
    public Flux<List<ServiceInstance>> get() {
        return Flux.just(instances);
    }
}

Next, create a class called WebClientConfig

package com.lb.example;

import org.springframework.cloud.client.loadbalancer.LoadBalanced;
import org.springframework.cloud.loadbalancer.annotation.LoadBalancerClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.reactive.function.client.WebClient;

@Configuration
@LoadBalancerClient(name = "api-gateway", configuration = LBExampleConfiguration.class)
public class WebClientConfig {
    @LoadBalanced
    @Bean
    WebClient.Builder webClientBuilder() {
        return WebClient.builder();
    }
}

Finally, within your application.yml

api:
  endpoints: "myservice.host1.com,myservice.host2.com,myservice.host3.com"

This is a comma delimited list where you define all of the endpoints in which you want your API calls to be load balanced. Don't include the http or https here, because functionality within the LBExampleConfiguration class takes care of this based on the port number specified in the call to new DefaultServiceInstance(...)

Setting up the Client Side

Well, there actually isn't much to do here. From your client application, whether it is Angular, React, Vue, or anything else. Just call your API(s) as if they exist on your backend. Your Java Spring Boot Load Balancer is essentially a simple backend application that will handle routes directed to /api/ and direct them to their actual external endpoints.

An Organic .NET Code Profiler/Logger Through IL Weaving

Corey Lasley — Tue, 21 Apr 2020 01:09:30 +0000

This article is sort of a Part II to my Poor Man’s Profiler article, in which I have taken profiling code to the next step. Initially, this seemed like it would have been a relatively easy task, but I ended up spending many more evenings than I expected tweaking things in order to find success. Often the thought went through my mind: “Why am I doing this, and is anyone actually going to find this useful?” Well I am not one to back down from a coding challenge, and even if I am the only one who gets anything out of this exercise, I wanted to finish what I started. Maybe this will be helpful to someone else out there.

Introduction to this Project

Despite being a developer for two decades, up until more recently, I had simply only known about CIL (Common Intermediate Language) often simply called “IL” and how the .NET compiler works, but never really gave it much thought beyond that, simply because (perhaps surprisingly,) I never really had a good reason to actually dig in and understand it. When I put together my proof of concept code and article on my Poor Man’s Profiler, I started doing some digging into more optimal approaches to profiling code and logging method consumption (i.e. ways to further limit code redundancy and eliminate unnecessary overhead.)

Of course I could go the easy route and use an existing product like PostSharp, but this exercise wasn't simply about finding an existing solution, rather, it was meant to be a learning experience to try to figure out how something like this could be accomplished. In all honesty, being an engineer, I really love a good challenge, and don’t like to just leave all the from-scratch fun programming to others, even if in this case, I may be reinventing the wheel a bit.

So this was when I ventured down a path I hadn't traveled before, discovering metaprogramming and IL weaving. It didn't take long after going down this path, before it clicked that I could disassemble .NET assemblies (via ildasm,) modify the IL code, and reassemble them in a modified state (via ilasm,) it was like I was really seeing for the first time, and the gears in my head really started cranking with ideas.

Granted, I cannot think of a whole lot of uses for directly modifying/writing IL code, that is, unless you are a glutton for punishment. It isn’t the easiest code to understand, as it looks very Assembler’ish. Because it is stack based (push / pop) it gets incredibly lengthy rather quickly, and if you are not careful, you can really dork things up.

I can at least think of a few reasons for needing to directly modify IL, albeit uncommon, such as writing an obfuscator, or create one’s own .NET programming language for academic purposes. The big reason for modifying/writing IL was the no-brainer reason why I was down this road to begin with: code profiling!

Two major aspects of my Poor Man’s Profiler solution didn’t sit well with me from the get-go, but I continued the project and article regardless to demonstrate the evolution of a concept from good to better. What was obviously not so great with my Poor Man’s Profiler:

Having to add additional code (albeit simple) all over an application where one wants method data logged
Having to rely heavily on boxing when logging parameter/variable values.

Through modifying IL code, an entirely different approach can be taken:

In very direct use, a static solution could be written so that no additional code would need to be added to application code, not even a new class library (But in this solution, we are going to take it beyond this.)
With a little additional code added to each class in an assembly, the weaving solution can become very customizable, by allowing the developer to handle logging events.
No more explicit boxing of parameter/variables.
Profiler logging calls are weaved into the methods of IL code post-build and could be used as part of a CICD process.

A Little Background on the Common Intermediate Language

Given the fact that IL code is the “low level” of what all .NET languages (i.e. C#/VB.NET/F#/etc.) compile down to, it’s a good idea to get a little familiar with it. You can take a deep dive into CIL by reading through the ECMA-335 Standards. Wikipedia has a good introduction to the Common Intermediate Language. An excellent book that I think belongs in every serious C# developers library is CLR via C# by Jeffrey Richter which is a great resource, and for a real deep dive, the book .NET IL Assembler by Serge Lidin (a book I do not yet have.)

With a couple resource available, I found that the best way for me to wrap my head around IL, was to simply write some C# code, such as in a console app, build it, and then use ildasm to disassemble it to readable IL. At this point I could look at my .NET code and compare it with its corresponding IL code, and identify patterns of translation. This gave me a solid idea of what I needed to do to generate my own IL code blocks that I could then inject into the code, and finally reassemble via ilasm.

Using a data/text comparison tool, such as WinMerge makes it much easier to see the differences between original code, and what code should look like after injecting new code. This takes quite a bit of guesswork out of things.

Writing my Own IL Parser, Code Generator, and Injector/Weaver Solution

The purpose of my exercise for this project and corresponding article was:

Log execution time of methods.
Log parameter names and values passed into methods, including all primitive types and types that implement IEnumerable.
Limited code needed to be added to a consuming solution, only simple attributes at Class level and/or Method level, depending on how fine tuned one wants things.
Only one method with a specific signature and an Attribute needs to be written in the consuming solution, giving the developer full control over the actual logging process. Method execution time, and parameter data is passed to this method.
Application assemblies are modified post-build to inject all profiling code into applicable methods.
Consumption of this solution should be extremely simple and straight forward.

Having gone through a 101 on IL; On a technical level, the final product will do the following:

Disassemble a DLL or EXE in IL code with ildasm.exe
Parse the IL code into an object structure making IL code analysis and injection possible.
Discover the location of the Logging Handler method (based on a Method level attribute)
Based on specific Class/Method level attributes, determine which methods need IL modification.
If any methods that will be profiled/logged are found to have a parameter which implements IEnumerable, generate a method to convert the value of the enumerable to a comma delimited list string.
Generate IL code to assign parameter names and values (i.e. “[parameter]=[value]; ”) to a single string. For IEnumerable types, call our generated conversion method, and inject at the beginning of the method.
Generate IL code to initiate and start a StopWatch object, and inject into the beginning of the method.
Modify the existing “maxstack” and “.locals init” of the method to accommodate for the StopWatch and Parameter list string objects.
Adjust the indexes on the existing “sdloc”, “ldloc”, and “ldloca” instructions of the method to accommodate the injected code above.
Generate IL code to stop the StopWatch, and inject at the end of the method.
Generate IL code to make a call to the discovered Logging Handler method, passing it the calling method name, the string holding our parameter names and values, and the StopWatch’s elapsed milliseconds, and inject at the end of the method.
Save the modified IL code to disk.
Assemble the modified IL code by calling ilasm.exe (including its resource file that was generated when disassembled if one exists).
Overwrite the original assembly.
Cleanup all temporary working files on disk.

After a couple of weeks, working for an hour or two in the evenings, I was able to successfully pull this off.

The CIL Weaver Solution

I am not going to go into the internals of the my code here because there is a lot to dissect. Rather, I will go into how what I have written can be used to weave profiling code into your assemblies. My CIL Weaver Profiler solution on GitHub contains the following projects:

Core Projects

CILWeaveProfiler (.NET Standard Class Library): This contains all the real beef, containing the code to automate disassembly/assembly, and IL code parsing/writing. There is quite a bit to take in here. Not only does this library contain the logic for modifying assemblies, but it is also the library that you will need to reference in your application that you want to be modified for profiling (more on this later)

CILWeaver.Console (.NET Core 3.1 Console): This application consumes the CILWeaveProfiler class library and performs the calls needed to disassemble an assembly (exe or dll), parse the IL, make modifications to it based on the contents of the existing IL, save the changes, and reassemble the assembly. This could be used manually, or as part of a CICD process. NOTE: If you want to use this, you may need to make some minor modifications.

Development Aids

ReferenceApp (.NET Core 3.1 Console): This is a sandbox application that I use to write test .NET (C#) code so that I could disassemble it, inspect the IL, and figure out what I needed to do within CILWeaveProfiler to accomplish the end goal.

ILStringify.Blazor (.NET Core 3.1 Blazor SPA): A quick tool that I whipped up to help me turn large blocks of IL code into C# string assignment code that I could use while developing CILWeaveProfiler. I figured that I might as well include this. Being so comfortable in the .NET world, I absolutely love how simple Blazor is, and how quickly I can slap a simple SPA together to build quick tools in .NET Core (a topic for another day.)

How the Solution is Applied to Your .NET Project

Prepping Your Code

Through the use of custom Attributes, which is essentially metadata applied to target elements in our .NET code (i.e. Classes and Methods,) we can customize our logging process. Attributes, as most .NET developers are familiar with, are generally used within the application or sometimes the compiler, however we will use these attributes only within the CILWeaver program (well actually the CILWeaveProfile library,) as they are, more or less, simply IL modification directives, used to determine which methods are going to be profiled and how (what sort of IL modifications need to be made.) Additionally, we point the IL modification process to the existing method that will consume the profiled data, so that the developer has the final say into to how the data is actually logged, while the cumbersome task of gathering the profiled data is taken care of and passed to this method.

First, you will want to reference the CILWeaveProfiler Class Library in your project.

In order to tell the CIL Weaver how to modify the application you will use Attributes which require the following reference:

    using CILWeaveProfiler.Attributes;

The two Attributes that can be applied are ProfilerClass and ProfilerMethod:

ProfilerClass: This is a class level attribute that can be used to apply how the CILWeaver program will treat all of it’s methods by default. If a method level attribute (see below) is applied to a class’s method, the method level attribute will override this attribute for the particular method.

    [ProfilerClass(LoggingType = CILWeaveProfiler.LoggingTypes.All)]    
    class  Program    
    {    
    …    
    }

For LoggingTypes you can specify:

All: Meaning, by default, unless a Method attribute states otherwise, profile the Execution Time and Parameters passed to the Method.
ExecutionOnly: Meaning, by default, unless a Method attribute states otherwise, only profile the Execution Time of the Method.
ParameterValuesOnly: Meaning, by default, unless a Method attribute states otherwise, only profile the Parameters passed to the Method.
None: Meaning, by default, unless a Method Attribute states otherwise, do not profile the Class’ methods.

If the Class is not marked with the Attribute, it is the same as if it was marked with None. The Class does not need to be marked with the Attribute for Methods to utilize the Attribute.

ProfilerMethod: This is a method level attribute that can be used to tell the CILWeaver program if it should profile the method and how if different from the class level attribute. If this attribute is NOT applied to a method, but the ClassAttribute is applied at class level, the attributes of the class level attribute will be applied to the method.

Little is actually gained by using ExecutionOnly or ParameterValuesOnly, as the modified code will simply pass in an empty string for parameters or a 0 for milliseconds to the logging method. The only gain would be in the small overhead required for instantiating and executing a StopWatch or converting parameter values to strings.

NOTE: Inspecting the code, you may notice that the ClassAttribute and MethodAttribute are identical in their properties, so why didn’t I just use one Attribute for both? This is
because I may want to eventually do more with these attributes that
are applicable more to one or the other, but not necessarily both.

Additionally, and importantly, through the use of another simple attribute, we mark the particular method in our code in which the weaver will inject calls to, so that we do not have to rely on a baked-in logging functionality, but rather, how profiled data is actually logged, is in full control of the developer.

LoggingMethodOverride: This can be applied to a method within the class, or inherited class, and the method can have any name. The method does however require a specific parameter signature. The parameters must be (string methodName, string parameters, long milliseconds) This method will never be profiled itself, or else endless recursion would occur until you reach stack overflow. Note this is not a true override, as consuming classes do not inherit from a special base class, or implement an interface in which a specific logging method needs to be overridden. However, if you do not mark a method with the required signature with this Attribute, absolutely nothing will be logged.

Example

    [LoggingMethodOverride]    
    static void LogIt(string methodName, string parameters, long milliseconds)    
    {    
    Console.WriteLine("LOGGING -> " + methodName + "(" + parameters + ") executed in: " + milliseconds);    
    }

As stated, we can define our logging method with the LoggerAttribute in a base class from which our other classes inherit, so that we have one definition per assembly.

    [ProfilerClass(LoggingType = CILWeaveProfiler.LoggingTypes.All)]    
    class Program    
    {

    [ProfilerMethod(LoggingType = CILWeaveProfiler.LoggingTypes.None)]    
    static void Main(string[] args)    
    {    
    // We do not want to profile this method    
    DoSomething();    
    DoSomethingElse(123, new List<DateTime> { DateTime.Now, DateTime.Now.AddMonths(2), DateTime.Now.AddYears(5) });    
    }

    static void DoSomething()    
    {    
    // Profiling of this method will inherit the Attribute of the Class since no method level Attribute is specified    
    }

    [ProfilerMethod(LoggingType = CILWeaveProfiler.LoggingTypes.ParameterValuesOnly)]    
    static  void DoSomethingElse(int id, List<DateTime> dates)    
    {    
    // For this method, we only want to profile Parameters    
    }

    [LoggingMethodOverride]    
    static void LogIt(string methodName, string parameters, long milliseconds)    
    {    
    // This method is marked with the Attribute of [LoggingMethodOverride], and the CIL will be modified    
    // in the other applicable methods to gather profile data and pass it to this method
        Console.WriteLine("LOGGING -> " + methodName + "(" + parameters + ") executed in: " + milliseconds);    
    }    
    }

Executing the IL Weaver on your Assembly

Included in the solution is the CILWeaver.Console program which can be used as the basis for your IL Weaver which utilizes the CILWeaverProfiler library. However with just a handful of lines of code you can quickly integrate the process into your own weaver app. Simply add a reference to the CLIWeaverProfiler, and get rolling.

    using CILWeaveProfiler;

This gives you access to the Weaver object which contains simple methods for executing the Disassembly, Modification, and Assembly process.

    // Disassemble the assembly...
    DisassembleInfo dIL = w.Disassemble("[PATH & FILE NAME]");
    string IL = dIL.CIL;

    // Parse the disassembled IL Code
    Assembly asm = w.ParseILCode(IL);
    string newIL = asm.GenerateAssemblyILCode();

    // Display the modified IL Code
    Console.WriteLine(newIL);

    // Assemble the modified IL Code            
    bool success = w.Assemble(IL, dIL, ReassemblyTypes.OverwriteOriginal);

    Console.WriteLine(success ? "SUCCESS!" : "Failed to reassemble");

Making Your Own Modifications to the IL Weaver

Consult a list of CIL instruction such as List of CIL instructions on Wikipedia and get familiar with what you are seeing in your IL code, so that you have a better understanding of how you need to make modifications.

Unless you really feel comfortable with IL code, I must emphasize the usefulness of a data/text comparison tool, such as WinMerge to aid in the process of writing the C# code to generate IL code.

For an IL novice such as myself, the best way to determine what sort of IL code was needed to be generated, and where to put it, I found to be in the following process:

Write a program which contains all the code I would expect to see in my final product.
Build it, and disassemble to IL code.
Save this IL code to a file somewhere.
Remove all the code that should not be in the original assembly (the code that will be injected later,)
Build it, and disassemble to IL code.
Save this IL code to a file.
Finally, load my two versions of IL code into a tool such as the aforementioned WinMerge, and compare them to get a good idea of changes that need to be made.

Sometimes after making IL changes, you may get ilasm to successfully assemble the code. However, after running the executable, you may receive a “System.InvalidProgramException: Common Language Runtime detected an invalid program” This means that syntactically the code was good, but there is an issue with it that ilasm clearly did not detect. With this, and if examining the IL doesn't reveal the issue, you should be able to debug the IL in Visual Studio. A really old article (like 18 years old!) on how this might be able to be done is found here I didn't actually attempt this however.

Another suggestion that I came across was to use the PEVerify.exe utility, however the results of this utility led me on a wild goose chase that wasted a ton of my time, and I was never able to get to the bottom of the issues I had. I discovered that using the utility PEVerify.exe, I consistently got the following 3 errors on a reassembled DLL, whether I modified its original disassembled IL code or do not. During my development process, at one point I was using this utility in my process for extra verification, and I ended up spending a lot of time assuming that something was wrong with my modified IL code in the wrong places, until I ran the utility on an assembly where I had simply disassembled a freshly built DLL to IL and then reassembled it back to a DLL without any modifications.

[MD]: Error: EntryPoint method has invalid number of arguments, expecting 3. [token:0x06000004]
[MD]: Error: EntryPoint method has invalid return type. [token:0x06000004]
[MD]: Error: EntryPoint method has invalid argument number 1. [token:0x06000004]

Despite the error messages above, the EXE consuming the DLL and its internals ran perfectly fine.

Some Quick Tips: Issues I ran into developing this code

When injecting IL code into existing methods, the sdloc, ldloc, and ldloc index values need to be adjusted accordingly, depending on code injection placement! In our case, it makes it somewhat straightforward that our code injections happen both at the beginning and ending of existing applicable methods. If you push and pull the wrong indexes at the wrong time, strange things can happen, and likely a hang or throw an unhandeled exception.
Make sure the maxstack values are adjusted appropriately based on the code you are injecting. If the modified application attempts to push more on to the stack than allocated in maxstack, the application will crash. It is a good idea to set the maxstack at a value that truly represents the max size of the items your method will push to the stack (as opposed to just putting some arbitrary high number there.)
This is what gave me the most grief…. In IL, a string doesn't like to be converted to a string. This seems obvious enough, but C# doesn't mind this, and will allow for code calling .ToString() on a string (the compiler makes adjustments for this in the IL). All other object types I have tested can be converted to a string without issue. In my code which concatenates all of the parameters as single string (thus essentially calling .ToString() on all parameters,) I needed to adjust my code generator to make an exception to this rule if the parameter type is already string. Let me elaborate on this...

As an example, lets say we have a method, with it’s first parameter being a string, and the parameter’s name is “str,” the following code block marked “No Good” was a pattern that works just fine on parameters of other types, but throws an error on string type parameters. Adjusting code to follow a different pattern for a string type parameter solves the problem:

/* No Good for a string */

IL_0011: ldc.i4.1


IL_0012: ldarga.s str


IL_0013: call instance string [System.Runtime]System.Object::ToString()


IL_0014: stelem.ref


IL_0015: dup

/* Good for a string */


IL_0011: ldc.i4.1


IL_0012: ldarg.0


IL_0013: stelem.ref


IL_0014: dup

Summary

To wrap things up here, I hope that what I have provided here within this article as well as the code in the associated GitHub solution can help to provide some insight as to how one might go about modifying a .NET executable post-build to inject additional functionality such as profiling.

Poor Man’s Profiler: An Approach to Profiling .NET Code with Code Block Execution Timing and Parameter Logging

Corey Lasley — Sat, 08 Feb 2020 02:07:42 +0000

Applies to: .NET Standard 2.2+

This article is based on my example class library project Poor Man's Profiler on GitHub. To see the internals of this class library, please grab it there.

Basic Profiling

At some point in time most of us have likely used one of the following methods for basic timing of code execution.

DateTime dt = DateTime.Now;
System.Threading.Thread.Sleep(2000);
TimeSpan ts = DateTime.Now - dt;
Debug.WriteLine("Execution took: " + ts.TotalMilliseconds + "ms");

Stopwatch sw = Stopwatch.StartNew();
System.Threading.Thread.Sleep(2000);
sw.Stop();
Debug.WriteLine("Execution took: " + sw.ElapsedMilliseconds + "ms");

This is some really basic code that works well if all you want to know is the execution time of a given block of code. However, if you want to log multiple code blocks (especially within a single method,) and/or you want to log values of various variables, your code can start to get messy quick as it becomes littered with DateTime/TimeSpan/Stopwatch declarations and calls to Debug.WriteLine(). Additionally, if you want to be able to easily toggle this profiling code on or off, so you don’t have to comment it out or remove it all together when you don’t want it to run (like when the application is in production,) you are going to have to write some additional boilerplate code to handle this.

More Advanced Profiling

There are several software vendors who provide some really powerful frameworks that can be utilized for profiling and logging code performance, with the most popular being PostSharp. Such products most certainly have their place among a developer’s arsenal. In no way do I think my project here even remotely compares to the offerings of such products, especially considering my solution here is the work of a few meager hours of brainstorming and development.

For the sake of this article, Let’s pretend that you have an idea of roughly where a problem might be, but don’t feel that it is necessary to purchase a license (of have the budget) for a product that isn’t going to be truly utilized.

Enter what I have dubbed as the Poor Man's Profiler (Poor Man, as in, there are no licensing costs in this open source solution, which lacks the rich features and high scalability of the big commercial products.) This is more or less a very simple idea I recently had, that sort of evolved and continues to evolve as I keep thinking of ways to enhance it.

An AOP (Aspect Oriented Programming) or IL Weaving approach would likely be a better overall approach when writing our own logger that would keep the consuming application more in line with the DRY Principal (i’ll explain this later.) In fact, AOP/ IL Weaving is exactly what the earlier mentioned PostSharp is known for having mastered. Of course this could be the direction we take in Poor Man’s Profiler, utilizing DispatchProxy, but for the sake of going through an evolutionary concept, I wanted the first version Poor Man’s Profiler to be fairly basic, so that we could see some pros and cons.

The End Goal

So what was my goal here? I wanted to be able to:

Time the executions of entire methods, and/or parts of method’s internals.
Have the ability to log parameter/variable values easily.
Identify both code block call frequency, and more importantly, repetitive code executions, whereas sometimes a block of code in itself when called once, isn’t necessarily causing a noticeable bottleneck, but when called multiple times unnecessarily, a performance issue begins to rears its ugly head.
Easily enable/disable this profiling with minimal effort, so that the profiling code could be left intact, but not executed or even in existence while running in a Release build.
Do the aforementioned with minimal code (as much as possible,) so as to not muck up and bloat the readability and flow of things that I wanted to profile.
Take a stab at writing some code a little out of the ordinary.

Some Points of Caution

The first generation of an idea is usually far from perfect, and the fist version of Poor Man’s Profiler is of no exception.

Before we go any further, I must state that if you choose to use Poor Man’s Profiler in an application, it is coded to be a debug only logging option to help a developer gain better insight into things while troubleshooting code in debug mode. There is of course overhead with the use of Poor Man’s Profiler since it obviously adds more to instantiate and process, and depending on how it’s used, utilizes some reflection and causes some boxing. Interestingly enough however, I have come across a few AOP Logger implementations online, that rely on reflection and boxing as well.

I found the performance hit to be surprisingly minor when not used excessively, in fact, relatively unnoticeable in most situations, even in more complex/lengthy logging situations. With that being said, lets discuss some of the cons of this approach.

Boxing

So What is this “Boxing” thing that I speak of? Boxing is a computationally expensive process of converting a value type to the type of object. You may have seen this done in code, or have done it yourself on many occasions and didn't even realize it was a potential issue. The following demonstrates boxing:

int i = 123;
object o = i;

Relying on boxing is a big no-no in applications that you expect to see operate with the best performance. Microsoft’s documentation describes the issue well: “Boxing is the process of converting a value type to the type object or to any interface type implemented by this value type. When the common language runtime (CLR) boxes a value type, it wraps the value inside a System.Object instance and stores it on the managed heap. Unboxing extracts the value type from the object.” If high-performance is one of the major goals of an application (which it should be,) you should limit code that converts to and from object as much as possible.

So while I am breaking my own advice here (that is, avoid boxing,) Poor Man’s Profiler provides a way around allowing such code to execute when in Release mode, which will be discussed later.

The DRY Principal

One of the concepts introduced in the 1999 work, “The Pragmatic Programmer” is the DRY (Don’t Repeat Yourself) Principal, which is pretty much self-explanatory. As far as programming goes, it is always best practice to write code in a way that you are not duplicating code. One good reason for this is that if you have a block of code duplicated all over the place, and then something needs to change in that block of code, you will have a lot of work on your hands making changes everywhere that code block is used.

As for Poor Man’s Profiler, you will find yourself duplicating some code wherever you want it to be used. This will consist of your declaration of the DebuggLogger class, and generally two simple method calls wherever you wish to Start and Stop a code profile. Though it may not be a huge deal if used conservatively, a very thorough use, i.e. if you have the StartDebug()/StopDebug() method calls plastered all over a large application, it could prove to cause a lot of potential future keyboard pounding refactoring, that is, if anything is changed in a future release concerning the way these core methods are called.

Getting Into the Code Behind Poor Man’s Profiler

One instance of DebugLogger class allows for the time monitoring and parameter logging of an unlimited number of code blocks, thus it is best to declare it at class level, where all methods in the class can use it.

The process of converting parameter values to a string

Unlimited parameter values can be logged. This first version supports primitive types and Lists of primitive types (with the addition of DateTime and Timespan) which tend to be the most common parameters I see in applications. The ParamValuesToString() method of the Logger base class, of course could easily be modified to support other types.

Dealing with Lists was a little uglier than I would have liked (see the the ParamValuesToString() method of the Logger base class,) and I have found no way to easily cast a boxed object representing a List<T> to an actual List<T> as the casting-to-type always needs to be known (i.e. “List<T> tl = (List<T>)object;” obviously won’t work, and “List tl = (List)object;” requires a type argument.) Our ultimate goal for logging is to extract the elements of a List<T> as strings, but because our object is a boxed representation a List of any possible type, there is no good way to cast the object to a List<string>, as this cannot be done with an explicit cast. One trick I have seen used to get around this is to serialize the object to JSON, then Parse the JSON and loop through the items as strings.

This will do the trick, but can be more difficult when working with Lists of more complex types. As far as performance goes, Initially I thought the performance of this approach was going to be horrendous, but with a quick test, I was surprised. The serialization process on a List of 7 integers for example, seems to be very comparable in speed, to a direct cast from object to List<int> then to List<string>, that is with the exception of the very first call to JsonConvert.SerializeObject() which lazy loads several dependencies, and takes more than 25x the amount of time it does with ensuing calls.

The following test, uses the Poor Man’s Profiler to compare these two ways to convert a List of int to a List of string.

private static DebugLogger dl = new DebugLogger();

private static void ListToStringListCompare()
{
    List<int> ints = new List<int> { 1, 2, 3, 4, 5, 6, 7 };

    // Box the ints object
    object obj = ints;

    List<string> stringList = new List<string>();

    logger.WriteMessage("Performing JSON Serialize/Parse to List<string>");
    logger.StartDebug("JSON Serialize");
    // using Newtonsoft.Json;
    // using Newtonsoft.Json.Linq;
    JObject j = JObject.Parse("{ items: " + JsonConvert.SerializeObject(obj) + "}");
    stringList = (j["items"]).Select(x => x.ToString()).ToList();
    logger.StopDebug("JSON Serialize");

    logger.WriteMessage("Performing Cast to List<int> -> List<string>");
    logger.StartDebug("Direct Test/Cast");
    Type type = obj.GetType();
    if (type == typeof(List<int>)) stringList = ((List<int>)obj).Select(x => x.ToString()).ToList();
    logger.StopDebug("Direct Test/Cast");
}

Executing this 3x, we can see that after the first execution, both methods perform the same.

******** Message  : Performing JSON Serialize/Parse to List<string>
******** Started  : Program -> JSON Serialize() ...
******** Completed: Program -> JSON Serialize() took [263ms] to execute.
******** Message  : Performing Direct Cast to List<int> -> List<string>
******** Started  : Program -> Direct Test/Cast() ...
******** Completed: Program -> Direct Test/Cast() took [5ms] to execute.

******** Message  : Performing JSON Serialize/Parse to List<string>
******** Started  : Program -> JSON Serialize() ...
******** Completed: Program -> JSON Serialize() took [2ms] to execute.
******** Message  : Performing Direct Cast to List<int> -> List<string>
******** Started  : Program -> Direct Test/Cast() ...
******** Completed: Program -> Direct Test/Cast() took [2ms] to execute.

******** Message  : Performing JSON Serialize/Parse to List<string>
******** Started  : Program -> JSON Serialize() ...
******** Completed: Program -> JSON Serialize() took [1ms] to execute.
******** Message  : Performing Direct Cast to List<int> -> List<string>
******** Started  : Program -> Direct Test/Cast() ...
******** Completed: Program -> Direct Test/Cast() took [1ms] to execute.

Using Poor Man’s Profiler in Your Application

In its most basic form, its use looks something like this:

private static DebugLogger logger = new DebugLogger { Prefix = "Program", MaxElementsIfNonNumericList = 5, MaxElementsIfNumericList = 25, MaxStringLength = 100 };

static void Main(string[] args)
{
  Test(123, new List<string> { "This", "Is", "A", "Test" }, new List<DateTime> { DateTime.Now, DateTime.Now.AddDays(5) }, new List<int> { 9, 8, 7, 6, 5, 4, 3, 2, 1 });
}  

private static void Test(int a, List<string> b, List<DateTime> c, List<int> d)
{
   // Start a stopwatch on the method
   logger.StartDebug("TestMethod");

   for (int x = 0; x < 10; x++)
   {
      // Start a stopwatch and log some values
      logger.StartDebug("loop" + x, x, a, b, c, d);

      // Some code here ...

      logger.StopDebug("loop" + x);
   }

   logger.StopDebug("TestMethod");
}

The Output window of the IDE will quickly display the following:

******** Started  : Program -> TestMethod() ...
******** Started  : Program -> loop0(0, 123, ["This", "Is", "A", "Test"], ["2/5/2020 8:08:13 PM", "2/10/2020 8:08:13 PM"], [9, 8, 7, 6, 5, 4, 3, 2, 1]) ...
******** Completed: Program -> loop0() took [1ms] to execute.
******** Started  : Program -> loop1(1, 123, ["This", "Is", "A", "Test"], ["2/5/2020 8:08:13 PM", "2/10/2020 8:08:13 PM"], [9, 8, 7, 6, 5, 4, 3, 2, 1]) ...
******** Completed: Program -> loop1() took [1ms] to execute.
******** Started  : Program -> loop2(2, 123, ["This", "Is", "A", "Test"], ["2/5/2020 8:08:13 PM", "2/10/2020 8:08:13 PM"], [9, 8, 7, 6, 5, 4, 3, 2, 1]) ...
******** Completed: Program -> loop2() took [1ms] to execute.
******** Started  : Program -> loop3(3, 123, ["This", "Is", "A", "Test"], ["2/5/2020 8:08:13 PM", "2/10/2020 8:08:13 PM"], [9, 8, 7, 6, 5, 4, 3, 2, 1]) ...
******** Completed: Program -> loop3() took [1ms] to execute.
******** Started  : Program -> loop4(4, 123, ["This", "Is", "A", "Test"], ["2/5/2020 8:08:13 PM", "2/10/2020 8:08:13 PM"], [9, 8, 7, 6, 5, 4, 3, 2, 1]) ...
******** Completed: Program -> loop4() took [1ms] to execute.
******** Completed: Program -> TestMethod() took [18ms] to execute.

Lets break this down a bit. Within the class level declaration of the DebugLogger object named “logger” we have some properties being set:

private static DebugLogger logger = new DebugLogger { Prefix = "Program", MaxElementsIfNonNumericList = 5, MaxElementsIfNumericList = 25, MaxStringLength = 100 };

Prefix: This is an optional string value that will precede the code block name with every log entry, this is helpful for helping to identify what class the profiler is currently logging from, if you so choose to use this.

MaxElementsIfNonNumericList: Since the contents of Lists of primitive types will be written to the log in a comma delimited fashion in between brackets, we want a way to restrict the number of elements to actually log, in the event the List contains a lot of elements. This property applies to primitive types that are non-numeric. This value will restrict the number of elements to be logged. Specify 0 if you want all elements listed.

MaxElementsIfNumericList: Works the same as the aforementioned property except it applies to Lists of primitive types that are numeric.

MaxStringLength: This determines at which length a string value in the log should be trimmed. This helps to avoid logging large strings that will then muddy up the logs. Specify 0 if strings should not be trimmed.

IndentNumber: Which isn't shown in the code example above. This is used to set the number of TABS you want to precede the code block name (and Prefix if one is set). This can be useful if you have the Profiler running in different layers of the application. An example would be any properly abstracted, where you might want to apply an IndentNumber on code blocks running in the Business Layer, to help display a sort of hierarchy in code execution.

Starting a Timer/StopWatch

In the above code example, you could see that a string value was passed in with the calls to StartDebug()/StopDebug(). Behind the scenes, Poor Man’s Profiler holds a collection of StopWatch items, and we must correlate which items to Stop from a Start based on a name.

logger.StartDebug("TestMethod");
...
logger.StopDebug("TestMethod");

Alternatively, an explicit name doesn't have to be passed in which essentially makes the code more generic, if null is passed as the first parameter, the name of the calling block will be the calling method name pulled from the Stack Trace (ahh reflection).

logger.StartDebug(null);

However, because the name isn't going to be known, you will need to grab the LastStartID so that it can later be used with a call to StopDebug().

private static void Test2()
{
    // Start a stopwatch on the method, let StartDebug get the name
    logger.StartDebug(null);

    // Since we didnt pass a name, lets get the ID so we can use it when stopping the stopwatch
    long p1 = logger.LastStartID;

    for (int x = 0; x < 10; x++)
    {
        // Some code here …                
    }

    logger.StopDebug(p1);
}

Your first observations of the code block above may be: “Why is he calling StartDebug() and then assigning a variable of type long with the value of LastStartID? Why not just return the ID with the StartDebug()?” The answer to that question is both simple, and somewhat frustrating (for a lack of better words.) This is because the StartDebug() method, along with other various methods are marked with the [Conditional(“Debug”)] attribute, which means that these methods as well as any code that calls them will be excluded when the code is compiled in Release mode. This is how we can guarantee that Poor Man’s Logger will not cause any performance degrading overhead when the application is run in production.

The frustrating aspect of this is that this attribute can only be applied to void return methods, which means there is a little more code that needs to be written. When you think about this limitation it makes sense, because one would not want code that is setting a value to a variable to suddenly disappear in production code, as this no doubt would lead to all sorts of problems.

My workaround here was to have a property in the DebugLogging class that will simply default to 0 if not assigned to. When code is compiled in Release mode, and all the Poor Man’s Profiler code vanishes, at worse there will be various long typed variables defined that will all simply be set to 0 and never actually used.

Logging Parameters

The StartDebug() method can also be called with an unlimited number of parameters values which will in turn be logged.

logger.StartDebug("CodeBlock1", a, b, c, d, e, f, g);

This of course is where most additional overhead comes into play, as the parameters passed in are all boxed into type object, so that any type of object can be passed in. Additionally reflection is used to obtain the actual type of the object parameters to best determine how to render their values in the logs. To top it off, a lot of string concatenation is also used here, which is another process that isn't known for high performance, and another thing to avoid when your end goal is high-performance code. The more parameter values you add, the more overhead you add, thus, add parameters sparingly.

Despite the use of performance killer code (where Microsoft’s Ben Watson would likely shake his head in disapproval) interestingly enough, in my tests, the performance reduction due to boxing, reflection, and string concatenation is rather insignificant for debugging purposes. However caution should be taken regardless, and since every application should strive to gain every fraction of a millisecond of speed that it can gain, we only allow this to be run in debug, which is why the StartDebug()/StartDebug() methods are given the aforementioned conditional attribute. They run when in debug mode, and disappear when in release mode.

With the first release of this library, using the logger to log parameter values is most useful if the parameters are:

Any object with a primitive type (with the addition of DateTime and Timespan)
An implementation of IList of any primitive type (with the addition of DateTime, Timespan).
Any object of a non-primitive type or IList of a non-primitive type which overrides the base.ToString() method (which provides useful info about the object).
Any object that does not meet the criteria above will simply be logged by its type name, or essentially the return of base.ToString().

Additional Methods

To stay on the safe side, I strongly suggest always using StartDebug() and StopDebug(), however I have exposed two alternative methods called Start() and Stop() that can be used with a little less code, however come with the caveat of not being able to be automatically removed when compiling in Release mode. The major downfall here is that if you use Start() and pass in parameter values, you will be triggering boxing, despite the fact that when in non-debug mode, the method calls will not actually be doing anything with these boxed objects, since the internals of these methods are marked with the precompiler directives to exclude code when not in debug mode. The benefit of using Start() is that you can get the ID for which to be used with Stop() from the call itself:

long p1 = logger.Start();

The benefit of using Stop() is that a double is returned representing total milliseconds in the event that you want to do something with this value besides the logging that is already happening internally.

double x = logger.Stop(p1);

I want to reiterate though, that these method calls will remain in Release code, so unless you don’t really care (i.e top performance isn't necessary) or you have a plan to remove them manually, just stick with StartDebug()/StopDebug().

There are a couple of other useful methods that are marked with the debug conditional, thus are safe to leave in code:

WriteParamValuesToString() which takes an unlimited list of parameter values to be logged which may be useful if you want to periodically log various variable values in strategic areas of code.

WriteMessage() which takes a string to be logged.

Making Sense of the Logs

Since the current working implementation of Poor Man’s Profiler writes to the Output window of the IDE, its likely going to be difficult to make whole lot of sense of things, especially since other events and messages are going to be logged in the same location.

It isn’t exactly pretty, but there is a way to use this log dump, and turn it into something of more use.

If there is interest in this, I could always write a log parser/aggregation application in Angular and/or Blazor as the topic of an upcoming article.

Summary

This article was meant to discuss a simple approach to .NET code profiling that can be used to time code block executions as well as log parameter/variable values. As noted, there are vendors who have created far more powerful frameworks for this need, as well as there are other ways to go about accomplishing this goal through AOP/IL Weaving to help reduce code redundancy.

Once again, this article is based on my example class library project that can be found on GitHub -> Poor Man's Profiler

An Old ASP.NET MVC Developer’s Transition to Angular

Corey Lasley — Sat, 25 Jan 2020 01:25:44 +0000

Applies To

Angular CLI: 7.3.9

Node: 10.1.0

OS: Windows 10 x64

Objectives

What is needed to get started developing with Angular.
Why we need Node.js.
Why we use NPM.
Why we use the Angular CLI.
Why VS Code instead of Visual Studio.
How to create our first project.
Understanding the basic structure of our project.
Understanding navigation.
How to build the project for deployment.

Heads Up!

Given the target audience of this article, an ASP.NET MVC Developer who wants to get a grasp on Angular, I am going to attempt to draw some comparisons between MVC and Angular that are loose at best. This is my effort to help bridge a gap of understanding. Clearly, these two technologies are very different, but both follow the Model View Controller pattern. Because of this, my comparisons between ASP.NET MVC and Angular are meant only as a guide, and not true representations of each other.

Introduction

I am ashamed to admit it, but I got really comfortable where I was. I had lived in the world of ASP.NET since .NET was birthed back in 2002. For many years I did everything Web Forms, but eventually transitioned into MVC. The design pattern between Web Forms and MVC was dramatic, to say the least, and it took me a bit before MVC felt natural.

So here I have been for many years, sitting in my comfort zone, writing and maintaining ASP.NET MVC web applications, with jQuery as my default client-side JavaScript framework, and Visual Studio, my IDE of choice.

I was introduced to Angular through a job change into a very forward-thinking and cutting edge driven organization that had made the decision to slowly move away from MVC, and transition into Single Page Applications (SPA) with Angular.

I suspect you may be here because like me, you found yourself facing a transition away from ASP.NET MVC to Angular. Being so ingrained in the ways of ASP.NET MVC, you may be feeling a bit overwhelmed at just how different things are in the world of Angular. The goal of this article is to try to help get you started on that transaction.

Tools Needed for the Job

I noticed many "Getting Started with Angular" articles either assumed the reader had some knowledge about Angular’s prerequisites, or that the reader likely didn’t care and/or need to know about the details as long as the reader was told what to install. Many articles are quick “do this, then that” type articles, which have their place, but for someone like myself who was jumping into completely new territory, I was looking for something more in depth that could give me more background on things.

So lets break things down into some detail! We will need the following in order to start developing Angular apps:

Node.js

An open-source, cross-platform JavaScript run-time environment which can be used for server side, client side, desktop & mobile applications. Node.js is comparable to the .NET run-time environment which executes C#, VB.NET and other supported languages. Node.js is the first thing we will need to install. This is used to serve our application and includes NPM.

NPM

Stands for “Node Package Manager”. It is a package manager for JavaScript built into Node.js. We will have NPM after we install Node.js. NPM is comparable to what NuGet is for .NET. NPM is needed to install the Angular CLI as well as various packages that will be used in the Angular app. NPM commands are prefixed with the term “npm”.

Angular CLI

Stands for “Angular Command Line Interface”. It simplifies creating angular apps by automating operations with quick commands so we don’t need to manually install and configure dependencies. Angular CLI commands are prefixed with the term “ng” (ng stands for a*ng*ular). For more information on Angular CLI click here. For a deep X 5 dive into the Angular compiler, see the following blog.

I highly recommend the following items to enhance our experience:

Visual Studio Code

Is optional, but the preferred editor. You can download it here. You may be asking yourself: Why use Visual Studio Code when we have Visual Studio?

One thing I noticed when attending my first Angular conference, NG-Conf 2019, was that among everyone I saw writing/reading code on their laptops, they were all using VS Code, both attendees, and presenters. I cannot say I saw a single instance of Visual Studio running on any machine there. I’m not a big fan of simply defaulting to something because “everyone else is doing it,” so here is my best guess.

	Visual Studio	Visual Studio Code (VS Code)
Cross Platform	No	Yes
Open Source	No	Yes
Cost	FREE (Community Edition)	FREE
Size	1 GB (bare minimum)	210 MB
Memory Consumption	Generally Higher	Generally Lower

VS Code may not have all the bells and whistles that Visual Studio has, however, being cross-platform and open source means that learning and teaching Angular development is going to be much easier when we have a common IDE that is available to all.

For us veteran Visual Studio users, there are tutorials out there for creating Angular apps in Visual Studio, however, it probably isn’t a bad idea to just start getting used to using VS Code.

Angular Essentials

This is an extension for Visual Studio Code put together by John Papa, which is essentially a collection some of the most popular extensions.

Once we have everything above ready to go, in order to execute NPM / Angular CLI commands we can (1) Open command prompt, PowerShell, Git Bash, or something similar, or (2) use the Terminal window in VS Code (which basically opens up the previous option within the VS Code environment)

Setting Things Up

Install Node.js. The installation can be downloaded here
NPM is built in to Node.js, so we are good to go here.
Install the Angular CLI. Open up a command prompt and execute the following:

npm install -g @angular/cli

A Basic Look Under the Hood

When Node.js and the Angular CLI are installed, you might be curious about how they integrate with your favorite command line interface. Entering the following command will give us a semicolon delimited list of all of the global paths we have defined in our system:

echo %path%

The two applicable paths that were setup during installation of Node.js are the following:

c:\Program Files\nodejs\npm.cmd

:: Created by npm, please don't edit manually.
@ECHO OFF

SETLOCAL

SET "NODE_EXE=%~dp0\node.exe"
IF NOT EXIST "%NODE_EXE%" (
  SET "NODE_EXE=node"
)

SET "NPM_CLI_JS=%~dp0\node_modules\npm\bin\npm-cli.js"
FOR /F "delims=" %%F IN ('CALL "%NODE_EXE%" "%NPM_CLI_JS%" prefix -g') DO (
  SET "NPM_PREFIX_NPM_CLI_JS=%%F\node_modules\npm\bin\npm-cli.js"
)
IF EXIST "%NPM_PREFIX_NPM_CLI_JS%" (
  SET "NPM_CLI_JS=%NPM_PREFIX_NPM_CLI_JS%"
)

"%NODE_EXE%" "%NPM_CLI_JS%" %*

C:\Users\[My Username]\AppData\Roaming\npm\npm.cmd

@IF EXIST "%~dp0\node.exe" (
  "%~dp0\node.exe"  "%~dp0\node_modules\@angular\cli\bin\ng" %*
) ELSE (
  @SETLOCAL
  @SET PATHEXT=%PATHEXT:;.JS;=;%
  node  "%~dp0\node_modules\@angular\cli\bin\ng" %*
)

We will also need the following file after we install the Angular CLI:

C:\Users\[My Username]\AppData\Roaming\npm\ng.cmd

@IF EXIST "%~dp0\node.exe" (
  "%~dp0\node.exe"  "%~dp0\node_modules\npm\bin\npm-cli.js" %*
) ELSE (
  @SETLOCAL
  @SET PATHEXT=%PATHEXT:;.JS;=;%
  node  "%~dp0\node_modules\npm\bin\npm-cli.js" %*

These are the paths that allow the command prompt, PowerShell, Git Bash, etc. to know where they can find the npm and ng commands (which ultimately point to
node.exe) no matter what directory we currently reside in, while putting our current directory in scope for the commands’ execution.

Get Comfortable with the Command Line

In my early days as a developer (way back in the late 1990’s early 2000’s,) GUIs were all the rage. Getting away from command line interfaces was the thing most users wanted, and most developers strived to produce. Visual Studio has done a great job over the years of providing a GUI for virtually everything, from project creation to interacting with Git repositories. Since becoming a developer, command-line tools seem to be more popular now due to open source. This makes sense, because an application needs to work within multiple platforms. Creating a GUI to support a wide range of commands and options can be time consuming with little real gain.

If you are anything like me, moving away from Visual Studio was definitely a move outside of my comfort zone. This was mostly due to shifting away from GUI's. GUI's provide menus, icons, and buttons to run commands. Whereas, you have to commit various commands to memory, or have a handy cheat sheet available, to use the command-line.

But alas! I quickly discovered that the move wasn't as bad as I thought it would be.

Creating Our First Project

Visual Studio users might expect to open up VS Code and find a File > New Project… menu item, and then complete a short wizard for setting up an Angular app, but this isn't how things are done in this general purpose lightweight IDE. In order to create a new Angular app, we need to use the Angular CLI. The Angular CLI will setup a basic project template with all of the basic files, configurations, and the structure needed to be able to immediatly run the application.

In the command prompt we will first want to change directory to the root folder where we want our Angular app set up. An example would be:

cd c:\Temp

Next, run the ng new command with the name of the app we want to create:

ng new my-app

We will be prompted to add Angular routing or not. For this project, please answer yes, since we will utilize this later.

We will then be prompted for which stylesheet format to use, I recommend selecting SCSS.

This will then create a new folder within the directory we are currently in, as
well as all the files needed.

*If we get an “Unexpected end of JSON input while parsing near” error after executing an ng new command, something in our cache is likely corrupt. This was the case for me at one point, because I had previously installed things without understanding what I was doing, and things went stale. One way I was able to solve this was by executing the following command:

npm cache clean --force

Once this is completed, we need to change directory into our new application folder:

cd my-app

We can now run the application with the following command:

ng serve --open

This will start the server, and then open the application in the browser.

When executing ng serve, the local server fires up and runs until you stop it, usually by typing [Ctrl][c] in the terminal window. One really nice feature of this local server is that it utilizes a File System Watcher mechanism (possibly fs.watchFile built into node.js) that detects when there are changes to files within your directory structure. When a change is detected, such as when a file is saved, the server automatically re-compiles the application, and then refreshes it within the browser.

Understanding Our First Project

To begin editing the code, we do the following: Within VS Code, on
the menu, select File > Open Folder…

When I opened up my first Angular project folder in VS Code, I was overwhelmed and confused by all of the files created, as you might be. There will be very little that will look familiar to the old ASP.NET MVC developer. Despite there being a lot to digest here, there really are only a handful of files that we need to be worried about, at least for now.

The first thing we might notice, is that there are no core project and solution files that we are used to having associated with our .NET applications. When it comes to Angular development, everything simply resides within a directory structure. So rather than having Angular project files that keep track of the other files, references, etc. that belong to the project; files reside within the base directory and its subdirectories. This structure is our project. Therefore, when we open up a project in VS Code, instead of opening up a file, we open up a directory.

Let’s get acquainted with the following files:

package.json

This file contains the list of node dependencies, and it can be somewhat likened to a list of References in any ASP.NET project.

src/index.html

This file is basically a simple placeholder for our core Component. This is similar to the application-wide shared layout View in MVC (often named _layout). This is the core html that will be rendered for the application. Rather than adding additional html pages, you will use Angular Components with Angular Routing which will be discussed later.

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>MyApp</title>
  <base href="/">

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="icon" type="image/x-icon" href="favicon.ico">
</head>
<body>
  <app-root></app-root>
</body>
</html>

Everything in here should look pretty standard, that is, with the exception of the non-standard <app-root> tag (line 12.) We will talk more about this next since it relates to Angular Components.

app/app.component.*

You will see several files in the app folder that starts with app.component. These files are all related to each other and make up an Angular Component. For comparison purposes, a component would look most like an MVC View or Partial View. Instead of creating additional HTML files on the root (in addition to Index.html) you will be creating more Components.

This particular component, however, is slightly different from other components you will create, because it is the root or parent component of the application and is generally associated with the Index.html. All other components that you create will exist in a parent/child hierarchy under this component. In a way, you could think of this component as the application’s core View or Layout, whereas all other components you add to the project would be like Partial Views.

The main component file is a TypeScript file called app.component.ts. This file is essentially the definition of the component where you set its various properties, wire up event handling, update the DOM, make calls to services, etc. This is like a Controller blended with a Model in MVC.

import { Component } from '@angular/core';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.scss']
})
export class AppComponent {
  title = 'my-app';
}

A key thing to note here is the @Component decorator section (line 3-7). This defines various properties of the component. When a project is first created with a default component, these three properties are established.

Looking at line 4 you see the selector property with a value of ‘app-root’. If you remember back to the Index.html file, there was that non-standard HTML tag <app-root>. This means that the rendered content of this component is going to be placed where this <app-root> tag resides in the Index.html. Basically it is a placeholder for a component's content. Within MVC, this <app-root> tag would be somewhat comparable to a @RendyBody() in a Layout or placehold tags within components. It would be comparable to an @Html.Partial() within a View.

Looking at line 5 you see the templateUrl property. Its value points to the app.component.html file. This file is where you can define the HTML that you want your component to render. Line 6 contains the styleUrls property. It maps to the app.component.css file which contains the CSS for this specific component.

src/app/app_module.ts

This is an important TypeScript file which is used to declare your components. If your components are not declared here, they will not be able to be consumed within your application.

import { BrowserModule } from '@angular/platform-browser';
import { NgModule } from '@angular/core';

import { AppRoutingModule } from './app-routing.module';
import { AppComponent } from './app.component';

@NgModule({
  declarations: [
    AppComponent
  ],
  imports: [
    BrowserModule,
    AppRoutingModule
  ],
  providers: [],
  bootstrap: [AppComponent]
})
export class AppModule { }

dist (Folder)

This is a folder that will not exist until you execute the ng build command (I will discuss ng build in detail later). This folder contains the output of the build and represents the content to be deployed to a web server.

So, to recap a bit on the descriptions above, the following image shows the contents of an Angular 7 application, loosely compared to an MVC application:

Get Comfortable with Angular CLI Commands

It is important to note that there is a lot more to the Angular CLI than simply instantiating, building and serving your project. The old ASP.NET developer, like me, might be tempted to use VS Code’s File menu to add new items to the application. In some cases you might create files manually, but for many types of Angular items you are going to want to let the Angular CLI to do the work for you. Adding certain types of items often requires adding and modifying multiple files. The Angular CLI is here to save us a considerable amount of time and headache.

An example of this would be adding a new component to our application. Lets say we want to add a new component called "about". To do this you would execute the following:

ng generate component about

This will add a new folder within the app folder representing a new component. It will generate default/template code for a .css, .html, .spec.ts, and .ts file for this component. Additionally it will modify app.module.ts to declare this new component.

A Quick Introduction to Angular Routing and Navigation

Not wanting to dig too deep into the rich features of Angular in this article, I feel that our time together here would be incomplete if I didn’t touch on Angular’s powerful routing capabilities. Routing is important for old ASP.NET developers who are used to writing applications that have mutiple “pages". In the world of ASP.NET MVC, we are used to having Controllers, Models, and Views to represent each page in our application. When it came to figuring out how to deal with multiple pages in Angular, my first thought was that our Index.html was sort of like our Home.chtml in MVC, and in order to create another page for something such as “About Us,” I would probably need to create an AboutUs.html page. This didn’t feel right, especially since I became accustomed to thinking of the Index.html more like our global layout/placeholder, so I asked my good friend and Angular expert, Wes Grimes about this. Wes told me to look at Routing. Well, it only took a quick scan at Angular’s official documentation on this, and I got really excited. While Routing in ASP.NET MVC is pretty slick, Routing in Angular steps up the game!

Angular’s implementation of routing is where you assign paths that map to various components. These are essentially our Views. The default app.component becomes the container for everything. When our URL matches a defined path, the router “directs” to the app.component, and replaces the contents inside the <router-outlet> tag with the rendered contents of the mapped component. In addition to this, the part that really got me pumped was instead of using href attribute in anchor tags (which you can do), we instead use the routerLink attribute. When the user clicks the link, the mapped component is simply swapped out in the DOM via lazy loading and the browser’s URL changes! There is no browser redirecting, and only the mapped component section is refreshed. This makes the flow seamless, dramatically speeds up navigation between “pages”, and reduces the amount of content coming down the pipe. It truly gives the end user the perception that the entire application exists on the same page (Well, I guess technically it does, but you know what I mean.)

So let’s break this down. If we remember back to earlier when we first created our application, I said we should select “yes” when asked if we wanted to set up Routing. This is where we are going to put this to use.

Lets say we have a few components created to represent several pages. We will have a home component, an about component, and a page-not-found component. Within our app.module.ts file, just under the import lines, add the following:

const appRoutes: Routes = [
  { path: 'about', component: AboutComponent },
  { path: 'home', component: HomeComponent },
  { path: '', component: HomeComponent },
  { path: '**', component: PageNotFoundComponent }
];

Notice the path values for each of these components. The 'about' and 'home' paths should be fairly obvious, but what about the '' path, and the path specified for the PageNotFoundComponent? As far as the '' path, this just means that if no particular path is defined, then route to the home component. As for the ** path, this simply means that if any other path is given that does not match one of the above, then route to the page-not-found component.

Next, within the @NgModule area we want to add the following within the imports section:

RouterModule.forRoot(
      appRoutes
    )

So it will look like this:

@NgModule({
  declarations: [
    AppComponent,
    PageNotFoundComponent,
    AboutContentComponent,
    HomeContentComponent
  ],
  imports: [
    RouterModule.forRoot(
      appRoutes
    ),
    BrowserModule,
    AppRoutingModule
  ],
  providers: [],
  bootstrap: [AppComponent]
})

Lastly, go to your app.component.html and place the following tag wherever it is you would like your home, about, and page not found components to render.

<router-outlet></router-outlet>

At this point, wherever you wish to place a link in any of your HTML, instead of using the standard anchor tag markup, it is going to look a little different. Lets say that we want to place a link to the about "page," our markup will look like the following:

<a routerLink="about" routerLinkActive="active">About Us</a>

Navigation is as simple as that!

Building for Deployment

For the last part of this article, I want to show you how to do an official build for deployment. There is so much more you can do, but lets wrap up this particular app.

We can always run our application locally with the ng serve command, but when it comes time to deploy our application, we need to perform a build. To do so, execute one of the following Angular CLI commands:

ng build --prod

ng b --prod

This will create a folder on the root called “dist”. This will contain your compiled application. The Angular CLI uses a JavaScript module bundler called Webpack. It compiles everything down to what the browsers understands: HTML, CSS, and JavaScript.

At this point, you might think that you can simply use file explorer to browse to C:\Temp\my-app\dist\my-app and double-click the Index.html to see the application work, however when you do, you will likely see a blank page. The index.html loaded, but the JavaScript didn’t execute.

The problem here resides with base-href mapping. If your application was sitting on the root of the system, the above build would run, however because it resides in a subfolder on the system, you need to specify the base-href location in the build command:

ng build --base-href "C:\Temp\my-app\dist\my-app\" --prod

Keep in mind that this also applies when deploying to a web server.

Summary

It is my hope that if you are an old ASP.NET MVC developer like me, who has been on the fence about checking out Angular, this article has given you some insight as to how things work, and helped to build some confidence in diving in to some Angular development. I am planning on a Part II in the near future. With this I will go into more depth with some things that you can do within your components.