DEV Community: Edward Chen

Weekly Quick Tips: AWS Lambda Deployments

Edward Chen — Tue, 31 Jan 2023 18:28:31 +0000

Hello folks! Been real deep in AWS as I learn more about the services and today's tip comes straight from the AWS team that I have been working with! If you don't want to read the rest, here's the quick TL;DR:

There's a known issue (as of 01/01/2023 is when we started dealing with it) with AWS Lambdas where even if the size is over the limit, sometimes it can still be deployed to your environments. The best way to ensure you never experience any issues is to be sure that you are under the size limit:

250 MB for uncompressed data (including layers)¹
50 MB for compressed data (including layers)

Now here's an overview of some notes and tips that I learned from my investigation! First off, it doesn't matter if this is done via CloudFormation, CodePipeline or any of their services, it all eventually has to go through Lambdas and that is when you will hit your size issue. The error message will appear as something like so:

Code uncompressed size is greater than max allowed size of 272629760. (Service: Lambda, Status Code: 400, Request ID: 99a3c30a-b444-4558-ba5f-13ce38922f5b)" (RequestToken: dc727a33-6f29-e240-1452-97c228fe2ad0, HandlerErrorCode: InvalidRequest)

The error message is pretty solid in that it immediately tells you that your code is larger than what they allow. However, welcome to the first obstacle that I encountered - depending on the OS and even how you unzip (GUI vs terminal), there's a slight size difference. Going with the terminal will show you the smallest size (command for that is du -h which stands for disk usage) while the GUI can be larger. The difference is negligible but something to note if you are wondering why there is a size difference.

Now comes to the biggest revelation (besides the known issue) that helped me solve my issue: AWS Lambda runtime environments have their own aws-sdk so you do not need to include that in your packages. Let me repeat that: You do not need to include aws-sdk in your packages.

For us, that immediately removed 80 MB and helped us well be under the limit. And then it took another couple weeks continuously talking to AWS support to determine that it is a known issue why even though we were over the size limit, we were still able to deploy sometimes.

Sources:

¹https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html

Just a friendly reminder: Don't stress about the layoffs!

Edward Chen — Thu, 19 Jan 2023 16:59:29 +0000

Here's another Weekly Tip! Today's topic is going to be a quick one and relevant to our careers but not really technical. DON'T STRESS ABOUT THE LAYOFFS!^*

This is more geared towards those earlier in your careers as I have found that generally the Senior+ Engineers are still in high demand. For those of you that are starting your careers as a Software Engineer or have been in the industry for 1-3 years now, I'm sure you are concerned and have been following the news on the layoffs.

But this is just a friendly reminder to not stress about your outlook! Keep learning, stay curious and do not take any rejections personally. Instead of applying to a FAANG (or MAANGA if we want to keep it more topical), you can get your foot in the door by applying to smaller companies or even a consultant agency (be wary of these though. Do not accept if they ask you to pay for training) as that will expose you to different tech stacks and experiences!

Also keep in mind that this could be a chance to reflect on yourself. Is this something that you want to do as a career or is it something that you enjoy more as a hobby? Would it be feasible to work as a Data Engineer even if you enjoy doing Front End work? You can spend some time to work on personal projects, figure out what you enjoy. Everyone has experienced (or are still experiencing) all the traumas from getting out of a global pandemic so give yourself a break. You earned it!

^* Of course take all of this with a grain of salt because I am a stranger on the internet and I do not know your situation.

Weekly Quick Tips: Pub/Sub vs Observer Pattern?

Edward Chen — Sat, 07 Jan 2023 18:48:05 +0000

In my attempts to blog more, I am going to do weekly tips on stuff that I think would be helpful or interesting to know for beginners. I will probably link to other articles but the intention of Weekly Quick Tips is for something someone can read in <5 min and walk away with a general sense of "wow that was interesting" or "nod good to know".

So to start it off, let's talk about Pub/Sub vs Observer Pattern. They're similar but just slightly different!

Starting off with the easier one, Observer Pattern. It's simple, there is a Subject that is being watched by many Observers. The Observers are subscribed to a Subject and eagerly wait for new information. There is a direct relationship between the Subject <-> Observer.

Pub/Sub has a middle layer called message broker/event bus that allows for decoupling between the Publisher and Subscriber. So both the Publisher and Subscriber only talk to the message broker and the two are completely oblivious to each other's existence.

You may ask, well what the heck - why would I want that? First off, the decoupling lets you minimize the risk of merge conflicts as people can work separately and since the two are independent of each other, if one happens to go down or encounters an issue it will not cause the other to immediately fail. Scaling can be easily done as well.

Disadvantages? Well that independence is a double edged sword my friend. It also is harder to track logging when something goes wrong and you have to be very sure that the Publisher is sending the message, a way for the subscriber to notify the broker that it received the message and handling the communication between the two. With a traditional client <-> server like in an Observer pattern, an error can just bubble up directly and it is easier to follow.

Thanks for tuning in, please feel free to comment if you have thoughts. At the end of the day, this is about learning so let's work together to help each other learn :D

Sources:
-https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern
-https://hackernoon.com/observer-vs-pub-sub-pattern-50d3b27f838c

What is OAuth 2.0?

Edward Chen — Wed, 05 Feb 2020 00:00:00 +0000

First off, there's a lot of resources out there on what is the OAuth2 protocol and how it came to be. What I'm going to try to do is collect it all in this post including where it originated from and what's the future of it. Please feel free to comment if something is incorrect. After all, the best way to learn is to teach right?

If you want a TL;DR, as always scroll to the bottom.

So what is the OAuth2 protocol?

By definition, OAuth is a protocol that allows a resource owner to authorize clients to access a website or non-website server. This is important because it allows the resource owner to determine what areas the client can see which is convenient if you have proprietary or confidential IP. Typically, the authorization is granted via Access Tokens, an encrypted string that represents the granted permissions (or scopes). The API will then determine what scopes the client is authorized for and return back the data. For example, a Menu app would accept differing levels of authorization:

creating item (scope read: item)
deleting item (scope delete: item)
updating item (scope update: item)

Within the OAuth 2.0 flow, there are the following roles that come into play.

Client: The application that is requesting access.
Resource Owner/Server: The entity that can grant access and the server that contains the protected resources.
Authorization Server: The server that authenticates whether or not the authorization grant is valid. If valid, an Access Token is granted.

This is taken from Auth0's page on OAuth 2.0 which I would highly recommend for further learning.

The Application (Client) asks for authorization from the Resource Owner in order to access the resources.
Provided that the Resource Owner authorizes this access, the Application receives an Authorization Grant. This is a credential representing the Resource Owner's authorization.
The Application requests an Access Token by authenticating with the Authorization Server and giving the Authorization Grant.
Provided that the Application is successfully authenticated and the Authorization Grant is valid, the Authorization Server issues an Access Token and sends it to the Application.
The Application requests access to the protected resource by the Resource Server, and authenticates by presenting the Access Token.
Provided that the Access Token is valid, the Resource Server serves the Application's request.

This is a high level overview of the flow and each step can be explained in further detail, enough to constitute it's own page so I will remain general and elaborate into further detail in another page. However, an important aspect of OAuth 2.0 is to know how the authentication works.

So first off, there is a difference between authentication and authorization. Authentication means that the entity has been validated to be who it claims to be. Authorization on the other hand refers to rules that determine who is allowed to do what. A good example of this is if you think of an application as a house. Authentication would be the owner of the house allowing Bob to enter the house. Once you have proven that you are Bob, you will be allowed to enter. Authorization in this case would be the owner telling Bob that he may enter the kitchen but not the master bedroom.

Now that we have defined the two, it will make more sense that OAuth 2.0 is an authentication protocol although due to scopes, it has some pseudo authorization aspects too. How the authentication works is that the protocol has two endpoints: Authorization and Token. What happens is that once a client has requested access to the resource server, the server will redirect the client to a consent screen where credentials can be entered. Assuming the credentials are valid, the client is given a response token and redirected back to the original page. I am sure you are familiar with this, this can be seen in many login screens where you can "Login via Facebook" or "Login via Github". Both of those options redirect you to the prospective login pages where you authorize the client's permission to access and then you are redirected back to the client.

Depending on the Authorization flow, the response token can return back a Refresh Token as well. This is generally used by web applications that are server side so the source code is not exposed to the public. Since the code is not exposed to the public, in this flow the response token will contain client_secret, client_id, scope, state, redirect_uri, access_token and refresh_token. So much information is exposed in this flow to allow a client and the resource server to interact with less, if any, user interaction. Given that information, the client can call the Authorization endpoint and use the refresh_token to renew the access_token since typically the access_token has a shorter duration. For example, the access token can expire every 24 hours whereas the refresh token can expire every 100 days.

TL;DR

Honestly, there is a lot of complexities and intricacies of OAuth 2.0. But Auth0 did an incredibly good job of simplifying a generic flow so I will just post their diagram here. They explain a lot of concepts easily and I did my best to summarize the information in a single page here for ease of use.

Sources:

C# Dictionary: Add or [key]=value?

Edward Chen — Fri, 22 Nov 2019 07:33:06 +0000

Recently I came across an interesting problem that threw me off initially. Now let me preface this with the fact that I have been using C# daily for almost three years now so I am by no means an expert like Jon Skeet but I am no beginner. If you want a TL;DR, as always scroll to the bottom.

So the problem was pretty simple, print out a list of the number of occurrences in an array.



var dict = new Dictionary<int, int>();
var nums = new int[]{1,1,2,3,3,3,4};

foreach (var num in nums)
{
    if(dict.TryGetValue(num,out int val)){
        dict[num] += 1; 
    }
    else{
        dict.Add(num,1);
    }
}

foreach (var num in dict)
{
    Console.WriteLine($"Key: {num.Key}, Occurence: {num.Value}");
}

This prints out what you would expect:

However when I tried to use the [key]=value way, I got an error, the ever so common KeyNotFoundException.



var dict = new Dictionary<int, int>();
var nums = new int[]{1,1,2,3,3,3,4};

foreach (var num in nums)
{
    dict[num] += 1; 
}

foreach (var num in dict)
{
    Console.WriteLine($"Key: {num.Key}, Occurence: {num.Value}");
}

That was odd to me because I had always thought that was equivalent to Add or update. Of course I could have gone my merry way and taken a mental note to always use the Add method in combination with TryGetValue which would have caught the exception but by golly, I had to know I was not going insane here with such a simple task like adding a value to a dictionary.

After digging into the reference source, the two look pretty identical.



public void Add(TKey key, TValue value)
{
    this.Insert(key, value, true);
}

public TValue this[TKey key]
{
    get
    {
        int index = this.FindEntry(key);
        if (index >= 0)
        {
            return this.entries[index].value;
        }
        ThrowHelper.ThrowKeyNotFoundException();
        return default(TValue);
    }
    set
    {
        this.Insert(key, value, false);
    }
}

The only difference was the third argument but after taking a look at Insert, that is really only used to throw an exception if there is a duplicate Key.

But then it dawned on me that I am shaving the yak and I needed to take a step. Lo and behold, with that step back I realized that it was the operator precedence. The issue wasn't with the dictionary, it was with the += that I was doing. Because of that, the code was doing a get and then setting the value to an increment of plus one. However, the value doesn't exist which is where the KeyNotFound exception was thrown.

Finally, I can push off the imposter syndrome for a little while longer. Thank you for reading and I hope everyone has fun programming!

TL;DR

First off, if I had checked StackOverflow first, I would have realized many people have thought of this and there's an excellent answer to this here.

But to summarize, [key]=value is not really Add or Update in my case because of the order of precedence with operators. Due to dict[num] += 1, the code is doing a get on the num value in the dictionary and throws an exception because the value does not exist.

Akka.NET Bootcamp Notes: Unit 1.2 - Defining and Handling Messages

Edward Chen — Wed, 16 Oct 2019 01:27:01 +0000

In an effort to reinforce my personal learning and summarize the wonderful Akka.NET bootcamp that is provided by Petabridge, I am going to cover each unit in a series as I go through it. I will continue to edit the page if the information is incorrect or not available. Since the topics can be complex, I will provide a TL;DR at the bottom and sources for people who are interested in learning more. So let's get right into it!

What is a message

I think Petabridge did a wonderful job of explaining a message so I am just going to put what they wrote below:

"Any POCO can be a message. A message can be a string, a value like int, a type, an object that implements an interface... whatever you want.
That being said, the recommended approach is to make your own custom messages into semantically named classes, and to encapsulate any state you want inside those classes"

In terms of sending and handling messages, you use the Tell() method to send a message and how an actor handles the message is up to the developer. One thing to note is that if an actor receives a message that it cannot handle, the actor will choose to ignore it. Whether or not the ignored message is logged is dependent on the actor type (for example, an UntypedActor would not handle it unless defined but a ReceiveActor will automatically send it to Unhandled). When handling messages, the value of the sender is always in the receiving actor's Sender property.

To organize the definitions of messages, it is recommended to put it in another file and reference the messages. A good example of the layout for Messages.cs would be something like this. This is taken from Akka.Net's Message.cs

#region Neutral/system messages
/// <summary>
/// Marker class to continue processing.
/// </summary>
public class ContinueProcessing { }
#endregion

#region Success messages
/// <summary>
/// Base class for signaling that user input was valid.
/// </summary>
public class InputSuccess
{
    public InputSuccess(string reason)
    {
        Reason = reason;
    }

    public string Reason { get; private set; }
}
#endregion

#region Error messages
/// <summary>
/// Base class for signalling that user input was invalid.
/// </summary>
public class InputError
{
    public InputError(string reason)
    {
        Reason = reason;
    }

    public string Reason { get; private set; }
}

/// <summary>
/// User provided blank input.
/// </summary>
public class NullInputError : InputError
{
    public NullInputError(string reason) : base(reason) { }
}

/// <summary>
/// User provided invalid input (currently, input w/ odd # chars)
/// </summary>
public class ValidationError : InputError
{
    public ValidationError(string reason) : base(reason) { }
}
#endregion

TL;DR

Messages and their types

A message is the simplest object an actor can pass to encapsulate any state. Actors use messages to communicate between each other and how the actor handles the message is dependent on the actor (either ReceiveActor or UntypedActor). Messages should be separated into specific categorical classes.

Sources:

Petabridge: ReceiveActor
Petabridge: UntypedActor
Petabridge Akka.NET Bootcamp

Akka.NET Bootcamp Notes: Unit 1 - Fundamentals of the Actor Model

Edward Chen — Sun, 29 Sep 2019 16:47:56 +0000

In an effort to reinforce my personal learning and summarize the wonderful Akka.NET bootcamp that is provided by Petabridge, I am going to cover each unit in the series as I go through it. I will continue to edit the page if the information is incorrect or not available. Since the topics can be complex, I will provide a TL;DR at the bottom and sources for people who are interested in learning more. So let's get right into it!

Unit 1.1 - How to create your own ActorSystem and actors

What is the Actor model?

In an actor model, everything is an actor (similar to OOP) and each actor does one task
Actors communicate via messages and message passing is asynchronous and immutable
Actors pass the message to ActorReferences, not directly to actors. The ActorReferences take the messages and place them in a mailbox (Queue data structure meaning FIFO) and pushes the messages to OnReceive() when the actor is ready to process them.

One of the guarantees of actors in Akka.NET is that an actor's context and internal state are always thread-safe when processing messages. This is because the messages are immutable and processed serially.
Actors can have an internal state which means when an actor is destroyed or restarted, the actor instance is lost and a new instance is created via Props. This is important to know because an Actor's Life Cycle can be rebooted whenever a problem is encountered:

In the event an Exception is thrown, the actor's mailbox is retained and processed with the new actor instance.
Every Actor has a Parent and Children
An actor has to be created by another actor. This is important because an actor is restarted by it's parent (the terminology is Supervisor). Every parent comes with a default SuperviserStrategy object (but a user can define it's own strategy) that decides to either restart, stop or escalate to the grandparent actor. With a restart, all the children of the parent will be restarted as well.

TL;DR

Concurrency and Location Transparency.

Since actors are lazy, cheap and single tasked, they can easily be scaled so even though you have one message per actor, you can have 10,000 actors -> 10,000 messages occurring at once. Due to messages being immutable and running asynchronously, this is really powerful.

Sources:

Petabridge: What is an Actor
Petabridge Akka.NET Bootcamp