DEV Community: Steve Bjorg

Lessons Learned on Optimizing .NET on AWS Lambda

Steve Bjorg — Fri, 14 Oct 2022 17:56:02 +0000

I learned a lot diving into the foundational parts of the AWS Lambda implementation for .NET, but I also have some questions left. Maybe I will pick this up sometime later or maybe someone will feel inspired to venture deeper into this topic.

Conclusions

Before sharing my conclusions, I want to stress again that you should use them as a starting point and benchmark your own code to see what makes sense for your situation.

Also, make sure you understand what optimal means for your application. There is no one-size-fits-all. Know what your objectives are ahead of time.

Tiered Compilation

Based on the data gathered by the benchmarks, Tiered Compilation should only be used if minimizing cold start duration is your top priority. From a cost perspective, it never makes sense to enable it.

ReadyToRun

This option is easy to recommend. If you know what the target CPU architecture is going to be then ReadyToRun is an obvious choice. Even with Tiered Compilation disabled, the Lambda function performs very well. This was quite a surprise, because ReadyToRun generates unoptimized code (Tier 0) and without Tiered Compilation enabled, that code will never be rejitted. However, from the measurements, the rejitting overhead is so onerous that 100 warm stars are not enough to make up the difference.

Pre-JIT .NET

I would recommend setting the AWS_LAMBDA_DOTNET_PREJIT environment variable to Always unless cold start duration is critical. If anything, I would explore how to pre-JIT even more of the code during the INIT phase since it's free and runs faster than the INVOKE phase for lower memory configurations.

CPU Architecture

The ARM64 architecture is the exciting new kid on the block for .NET and AWS Lambda, but the venerable x86-64 architecture should not be discounted. In these benchmarks, it has often fared better in performance, but at increased cost. There are also more issues with the ARM64 architecture. Make sure to check the issue tracker to see if any of them might affect your project.

Future Work

Here are some areas I would like to explore further when time permits.

Are all AWS Lambda regions the same?

My benchmarks were conducted in us-west-2. I would assume all regions perform the same, but it would be interesting to confirm this is the case indeed.

Benchmarking with I/O Operations

I received some feedback that my benchmarks were not representative because they lacked I/O operations, such as when interacting with other services. That is correct and it was intentional. I/O operations are at least an order of magnitude slower than compute operations. My interest was driven by understanding the interplay of various compiler options, CPU architectures, and memory configurations. Adding I/O into the mix would have prevented establishing a clean baseline. That said, I agree that benchmarking these scenarios with I/O operations is both interesting and valuable.

.NET 7 Native

One of the features I'm most excited about in .NET 7 is native compilation. Preliminary results shared by others have shown very promising improvements to performance. However, because native compilation uses the generic Lambda runtime, the INIT phase is no longer free. Does that also mean, the INIT phase no longer runs at full speed? If so, it changes everything about how we need to approach minimizing execution costs. Still, the promise of much faster execution is tantalizing to say the least.

Self-Hosted .NET

I thought about benchmarking self-hosted .NET Lambda functions but given that native compilation is a much better alternative, I did not bother to do so. From my experience, self-hosted functions are large and slow. The only time it made sense to consider using them was with .NET 5 to access newer features before .NET 6 was supported by AWS Lambda. For .NET 7, I would focus on native compilation instead and ignore the self-hosted option.

More Pre-Jitting during INIT Phase

As mentioned, I think there is something to be said about pre-jitting more code during the INIT phase. I don't know what else would make sense, but I would explore this area to shift some billable execution time to the free INIT phase. I feel there is some untapped potential here.

Custom Amazon.Lambda.RuntimeSupport Package

I can't shake the feeling that there are some opportunities to specialize the Amazon.Lambda.RuntimeSupport package for greater flexibility and better performance.

To better understand how Lambda functions interact with the AWS Lambda service, I created a mock implementation of the service and its 4 APIs:

/2018-06-01/runtime/invocation/next: This endpoint returns the payload for the next Lambda invocation request.
/2018-06-01/runtime/invocation/{awsRequestId}/response: This endpoint receives the response of a successful Lambda invocation.
/2018-06-01/runtime/invocation/{awsRequestId}/error: This endpoint receives the error message of a failed Lambda invocation.
/2018-06-01/runtime/init/error: This endpoint receives the error message of a failed Lambda initialization.

The Lambda invocation is suspended on /2018-06-01/runtime/invocation/next if no other payload is available. That means, it is technically possible to respond to a request using /2018-06-01/runtime/invocation/{awsRequestId}/response and then do some additional post response clean-up work that does not impact the responsiveness of the Lambda function. For example, the garbage collector could be explicitly triggered. I don't know if that makes sense in practice, but it's an interesting notion to explore.

It also bugs me that baseline .NET Core 3.1 runs faster than .NET 6. With all the hard work that went into optimizing performance of .NET 6, it feels wrong. Maybe something could be done at the custom runtime level to improve things further.

More C# Compiler Options

I only benchmarked the most obvious C# compiler options, such as Tiered Compilation and ReadyToRun, but there are more options that might be interesting to explore.

Profile-Guided Optimizations (PGO)

This is one of the exciting features that got away this time. Profile Guided Optimization (PGO) enables the .NET runtime to gather execution information that can be fed back into the C# compiler to produce a better executable. In essence, it's a smart optimizer that looks at real-world data to produce the best possible code.

I don't know how one would instrument a Lambda function to collect the profile data, but if it possible, it would be very interesting to make it part of a CI/CD pipeline. Something akin to the following steps:

Build and deploy unoptimized version of Lambda function
Run integration tests against deployed Lambda function and collect profile data
Re-build Lambda function with profile information

Parting Thoughts

While there are stones left unturned on this journey, I hope that some of this work can already been put to good use. I find measuring performance very rewarding, because it can be objectively assessed. I also think it's important because the faster our code runs, the less harm we do to the environment. Last, but not least, there is also an attractive notion of minimalism that is easily capture as only execute what is needed and nothing more.

If you have any questions, suggestions, or corrections, please leave them in the comments and I will update these posts accordingly.

Baseline Performance for AWS Lambda .NET using Top-Level Statements

Steve Bjorg — Wed, 12 Oct 2022 18:03:35 +0000

.NET 6 introduced top-level statements, which simplify the entry point of the application code. Unlike the previous style of Lambda definitions, this project creates an executable instead of an assembly. That means we have to provide our own Lambda host implementation. Fortunately, AWS already provides one in the Amazon.Lambda.RuntimeSupport package.

Minimal Top-Level Lambda Function

The MinimalTopLevel project defined a Lambda function that takes a stream and returns an empty response. It has no business logic and only includes required libraries. There is also no deserialization of a payload. This is the Lambda function using top-level statements with the least amount of overhead.

using Amazon.Lambda.Core;
using Amazon.Lambda.RuntimeSupport;
using Amazon.Lambda.Serialization.SystemTextJson;

await LambdaBootstrapBuilder.Create(Handler, new DefaultLambdaJsonSerializer())
    .Build()
    .RunAsync();

Task Handler(Stream request, ILambdaContext context)
    => Task.CompletedTask;

Benchmark Data for .NET 6 on x86-64

Again, we see that the duration of the INIT phase is not impacted until we exceed the 3,008 MB threshold, which also drives up cost.

However, compared to the Minimal baseline project, this Lambda function has 20% to 100% longer cold start durations.

Memory Size	Init	Cold Used	Total Cold Start	Total Warm Used (100)	Cost (µ$)
128MB	235.420	1,484.898	1,720.318	367.174	24.05849
256MB	236.617	744.375	980.992	151.328	23.93210
512MB	236.002	353.587	589.589	120.821	24.15341
1024MB	238.403	163.425	401.828	115.392	24.84696
1769MB	234.304	96.894	331.198	115.843	26.32520
5120MB	216.870	92.632	309.502	117.367	37.69996

Fullsize Image

Minimum Cold Start Duration

Fortunately, the situation improves quite a bit once we look at the optimal configuration for minimum cold start duration. Using top-level statements shift some of the overhead from the INIT phase to the first INVOKE phase, but otherwise the total duration is very close to what was measured for the Minimal baseline project.

Architecture	Memory Size	Tiered	Ready2Run	PreJIT	Init	Cold Used	Total Cold Start
arm64	5120MB	yes	yes	no	178.743	70.947	249.690
x86_64	5120MB	yes	no	no	190.382	67.500	257.882
x86_64	1769MB	yes	yes	no	189.152	58.625	247.777
x86_64	5120MB	yes	yes	no	176.368	57.039	233.407

Fullsize Image

Minimum Execution Cost

Again, the ARM64 architecture is the most cost-effective approach. ReadyToRun, Tiered Compilation, and the PreJIT settings all contribute to reduce cost a bit further for the minimal top-level project. That said, the minimum execution cost is ~8.5% higher when using top-level statements. This increased cost is most likely due to the higher memory configuration, which is required to compensate for the increased overhead of the INIT and first INVOKE phases.

Architecture	Memory Size	Tiered	Ready2Run	PreJIT	Init	Cold Used	Total Warm Used (100)	Cost (µ$)
arm64	1024MB	no	yes	no	234.381	168.534	122.581	24.08155
arm64	1024MB	no	yes	yes	252.593	167.558	124.272	24.09109
arm64	1024MB	yes	yes	no	202.952	124.406	152.314	23.88962
arm64	1024MB	yes	yes	yes	218.905	118.831	152.727	23.82079

Fullsize Image

What's Next

It's been fun diving into the fundamentals of AWS Lambda for .NET functions. The post is summarizing my findings and thoughts on future projects that might be interesting to explore.

The Surprising Cold Start Penalty in the AWS SDK for .NET

Steve Bjorg — Fri, 09 Sep 2022 21:07:17 +0000

This post is about raising awareness of a performance penalty when initializing the AWS SDK for .NET.

This is a startup tax incurred by all AWS Lambda functions using .NET. Fortunately, it's trivial to make it happen during the INIT phase where it's free. However, there is no way of avoiding it during a cold start. I can't help but think the initialization overhead should be much lower.

For this benchmark, our code is almost identical with the baseline function, except that we initialize an S3 client in the constructor.

using System.IO;
using System.Threading.Tasks;
using Amazon.S3;

namespace Benchmark.AwsSdk {

    public sealed class Function {

        //--- Fields ---
        private IAmazonS3? _s3Client;

        //--- Constructors ---
        public Function() {

            // initialize S3 client
            _s3Client = new AmazonS3Client();
        }

        //--- Methods ---
        public async Task<Stream> ProcessAsync(Stream request) {
            return Stream.Null;
        }
    }
}

Cold Start Durations

The following table shows the new measurements using .NET 6, Tiered Compilation, and ReadyToRun with 3 different memory configurations: 1,024 MB, 1,769 MB, and 5,120 MB.

Initializing the AWS SDK for .NET adds a 120+ ms penalty to the cold start duration. The impact remains the same, no matter the memory configuration since the initialization is happening during the INIT phase.

Architecture	Memory Size	Tiered	Ready2Run	PreJIT	Init	Cold Used	Total Cold Start	Penalty
arm64	1024MB	yes	yes	no	380.461	41.773	422.234	147.21
arm64	1769MB	yes	yes	no	377.036	29.365	406.401	141.856
arm64	5120MB	yes	yes	no	349.227	28.428	377.655	136.467
x86_64	1024MB	yes	yes	no	344.643	29.678	374.321	128.797
x86_64	1769MB	yes	yes	no	339.3	23.058	362.358	122.975
x86_64	5120MB	yes	yes	no	325.178	22.468	347.646	124.422

Potential Cause

Looking at the code for the AWS SDK for .NET, I suspect the culprit of the slow initialization is the endpoints.js file. This 700+ KB megalodon JSON file is parsed every time the AWS SDK is initialized. Since this happens in the AWS Core assembly, this penalty is incurred by all AWS service clients.

I hope this is something the AWS team fixes in the future. As a clueless layman, I would expect the endpoint definitions to belong to their respective packages. This would also produce a smaller Lambda deployment package as the current AWSSDK.Core.dll assembly size is an eyewatering 1.5 MB!

What's Next

In the next post, I'm benchmarking the impact of using top-level statements with Lambda functions. This new way of writing Lambda code is aesthetically pleasing, but does it have a hidden cost?

Benchmarking .NET JSON Serializers on AWS Lambda

Steve Bjorg — Wed, 31 Aug 2022 18:31:02 +0000

Virtually all .NET code on AWS Lambda has to deal with JSON serialization. Historically, Newtonsoft Json.NET has been the go-to library. More recently, System.Text.Json was introduced in .NET Core 3. Both libraries use reflection to build their serialization logic. The newest technique, called source generator, was introduced in .NET 6 and uses a compile-time approach that avoids reflection.

So, now we have three approaches to choose from, which begs the question: Is there a clear winner or is it more nuanced?

For these benchmarks, the code deserializes a fairly bloated JSON data structure taken from the GitHub API documentation and then returns an empty response.

Newtonsoft Json.NET

This library has been around for so long and has been so popular that it broke the download counter when it exceeded 2 billion on nuget.org. The counter has been fixed since, but this impressive milestone remains!

using System.IO;
using System.Threading.Tasks;
using Amazon.Lambda.Core;
using Amazon.Lambda.Serialization.Json;

[assembly:LambdaSerializer(typeof(JsonSerializer))]

namespace Benchmark.NewtonsoftJson {

    public sealed class Function {

        //--- Methods ---
        public async Task<Stream> ProcessAsync(Root request) {
            return Stream.Null;
        }
    }
}

Minimum Cold Start Duration

The 4 fastest cold start durations use the x86-64 architecture and ReadyToRun. The fastest uses Tiered Compilation as well. The PreJIT option is always slower when enabled, but still makes the top 4 cut.

Architecture	Memory Size	Tiered	Ready2Run	PreJIT	Init	Cold Used	Total Cold Start
x86_64	1769MB	no	yes	no	262.942	186.097	449.039
x86_64	1769MB	no	yes	yes	317.328	151.456	468.784
x86_64	1769MB	yes	yes	no	236.714	170.028	406.742
x86_64	1769MB	yes	yes	yes	295.209	137.727	432.936

Fullsize Image

Minimum Execution Cost

I'll admit, I was a bit surprised here. I would have expected ARM64 to be the obvious choice since the execution cost is 20% lower. However, that was not the case. Instead, we have a 50/50 split with x86-64 winning ever so slightly.

Also interesting is that the cheapest execution cost always uses the PreJIT option. That makes intuitively sense since this option shifts some cost from the first INVOKE phase to the free INIT phase and only has a small overhead penalty otherwise.

Similarly, Tiered Compilation is disabled for all because it introduces additional overhead during the warm INVOKE phases.

Most fascinating to me is that ARM64 is cheaper with 512 MB memory, while x86-64 is cheaper with 256 MB. This is probably just an oddity, but it serves to highlight that nothing is ever obvious and why benchmarking the actual code is so important!

Architecture	Memory Size	Tiered	Ready2Run	PreJIT	Init	Cold Used	Total Warm Used (100)	Cost (µ$)
arm64	256MB	no	yes	yes	346.884	1598.711	406.117	26.88279408
arm64	512MB	no	yes	yes	348.615	753.974	238.541	26.81680042
x86_64	256MB	no	yes	yes	317.574	1186.12	377.718	26.71600553
x86_64	512MB	no	yes	yes	314.298	562.768	234.544	26.84427746

Fullsize Image

System.Text.Json - Reflection

System.Text.Json was introduced in .NET Core 3. The initial release was not feature-rich enough to be a compelling choice. However, that is no longer the case. By .NET 5, all my concerns were addressed, and it has been my preferred choice since. Sadly, we had to wait until .NET 6, which is LTS, for it to become supported on AWS Lambda.

using System.IO;
using System.Threading.Tasks;
using Amazon.Lambda.Core;
using Amazon.Lambda.Serialization.SystemTextJson;

[assembly:LambdaSerializer(typeof(DefaultLambdaJsonSerializer))]

namespace Benchmark.SystemTextJson {

    public sealed class Function {

        //--- Methods ---
        public async Task<Stream> ProcessAsync(Root request) {
            return Stream.Null;
        }
    }
}

Minimum Cold Start Duration

Similar to Json.NET, the 4 fastest cold start durations use the x86-64 architecture. Unlike the previous benchmark, all of them have Tiered Compilation enabled. ReadyToRun provides an ever so slight benefit, but not much. That's likely due to the fact that the JSON serialization code lives in the .NET framework. Same as before, PreJIT makes things slower, but it's still among the 4 fastest configurations.

Architecture	Memory Size	Tiered	Ready2Run	PreJIT	Init	Cold Used	Total Cold Start
x86_64	1769MB	yes	no	no	231.55	97.37	328.92
x86_64	1769MB	yes	no	yes	276.791	74.063	350.854
x86_64	1769MB	yes	yes	no	226.864	93.64	320.504
x86_64	1769MB	yes	yes	yes	273.615	71.244	344.859

Fullsize Image

Minimum Execution Cost

Identical to the Json.NET benchmark, the 4 cheapest execution costs disable Tiered Compilation and enable the PreJIT option. Also, results are evenly split between ARM64 and x86-64.

Again, the optimal configuration uses the x86-64 architecture with ReadyToRun enabled. However, this time, all 4 optimal configurations agree on 256 MB for memory.

Architecture	Memory Size	Tiered	Ready2Run	PreJIT	Init	Cold Used	Total Warm Used (100)	Cost (µ$)
arm64	256MB	no	no	yes	335.019	977.84	344.601	24.60815771
arm64	256MB	no	yes	yes	330.424	966.123	347.232	24.57787356
x86_64	256MB	no	no	yes	302.287	688.363	341.735	24.49208483
x86_64	256MB	no	yes	yes	293.871	679.57	299.889	24.28108858

Fullsize Image

System.Text.Json - Source Generator

New in .NET 6 is the ability to generate the JSON serialization code during compilation instead of relying on reflection at runtime.

Personally, as someone who cares a lot about performance, I find source generators a really exciting addition to our developer toolbox. However, I don't consider this iteration to be production ready, because it is missing some features I rely on. In particular, the lack of custom type converters to override the default JSON serialization behavior is a blocker for me. That said, for some smaller projects, it might be viable. My biggest recommendation here is to thoroughly validate the output to ensure any behavior changes are caught during development.

using System.Text.Json.Serialization;
using Amazon.Lambda.Core;
using Amazon.Lambda.Serialization.SystemTextJson;
using Benchmark.SourceGeneratorJson;

[assembly: LambdaSerializer(typeof(SourceGeneratorLambdaJsonSerializer<FunctionSerializerContext>))]

namespace Benchmark.SourceGeneratorJson;

[JsonSerializable(typeof(Root))]
public partial class FunctionSerializerContext : JsonSerializerContext { }

public sealed class Function {

    //--- Methods ---
    public async Task<Stream> ProcessAsync(Root request) {
        return Stream.Null;
    }
}

Minimum Cold Start Duration

This time, the 4 fastest cold starts all use Tiered Compilation and ReadyToRun. Since source generators create more code to jit, it makes sense that these options improve cold start performance, since that's their purpose. Also, unlike the previous benchmarks, ARM64 and x86-64 are now competing for the top spot. PreJIT again slows things down a bit, but still makes it into the top 4.

Despite ARM64 finally making an appearance in the Minimum Cold Start Duration benchmark, the x86-64 architecture still secures the top two spots.

Architecture	Memory Size	Tiered	Ready2Run	PreJIT	Init	Cold Used	Total Cold Start
arm64	1769MB	yes	yes	no	249.244	65.429	314.673
arm64	1769MB	yes	yes	yes	276.097	60.221	336.318
x86_64	1769MB	yes	yes	no	240.88	53.104	293.984
x86_64	1769MB	yes	yes	yes	265.776	46.327	312.103

Fullsize Image

Minimum Execution Cost

The results for this benchmark are a bit more complicated to parse. For the first time, we don't have a symmetry across options. Instead, ARM64 secures the 3 out of 4 cheapest spots. The same is true for the PreJIT option and the 256 MB memory configuration.

Similar to the Json.NET benchmark, the cheapest configurations use ReadyToRun and, as for all execution cost benchmarks, Tiered Compilation is disabled.

Architecture	Memory Size	Tiered	Ready2Run	PreJIT	Init	Cold Used	Total Warm Used (100)	Cost (µ$)
arm64	256MB	no	yes	no	287.093	702.015	294.423	23.52147561
arm64	256MB	no	yes	yes	311.507	660.822	295.178	23.38668193
arm64	512MB	no	yes	yes	312.017	315.322	204.109	23.66288998
x86_64	256MB	no	yes	yes	294.279	519.965	298.581	23.61061349

Fullsize Image

Summary

Here are our observed lower bounds for the JSON serialization libraries, as well as the baseline performance on .NET 6 for comparison. I've omitted .NET Core 3.1 since I don't consider a viable target runtime anymore. However, you can explore the result set in the interactive Google spreadsheet.

Baseline for .NET 6
- Cold start duration: 223ms
- Execution cost: 21.94µ$
Newtonsoft Json.NET
- Cold start duration: 433 ms
- Execution cost: 26.72 µ$
System.Text.Json - Reflection
- Cold start duration: 321 ms
- Execution cost: 24.28 µ$
System.Text.Json - Source Generator
- Cold start duration: 294 ms
- Execution cost: 23.39 µ$

It shouldn't be a surprise that Json.NET, which has been around for a long time, has accumulated a lot of cruft. Json.NET is truly a Swiss army knife for serialization and this flexibility comes at a cost. It adds at least 210 ms to our cold start duration and it's also the most expensive JSON library to run.

The newer System.Text.Json library has a compelling performance and value benefit over Json.NET. It only adds 100 ms to our cold start duration and is 9% cheaper to run compared to Json.NET.

However, the clear winner is the new JSON source generator with only 70 ms in cold start overhead compared to our baseline. Cost is also 12% lower than Json.NET. That said, the lack of features may not make it a good choice just yet.

When it comes to minimizing cold start duration, the more memory, the better. These benchmarks used 1,769 MB, which unlocks most of the available vCPU performance, but not all of it. Full vCPU performance is achieved at 3,008 MB, which almost doubles the cost for a 10% improvement (source).

For minimizing cost, 256 MB seems to be the preferred choice. Tiered Compilation should never be used, but ReadyToRun is beneficial. The weird thing about this configuration is that ReadyToRun produces Tier0 code (i.e. dirty JIT without inlining, hoisting, or any of that delicious performance stuff). With Tiered Compilation disabled, our code will never be optimized further, as far as I know.

What's Next

For the next post, I'm going to investigate the overhead introduced by the AWS SDK. Since most Lambda functions will use it, I thought it would be useful to understand what the initialization cost is.

Baseline Performance for .NET on AWS Lambda

Steve Bjorg — Tue, 30 Aug 2022 15:29:55 +0000

I always like to understand what the lower bound looks like. What is the absolute fastest performance we can hope for? I find it insightful as it sets a baseline for everything else.

A necessary warning here is the risk of extrapolating too much from such a trivial sample. We need to take the data for what it is: a baseline. It's not representative of real-world business logic. Simply adding some I/O operations would greatly increase the processing time. Usually I/O is 1,000x to 1,000,000x slower than code.

Minimal Lambda Function

The Minimal project defines a Lambda function that takes a stream and returns an empty response. It has no business logic and only includes required libraries. There is also no deserialization of a payload. This is the Lambda function with the least amount of overhead.

using System.IO;
using System.Threading.Tasks;

namespace Benchmark.Minimal {

    public sealed class Function {

        //--- Methods ---
        public async Task<Stream> ProcessAsync(Stream request)
            => Stream.Null;
    }
}

Benchmark Data for .NET 6 on x86-64

The data neatly shows that the INIT phase is approximately the same for all memory configurations under the 3,008 MB threshold. As mentioned in the Anatomy of the AWS Lambda Lifecycle post, the INIT phase always runs at full speed.

The cold INVOKE phase is about 10x slower for 128 MB than it is for 1,024 MB. However, the sum of all warm INVOKE phases is only ~3x slower. Yet, the cost is less than 5% higher for the improved performance.

It's surprising that even for such a trivial example, we can already appreciate the delicate balance between performance and cost.

Memory Size	Init	Cold Used	Total Cold Start	Total Warm Used (100)	Cost (µ$)
128MB	235.615	620.921	856.536	365.519	22.25509
256MB	238.296	315.731	554.027	150.124	22.14107
512MB	241.193	136.89	378.083	124.686	22.37980
1024MB	239.972	60.804	300.776	115.53	23.13891
1769MB	241.005	37.623	278.628	116.322	24.63246
5120MB	218.112	37.009	255.121	119.559	33.24730

Fullsize Image

Minimum Cold Start Duration for .NET 6

Not surprisingly, the lowest cold start duration was achieved using the highest memory configuration. Tiered Compilation also helped lower the number. However, ReadyToRun did not make much of an impact, which is expected since our minimal project has almost no code.

More notable is that the ARM64 architecture was slower for comparable memory configurations than the x86-64 architecture.

Architecture	Memory Size	Tiered	Ready2Run	PreJIT	Init	Cold Used	Total Cold Start
arm64	5120MB	yes	no	no	211.006	30.165	241.171
x86_64	1024MB	yes	no	no	213.085	33.173	246.258
x86_64	1769MB	yes	no	no	215.754	24.164	239.918
x86_64	5120MB	yes	no	no	198.771	24.094	222.865

Fullsize Image

Minimum Execution Cost for .NET 6

Another unsurprising result is that the ARM64 architecture yields the lowest execution cost since its unit price is 20% lower. Similarly, the memory configuration is towards the bottom end at only 256 MB.

More interesting is that Tiered Compilation is always more expensive to operate. This makes intuitively sense since it requires additional processing time to re-jit code. After that, it's a bit of tossup between the ReadyToRun and PreJIT settings.

Architecture	Memory Size	Tiered	Ready2Run	PreJIT	Init	Cold Used	Total Warm Used (100)	Cost (µ$)
arm64	256MB	no	no	no	266.026	378.676	158.064	21.98914228
arm64	256MB	no	no	yes	288.025	371.274	161.529	21.97601788
arm64	256MB	no	yes	no	264.304	361.657	164.619	21.95426344
arm64	256MB	no	yes	yes	287.762	361.285	160.248	21.93844936

Fullsize Image

What about .NET Core 3.1?

I struggled if I should mention this since .NET Core 3.1 is reaching end-of-life in December 2022, but the performance delta for the baseline case is just staggering.

A Lambda function using .NET Core 3.1 with 512 MB is 40% faster on cold start than one using .NET 6 with 5,120 MB!

I'm just flabbergasted by this outcome. All I can do is remind myself that this baseline test is not representative of real-world code.

Architecture	Memory Size	Tiered	Ready2Run	PreJIT	Init	Cold Used	Total Cold Start
x86_64	512MB	yes	no	no	150.129	6.903	157.032
x86_64	1024MB	yes	no	no	148.376	6.081	154.457
x86_64	1769MB	yes	no	no	148.338	5.972	154.31

Fullsize Image

Similarly, execution cost is lower with .NET Core 3.1, but not as dramatically. Still, for .NET 6 there were just 4 configurations that achieved a cost under 22µ$. For .NET Core 3.1, there are 39 configurations under 21µ$!

Interestingly, the 4 lowest cost configurations follow a similar pattern: ARM64, 128 MB, no Tiered Compilation, and tossup for ReadyToRun and PreJIT.

Architecture	Memory Size	Tiered	Ready2Run	PreJIT	Init	Cold Used	Total Warm Used (100)	Cost (µ$)
arm64	128MB	no	no	no	162.366	102.693	110.096	20.55465044
arm64	128MB	no	no	yes	186.627	98.641	112.327	20.55161642
arm64	128MB	no	yes	no	161.989	88.677	110.391	20.53178133
arm64	128MB	no	yes	yes	185.923	85.289	117.811	20.53850086

Fullsize Image

Conclusion

Based on the benchmarks, we can establish these lower bounds.

For .NET 6:

Cold start duration: 223ms
Execution cost: 21.94µ$

For .NET Core 3.1:

Cold start duration: 154ms
Execution cost: 20.53µ$

Unless anything fundamental changes, we should not expect to do better than these baseline values.

What's Next

In the next post, I'm going to benchmark JSON serializers. Specifically, the popular Newtonsoft JSON.NET library, the built-in System.Text.Json namespace, and the new .NET 6 JSON source generators.

Benchmarking .NET on AWS Lambda

Steve Bjorg — Mon, 29 Aug 2022 17:50:11 +0000

My motivation for benchmarking all the compiler, deployment, and runtime options has been to feed my curiosity. I've seen various blog posts recommending one setting over another, but they never provided a justification.

Unfortunately, we can't use the outstanding BenchmarkDotNet tool with AWS Lambda. So, I built a benchmarking harness to collect data for all the deployment options, and hopefully, determine an "optimal" combination.

As the first blog post of this series mentioned, there are two distinct cases we can optimize for: "Minimize Cold Start Duration" or "Minimize Operating Cost". Now that the groundwork has been laid, we can formally capture what that means.

Optimal Strategies

To Minimize Cold Start Duration, we need to minimize the INIT and the first INVOKE phase. The optimal configuration yields the lowest duration, measured in milliseconds (ms), to process the request. For this measurement, we rely on the data reported by AWS Lambda in the logs. This is the same data that is used for billing and is the most accurate we have access to.
To Minimize Execution Cost, we need to minimize the sum of all INVOKE phases (cold and warm) while taking into account the Lambda memory configuration and CPU architecture. The optimal configuration yields the lowest execution cost for 1 cold start followed by 100 warm invocations. AWS Lambe execution has an extremely low unit cost. To make the number more intuitive, I opted to report execution cost in millionth of a dollar, or micro-Dollars (µ$).

Benchmarked Options

To leave no stone unturned, the benchmarking harness executes all possible permutations of the following options.

Tiered Compilation: On and Off
ReadyToRun: On and Off
Lambda Memory: 128 MB, 256 MB, 512 MB, 1024 MB, 1769 MB, and 5120 MB
CPU Architecture: x86-64 and ARM64
.NET Runtime: .NET Core 3.1 and .NET 6
Pre-JIT .NET: On and Off

These options produce 192 unique combinations that are benchmarked.

Benchmarking Approach

The Lambda function for each project is measured performing 100 cold starts. Each cold start is followed by 100 warm invocations. The results are then averaged.

I debated using the median or a percentile value instead of the average value. The challenge is when these values are further combined. For example, summing the p99 value for the INIT phase with p99 value of the first INVOKE phase doesn't seem right. Furthermore, outliers happen in real life. I'm hoping that 100 cold and warm invocations are sufficient to fairly represent what should be expected in real world situations.

That said, the raw measurements have been captured for each benchmark. That way, alternative analyses can be done without having to collect the data again.

Benchmarked Projects

My interest was in studying how the options are impacting the compute aspect of Lambda functions. Therefore, I opted to only benchmark projects that do not perform I/O operations.

Minimal Baseline

The Minimal project establishes a baseline for all projects. It has no business logic and only includes required libraries.

JSON Serializers

JSON serialization is necessary for virtually all Lambda functions. With .NET 6, there are 3 common approached to handle this task.

NewtonsoftJson: using Newtonsoft JSON.NET
SourceGeneratorJson: using .NET 6 source generators for JSON parsing
SystemTextJson: using System.Text.Json

AWS SDK

Most Lambda functions will interact with other AWS services via the AWS SDK. The AwsSdk project is used to benchmark the cost of initializing the SDK.

Top-Level Statements

Available starting in .NET 6, Lambda functions can use top-level statements instead of declaring a class. What is the performance impact when doing so?

AwsNewtonsoftJson: using AWS .NET SDK and Newtonsoft JSON.NET
SampleAwsNewtonsoftTopLevel: using AWS .NET SDK, Newtonsoft JSON.NET, and Top-Level Statements
SampleAwsSystemTextJsonTopLevel: using AWS .NET SDK, System.Text.Json, and Top-Level Statements

Minimal API

Also new in .NET 6 is a new approach to express ASP.NET routes using top-level statements. This sample was taken from the .NET 6 support on AWS Lambda announcement blog post.

SampleMinimalApi: using ASP.NET Core Minimal API

Benchmark Explorer

The results from the benchmarks have been compiled into an interactive Google spreadsheet. Feel free to explore the data anyway you like and draw your own conclusions. Any feedback on improving the analysis or visualization is welcome.

What's Next

Finally, we can dive into the results and see what new insights we can gain! First up is the baseline performance measurement. While this is not critical for production code, it gives us a foundation to work on.

Compile, Deploy, and Runtime Performance Options

Steve Bjorg — Tue, 09 Aug 2022 15:39:32 +0000

There are lots of options that impact performance. I actually found it a bit overwhelming, which is why I wanted to benchmark all the possible combinations.

C# Compiler Options

Tiered Compilation

This is a .NET runtime option that can be set at compile time. It instructs the .NET runtime to perform a dirty JIT (called Tier0) that is generated faster but leads to less performant code. If the code is run often enough, it is replaced by an optimized version (called Tier1) later on.

Without Tiered Compilation, the jitter emits all code as Tier1. Optimizing start-up code can be wasteful, especially if the code is only run once. With this option enabled, the jitter waits 100ms before it starts optimizing methods that invoked 30 times or more. This means that the time savings gained during the Lambda cold start impact subsequent warm invocations.

The whole process is quite complex and fascinating. For more details, check out the Tiered Compilation specification.

ReadyToRun

This .NET compiler option instructs the compiler to include pre-jitted code in the produced assembly. Note this option can increase the assembly size by 200% to 300%.

During startup, the runtime uses the pre-jitted code, but only when it matches the CPU architecture of the execution environment. The pre-jitted code is not optimized and equivalent to that of a dirty JIT (Tier0). When Tiered Compilation is also enabled, the pre-jitted code is eventually optimized when it is invoked often enough.

For more details, check out the official page about ReadyToRun Compilation.

Lambda Function Options

Memory

Performance of a Lambda execution environment is directly tied to its memory configuration. However, the relationship is not linear. Single-threaded performance maxes out at 3,008 MB, which provides 100% capacity of 2 vCPU cores. After that, additional fractional cores are added until the maximum of 10,240 MB is reach, which provides 6 cores.

For an in-depth analysis for Lambda memory configuration and the impact on performance, check out Optimizing Lambda Cost with Multi-Threading.

An important detail is that performance is boosted during the INIT phase of the execution environment. It makes no difference if the Lambda function is configured for 128 MB or 3,008 MB. In both cases, the duration of the INIT phase will be the same and perform as if the Lambda had been configured for 3,008 MB. Only if it exceeds that threshold, will the INIT phase run faster, assuming it can use more than two cores.

CPU Architecture

AWS Lambda supports two CPU architectures, depending on the region: x86 64-bit and ARM64. Cost for ARM64 is 20% lower than x86 for the same memory configuration. This makes it a very appealing choice, when available.

As of this writing, the following regions don't yet support ARM64 for Lambda.

US West - Northern California
Africa - Cape Town
Asia Pacific - Hong Kong, Jakarta, Osaka, and Seoul
Canada - Central
Europe - Milan, Paris, and Stockholm
Middle East - Bahrain
South America - Sao Paulo
AWS GovCloud - US-East, and US-West

.NET Runtime

At this time, only .NET Core 3.1 and .NET 6 runtimes can be used for new function deployments. However, in typical AWS fashion, old functions continue run. I can attest to that, as I still have some old .NET Core 1.0 functions chugging away.

Beware that .NET Core 3.1 will reach end-of-life on December 13th, 2022. At some point thereafter, it will not be possible to create new .NET Core 3.1 functions.

.NET Host for AWS Lambda

Pre-JIT .NET - `AWS_LAMBDA_DOTNET_PREJIT`

When this environment variable is set to "Always", it instructs the .NET host for AWS Lambda to prepare code during the INIT phase of the execution environment rather than to wait for the INVOKE phase.

It's default use-case is for Provisioned Concurrency, which allows one or more Lambda execution environments to be pre-initialized to avoid cold starts. However, it can also be set to always perform the code preparation.

The interesting property of this environment variable is that it moves some the code jitting overhead from the INVOKE phase to the INIT phase. The INIT phase always runs at the performance level of a 3,008 MB memory configuration, unless set higher. In addition, the INIT phase is also free of charge, unless it exceeds 10 seconds.

What's Next

In the next post, I'm covering how the benchmarking was performed methodology.

Anatomy of the AWS Lambda Lifecycle

Steve Bjorg — Mon, 08 Aug 2022 15:40:37 +0000

Before we dive into the nitty gritty, let's define some AWS Lambda terminology. Feel free to skip to the next post and come back to it later if need be.

AWS Lambda

This is the name of the AWS service that allows us to run code in a serverless manner. That means, we don't have to think about servers, virtual or physical. No need to worry about backups, patching, or maintenance time (woohoo!).

Lambda Function

This the definition of our deployment. It contains configuration options and a zip package with our code. However, this is not running code, just the definition for it.

Lambda Execution Environment

This is a runnable instance of our code that has been deployed into a secure, isolated runtime environment using the configuration of our Lambda Function. Additional execution environments get created as requests come in and we need more capacity to handle them. AWS Lambda will create them as needed and shut them down when they idle for too long. We can have one execution environment, 100s of them, or none. All managed automatically. Each execution environment gets the amount of memory and processing power assigned to it by its definition, as well as 512 MB of ephemeral storage. It's important to note that these execution environments are completely independent of each other as they share absolutely no state.

For a more detailed description, check out the official page about the AWS Lambda execution environment.

Lambda Invocation

This is an invocation of one of our Lambda execution environments in response to a request. Although an execution environment can handle multiple invocations in a row, it only ever handles one invocation at a time. Hence our code does not need to be thread-safe (unless we explicitly make it multi-threaded). Since the execution environment is reused between invocations, state in the ephemeral storage is shared from one invocation to the next. That means, if we're not careful, we can run out of ephemeral storage. Similarly, in-memory state is also kept between consecutive invocations. This allows us to initialize components we can reuse for the lifetime of the execution environment. An invocation is instantaneously aborted if it exceeds its duration or memory limits, as defined by the Lambda function. It's a situation we should strive to avoid! Finally, it's important to note that the execution environment is suspended, along with any background threads, as soon as our code returns a response to the requests.

For more details, check out this fantastic write-up on understanding AWS Lambda scaling and throughput.

Lifecycle & Billing

The following graphic from the AWS Lambda execution environment documentation.

The billing of a Lambda invocation is based on the duration of the INVOKE phase only. The INIT phase is not billed unless provisioned concurrency is used, or the INIT phase exceeds 10s.

Check out this excellent write-up on the INIT phase billing for more details.

Additionally, the performance of the INIT phase is boosted for low memory configurations. Thus, unless the Lambda function is given a high memory limit (e.g. 5GB), the INIT phase takes the same amount of time for all configurations.

What's Next

The next post explores the compilation and deployment options that impact performance.

Optimal Strategies for .NET on AWS Lambda

Steve Bjorg — Mon, 08 Aug 2022 15:39:00 +0000

I care a lot about performance. Making my code faster makes me happy and the metric for success is trivial. It's especially trivial in AWS Lambda since execution time is automatically reported. There's also a financial incentive as it is tied to the billing model.

And so, I've been wondering: Is there an optimal strategy for my .NET code on AWS Lambda?

AWS Lambda is its own little beast to understand and master. There are a variety of deployment options available that impact performance. Combined with the various compilation and runtime options of .NET code, it presented an interesting challenge. There is also the fundamental question of what does "optimal" mean?

Well ⟪spoiler alert⟫ there isn't just one "optimal" strategy! Instead, we have to choose what we want to optimize for. I settled on examining two strategies. Both have their place and it's rarely (never?) possible to achieve both. Note that if some of the terminology is unfamiliar, fear not. I'm going to explain each concept as we dive into the material. And, if I miss something, please let me know in the comments and I'll amend the posts.

Strategy 1: Minimize Cold Start Duration

This strategy seeks to achieve the shortest response time in the worst-case scenario: a cold start of our AWS Lambda function. In this case, a new execution environment must be created to handle our request. The execution environment has to be initialized, our code has to be loaded into it, and then our logic needs to be run to produce a response. All of this takes additional time when compared to subsequent requests. Correspondingly, this strategy trades faster initialization for absolute performance in later invocations, which may impact execution cost.

Strategy 2: Minimize Execution Cost

This strategy minimizes the execution cost of the AWS Lambda instance, including cold start followed by some threshold of warm invocations. In this case, we are willing to have slower cold starts for better performance once the everything is warmed up. This strategy leverages one of the lesser-known quirks of the AWS Lambda billing model works (more on this later). This strategy works best when we know that our code will be invoked frequently. For the purpose of this study, I assume that our Lambda function is invoked at 100 times after a cold start.

Choosing a Strategy

The preferred strategy depends on the purpose of our code. For handling synchronous invocations, such as API requests, minimizing cold starts can be preferrable. Especially when a human is waiting at the other end. But for asynchronous invocations, such as with EventBridge, then minimizing execution cost is more important.

What's Next

In the next post, I'm going to dive into the lifecycle of an AWS Lambda instance and associated terminology. I'll then cover some of the compilation and runtime options for .NET code. Then I'll introduce the benchmarking methodology. Finally, I'll present my findings and conclusion. As with all studies, peer review and independent confirmation is critical. Therefore, all my code is also made available under a permissive open-source license in the LambdaSharp.Benchmark GitHub repository.

Disclaimer: This is stating the obvious, but please check the date of this post. If it's older than 3 years, it's probably out-of-date!

CloudWatch Logging for Web Apps (Part 3)

Steve Bjorg — Wed, 23 Sep 2020 23:16:29 +0000

In the previous two posts, I covered the CloudFormation template for creating a REST API to log to CloudWatch from a frontend app and then the communication protocol of the REST API. This final post in the series covers the implementation of the frontend client to use the logging REST API.

Overview

The frontend client is responsible for creating a CloudWatch log stream for each app session and then writing log messages to it in sequential batches. Each batch must include a sequence number from the previous batch, unless it is the first batch.

For Blazor WebAssembly apps built with LambdaSharp, the implementation of the client resides in the LambdaSharpAppClient class. The class is instantiated as a singleton and can be injected either directly or indirectly via the ILogger interface. Only one of the following statement is needed for the frontend app to log to CloudWatch. The choices comes down to personal preference.

@inject LambdaSharpAppClient AppClient
@inject ILogger<Index> Logger

Implementation

CloudWatch Logs organizes log entries into log streams. A log stream is a chronological sequence of entries. Many log streams can exist concurrently. For Blazor WebAssembly apps--and other single-page apps--it is recommended to create a log stream on-demand when the first log message is generated.

Note that the code in this post is modified for simplicity. The actual implementation covers a few more edge-cases that would be distracting. The complete implementation can be found here.

Sending Log Messages

The app client queues messages into an internal accumulator. This enables the implementation to send multiple messages at once to avoid being throttled.

private readonly List<PutLogEventsRequestEntry> _logs = new List<PutLogEventsRequestEntry>();
private void SendMessage(string message) {

    // queue message for server-side logging
    _logs.Add(new PutLogEventsRequestEntry {
        Timestamp = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(),
        Message = message ?? throw new ArgumentNullException(nameof(message))
    });
}

Timed Accumulator

The accumulator is checked every second by a timer for pending messages. The timer callback first makes sure that any previous asynchronous operation is completed. It then attempts to flush any accumulated messages.

private Task _previousOperationTask;
private void OnTimer(object _) {
    if(!(_previousOperationTask?.IsCompleted ?? true)) {

        // previous operation is still going; wait until next timer invocation to proceed
        return;
    }

    // initialize invocation to FlushAsync(), but don't wait for it to finish
    _previousOperationTask = FlushAsync();
}

Batch Sending

Before the first batch of accumulated messages can be sent, the client must ensure that a log stream was created. It then chunks the accumulated messages into batches constrained to 1MB in size or 10,000 messages, whichever is lower. If the operation fails, because the client is possibly offline, the messages are inserted back into the accumulator.

private string _logStreamName;
private string _sequenceToken;
private async Task FlushAsync() {

    // check if any messages are pending
    if(!_logs.Any()) {
        return;
    }

    // check if a log stream must be created
    if(_logStreamName == null) {
        _logStreamName = AppInstanceId;
        var response = await CreateLogStreamAsync(new CreateLogStreamRequest {
            LogStreamName = _logStreamName
        });
        if(response.Error != null) {
            Console.WriteLine($"*** ERROR: unable to create log stream: {_logStreamName} (Error: {response.Error})");
            return;
        }
    }


    // NOTE (2020-08-06, bjorg): we limit the number of log message we send in the unlikely event that we have too many
    //  See: https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutLogEvents.html
    const int MaxPayloadSize = 1_048_576;
    const int MaxMessageCount = 10_000;

    // consume as many accumulated log messages as possible
    var payloadByteCount = 0;
    var logs = _logs.Take(MaxMessageCount).TakeWhile(log => {
        var logMessageByteCount = Encoding.UTF8.GetByteCount(log.Message) + 26;
        if((payloadByteCount + logMessageByteCount) >= MaxPayloadSize) {
            return false;
        }
        payloadByteCount += logMessageByteCount;
        return true;
    }).ToList();
    _logs.RemoveRange(0, logs.Count);

    // send log messages to CloudWatch Logs
    try {
        var response = await PutLogEventsAsync(new PutLogEventsRequest {
            LogStreamName = _logStreamName,
            LogEvents = logs,
            SequenceToken = _sequenceToken
        });

        // on error, re-insert the log messages and try again later
        if(response.Error != null) {
            _logs.InsertRange(0, logs);
            return;
        }
        _sequenceToken = response.NextSequenceToken;
    } catch {

        // on exception, re-insert the log messages and try again later
        _logs.InsertRange(0, logs);
    }
}

Disposal

Finally, the client performs a final flush operation when being disposed to ensure that all pending messages in the accumulator are sent to CloudWatch.

async ValueTask IAsyncDisposable.DisposeAsync() {

    // stop timer and wait for any lingering timer operations to finish
    await _timer.DisposeAsync();

    // wait for any in-flight operation to complete
    if(!(_previousOperationTask?.IsCompleted ?? true)) {
        await _previousOperationTask;
    }

    // flush all remaining messages
    while(_logs.Any()) {
        await FlushAsync();
    }
}

Conclusion

That's it. I hope you enjoyed this behind-the-scenes series of how LambdaSharp implemented CloudWatch Logs support for Blazor WebAssembly frontend apps. Hopefully you may find it useful in your own endeavors.

Happy Hacking!

CloudWatch Logging for Web Apps (Part 2)

Steve Bjorg — Tue, 15 Sep 2020 19:46:55 +0000

In the previous post, I covered the CloudFormation template for creating a REST API to log to CloudWatch from a frontend app. This post covers the communication protocol of the REST API.

Overview

Each app has a dedicated log group, created by the CloudFormation template, to make it easy to track all log messages for the app. Log groups contain log streams, which themselves contain chronologically ordered log entries.

The app is responsible for creating the log stream. For single page apps (SPA), such as Blazor WebAssembly, the log stream can be created when the app is loaded. Creating a log stream per app session has the benefit that the log entries show the chronological sequence of operations done by a user.

Once a log stream is created, the app can then send log entries to it. Log entries are sent as a batch operation. After the first batch, the CloudWatch API requires a sequence token for each subsequent batch. The sequence token is obtained in the response from the preceding batch operation.

I will leave the details of how to batch log entries for the next and final post. This post merely focuses on the protocol we will need to implement for it.

API Key

The CloudFormation template from the previous post created an API key using the CloudFormation stack ID to limit access to the REST API. This API key needs to be communicated to the frontend app via a JSON configuration file, for example. By basing the API key on the stack ID, the API key is different for each deployment, but ultimately a malicious actor could load the JSON file with the API key and spam the REST API. Unfortunately, for frontend apps, there is no technique to safely pass an API key without someone else getting a hold of it.

In LambdaSharp, the API key is made of two parts: the stack ID and the build GUID of the app assembly. For this tutorial, I skipped the build GUID part, because it can only be done reliably with tooling and only applies to .NET apps. Note the LambdaSharp approach does not make it safer, only harder for a third party to obtain the API key.

REST API

The logging REST API has two endpoints: one for creating a log stream and another for sending log entries to it.

POST:/.app/logs - Create Log Stream

The POST:/.app/logs endpoint creates a new log stream in the app log group. A log stream is a sequence of log events that originate from an app instance.

There is no limit on the number of log streams that can be created. There is a limit of 50 requests-per-second on this operations, after which requests are throttled.

The log stream name must match the following guidelines:

Log stream names must be unique within the log group.
Log stream names can be between 1 and 512 characters long.
The ':' (colon) and '*' (asterisk) characters are not allowed.

Request Syntax

{
   "logStreamName": "string"
}

Request Parameters

The request accepts the following data in JSON format.

logStreamName (required): The name of the log stream. Minimum length of 1. Maximum length of 512. Value must match pattern: [^:*]*

Success Response (HTTP Status Code: 200)

On success, the API responds with an empty JSON document.

{ }

Bad Request Response (HTTP Status Code: 400)

On a Bad Request response, the body contains a message describing why the request was rejected. Additional details can be found in the API logs when enabled.

Example: request body is missing required fields

{
    "error": "Invalid request body"
}

Example: request validation error

{
  "error": "1 validation error detected: Value \'\' at \'logStreamName\' failed to satisfy constraint: Member must have length greater than or equal to 1"
}

Internal Error Response (HTTP Status Code: 500)

On an Internal Error response, the body contains a generic message. The actual reason can be found in the API logs when enabled.

{
   "error": "Unexpected response from service."
}

PUT:/.app/logs - Put Log Messages

The PUT:/.app/logs endpoint uploads a batch of log messages to the specified log stream.

The request must include the sequence token obtained from the response of the previous call, unless it is the first request to a newly created log stream. Using the same sequenceToken twice within a narrow time period may cause both calls to be successful or one might be rejected.

The batch of events must satisfy the following constraints:

The maximum batch size is 1,048,576 bytes. This size is calculated as the sum of all event messages in UTF-8, plus 26 bytes for each log event.
None of the log events in the batch can be more than 2 hours in the future.
None of the log events in the batch can be older than 14 days or older than the retention period of the log group.
The log events in the batch must be in chronological order by their timestamp. The timestamp is the time the event occurred, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.
A batch of log events in a single request cannot span more than 24 hours. Otherwise, the operation fails.
The maximum number of log events in a batch is 10,000.
There is a quota of 5 requests per second per log stream. Additional requests are throttled. This quota cannot be changed.

Request Syntax

{
   "logEvents": [
      {
         "message": "string",
         "timestamp": number
      }
   ],
   "logStreamName": "string",
   "sequenceToken": "string"
}

Request Parameters

The request accepts the following data in JSON format.

logEvents (required): The log events. Minimum number of 1 item. Maximum number of 10,000 items.
- message (required): The raw event message. Minimum length of 1.
- timestamp (required): The time the event occurred, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC. Minimum value of 0.
logStreamName (required): The name of the log stream. Minimum length of 1. Maximum length of 512. Value must match pattern: [^:]
sequenceToken (optional): The sequence token obtained from the response of the previous call. An upload in a newly created log stream does not require a sequence token. Using the same sequenceToken twice within a narrow time period may cause both calls to be successful or one might be rejected.

Success Response (HTTP Status Code: 200)

On success, the API responds with a JSON document containing the sequence token for the next request.

{
  "nextSequenceToken": "49608818592289528730168753288679022865213175397425034930"
}

Bad Request Response (HTTP Status Code: 400)

Example: request body is missing required fields

{
    "error": "Invalid request body"
}

Example: request validation error

{
  "error": "1 validation error detected: Value \'\' at \'logStreamName\' failed to satisfy constraint: Member must have length greater than or equal to 1"
}

Example: The sequenceToken field is either missing or reusing a previous token value

{
  "error": "The given batch of log events has already been accepted. The next batch can be sent with sequenceToken: 49608818592289528730168753288679022865213175397425034930",
  "nextSequenceToken": "49608818592289528730168753288679022865213175397425034930"
}

Internal Error Response (HTTP Status Code: 500)

On an Internal Error response, the body contains a generic message. The actual reason can be found in the API logs when enabled.

{
   "error": "Unexpected response from service."
}

Conclusion - To be concluded...

In this post, we covered the communication protocol for a frontend apps to log to CloudWatch directly. In the next post, we will conclude this series with how to implement it in the frontend.

Happy Hacking!

CloudWatch Logging for Web Apps (Part 1)

Steve Bjorg — Thu, 10 Sep 2020 19:11:27 +0000

In this post, I'm describing how to replicate the LambdaSharp app capability to log to CloudWatch Logs using an Amazon API Gateway REST API.

Observability is a critical building block for developers. Therefore, it is an integral part of the LambdaSharp developer experience. For reference, this is how a Blazor WebAssembly app is created and automatically wired for CloudWatch logging in LambdaSharp.

Module: Sample.BlazorWebAssembly
Items:
  - App: MyBlazorApp

Yes, that is really it! Nothing additional is needed, but there are plenty of additional capabilities. However, non-LambdaSharp developers may want to achieve the same capability for their apps using their preferred framework. That is the purpose of this post. It shows how to build the CloudWatch logging capability for any frontend app using any framework.

Overview

This implementation does not use any Lambda functions. Instead, we enable logging to CloudWatch by directly integrating the API Gateway REST API with the CloudWatch Logs API using Apache Velocity templates. This design means there is only minimal code involved, no Lambda cold-start latencies, and no Lambda invocation costs.

The implementation is described in terms of CloudFormation resources using the YAML notation, but the same outcome can be achieved by using AWS Console instead.

Logging REST API

First, we need to create a new API Gateway resource. We define the top-level .app resource to anchor our API. In LambdaSharp, this top-level resource name is configurable via CloudFormation parameters, which are omitted here.

RestApi:
  Type: AWS::ApiGateway::RestApi
  Properties:
    Name: !Sub "${AWS::StackName} App API"

RestApiAppResource:
  Type: AWS::ApiGateway::Resource
  Properties:
    RestApiId: !Ref RestApi
    ParentId: !GetAtt RestApi.RootResourceId
    PathPart: .app

CloudWatch Log Group

The CloudWatch Log Group should be created explicitly for each app to make it is easy to distinguish them across apps. In addition, a log retention policy should be set to limit the amount of storage the log group uses to avoid being billed indefinitely for it.

LogGroup:
  Type: AWS::Logs::LogGroup
  Properties:
    RetentionInDays: 90

API Gateway IAM Role

API Gateway needs permission to create log streams in the log group and write to them. This is achieved by the following IAM role definition.

RestApiRole:
  Type: AWS::IAM::Role
  Properties:
    AssumeRolePolicyDocument:
      Version: 2012-10-17
      Statement:
        - Sid: ApiGatewayPrincipal
          Effect: Allow
          Principal:
            Service: apigateway.amazonaws.com
          Action: sts:AssumeRole
    Policies:
      - PolicyName: ApiLogsPolicy
        PolicyDocument:
          Version: 2012-10-17
          Statement:
            - Sid: LogGroupPermission
              Effect: Allow
              Action:
                - logs:CreateLogStream
                - logs:PutLogEvents
              Resource:
                - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:${LogGroup}"
                - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:${LogGroup}:log-stream:*"

API Validation

A neat feature of API Gateway REST API is that it can validate requests against a JSON schema model. This capability prevents unnecessary invocations of the API when the incoming payload is not valid. Validation is enabled by associating each API Gateway method with the following validator declaration.

RestApiValidator:
  Type: AWS::ApiGateway::RequestValidator
  Properties:
    RestApiId: !Ref RestApi
    ValidateRequestBody: true
    ValidateRequestParameters: true

REST API

This next section is a bit heavy, because how API Gateway resources, methods, and integrations are built.

First, we create the logs resource associated with the API methods.

RestApiAppLogsResource:
  Type: AWS::ApiGateway::Resource
  Properties:
    RestApiId: !Ref RestApi
    ParentId: !Ref RestApiAppResource
    PathPart: "logs"

Next, we need to create the OPTIONS method to handle CORS requests. Note this implementation uses Allow-Origin: '*', which should be replaced with the actual host scheme and name from which the application is served. LambdaSharp uses CloudFormation parameters to make it configurable, but these were omitted for brevity.

RestApiAppLogsResourceOPTIONS:
  Type: AWS::ApiGateway::Method
  Properties:
    AuthorizationType: NONE
    RestApiId: !Ref RestApi
    ResourceId: !Ref RestApiAppLogsResource
    HttpMethod: OPTIONS
    Integration:
      IntegrationResponses:
        - StatusCode: 204
          ResponseParameters:
            method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
            method.response.header.Access-Control-Allow-Methods: "'OPTIONS,POST,PUT'"
            method.response.header.Access-Control-Allow-Origin: "'*'"
            method.response.header.Access-Control-Max-Age: "'600'"
          ResponseTemplates:
            application/json: ''
      PassthroughBehavior: WHEN_NO_MATCH
      RequestTemplates:
        application/json: '{"statusCode": 200}'
      Type: MOCK
    MethodResponses:
      - StatusCode: 204
        ResponseModels:
          application/json: 'Empty'
        ResponseParameters:
            method.response.header.Access-Control-Allow-Headers: false
            method.response.header.Access-Control-Allow-Methods: false
            method.response.header.Access-Control-Allow-Origin: false
            method.response.header.Access-Control-Max-Age: false

Once the browser is authorized via the OPTIONS request, we need to provide two additional endpoints: one for creating a new log stream using POST and another for writing to a log stream using PUT. In addition, we define a JSON schema model for each endpoint to validate requests before they are executed.

Note that the app is responsible for creating a new log stream. For single page apps (SPA), a new log stream should be created each time the app loads. This is also the behavior for Blazor WebAssembly apps built with LambdaSharp.

Create LogStream - POST:/.app/logs

The POST method creates a new log stream in the associated log group. Some of the response handling relates to how errors are returned to the calling application. Emphasis of the handling is on providing useful feedback without revealing too many internal details.

Similar to the OPTIONS method, this configurations uses Allow-Origin: '*', which should be replaced with the actual host scheme and name from which the application is served. LambdaSharp uses CloudFormation parameters to make it configurable, but these were omitted for brevity.

RestApiAppLogsResourcePOST:
  Type: AWS::ApiGateway::Method
  Properties:
    OperationName: CreateLogStream
    ApiKeyRequired: true
    RestApiId: !Ref RestApi
    ResourceId: !Ref RestApiAppLogsResource
    AuthorizationType: NONE
    HttpMethod: POST
    RequestModels:
      application/json: !Ref RestApiAppLogsResourcePOSTRequestModel
    RequestValidatorId: !Ref RestApiValidator
    Integration:
      Type: AWS
      IntegrationHttpMethod: POST
      Uri: !Sub "arn:${AWS::Partition}:apigateway:${AWS::Region}:logs:action/CreateLogStream"
      Credentials: !GetAtt RestApiRole.Arn
      PassthroughBehavior: WHEN_NO_TEMPLATES
      RequestParameters:
        integration.request.header.Content-Type: "'application/x-amz-json-1.1'"
        integration.request.header.X-Amz-Target: "'Logs_20140328.CreateLogStream'"
      RequestTemplates:
        application/json: !Sub |-
          #set($body = $input.path('$'))
          {
            "logGroupName": "${LogGroup}",
            "logStreamName": "$body.logStreamName"
          }
      IntegrationResponses:
        - SelectionPattern: "200"
          StatusCode: 200
          ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: "'*'"
          ResponseTemplates:
            application/x-amz-json-1.1: |-
              { }

        - SelectionPattern: "400"
          StatusCode: 400
          ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: "'*'"
          ResponseTemplates:
            application/x-amz-json-1.1: |-
              #set($body = $input.path('$'))
              {
              #if($body.message.isEmpty())
                "error": "Unknown error"
              #else
                "error": "$util.escapeJavaScript($body.message).replaceAll("\\'","'")"
              #end
              }

        - StatusCode: 500
          ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: "'*'"
          ResponseTemplates:
            application/x-amz-json-1.1: |-
              {
                "error": "Unexpected response from service."
              }

    MethodResponses:
      - StatusCode: 200
        ResponseModels:
          application/json: Empty
        ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: false

      - StatusCode: 400
        ResponseModels:
          application/json: Empty
        ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: false

      - StatusCode: 500
        ResponseModels:
          application/json: Empty
        ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: false

RestApiAppLogsResourcePOSTRequestModel:
  Type: AWS::ApiGateway::Model
  Properties:
    Description: CreateLogStream
    ContentType: application/json
    RestApiId: !Ref RestApi
    Schema:
      $schema: http://json-schema.org/draft-04/schema#
      type: object
      properties:
        logStreamName:
          type: string
      required:
        - logStreamName

Append to Log Stream - POST .app/logs

Similar to the POST method, the PUT method validates incoming requests and limits what internal details are exposed when errors occur.

RestApiAppLogsResourcePUT:
  Type: AWS::ApiGateway::Method
  Properties:
    OperationName: PutLogEvents
    ApiKeyRequired: true
    RestApiId: !Ref RestApi
    ResourceId: !Ref RestApiAppLogsResource
    AuthorizationType: NONE
    HttpMethod: PUT
    RequestModels:
      application/json: !Ref RestApiAppLogsResourcePUTRequestModel
    RequestValidatorId: !Ref RestApiValidator
    Integration:
      Type: AWS
      IntegrationHttpMethod: POST
      Uri: !Sub "arn:${AWS::Partition}:apigateway:${AWS::Region}:logs:action/PutLogEvents"
      Credentials: !GetAtt  RestApiRole.Arn
      PassthroughBehavior: WHEN_NO_TEMPLATES
      RequestParameters:
        integration.request.header.Content-Type: "'application/x-amz-json-1.1'"
        integration.request.header.X-Amz-Target: "'Logs_20140328.PutLogEvents'"
        integration.request.header.X-Amzn-Logs-Format: "'json/emf'"
      RequestTemplates:
        application/json: !Sub |-
          #set($body = $input.path('$'))
          {
            "logEvents": [
          #foreach($logEvent in $body.logEvents)
                {
                  "message": "$util.escapeJavaScript($logEvent.message).replaceAll("\\'","'")",
                  "timestamp": $logEvent.timestamp
                }#if($foreach.hasNext),#end
          #end
            ],
            "logGroupName": "${LogGroup}",
            "logStreamName": "$body.logStreamName",
            "sequenceToken": #if($body.sequenceToken.isEmpty()) null#else "$body.sequenceToken"#end
          }
      IntegrationResponses:
        - SelectionPattern: "200"
          StatusCode: 200
          ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: "'*'"
          ResponseTemplates:
            application/x-amz-json-1.1: |-
              {
                "nextSequenceToken": "$input.path('$.nextSequenceToken')"
              }

        - SelectionPattern: "400"
          StatusCode: 400
          ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: "'*'"
          ResponseTemplates:
            application/x-amz-json-1.1: |-
              #set($body = $input.path('$'))
              #if($body.expectedSequenceToken.isEmpty())
              {
              #if($body.message.isEmpty())
                "error": "Unknown error"
              #else
                "error": "$util.escapeJavaScript($body.message).replaceAll("\\'","'")"
              #end
              }
              #else
              {
              #if($body.message.isEmpty())
                "error": "unknown error",
              #else
                "error": "$util.escapeJavaScript($body.message).replaceAll("\\'","'")",
              #end
                "nextSequenceToken": "$body.expectedSequenceToken"
              }
              #end

        - StatusCode: 500
          ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: "'*'"
          ResponseTemplates:
            application/x-amz-json-1.1: |-
              {
                "error": "Unexpected response from service."
              }

    MethodResponses:
      - StatusCode: 200
        ResponseModels:
          application/json: Empty
        ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: false

      - StatusCode: 400
        ResponseModels:
          application/json: Empty
        ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: false

      - StatusCode: 500
        ResponseModels:
          application/json: Empty
        ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: false

RestApiAppLogsResourcePUTRequestModel:
  Type: AWS::ApiGateway::Model
  Properties:
    Description: PutLogEvents
    ContentType: application/json
    RestApiId: !Ref RestApi
    Schema:
      $schema: http://json-schema.org/draft-04/schema#
      type: object
      properties:
        logEvents:
          type: array
          items:
            - type: object
              properties:
                message:
                  type: string
                timestamp:
                  type: integer
              required:
                - message
                - timestamp
        logStreamName:
          type: string
        sequenceToken:
          type:
            - string
            - "null"
      required:
        - logEvents
        - logStreamName

API Key & Usage Plan

The following resources declare an API key and usage plan. The API key is set by default to the Base64 value of the CloudFormation stack GUID. It is recommended to explicitly set the API key since the frontend app will need access to it to use the logging REST API. The API key can be further obfuscated by combining it with an internal value of the app. In LambdaSharp, the API key is generated by combining the CloudFormation stack GUID and the compiled .NET Core assembly identifier GUID.

RestApiKey:
  Type: AWS::ApiGateway::ApiKey
  Properties:
    Description: !Sub "${AWS::StackName} App API Key"
    Enabled: true
    StageKeys:
      - RestApiId: !Ref RestApi
        StageName: !Ref RestApiStage
    Value:
      Fn::Base64: !Select [ 2, !Split [ "/", !Ref AWS::StackId ]]

RestApiUsagePlan:
  Type: AWS::ApiGateway::UsagePlan
  Properties:
    ApiStages:
      - ApiId: !Ref RestApi
        Stage: !Ref RestApiStage
    Description: !Sub "${AWS::StackName} App API Usage Plan"
    Throttle:
      BurstLimit: 200
      RateLimit: 100

RestApiUsagePlanKey:
  Type: AWS::ApiGateway::UsagePlanKey
  Properties:
    KeyId: !Ref RestApiKey
    KeyType: API_KEY
    UsagePlanId: !Ref RestApiUsagePlan

API Deployment

Finally, we define a stage called LATEST, which is used by the deployment resource. Note that CloudFormation only runs the deployment once. Subsequent CloudFormation stack updates need to be manually deployed when the REST API changes. LambdaSharp uses the Finalizer construct to allow for configuration changes to be always applied automatically.

RestApiStage:
  Type: AWS::ApiGateway::Stage
  Properties:
    DeploymentId: !Ref RestApiDeployment
    Description: App API LATEST Stage
    RestApiId: !Ref RestApi
    StageName: LATEST

RestApiDeployment:
  Type: AWS::ApiGateway::Deployment
  Properties:
    Description: !Sub "${AWS::StackName} App API"
    RestApiId: !Ref RestApi
  DependsOn:
    - RestApiAppLogsResource
    - RestApiAppLogsResourcePOST
    - RestApiAppLogsResourcePOSTRequestModel
    - RestApiAppLogsResourcePUT
    - RestApiAppLogsResourcePUTRequestModel

Conclusion - To be continued...

In this post, we created the resources required to enable a frontend apps to log to CloudWatch directly. In the next post, we will cover the protocol for logging via this REST API.

Happy Hacking!

DEV Community: Steve Bjorg

Lessons Learned on Optimizing .NET on AWS Lambda

Conclusions

Tiered Compilation

ReadyToRun

Pre-JIT .NET

CPU Architecture

Future Work

Are all AWS Lambda regions the same?

Benchmarking with I/O Operations

.NET 7 Native

Self-Hosted .NET

More Pre-Jitting during INIT Phase

Custom Amazon.Lambda.RuntimeSupport Package

More C# Compiler Options

Profile-Guided Optimizations (PGO)

Parting Thoughts

Baseline Performance for AWS Lambda .NET using Top-Level Statements

Minimal Top-Level Lambda Function

Benchmark Data for .NET 6 on x86-64

Minimum Cold Start Duration

Minimum Execution Cost

What's Next

The Surprising Cold Start Penalty in the AWS SDK for .NET

Cold Start Durations

Potential Cause

What's Next

Benchmarking .NET JSON Serializers on AWS Lambda

Newtonsoft Json.NET

Minimum Cold Start Duration

Minimum Execution Cost

System.Text.Json - Reflection

Minimum Cold Start Duration

Minimum Execution Cost

System.Text.Json - Source Generator

Minimum Cold Start Duration

Minimum Execution Cost

Summary

What's Next

Baseline Performance for .NET on AWS Lambda

Minimal Lambda Function

Benchmark Data for .NET 6 on x86-64

Minimum Cold Start Duration for .NET 6

Minimum Execution Cost for .NET 6

What about .NET Core 3.1?

Conclusion

What's Next

Benchmarking .NET on AWS Lambda

Optimal Strategies

Benchmarked Options

Benchmarking Approach

Benchmarked Projects

Minimal Baseline

JSON Serializers

AWS SDK

Top-Level Statements

Minimal API

Benchmark Explorer

What's Next

Compile, Deploy, and Runtime Performance Options

C# Compiler Options

Tiered Compilation

ReadyToRun

Lambda Function Options

Memory

CPU Architecture

.NET Runtime

.NET Host for AWS Lambda

Pre-JIT .NET - AWS_LAMBDA_DOTNET_PREJIT

What's Next

Anatomy of the AWS Lambda Lifecycle

AWS Lambda

Lambda Function

Lambda Execution Environment

Lambda Invocation

Lifecycle & Billing

What's Next

Optimal Strategies for .NET on AWS Lambda

Strategy 1: Minimize Cold Start Duration

Strategy 2: Minimize Execution Cost

Pre-JIT .NET - `AWS_LAMBDA_DOTNET_PREJIT`