DEV Community: Bastian

Deploy a .NET Core Web API with AWS Lambda and the serverless framework

Bastian — Wed, 03 Oct 2018 20:16:32 +0000

One of the cool things you can do with lambdas triggered by API Gateway is that you can simply let your code handle the routing. This should work for all serverless stacks and I have already used it for nodejs (check out this dev.to article explaining how this works for nodejs).

Of course you could also go and define a lot of separate functions and create a new API Gateway Resource/Method for each. What I really like about running your entire application instead of function by function is the fact that you can firstly run existing apps without a huge change. You can actually simply run the same app in the lambda run time, in a container or by hitting F5...

Secondly you will not have to reinvent the wheel when it comes to standards you already have established in your .net core Web APIs e.g. health check endpoints, logging standards and all the other middleware you really appreciate.

There are already a bunch of tutorials out there that help you set up your .net core Web API with lambda. They do however use Visual Studio templates or .net core cli templates. The problem for me with this templates is that they don't rely on one tool that is a must for me when it comes to serverless - the serverless framework. So in this post I would like to explain how to set up a .net Core Web API on lambda with the serverless framework. This is not very complicated, but I will not go into detail about things covered in other tutorials.

The final result will be something like this:

Prerequisites

You should have a couple of things set up already

Creating the API

Create a new folder you want your code to reside in and run:

dotnet new webapi

You can test that it works locally by running the following commands and then opening http://localhost:5000/api/values:

dotnet restore
dotnet run

Adding the serverless framework to it

Add a file called serverless.yml with the following content:

service: exampleservice

provider:
  name: aws
  runtime: dotnetcore2.1
  region: eu-west-1

package:
  artifact: bin/release/netcoreapp2.1/deploy-package.zip

functions:
  api:
    handler: exampleservice::exampleservice.LambdaEntryPoint::FunctionHandlerAsync
    events:
     - http:
         path: /{proxy+}
         method: ANY

If you already are using the serverless framework you should be familiar with most of the file's content.
We are adding an API Gateway that has a wildcard path => {proxy+} and the handler points to a not yet existing function.
(Change the region to something that makes sense for you).

Adding additional dependencies

Open your csproj file and change it to something like this (don't forget to run dotnet restore afterwards):

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.App" Version="2.1.0" />
    <PackageReference Include="Amazon.Lambda.Core" Version="1.0.0" />
    <PackageReference Include="Amazon.Lambda.Serialization.Json" Version="1.4.0" />
    <PackageReference Include="Amazon.Lambda.AspNetCoreServer" Version="2.1.0" />
  </ItemGroup>

</Project>

! It is important to explicitly point to a specific version of Microsoft.AspNetCore.App. The dotnet new webapi command will not do that by default.

Add the lambda entry point (handler)

Additionally to your Program.cs file which handles the initialization of your app when you start it locally we need to add an handler for the lambda to hook into. Check the serverless.yml file we created earlier for reference. The handler in the serverless.yml has to be exactly correct -> {assemblyname}::{namespace}.{class}::FunctionHandlerAsync. Create a file called LambdaEntryPoint.cs:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Hosting;
using System.IO;

namespace exampleservice
{
    public class LambdaEntryPoint : Amazon.Lambda.AspNetCoreServer.APIGatewayProxyFunction
    {
        protected override void Init(IWebHostBuilder builder)
        {
            builder
                .UseStartup<Startup>();
        }
    }
}

Add build/package scripts for your application

Before you can package your lambda you will have to install the dotnet-lambda cli tool:

dotnet tool install --global Amazon.Lambda.Tools --version 3.0.1

for windows systems add a file called build.ps1 :

dotnet restore
dotnet lambda package --configuration release --framework netcoreapp2.1 --output-package bin/release/netcoreapp2.1/deploy-package.zip

if you run this in debian (e.g. docker image microsoft/dotnet) add build.sh :

#!/bin/bash

#install zip on debian OS, since microsoft/dotnet container doesn't have zip by default
if [ -f /etc/debian_version ]
then
  apt -qq update
  apt -qq -y install zip
fi

dotnet restore
dotnet lambda package --configuration release --framework netcoreapp2.1 --output-package bin/release/netcoreapp2.1/deploy-package.zip

Test the build by running the build script for your environment e.g.

.\build.ps1

! You will have to run this every time your code changes. Perfectly fine when you run this as part of a CI/CD pipe since you will have to execute that script anyways, but easy to forget if you just play around with it and deploy from your machine.

Publish your application!

This should be it! It is time to deploy the app. This assumes that you have followed the serverless install instructions and set up your AWS account (may mean that you have to adjust your serverless.yml file to make that work for you):

serverless deploy -v

This should result in something like this:

Copy the highlighted link, replace {proxy+} with api/values and you should get the same result as before! Congrats!

This was a quick write up! If you miss information for some of the steps please add a comment so I can be more clear.

The example repository can be found here: https://github.com/schwamster/lambda_webapi_serverless

Deploying a asp.net core container with Zeit "now"

Bastian — Thu, 11 May 2017 22:01:44 +0000

I just listened to a podcast hanleminutes about "now" from Zeit. The promise is, that you create new folder with an node.js app, a static website or any app that you can containerize with docker. You then just type "now" in the shell and your app gets deployed.

You can find the related code in this repo. If you would like to find out more about Asp.Net Core and docker check this tutorial out.

This sounds nice. So I gave it a try. Here is a short write up of what I did.

What you will need

Creating the project

In the commandline of your choice run the following:

mkdir aspnetcore-now
cd ./aspnetcore-now
dotnet new webapi
dotnet restore

We now have a sample web api. We can test if everything works locally by running:

dotnet run

Now point a browser to http://localhost:5000/api/values and you should see a sample response like this:

["value1","value2"]

Add a Dockerfile

FROM microsoft/aspnetcore-build
ENV ASPNETCORE_URLS=http://+:80
EXPOSE 80
COPY . /opt/app
WORKDIR /opt/app
RUN dotnet restore
CMD dotnet run

Also add a .dockerignore file to your project, so bin and obj are not part of your build context:

bin
obj

This is a pretty straight forward dockerfile. We copy our sources to the container, expose port 80, run dotnet restore to fetch the dependencies and finally set the default command to dotnet run to execute our application.
Yes, we are using aspnetcore-build as an image and not an optimized runtime image. The deployment will take longer, but firstly it is easier for the purpose of this article and secondly it helps work around some restrictions with the free OSS plan of now. I wanted everybody to be able to give this a try without having to use their credit card. But one of the restrictions that I quite frankly don't really understand, is that an individual file that is part of your container build context ( in this case only the very small source files), cannot be larger than 1 MB if you run in the free plan. If you build/publish your dotnet core project on your local machine or you use the build image to compile your app you will see that some files in the output are larger than 1 MB. So copying the compiled app into the runtime image (microsoft/aspnetcore) will not work, since now will then complain with the following errors:

There are 3 ways to work around the problem (maybe more):

Use the build image

Use the build image instead like we do it in the article. This is creating a much lager image though. I cannot really see the benefit for Zeit forcing me to do this. A maximum total image size or a maximum total build context would have made more sense in my eyes. Every tech stack is different, in some there are fewer larger in others there is a insane amount of very small files. Zeit should not drive away devs of stacks they did not have on their radar... Of course I also might be misunderstanding something and this restriction makes perfect sense.

Remove the large files

The offending files (refs/Microsoft.CodeAnalysis*) are actually not needed at runtime for this project - add the following to your csproj

Use a registry

Another way to work around this would be to build the image however you like, push it to e.g. hub.docker.com and then just "now" a simplified dockerfile:

FROM myusername/myapp
ENV ASPNETCORE_URLS=http://+:80
EXPOSE 80
CMD ["dotnet", "myapp.dll"]

This works fine but it kind of takes the fun away. The thing I really like about "now" is, that i don't have to worry about a build pipeline.

Back to the original plan. We just use the build-image, that includes runtime and sdk as stated in the first dockerfile above.

Install now

There are different ways to install now as described https://zeit.co/docs/getting-started/installing-now.

Here is how you can do it with npm:

npm install -g now

Deploy

We are now ready to deploy our application. All you need to do is run:

now

Follow the instructions. If this is the first time you run now, you will have to register with your email address.

In the beginning of the output you will find a url now creates for you.
Open that address in your browser and append /api/values. The result should be the same as running it locally.

Creating aliases

Every time we run now, we will create a new deployment with a new url. What is really helpful is to create aliases with more memorable and lasting urls or even custom domains. In this way, a new version can always be deployed side by side with the old one. Once happy with the result you can point the alias to the new deployment. Have a look at the docs to find out more.

Whats next?

Check out the cli to find out more about the nice features of now. Scale, Logs, Alias, DNS, Domains, Certs...

Summary

Except for the file size limitation that kind of sucks I can highly recommend "now". This is just a first impression though. I will try this for one of my pet projects and see how it goes. I am especially interested in finding out how the whole "we only offer cpu fronted by web apps" limitation will hinder me, since I am kind of used to have certain parts of the application hidden away inside the cloud providers eco system, inside private subnets, protected by policies...

Beginners Tutorial: Docker with ASP.NET Core

Bastian — Thu, 06 Apr 2017 13:23:35 +0000

Beginners Tutorial: Docker with ASP.NET Core

--updated to .net core 2.0--
--updated with multi-stage builds --

In this tutorial, you will learn how to build and run your first asp.net core docker image. We start of with a very short general docker introduction.
After that we choose the "right" images. We will first create a docker container that is responsible for building our source files. For that we copy our source files to the build container. When the build is done we will copy the published project back to the host system and create a runtime image. After that we explore the handy additon "multi-stage" build to simplify the build.

You can find all the related code here: github

Your will need to install dotnet core and docker on your machine before you begin this tutorial.

If you are running behind a proxy some of the commands might not work, so be sure to check out the Proxy-Section below.

The Dockerfile

If you already have basic knowledge of Docker skip this introduction and go straight to "Choose an image".

You can run one of the many images that exist ready for usage on hub.docker.com. You can for example
run a command on an instance of Debian a popular Linux Distro with the following command:

docker run debian echo "Welcome to Docker"

This might take a while the first time, since docker has to pull the image. A second run should start the command in a fraction of a second.

Instead of running a "throw away"-container you can also use an container interactively like so:

docker run -it debian /bin/bash

Check out the docker run reference to find out more: docker run

You can exit the container by typing "exit" and hitting enter.

But you can not only run other peoples images, you can also create your own images. For that you will need to create a Dockerfile. The Dockerfile describes an image and all its dependencies in steps.

We can start with a simple Dockerfile that extends our hello world example.

Create a new folder called cowsay and add a file called Dockerfile. Add the following content to the file:

FROM debian

RUN apt-get update && apt-get install -y cowsay

ENTRYPOINT ["/usr/games/cowsay"]

In this dockerfile we are doing the follwing:

defining what base image we want to use => debian
running a command in the image that updates the packagemanager and installs an app called cowsay
defining what app to run when the image is run

For a full reference of the available instructions in Dockerfile go here Dockerfile

Now let's build the image with the build command from the created folder:

docker build -t cowsay .

If this hangs and you are running behind a proxy check this out.

Now that we have build our image we can run it:

docker run cowsay "Welcome to Docker"

Choose an image

Go to hub.docker.com and search for aspnetcore
You will find many different choices. If there are no very special reasons i would opt for official images or images uploaded by the involved companies. Two images are interesting:

There are two different images provided by microsoft. One of them only contains the runtime and the other contains the SDK as well - see the following descriptions

ASP.NET Core Docker Image

This repository contains images for running published ASP.NET Core applications. These images use the
microsoft/dotnet image as its base.

ASP.NET Core Build Docker Image

This repository contains images that are used to compile/publish ASP.NET Core applications inside the container. This is different to compiling an ASP.NET Core application and then adding the compiled output to an image, which is what you would do when using the microsoft/aspnetcore image. These Dockerfiles use the microsoft/dotnet image as its base.

Create a asp.net core project

create a folder called docker-tutorial and navigate to it, then execute the following command:

dotnet new webapi

First Build

Let's start easy and compile the app on our computer and then add the output to the runtime image.

Run the following commands in the root of your project:

dotnet restore
dotnet publish -o ./publish

You should now have a publish folder, that contains your compiled application.

Now create a new Dockerfile in the root of the application

FROM microsoft/aspnetcore:2.0
WORKDIR /app
COPY ./publish .
ENTRYPOINT ["dotnet", "docker-tutorial.dll"]

This Dockerimage will copy the contents of the publish folder in the root of your project into the app folder on the image.

Build the image:

docker build -t docker-tutorial .

You can find out more about the build command here

Test the image:

docker run -p 8181:80 docker-tutorial

Now you can navigate to the hosted application: http://localhost:8181/api/values

You should get a response like this:

["value1","value2"]

Your docker engine might not be reachable through localhost. If so change to the correct url. If you
are using the docker toolbox with docker-machine you can get the ip with the following command:

docker-machine ip default

Compiling within the aspnetcore-build image

It is recommended to compile your project within the docker image, since this will produce a more reliable build pipeline. The build on the development machine will work the same way as the build in the build server.

So let's create another Dockerfile called Dockerfile.build

FROM microsoft/aspnetcore-build:2.0
WORKDIR /app

COPY *.csproj .
RUN dotnet restore

COPY . .
RUN dotnet publish --output /out/ --configuration Release

The new instruction we use here is COPY. This copies files from our host into the image.
Also note what happens when you rebuild the image. If you don't change anything nothing will be done. If you change something in the code the publish instruction will be executed but not dotnet restore. Only if you change some dependency will the dotnet restore instruction be executed.
For a more detailed description of this "layered" build process check this out.

Before we build the "build-image" we need to add one more file to avoid that dotnet commands on our host (dotnet restore/build) interfere with the build context. See this for more information. Add a file called .dockerignore with the following content to the root of the project:

bin
obj

Now let's build the image. Note we have to explicitly specify what Dockerfile we want to use:

docker build -f Dockerfile.build -t docker-tutorial-build .

We have now build the app in the image. We could now run the container from that image but run the following command first:

docker image ls | sls docker-tutorial

As you can see the build image is dramatically larger than the image we created before. This is because the build images has absolutely everything you need to build your images (SDK). We don't need that when we run our container. The solution is to create a runtime image.

So next step is to get the compiled app out of the build image. First we create the container with the create command. This is almost like docker run just that the container is never really started. We can however copy out the compiled app.

docker create --name docker-tutorial-build-container docker-tutorial-build

! delete the earlier created publish folder - we will now copy the containers compiled result into that folder:

docker cp docker-tutorial-build-container:/out ./publish

Great now we can build the runtime image just like before:

docker build -t docker-tutorial .

And of course run it:

docker run -p 8181:80 docker-tutorial

! You probably have to stop the container we started earlier, since that one is already using port 8181. To do so first list the running processes:

docker ps

Copy the container ID, e.g. ba51e5dc4036

and run the following command:

docker rm $(docker stop ba51e5dc4036)

Multi-Stage Builds

With Docker Version 17.05 we got a new featuer that makes the build process much easier. The reason we have a build image and a runtime images is,
because we want a slimer image at runtime. Since this is a very common requirement for a lot of languages Docker provided us with multi-stage
builds => see documentation. This means we can now define build and runtime image in one single Dockerfile and we can copy the produced binaries from the build image into
our runtime image.

First stop the container if it is still running:

docker stop $(docker ps --filter "ancestor=docker-tutorial" -q)

Add a new Dockerfile with the name Dockerfile.multistage with the following content:

# build image
FROM microsoft/aspnetcore-build:2.0 as build
WORKDIR /app

COPY *.csproj .
RUN dotnet restore

COPY . .
RUN dotnet publish --output /out/ --configuration Release

# runtime image
FROM microsoft/aspnetcore:2.0
WORKDIR /app
COPY --from=build /out .
ENTRYPOINT [ "dotnet", "docker-tutorial.dll" ]

Building the image is now much easier. All you got to do is to run the follwing:

docker build -f .\Dockerfile.multistage -t docker-tutorial .

Check the image size of the image again:

docker image ls | sls docker-tutorial

The resulting runtime image still has the much smaller footprint, since the intermediate images in a multi-stage build dont make it into the resulting image.

Run it:

docker run -p 8181:80 docker-tutorial

Great! Now we massivly simplified the build process and still kep the image small! At this point I would get rid of the existing Dockerfile and Dockerfile.build and rename Dockerfile.multistage to Dockerfile. Then the build command looks like this:

docker build -t docker-tutorial .

Publishing & Pulling Images

Now that we build the image it would be nice if we could use that on other docker hosts. To do so we can upload our image to a docker registry.
There are many different choices for docker registries: hub.docker.com, AWS ECR, Artifactory...
For simplicity we will be using hub.docker.com which is free for public images.

If you havent done so yet, create an account .

You can then logon in powershell:

docker login

To be able to upload (push) our image we have to prefix our image with our username. My username is schwamster so I would have to run the following command:

docker tag docker-tutorial schwamster/docker-tutorial

Now I can push the image

docker push schwamster/docker-tutorial

After the image is pushed I can verify that it worked by opening the following url: https://hub.docker.com/r/schwamster/docker-tutorial/

To pull the image run the following command:

Now you can also run my image like this:

docker run -p 8182:80 schwamster/docker-tutorial

My image will now be pulled and run on your machine. Check it out under http://localhost:8182/api/values (Changed port to 8182)

Proxy

If you are forced to go through a proxy you will have to adjust some of the commands we used above.

Proxy and docker run

If you need to have internet access from within your container you will have to add the proxy settings to respective environment variables of the container instance. Those can be different depending on what application you use. In general I would set the following:

http_proxy
https_proxy

I noticed that some applications want the variables to be set uppercase -> HTTP_PROXY, HTTPS_PROXY. Other apps might need dedicated environment variables or even changes in config files.

To add the proxy environment variables add each environment variable with the -e argument

Here is an example:

docker run -it -e https_proxy=http://someproxy:8080 -e http_proxy=http://someproxy:8080 debian /bin/bash

To test this run the following command in your container:

apt-get update

apt-get update should now work and not run into a timeout.

Proxy and docker build

If you need internet access while building the image you need to pass the environment variables with the --build-arg argument just like you do it with run time environment variables. Its just the argument that is called different.

Example:

docker build --build-arg http_proxy=http://someproxy:8080 --build-arg https_proxy=http://someproxy:8080 -t cowsay .

Acknowledgement

Please check out this great book "Using Docker" by Adrian Mouat (OÂ´Reilly) ISBN 978-1-491-91576-9

You can find me on GitHub as schwamster