DEV Community: Steve Roberts

Adopting the Lambda Annotations Framework

Steve Roberts — Fri, 03 Mar 2023 04:03:56 +0000

If you're a .NET developer working with AWS Lambda I hope you've heard about the new Lambda Annotations Framework. It's an open-source framework for .NET-based Lambda functions which (in my opinion) simplifies your function code, reducing boilerplate. In this blog post, published with the release of the original preview, you'll find the ideas behind the framework discussed in more detail.

The development team's been hard at work since that initial release, and recently added support to make it really simple to return HTTP status data. So, I thought the time was right to look at how simple it is to update an existing function that has a traditional Lambda function signature to use the framework.

The function I'm going to work on, called Backmask, is part of a sample serverless application used in the AMster & the Brit's Code Corner show, broadcast on AWS on Air. The application contains an API and backing Lambda function that accepts a text string message, converts it to audio using Amazon Polly, then reverses the audio and finally returns the Base64-encoded text version of the audio. You might be familiar with this idea from music, where an artist embeds reversed messages inside the main audio (or so my co-host, AM, on Code Corner tells me!). In the latest episode of Code Corner, I work through the steps of converting the function to use the new framework live. This post represents the condensed version without all the chit-chat.

You can find the sample application in this GitHub repo. It comprises a Serverless Application Model (SAM) template, a Lambda function using .NET 6, and an API Gateway endpoint. The function runs when the endpoint receives a request. A Lambda layer carries the dependency on ffmpeg for processing of the audio created by Polly. You can also find the original Python version, used in an earlier Code Corner episode, there too.

Below is the function signature, and, while it's perfectly good C# it could be more expressive in terms of what the function expects and returns. As written, it returns an object of type APIGatewayHttpApiV2ProxyResponse that will contain the encoded text and status code, and accepts two objects as function parameters. One is the Lambda context, providing information about the running environment of the function. This includes a Logger object that the function makes use of to log activity. The other, and more important parameter, is an API Gateway event object, APIGatewayHttpApiV2ProxyRequest. This contains the event payload from the API - the request body, headers, and any query and path parameters:

The text to backmask is supplied in the request body. An optional query string parameter, voiceOverride, can select the voice that Polly should use to create the audio. Inside the function body, there's a lot of boilerplate code processing the event data sent to the function from the API endpoint, before we actually get to anything useful, highlighted below.

Converting this function to use the new annotations framework will make it easier to see what the function accepts as parameters, and what the function is actually doing, and it only takes a few steps.

Step 1: Add the annotations package

The Amazon.Lambda.Annotations NuGet package contains the new framework, so I start by adding it to the project. If you're following along, open a command line shell and run the command below from the folder containing the project file, or use your favorite environment to add the package:



dotnet add package Amazon.Lambda.Annotations

In the function's code file (function.cs, if you're following along), add a couple of using statements for the namespaces in this new package. The first is for the attributes we'll use from the framework, the second relates to the HttpApi types used with the function's API Gateway resource.



using Amazon.Lambda.Annotations;

using Amazon.Lambda.Annotations.APIGateway;

Step 2: Annotate the function

Before changing the function signature to better represent what the function accepts, first add the LambdaFunction attribute above the function. Note that I'm using this attribute with no arguments, but you can use some to set custom data such as desired timeout, memory size, role, policies, and more.

Second, the function runs when a request arrives at a HttpApi endpoint in API Gateway, so I add the HttpApi attribute too. For this function, I specify the verb (POST) and a route for the endpoint (/backmask in this case).

Next, I need to change the function signature. I still want to return an HTTP result containing both status and the encoded text, so the return type will be IHttpResult (this is the new functionality added recently to the framework). I could, if I wanted, change the return type to be just a string representing the text-encoded audio, but I want to keep the initial payload validation and associated HTTP status code results, so an IHttpResult result is ideal.

Now I update the parameters of the function. Instead of the original APIGatewayHttpApiV2ProxyRequest type, I want the function signature to express the true intent; it should accept a string representing the id of the voice to use when generating the audio, and a second string containing the text to encode. The voice parameter will come from the query string, so I decorate it with the FromQuery attribute and make sure we use the same name as we expect in the request URL. The text to encode will come from the body so similarly I use a FromBody attribute to express the body as the source.

Note that I'm keeping the Lambda context parameter as I want to keep the logging code making use of it, but could remove it otherwise.

Next, I can clean up the boilerplate code seen earlier that extracted the real parameters needed from the original APIGatewayHttpApiV2ProxyRequest parameter. Instead of constructing APIGatewayHttpApiV2ProxyResponse objects, if validation fails, l use the newly added HttpResults type in the framework.

The result is below, which you can compare with the original above. I think you'll agree it's a lot simpler and easier to read (for context, I've once again highlighted the "actual work" part of the function).

Those few steps were all that were required to convert the function's code. The next step is to build the project. Once we've done that, inspection of the serverless template file for the project shows a change - the source generator implemented inside the annotations framework has output a new AWS::Serverless::Function resource for the function into the template. It's easy to spot, as the logical Lambda function name has the suffix Generated. This generated function resource has an added benefit in that I no longer have to keep the function handler string in the resource definition in sync with the function code. If I decide to change the name of the actual method implementing the function (the one I attributed with [LambdaFunction]), or the class, or the namespace containing the class then, when I build the code, the source generator in the framework will keep the related resource in the template up-to-date for me.

Step 3: Copy function settings to the generated function definition

I'm almost done with all the conversion steps. In my case, the function resource declared in the template had a custom permissions policy specified, allowing the code to call the polly:SynthesizeSpeech API. It also referenced a Lambda layer (also declared in the template) that contains the ffmpeg binaries used to reverse the audio generated by Polly. So, I simply copy the relevant items to the generated function resource definition, and then delete the old function resource.

The highlighted boxes in the image below show the data that I simply copied from the original function resource definition. Note that I also deleted the MemorySize and Timeout elements from the generated function definition. I did this since I have them specified (as the same default values) in a Globals section of my template. If, however, I'd specified them as arguments to the [LambdaFunction] attribute I attached to the code I would have kept them. As I noted earlier, I can also set these values, along with role and policy information, in that attribute. If I do so, the values I set get emitted by the source generator into the template and kept in sync with changes to the attribute.

Step 4: …there is no step 4

That's it, "job done" for this function! We can now deploy the serverless application and test it out or, if it contains more functions, convert those to use the new annotations framework. If you want to look at the project and the changes in their entirety, checkout the s1e2_backmask.net-annotations branch in the Code Corner repository on GitHub.

Hopefully, this post will have intrigued you enough to check out the new framework, and try converting some of your existing functions into what I think is a much more readable and easier to comprehend format. Also, the development team is always interested in feedback, which you can supply in the issues folder of their repo. You can also find the design document for the annotations in this issue.

Happy annotating!

AMster & the Brit's Code Corner

Steve Roberts — Tue, 19 Jul 2022 20:06:07 +0000

AWS on Air is a livestream show, broadcast from AWS events in North America as well as regular Friday shows when the team isn't at an event. You can find past episodes on YouTube. The show features guests from AWS teams, talking about service launches, updates, and other news. We also occasionally feature special guests from outside AWS to discuss industry topics.

During the broadcast from the recent Amazon re:MARS conference I teamed up with fellow AWS on Air host AM Grobelny, a Solutions Architect at AWS (Twitter: @amsxbg), to start a new series of informal, developer-oriented segments we're calling AMster & the Brit's Code Corner. We were hoping to do more at the AWS Summit in New York but sadly I had to skip the event. We'll be back with more in the future so in this post, I thought I'd highlight the three initial videos from re:MARS that have now been uploaded to YouTube.

Using AI Services from your Code - Amazon Rekognition

When we think of AWS services, we tend to focus on using them from code we've deployed to the cloud. However, they can equally well be used from code running locally on our machines. In this segment, we look at how I use Amazon Rekognition to support tagging images in my photography hobby, so that I can upload them to stock galleries. I'm somewhat lazy, so leaning on a service to do the basic tagging for me is an ideal scenario! The segment also takes a look at how using the various language SDKs that AWS provides (in this case for .NET) makes calling services super-easy.

Tour of the AWS IDE Toolkits

Did you know that AWS provides a number of freely available integrations with common Integrated Development Environments (IDEs)? In our second segment, we take a quick look at three integration, called toolkits, for Visual Studio, Visual Studio Code, and JetBrains Rider. While Visual Studio and Rider are focused on .NET development, the VS Code toolkit can also be used by developers working in other languages such as Typescript. By the way, you can find details on the full collection of free tools from AWS, for .NET development, here.

Using AI Services from your Code - Amazon Polly and Amazon Transcribe

In this final segment from the show, AM takes the demo hot seat to showcase a text-to-audio, audio-to-text sample he created, using Amazon Polly and Amazon Transcribe, that again uses the AWS SDKs to handle the work of calling services from your code (secretly, he loves C# and PowerShell...don't tell anyone).

We had a lot of fun recording these sessions, and are looking forward to more AMster & the Brit Code Corner sessions in future shows. If you've ideas on things you'd like to see covered, let us know on Twitter or in comments!

Finding your way around the AWS Tools for PowerShell

Steve Roberts — Thu, 17 Feb 2022 00:59:43 +0000

I’ve been writing and presenting on the AWS Tools for PowerShell recently and, out of curiosity, decided to take a look at how many cmdlets the tools now support. Back when we shipped version 1, in 2013, the (then single) module contained around 500 cmdlets spanning 20 or so services.

Having maintained and enhanced the tools for several years, and seen them grow to support the expanding set of services for AWS, I was however somewhat unprepared for the scale of what I found. On a new EC2 instance, running Windows Server 2019 with the monolithic AWSPowerShell module pre-installed (like all Windows images provided by the EC2 team), Get-Command reported over 11000 cmdlets!

When I moved on from being the lead developer a couple years back, the tools had grown to over 5000 cmdlets, and expansion has obviously continued. Back then, we broke our ability to publish the monolithic modules (AWSPowerShell and AWSPowerShell.NetCore) to the PowerShell Gallery a couple times, as we’d stumbled over a previously undocumented limit on the number of cmdlet names you could include in the module manifest. In order to publish, we had to remove the cmdlet names from the module manifests, which is why tab-completion of cmdlet names failed periodically every time we broke through the limit.

Although the PowerShell team did us a favor and raised the limit a few times, allowing us to reinstate the exported names, it was clear this wasn’t going to be a workable approach going forward. The sheer size of the monolithic modules was why the AWS team refactored to per-service modules, known as AWS.Tools.*, on the gallery. The per-service modules load faster, always have tab completion for cmdlet names, and offer the additional benefit of not having to have all 11000+ cmdlets installed if you're only working with a handful of services.

All editions of the tools (AWSPowerShell, AWSPowerShell.NetCore, and AWS.Tools.*) contain the same overall set of cmdlets – it’s just a packaging variation. The monolithic modules are kept around for backwards compatibility as there are scripts out there that import the modules from their pre-installed location on the EC2 instances.

So, this all begs a question – given over 11000 cmdlets spanning 200 services, how do you find your way around? How do you locate the cmdlets for a given service and, within that service, how do you find the cmdlet you need for a given service API? (Pretty much all of the cmdlets map 1:1 with AWS service APIs.) Let’s take a look.

The AWS cmdlet naming convention

When we originally created the tools, we had one module but needed to find a way to distinguish cmdlets between different services that used the same underlying API name. For example, several services have an API named DescribeInstances (EC2 being one such service). This would ordinarily map to a cmdlet name like Get-Instance, but only one service could then use this in our single module, unless we add a parameter to distinguish the service. That’s not the route we took. Instead, we chose to prefix the noun portion of the cmdlet name with a short 2-5 letter "tag" identifying the parent service – so Get-EC2Instance, Get-GMLInstance (GameLift), and Get-OPSInstance (OpsWorks), for example.

The prefix tags for some services are obvious – S3, EC2, ECS, etc., while others are less so. Interestingly, when we started to look at refactoring to per-service modules and asked our user base about potentially dropping the convention we got a 50/50 split between keeping and dropping the prefix. Ensuring backwards compatibility also played a large part in our decision making to keep the prefix.

The prefix tags allow us to isolate cmdlets for a service, but how to find the tag used for a service? The cmdlets contain a custom version cmdlet, Get-AWSPowerShellVersion (I don’t recall why we added this now, but it has its uses). When invoked with the -ListServiceVersionInfo parameter, one of the data items it outputs is the noun prefix:

I have the AWS.Tools modules installed, so you can also see the corresponding per-service module. If you’re using the monolithic versions, you’ll see AWSPowerShell or AWSPowerShell.NetCore in that third column. For the AWS.Tools edition, the version cmdlet reports the prefix tags of all supported services, even if you don’t have the module for a particular service installed.

Finding cmdlets for a service

Now that we know how to identify the prefix tag for a service, how do you list the cmdlets for that service? For the AWS.Tools modules, Get-Command will suffice if you have the corresponding module installed (for example, Get-Command -Module AWS.Tools.EC2). This isn’t so clear-cut with the monolithic modules however, and only works with the per-service modules if the module is installed.

Back in the mists of time we added the cmdlet Get-AWSCmdletName to help with discovery. This cmdlet takes advantage of additional metadata that we compile into every cmdlet, enabling users to slice-and-dice the set of cmdlets based on service, API, and AWS CLI command.

To find the set of cmdlets for a service, use the -Service parameter. This nets you the set of cmdlets, and corresponding APIs (as well as service name and module name, not shown in this example), for the specified service. Again, with the AWS.Tools modules, this works even if you don’t have the corresponding service module installed:

Note: all the images in this post use a Windows command line prompt, however all these commands can be used on Linux and macOS too. For those of you that might be unaware, PowerShell is no longer tied to Windows!

If you don’t know the service tag, you can use all or part of the service name to get the same kind of result. However, as some services use similar words (“elastic” for example) this can be a little less focused:

The -Service parameter can also be used with multiple words (-Service "Compute Cloud") and also supports a -MatchWithRegex parameter. Note that the -MatchWithRegex parameter automatically surrounds the regex you provide with ^ and $ terms. Also, in the examples I'm showing in this post I’m using mixed case, but case is actually insignificant.

Finding cmdlets that implement a service API

Having found cmdlets that map to a service, how about the inverse – finding cmdlets that map to AWS service APIs? This is useful for developers used to working with one of the AWS SDKs and who have some familiarity with the underlying services. As an ex-SDK developer, this is my usual go-to approach especially as I’m usually starting from a service’s API reference material.

To map from API to cmdlet, use the-ApiOperation parameter to Get-AWSCmdletName, and provide the name of the API (case is not significant):

If you want to restrict the search to a particular service, add the -Service parameter. Modifying the example above to be Get-AWSCmdletName -ApiOperation -Service EC2 would output only one result.

You can also perform a regex-based search with -ApiOperation:

Mapping from an AWS CLI command to a cmdlet

It’s no secret that there are more help examples available for the AWS CLI than PowerShell. Get-AWSCmdletName has an additional mode, invoked with the -AwsCliCommand parameter, that parses a CLI command to output the name of the corresponding cmdlet. This is based on how the CLI command hyphenates the service API name so might not be 100% correct if this convention ever changes. To use it, take the CLI command with the service indicator and operation name and pass it to the cmdlet. You don’t need the initial "aws" component in the CLI command example, and if you pass it, it’s ignored. For example, below I could also have specified -AwsCliCommand "aws ec2 describe-instances" to achieve the same result.

Note the deprecation warning; personally, I hope this feature isn’t ever removed but, since it’s based on a naming convention rather than internal metadata added when the cmdlets are built, it’s possible it could fall victim to a future change.

The CLI mode doesn’t translate parameters (and ignores them if you include them in the parameter value). Since the CLI and the PowerShell cmdlets map to AWS service APIs, I usually head to the cmdlet reference, or the respective service reference documentation, at that point.

With over 11000 cmdlets available, the tools provide a lot of scope to work with AWS services and resources from PowerShell, but finding out what’s available can be daunting. Hopefully, from this post you can now see how you can find your way around to become productive quickly.

Happy scripting!

Addendum

AWS Solutions Architect Vlad Hrybok informed me of a site that you might also find useful when mapping from AWS CLI commands to the PowerShell cmdlet, AWS CLI -eq PowerShell. Clicking entries in the Services column takes you to the product details, whereas clicking in the CLI command column takes you to a command/cmdlet list for that service. Super useful!

Working with Amazon S3 presigned URLs in Visual Studio

Steve Roberts — Mon, 07 Feb 2022 16:58:00 +0000

I noticed a little flurry of interest on Twitter recently, after it was noticed that it’s now possible to create presigned URLs for objects in Amazon S3 storage buckets using the AWS Management Console. Whilst it’s always cool to get a positive reaction for customer-facing enhancements, I do recall thinking to myself “Pffft, .NET developers using the AWS Toolkit for Visual Studio have had this ability since v1 of our toolkit, back in 2011!”.

What's a "presigned URL"?

Objects (files) in an S3 storage bucket are private by default. While you could share objects in a bucket by relaxing permissions, on either specific objects or the entire bucket. this really isn’t a good practice. Who knows what data you’ll put into the bucket in future, having forgotten that you relaxed permissions….and now you have a data leak ☹.

A presigned URL is simply a generated, time-limited URL to an object that you can use to share the object with others. When you create a presigned URL, you supply the bucket and object name (the object key, in AWS parlance), the allowed HTTP method, an expiration date and time, and your security credentials (which can be temporary, token-based time limited credentials too). If you use token-based credentials, the link expires when the token expires, even if this is earlier than the requested link duration. The maximum duration you can request for a presigned URL is 7 days.

The URL you get back can be shared with others, who can then access the object – even it is otherwise private. Therefore, you should obviously be careful when sharing them. Presigned URLs can be used to download, upload, and delete objects depending on the method encoded in the URL. I use presigned URLs frequently, to share large files with colleagues. This allows me to follow best practices and otherwise keep all my buckets and objects in them private and accessible only by me.

Essentially, a presigned URL contains a bearer token whose permissions are scoped by the permissions granted to whoever (or whatever) created the URL. This means a presigned URL grants no additional permissions to the consumer beyond those of the creator. For example, if my account doesn’t have permission to get (download) an object, a presigned URL I create, with HTTP method GET, to the object will fail to work - for me and anyone I share it with. It’s also possible to further restrict usage of a presigned URL to specific network paths, although not when creating them through the toolkits. See Limiting presigned URL capabilities in the S3 user guide.

The AWS Toolkit for Visual Studio

Ok, now to the toolkit. If you’ve not seen or heard of it the toolkit is a free extension, available on the Visual Studio marketplace, that supports Visual Studio 2017/2019, and Visual Studio 2022 (there’s also a free toolkit for Visual Studio Code, which also supports creating presigned URLs without leaving the editor).

The Visual Studio toolkit surfaces an explorer window in the IDE that allows you to work with several AWS services (S3, EC2, DynamoDB, Lambda, et al) from within the convenience of the IDE and not have to jump out to a web browser to use the console, and several wizards making it easy to deploy code to Elastic Beanstalk (web apps), Lambda (serverless functions and apps), Elastic Container Service, and CloudFormation (infrastructure as code).

Generating a presigned URL using the toolkit

To create a presigned URL to an object using the toolkit, you first need to open a view onto the bucket’s objects. From the AWS Explorer window (View -> AWS Explorer, if the window’s not visible), expand the Amazon S3 hierarchy and locate the entry for the bucket. Double-click it, or use the context menu, to open the view:

Although S3 is a flat object store, not a hierarchical file system, the toolkit’s organizes objects in the bucket into “folders”, parsing on the / character in the object keys. So, you might need to traverse down one or more of those simulated folders to find the object you want. Let's generate a presignedURL for an object at the root of the bucket.

To do that, right-click on the object to share a link to and select “Create Pre-Signed URL…”:

In the resulting dialog I select the HTTP method (GET or PUT, the toolkit doesn’t support DELETE) and the expiry date and time. I can also set a content-type if needed. Then I click the Generate button to obtain the URL:

The Copy button copies the generated URL to the clipboard. I can now click OK to dismiss the dialog and head back to whatever content I was writing (email, etc.), and paste the link. The recipient will then be able to use the link to download the object, even though it’s private, until 12PM on Feb 7th in this example.

If you’re a Visual Studio Code user, you can do this too but the process is a little different. In the VS Code toolkit, you navigate objects in a bucket directly within the explorer pane. The context menu on an object shows a command to generate a URL (here I’m going to generate a URL for an object in a “subfolder“):

You’re then asked for the validity duration, in minutes – there’s no dialog as in Visual Studio, and you can’t change the method (it’s always GET) or set a custom content type:

Press Enter once you’ve set the duration and the toolkit creates a URL, automatically copies it to the clipboard, ready to paste elsewhere. It also displays confirmation in the status bar:

I don’t use an IDE toolkit, now what?

To wrap up the post, I wanted to note that it’s also possible to create presigned URLs from command line tools. Using the AWS Tools for PowerShell, the Get-S3PresignedUrl is what you need (docs here). It accepts a DateTime object to set the expiry, defaulting to HTTPS protocol and GET as the method:

Get-S3PresignedUrl -BucketName steve-presignedurldemo -Key dotNETAndPowerShellOnAWSBoothVideo2021.mp4 -Expire 2022-02-06
https://steve-presignedurldemo.s3.us-west-2.amazonaws.com/dotNETAndPowerShellOnAWSBoothVideo2021.mp4?X-Amz-Expires=217008&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-SignedHeaders=host&X-Amz-Signature=5982...fb4b

If you use the AWS CLI, you use aws s3 presign (docs here), which defaults to a one-hour expiry if not specified:

aws s3 presign s3://steve-presignedurldemo/dotNETAndPowerShellOnAWSBoothVideo2021.mp4
https://steve-presignedurldemo.s3.us-west-2.amazonaws.com/dotNETAndPowerShellOnAWSBoothVideo2021.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=f7b8...29bdf

Happy presigning – and welcome to the party, (management console) pal!

Retrieve Amazon EC2 instance tags using PowerShell

Steve Roberts — Wed, 12 Jan 2022 20:04:04 +0000

In a recent What’s New post, the Amazon EC2 team announced that you can now retrieve, from the instance metadata service (IMDS), tags that you’ve applied to your EC2 instances (virtual machines) if you opt-in to making them available via IMDS during instance launch. You can opt-in when launching instances from the AWS Management Console, programmatically using EC2’s RunInstances API, or other tooling you might use to launch instances, such as CloudFormation templates. Below is a screenshot of the management console option to in the EC2 launch wizard to make tag data available on the instance:

The IMDS is a private endpoint that’s only accessible using a non-routable IPv4 or IPv6 address from within the instance, and provides a variety of metadata related to the instance. Some examples are the ID of the Amazon Machine Image (AMI) used to launch the instance, the ID of the instance, security role information, the region in which the instance is running, block device mappings, and much more. The instance metadata categories topic in the EC2 User Guide details the kinds of data available from IMDS on an instance, which can be queried on the instance without needing credentials.

Tags are an additional form of metadata that you can apply to your resources in the AWS cloud. A tag is simply a label that consists of two parts, a user-defined key and its associated value. Tagging resources is considered a best practice, as it can help with identifying, grouping, and categorizing related resources. For example, let’s say I have code running on an instance but the code needs to know what staging environment (dev, beta, gamma, prod, etc.) the instance is in. I can apply a tag to the instance that denotes the staging environment and then, from within the code on the instance, query the stage and adjust what the code is doing appropriate to that environment. By the way, EC2 instances are just one type of AWS resource that you can apply tags to.

Reading tags in the "before times"

Prior to EC2 making tags available from IMDS, to access them from code running on an instance you needed to make use of EC2’s DescribeTags API. This API is exposed by the Get-EC2Tag cmdlet in the AWS Tools for PowerShell modules.

You’ll find the monolithic AWSPowerShell module pre-installed on Windows images provided by Amazon EC2. For Linux and macOS there's an equivalent cross-platform (and also monolithic) AWSPowerShell.NetCore module. If you're using an image that doesn't have the modules pre-installed it's easy to add them yourself and, if you go this route, I recommend using the modular (AWS.Tools.*) variant instead of the monolithic modules. In the modular variant, each AWS service has its own PowerShell module making the individual modules much faster to load (the monoliths each contain thousands of cmdlets). So, for example, there are AWS.Tools.EC2, AWS.Tools.S3, and other modules. The modular variant can be used on Windows, Linux, and macOS systems. Note that the AWSPowerShell, AWSPowerShell.NetCore, and AWS.Tools.* modules are all just packaging variations, and provide access to the same set of cmdlets.

The example below shows how I can retrieve the tags attached to a given instance using Get-EC2Tag (this code runs on the instance):



PS C:\> Get-EC2Tag -Filter @{Name="resource-type";Values="instance"},@{Name="resource-id";Values=" i-0f53ee144573513c5"}

Key     ResourceId          ResourceType Value
---     ----------          ------------ -----
Name    i-0f53ee144573513c5 instance     App1
Project i-0f53ee144573513c5 instance     App1Project
Purpose i-0f53ee144573513c5 instance     Dev team app testing
Stage   i-0f53ee144573513c5 instance     Beta
Team    i-0f53ee144573513c5 instance     WinDev

The output echoes the five keys, Name, Project, Purpose, Stage, and Team, and their associated values, of the tags I attached to my instance. The filters used in the command restrict the lookup to, firstly, tags applied to EC2 instances, and secondly the specific instance in question. This is because the underlying API is able to return tags for other resources types. I can run this cmdlet anywhere, not just on the instance. It also, by the way, works irrespective of whether I elect to make tags available from IMDS.

There are, however, two problems with this approach when it comes to running it on an instance. First, for the cmdlet to be able to access the DescribeTags API, it needs credentials so I’d have to launch my EC2 instances with a role that has a trust relationship with ec2.amazonaws.com to allow the tools to obtain temporary credentials on demand. This role would also have needed to grant permissions to call the DescribeTags API. This isn’t so bad, as launching instances with an attached role is a fairly standard practice.

The second problem relates to the need to specify the instance ID and this makes it much more awkward – how do I get the ID of the instance I’m running the cmdlet on? In this scenario, I was connected to my instance using RDP and so was able to paste the instance ID into the command. This won’t work from automation though so we need another way.

Retrieving the instance ID

I noted earlier that one of the items of data available from IMDS is the instance ID. The question is, how to get it. If you read the AWS documentation, you’ll find examples showing how to ‘curl’ the IMDS endpoint (http://169.254.169.254 in IPv4, or http://[fd00:ec2::254] in IPv6) for data. However, there’s a simpler way using the Get-EC2InstanceMetadata cmdlet, present in all variants of the AWS Tools for PowerShell modules (in the modular variant, you’ll find it in the AWS.Tools.EC2 module).

Get-EC2InstanceMetadata handles the work of accessing the underlying IMDS endpoint for you, and outputs the requested metadata. The cmdlet accepts a -Category parameter that you use to select which data you want output. There’s also a switch, -ListCategory, that displays the set of categories that whatever version of the cmdlet had built-in support for when it was released (new categories are added over time).

For the earlier example, we could rewrite it to use Get-EC2InstanceMetadata cmdlet with the InstanceId category value to get the same output:



PS C:\> Get-EC2Tag -Filter @{Name="resource-type";Values="instance"},@{Name="resource-id";Values=(Get-EC2InstanceMetadata -Category InstanceId)}

Key     ResourceId          ResourceType Value
---     ----------          ------------ -----
Name    i-0f53ee144573513c5 instance     App1
Project i-0f53ee144573513c5 instance     App1Project
Purpose i-0f53ee144573513c5 instance     Dev team app testing
Stage   i-0f53ee144573513c5 instance     Beta
Team    i-0f53ee144573513c5 instance     WinDev

This is better, and works with automation scenarios on the instance, but we still need credentials to be able to run Get-EC2Tag. So, let’s look at solving that problem using the new support for retrieving tags from the instance metadata service.

Retrieving tags from IMDS

If you run the Get-EC2InstanceMetadata cmdlet with the -ListCategory switch (or check the help docs), you’ll see the set of category values the cmdlet is aware of in whatever version you have available:



PS C:\> Get-EC2InstanceMetadata -ListCategory
AmiId
LaunchIndex
ManifestPath
AncestorAmiId
BlockDeviceMapping
InstanceId
InstanceType
LocalHostname
LocalIpv4
KernelId
AvailabilityZone
ProductCode
PublicHostname
PublicIpv4
PublicKey
RamdiskId
Region
ReservationId
SecurityGroup
UserData
InstanceMonitoring
IdentityDocument
IdentitySignature
IdentityPkcs7

These built-in category values exist simply to make retrieving certain data more convenient at the command line. You’ll notice though that there’s no mention of tags. Not a problem! The cmdlet also accepts a -Path parameter and using this, you can supply the path of any metadata item and retrieve the value. This is exactly what happens when you specify a category value - the cmdlet knows the path associated with the requested category and internally requests the data using the path. So, for our purposes, we can retrieve instance tags using the /tags/instance path, shown below:



PS C:\> Get-EC2InstanceMetadata -Path /tags/instance
Name
Project
Purpose
Stage
Team

That gives us the tag keys. To retrieve the value for a tag, we simply append the key to the path (note that the keys are case-sensitive):



PS C:\> Get-EC2InstanceMetadata -Path /tags/instance/Name
App1

PS C:\> Get-EC2InstanceMetadata -Path /tags/instance/Stage
Beta

PS C:\> Get-EC2InstanceMetadata -Path /tags/instance/Purpose
Dev team app testing

If you’re already using the AWS cmdlets on your instances, then accessing metadata and tags using the Get-EC2InstanceMetadata cmdlet is simple and convenient. However, as I noted earlier the monolithic variants of the modules (AWSPowerShell and AWSPowerShell.NetCore) contain thousands of cmdlets and do take some time to load (this is not a problem with the AWS.Tools.EC2 module, which is why I recommend you choose that if you can). If all you need to do is access the tag data and not make use of the other AWS cmdlets, you might prefer to retrieve tags using Invoke-RestMethod, shown in the example below:



PS C:\> Invoke-RestMethod -UseBasicParsing -Uri '169.254.169.254/latest/meta-data/tags/instance'
Name
Project
Purpose
Stage
Team

PS C:\> Invoke-RestMethod -UseBasicParsing -Uri '169.254.169.254/latest/meta-data/tags/instance/Name'
App1

Whichever approach you use, automation scripts running on your instances can make use of tags to reason about the runtime environment and adjust behavior accordingly. Happy tagging!