DEV Community: Nicolò Marchesi

IAM policies and Service Control Policies (SCPs): How to master and secure access and permissions in an AWS Landing Zone

Nicolò Marchesi — Mon, 03 Apr 2023 13:51:11 +0000

Introduction

Hello cloud fellows! Today we’ll build up on the cloud landing zone series to explore a fascinating but often ignored context… Service Control Policies! SCPs are powerful, but there are a few caveats and tricks to make them work efficiently with every different kind of IAM policy. In this blog post, we will see how the whole IAM ecosystem interacts and how we can effectively leverage the tools to deploy a strong IAM strategy in our Cloud Landing Zone.

There is a lot of ground to cover, so let’s start immediately!

⚠️ I don’t want to bother you with all the details about AWS Organizations so that we can keep our focus on permissions. If that’s not the case, refer to one of the thousands of articles on the web or check THIS one I wrote.

Policies

So, let’s get the foundations covered (for the laziest of you, skip after the graph, there’s a neat bullet-point recap). Before jumping directly to the interactions, we need to understand what we have at our disposal in our IAM strategy and all the different kind of tools in IAM to make it work:

Identity-based policies are the most common type of policy. They are tied to IAM (Identity and Access Management) users, groups, and roles and specify what actions those entities can perform on certain AWS services.
Resource-based policies, on the other hand, are policies that are directly associated with an AWS resource, such as an S3 bucket or an EC2 instance. Resource-based policies specify who has access to the resource and what actions they can take. Not all AWS services support them (for a comprehensive list, check the AWS services that work with IAM page), and they are usually used for very specific scenarios.
Permission boundaries (PBs) are policies that define the maximum permissions that an IAM entity can have. These policies limit an entity's permissions, ensuring they cannot perform actions that exceed their authorized scope. Permission boundaries are also written in AWS policy language and can be attached to IAM entities.
Service control policies (SCPs) enforce restrictions on AWS accounts within an AWS organization. SCPs are hierarchical policies applied to the entire organization or specific organizational units. SCPs can be used to limit the actions that an AWS account can perform, preventing them from performing activities that are outside their authorized scope.
Session policies are applied to temporary credentials created by IAM roles. Session policies limit the permissions of a temporary credential set, ensuring that it cannot perform actions that exceed its authorized scope. However, they primarily work like PBs and SCPs: they do not grant permissions and are applied only for the duration of the token. For the sake of simplicity, we’ll introduce this only at the end, so for now, let’s not consider it.

The caveat

As you may be starting to get, there is a fundamental difference between those policies, and it's laid out in the diagram by the action that connects the policy to the actual permissions.

⚠️ Permission boundaries and Service control policies do NOT grant ANY permission

An Identity can access a Resource only through Identity-based and Resource-based policies. Permission boundaries and SCPs can only limit the aforementioned permissions. That means we need an Identity-based or Resource-based policy that explicitly allows permission to let the policy evaluation engine.

In short, Identity-based and Resource-based policies define who can access resources and what actions they can perform. Permission boundaries and Service Control Policies limit the scope of those permissions, ensuring that entities cannot perform unauthorized actions.

A visual representation of policy interaction

So let’s get a visual representation of our policies: start by seeing how it works in a single account, and then see how things change by adding AWS Organizations to the equation.

Yeah, I know the icons differ from the AWS framework but bear with me; I want to highlight the differences and that we’re working with fundamentally different policies.

Single account (without Organizations)

As you can see, the three elements we can leverage are Identity-based, Resource-based policies, and Permission boundaries:

We can see that while the policies going in and out of the Identity and the Resource grants access, the Permission Boundaries limit that set of permission instead.

It’s impossible to grant additional permissions through the use of Permission Boundaries.

Organization structure

When integrating the AWS Organization service, we gain the ability to use Service control policies to have better control over our environment:

The behavior is practically the same as Permission Boundaries, but we’ll see soon that it has a slight difference. So, to briefly recap:

AWS Identity-based policies:
- Most common policy type in AWS.
- Attached to IAM entities (users, groups, and roles)
- Specify actions that entities can execute on specific AWS resources
- Goes from Identity to Resource
AWS Resource-based policies:
- Attached to an AWS resource (e.g., S3 bucket or EC2 instance)
- Determine which users have access to the resource
- Define what actions the authorized users can perform on the resource
- Not all AWS services support them
- Used in very specific scenarios
- Goes from Resource to Identity
Permission boundaries:
- Attached to IAM entities.
- Defining maximum permissions for an IAM entity
- Limits the entity's permissions to the authorized scope
Service control policies (SCPs):
- Hierarchical policies for AWS accounts in an Organization
- Used to restrict the actions of AWS accounts
- Limits activities outside the scope of permissions.
Session policies:
- Used with identity-based policies and permission boundaries
- Applied to temporary credentials of IAM roles
- Limits the permissions of temporary credentials
- Ensures authorized scope is not exceeded

The policy evaluation flow

Behind all these definitions, a policy evaluation engine goes through all the policies we have seen before and evaluates if the specific action performed has to be allowed or denied.

Here I condensed the logic to understand the decision flow better:

It’s interesting to note that Resource and Identity-based permissions are evaluated at the center of the evaluation flow. Around them, SCPs and PBs are evaluated, which we clustered in control policies.

Aside from Resource-based and Session policies, it turns out that it’s pretty straightforward; the trick here is to focus on the edge cases.

With resource-based policies

Resource-based policies result in a deny when the Principal making the request is different than the Principal we’re granting access to through the Resource-based policy.
I’ve highlighted the affected case in the schema proposed by AWS at this link.

With session policies

Even in this case, it’s simpler than it looks. Session policies kick in only when they are present.

If you’re not using them in your request, you shouldn't care, but when the time comes that you’re leveraging them, you need to remember the following:

there is no allow → implicit deny
there is no explicit deny → implicit deny

A side note on cross-account access

Until now, we’ve considered only access within the same account, but what would happen if we need to evaluate also cross-account access? It’s simpler than it looks: request is evaluated on policies and permission from the perspective of both the trusted and trusting account and allowed only if both are evaluated as an allow!

If you think about this, it’s more restrictive than single access. Since AWS permission starts with an implicit deny, you must explicitly set the permissions on both accounts before evaluating the request as an allow.

Permission intersections

After understanding what is involved in deciding when a request is allowed, we can move on to how they interact.

This led to two practical scenarios, one with and one without Resource-based policies. The main point here is to understand that the only case in which one’s effective permissions can exceed that of the whole intersection is when Resource-based policies are involved.

When Resource-based policies are involved, if they evaluate as an allow, you’re taking the final decision before the policy evaluation flow would evaluate Identity-based, Permission Boundaries, and Session policies. This results in permissions granted that can exceed the ones explicitly allowed by those policies.

About SCPs inheritance

The last thing to say is about Service Control Policies and the fact that they can be inherited.

Strangely, this doesn’t work as one should expect (but it’s for the better, trust me).

When one thinks about inheritance, usually in programming, but even in other fields, one thinks about getting the parent's configurations to the child. If we apply this to SCPs, Organizational Units, and accounts, one can define the SCPs at the root level and then inherit everything. Well, that’s NOT how it works.

Since DENY statements are evaluated first, in practice, only DENY statements are inherited.

If you deny a service in an SCP, there is no way to grant it in a lower-level OU or Account.

So, when an SCP is evaluated, there are actually only two SCPs that concur with the outcome:

The parent SCP — it’s the SCP evaluated of the next higher-level SCP; this needs to be navigated to the Root to the Organization, so it just needs to add the next layer, one step at a time
The current SCP — the SCP which is currently being evaluated

From this perspective, you can see that ALLOW statements are not inherited but need to be explicitly set in the SCP of the Account or Organization Unit. This is for security purposes, as we want to explicitly set the boundaries of accounts and organizational units to avoid implicit significant permissions.

Deny strategy

With a deny strategy, actions are allowed by default through a FullAccess SCP managed directly by AWS. You attach that SCP to all Accounts and OU and you need to specify what services and actions are prohibited:

This is the default configuration of AWS Organizations so that account admins can delegate all services and actions until you create and attach an SCP that denies a specific service.

The benefit of this approach is that deny statements require less maintenance because you don't need to update them when AWS adds new services, and it’s supported out of the box. You can also restrict access to specific resources or define conditions for when SCPs are in effect.

This is a great way to start for smaller organizations that need to act fast and don’t have strict requirements over governance and security.

Allow strategy

With an allow strategy, you must remove the AWS-managed FullAWSAccess SCP. Now all actions for all services are implicitly denied, and you need to create an SCP that explicitly permits only those services and actions you want to allow. This case is the same schema as the inheritance evaluation.

The benefit of this approach is that you have more control over what is allowed. Since everything is implicitly denied, this way is easier to scope down the actual boundaries of an account. Since deny permissions are inherited, you don’t have to specify it everywhere.

However, it requires more maintenance since you have to manually allow each service (and this includes new services). And it comes with some limitations on the allow statements: Resource elements can only have a ”*” entry, and they can't have a Condition.

This approach's value shines when services may not be needed by a large portion of the organization but may still be required for specific use cases. In this scenario, it’s simpler to elevate permissions and satisfy security and governance concerns while allowing flexibility and exceptions.

Just a few examples

So let’s see those SCPs in action; here, I put some examples so you can better picture what an SCP looks like in a real-world scenario.

Pipeline only account

In this case, we’ve created an SCP to deny everything except changes that run through pipelines. The use case is to create an automation-only account where no manual action is allowed, but changes are always deployed through pipelines.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyAllExceptPipelines",
      "Effect": "Deny",
      "NotAction": [
        "codepipeline:*",
        "codebuild:*",
        "codecommit:*",
        "codedeploy:*",
        "codestar:*",
        "cloudformation:*",
        "iam:*",
        "s3:*",
        "logs:*",
        "cloudwatch:*",
        "cloudtrail:*",
        "codestar:*",
        "codestar-notification:*",
        "codeartifact:*",
        "kms:*",
        "tag:*",
        "access-analyzer:*",
        "codestar-connections:*",
        "ssm:GetParameter*",
        "sts:*",
        "events:*"
      ],
      "Resource": "*",
      "Condition": {
        "StringNotLike": {
          "aws:PrincipalArn": [
            "arn:aws:iam::MY-ACCOUNT-ID:role/MY-ROLE"
          ]
        }
      }
    }
  ]
}

Backup protection

In this SCP, we implemented a way to protect all backups made through AWS backup by deletion. Notice that both S3 and Backup vaults are protected and the service cannot be turned down.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DenyS3BackupDelete",
            "Action": [
                "s3:DeleteObject",
                "s3:DeleteObjectVersion",
                "s3:DeleteObjectTagging",
                "s3:DeleteBucket"
            ],
            "Resource": [
                "arn:aws:s3:::MY-S3-BACKUP*/*"
            ],
            "Effect": "Deny"
        },
        {
            "Sid": "DenyBackupDelete",
            "Action": [
                "backup:DeleteBackupVault",
                "backup:DeleteBackupVaultAccessPolicy",
                "backup:PutBackupVaultAccessPolicy"
            ],
            "Resource": [
                "arn:aws:backup:*:*:backup-vault:MY-BACKUP-VAULT"
            ],
            "Effect": "Deny",
            "Condition": {
                "StringNotLike": {
                    "aws:PrincipalARN": "arn:aws:iam::*:role/MY-EXECUTION-ROLE"
                }
            }
        },
        {
            "Sid": "DenyBackupTurnoffService",
            "Action": [
                "backup:UpdateRegionSettings"
            ],
            "Resource": [
                "*"
            ],
            "Effect": "Deny",
            "Condition": {
                "StringNotLike": {
                    "aws:PrincipalARN": [
                        "arn:aws:iam::*:role/MY-EXECUTION-ROLE"
                    ]
                }
            }
        }
    ]
}

Conclusion

Congratulations, you’ve managed to get to the end of the blog post! We’ve gone from the policy evaluation flow to the actual implementation of SCPs, passing through the understanding of all the details that concur in policy and boundary interaction.

From seeing the SCP examples, as with any governance tool, their implementation is extremely tied to how your organization is structured and works. So I truly believe this knowledge should be well internalized and widely understood within the organization.

Now you’re on your way to mastery; see you next time, and let me know how your cloud journey is going!

If you're curious about my open-source project visit the Leapp GitHub repository (be sure to drop a star if you like it!) or read more about IAM and Cloud Access Management on our blog.

AWS multi-account strategy explained

Nicolò Marchesi — Fri, 16 Dec 2022 09:54:03 +0000

If you ever wanted to understand how an AWS multi-account strategy works, you’re in the right place! In this blog post, we’ll explore a bottom-up approach where we’ll see how to secure your single AWS account and then move to a multi-account organization with a fun take.

One box to rule them all

So, first of all, we need to understand what is an AWS account and let’s deep dive into the topic with a definition:

💡 An AWS account acts as a resource container and resource isolation boundary.

The two essential keywords here are container and isolation boundary, and I don’t know about you, but to me, they both contribute to forming a pretty clear image:

Disappointed? You were thinking of something more sturdy, right? Don’t you worry now; we’ll get there eventually. So, what can you do with this box? You can put things inside, and… that’s it.

The container, by itself, acts like any other container: to keep things from being scattered around. It can still be wet by a sudden flood, gnawed by rodents, or stolen by someone else!

We must make a little effort to make it more robust and secure.

And everything because of the shared responsibility model.

The shared responsibility model

As you may already know, when moving to the cloud, you stop thinking about things like managing the low-level components, such as the host operating system and virtualization layer, and down to the physical security of the facilities where the service operates. But not everything goes away.

You are responsible for the security of your workloads, and where does our brittle cardboard box we just discussed fit in? Precisely! That’s your job to keep it safe.

Making the box sturdy: let’s secure it

So let’s start by making your AWS account more secure. The magic happens in the AWS Identity and Access Management (IAM) service, where you can tell anyone in your organization what they can access.

Given that we’re talking about a single account structure, there is a high chance of a root user in your account, the one identity with complete access to all AWS services and resources. The very first thing you should do is to activate multi-factor authentication (MFA) on the AWS account root user.

Aside from that, the actions you should always take are:

Activate MFA to any users with interactive access to AWS IAM
Limit AWS account root user access to your resources
Use AWS IAM roles and learn about granular permissions
Enable CloudTrail monitoring for your account and its resources

But this is just the starting point; from there on, you’re embarking on a journey where it’s up to you to understand how to define your security policies. The essential takeaway is that there will always be new things to look out for, so integrate them into your cloud operations process.

At least the box looks more like this now!

More boxes! Scaling out your accounts

Now that we’ve learned how to safeguard a single account, why should we ever move away? Let’s continue with the boxes example to know how. So right now, we have made our box sturdy and safe, but a few things could happen:

I need more space! Yeah, the box could virtually expand indefinitely (we’re still talking Cloud here, remember?), or I could purchase a giant box, but at some point, it will become a mess to organize. Ever tried moving to a new house and using just one box for all your belongings? Yeah, that’s not pretty.
I need to share the box's content with someone else; I trust myself since I know well how I manage my secret code, but what about others? A burglar could steal everything from the box if he knew the code, regardless of who got it from.

I think you’re starting to get the point. The isolation of our single AWS account can let us leverage the two essential concepts of organization and privilege.

And AWS Organization is the thing that ties everything together. You don’t have to implement this, but you can leverage it for the best benefits with the least friction.

To sum up what you can do:

Create new AWS accounts using a hierarchy pattern;
Put out-of-order AWS accounts that aren’t used anymore;
Cluster accounts with your logic and apply the same set of rules to them;
Manage fine-grained permission and access to single accounts or collections;
View all your accounts billing in one place.

Let’s get organized

If you try to fit everything in one AWS account, you could have problems differentiating your workloads, while the AWS account itself poses boundaries on the content you decide to put in. Want to distinguish between development and production? Sure! Have you a machine learning division? Give them their account! Want to deploy some microservices? Here you are!

But how should you get organized? That’s one of the most common questions I hear in this field, and for sure, there isn’t a single true answer because it depends on what you’re trying to achieve.

What AWS suggest in their best practices is to organize based on function. This means that instead of trying to mirror the actual organization that you have, you should group based on security and operational needs. With AWS Organization, you can cluster accounts into Organizational Units (OUs) and make them follow the same rules. Also, it’s possible to use a hierarchy to make child accounts inherit the same rules with the same Service Control Policies (SCPs) as their parent. Like this, securing things down the line is more manageable if you allow specific services from the top.

Let’s share access

After getting organized into a beautiful pattern we choose, we still need a way to get our colleagues in. We don’t want to do all the work by ourselves, no?

When managing access, AWS best practices tell us that you should always prefer a federated access method, leaving us with AWS Identity Center (former AWS SSO) or a SAML-based approach. Which one to pick largely depends on how you usually set up accounts and organizational units, so let's overview the two.

For AWS Identity Center to work, you need to enable it with your AWS Organization's root account. Once enabled, you can manage all directory users from the AWS SSO portal and give access to single persons or groups through roles. You can use the internal user directory of AWS SSO or link an external guide with some third-party identity provider.

AWS SSO can be the most straightforward way to access your multi-account environment if you have relatively few accounts and users. It's easy to use, takes minutes to set up, and has a neat integration with the AWS CLI to impersonate your roles directly from your local environment.

But it's when you reach a certain amount of users and accounts you start to have some problems in management, and it becomes increasingly hard to keep up with that amount of resources. In my opinion, that threshold is reached at 20-30 accounts and users, at which point AWS SSO can't keep up with everything, and you need another solution.

On the opposite side, the SAML-based approach is the manual way of doing AWS SSO. This approach's downside is setting up the federation process and syncing between your user directory and roles. You have strict control over your identity provider and AWS accounts and less magic on the advantage side. And you don't need access to your AWS Organization root account because all setup can be done in IAM.

And now, starting operations in AWS

Still, there is an elementary problem that we need to address, and it’s more on the operational side of things. Once we secured and implemented a tremendous multi-account strategy, how do people access AWS accounts? It turns out there is a fantastic open-source tool that lets you handle that with no effort, and its name is Leapp.

Since there wasn’t a tool that solved the full spectrum of the complexity of this problem, my team and I implemented it. Leapp is designed to rotate temporary, short-lived credentials generated from sensitive data encrypted in your local system vault; and for you to use with any AWS-compatible tools (i.e., AWS CLI, Terraform, CDK, etc.) and access your AWS console too.

Conclusions

Now that we’ve started implementing our multi-account structure with AWS Organizations, our situation looks more like this. Seems a lot better now.

So, to recap everything we’ve seen today: we’ve used the AWS account definition to navigate the critical concepts of isolation and security; the first for organizing your account structure and the second to understand who is responsible for its security. Then we have seen a few best practices and learned that account security is a journey rather than a few spot actions.

Then we scaled out to apply the same model to multiple AWS accounts and saw an implementation for organizing and sharing through AWS Organizations. We’ve learned about Service Control Policies to define boundaries and Organizational Units to structure the accounts hierarchy, reaching our ultimate goal for the implementation of a great multi-account strategy.

I hope this was interesting to read; we can go more in-depth next time.

Is automating processes also your thing? Do you like to find solutions to your everyday problems and want to share them with others? Then join our community.

Until next time! Thanks for reading, and have a great day!

Leapp and the Windows Certificate expiry

Nicolò Marchesi — Thu, 09 Jun 2022 14:28:49 +0000

In the last two weeks, we had a few problems with the Windows certificate of our open-source project Leapp. In the spirit of complete transparency, I'll try to highlight what went wrong and why.

Many thanks to Alessandro Gaggia for the support in writing this.

Introduction

As many of you are aware, Windows implements a security system named User Account Controls (UAC) that requires the administrator access token must prompt for consent. To pass UAC on Windows and results as a verified application, one must sign its executables with a .p12 (.pfx) code signing certificate. This goes for the installer, app, and uninstaller as well.

These certificates are issued by specific companies that have the authority to validate your company and your Brand from a legal perspective, ensuring that you are what you declare to be and that the software is not malevolent.

In our case, we used Sectigo, formerly Comodo, as our certification authority. But, as you may guess, the journey was a bit bumpy.

Phase 1: Generating .p12 certificate

Every year we renew our code signing certificate, typically in May. What are the steps to acquire a new certificate? Let's start by obtaining a new .crt certificate:

We register and log in to our certificate authority website (sectigo in our case)
Inside your dashboard is a voice called certificates: Go to the "code signing" sub-item.
Here you'll eventually find the old certificates and a green button to buy a new one
On the new page, we need the first type of certificate, also known as the "legacy type certificate" or OV (organization validated) certificate. Currently, we are not using EV (Extended Validation) certificates, but we're opting to change the next time we'll have to renew.
After completing the order, you'll find the new certificate requested in your account's certificate section. And now the fun part begins.
You must also generate a CSR (certificate signing request) on your machine when requesting the certificate. When asked, use "Generate CSR" under the personal info form (which will do via your default browser). This is very important as it contains the certificate's public key and your organization information and is used to request your new certificate.
After "generate CRT", we waited for validation. And here started the delays.

Phase 2: Speeding up validation

When we requested our certificate, we didn't do anything particular the first days, believing that the process would have gone by itself; however, after a few days, we saw that the process was stuck. What did we miss?

We had to verify our organization's identity by passing two critical things to the form that comes with the receipt email and that tracks the validation process:
the organization DUNS' number
A document that demonstrates the existence of the company. We chose the Chamber of Commerce registration document. However, we needed to contact the support team 2 times to make them accept this document. For reference, in many countries with a good volume of imports/exports with Italy, some places can validate this document, even abroad.

After wasting six days of back and forth, we managed to make them accept our document but still, we needed a photo of the face of one of us with our ID card near us for validation. So we needed to take the picture, upload it and wait another day for it to be accepted. And this was another silly problem: the first time, the ID card was not near enough to our colleague's face, which was invalidated. We had to retake a picture, and we wasted another day.

After another call with the support team, we obtained the certificate, which was already in .p12 format and could be renamed to .pfx (they are the same except for the extension name). Finally, we had a certificate to be used to sign the window executable.

Side thoughts on what happened

During this process, we felt primarily pressured by the delays of the supplier but also by our user base. The sudden problems with the application meant that we were flooded with reports and requests to fix this problem for about two weeks.

Mind me; I try to see this in the brightest light: people were so much in pain not being able to use Leapp that they forgot that the project is entirely free and open-source (and they can build and fully utilize the application themselves). Luckily, when we arrived at the thought of buying another certificate, we resolved the issue. But what pained me the most was that few requests weren't exceptionally polite.

We have a company support program that can take care of like-wise situations. We strongly suggest adhering to it if organizations and companies rely heavily on Leapp to manage their access. The support program is designed to take into account situations like this and gives us the margin to continue providing for the community and free users. Everybody wins.

Also, you get to have a privileged interaction with the development team along with ours and the community's gratitude :)

Top 10 uncommon DevOps tools you should know

Nicolò Marchesi — Thu, 24 Mar 2022 10:43:00 +0000

Hello everyone! You clicked here to take a laugh at yet another boring list, right? I’ll try to surprise you with a list of unique tools that you probably still don’t know! I know you already know about Git, Terraform, Jenkins, and so on, so let’s focus on the tiny pearls you can find out there! I’ve chosen only open-source tools to spread some love; if you find anything helpful, you can contribute back.

1. Chaostoolkit

As a DevOps (and software engineer), I find chaos engineering one of the most relevant and thrilling topics to study right now. Chaostoolkit it’s a perfect way to introduce into this world through a straightforward and flexible command-line tool and a way to define experiments with declarative files that you can version.

2. Insomnia

I bet you all know Postman for API design, right? I prefer Insomnia.rest since they’ve taken the extra mile to open-source the complete application and integrations through their plugin hub. I know it’s a bit more popular nowadays, but I started using this when there were just a few thousand stars, and it got better and better over the years. I think the trend will keep up.

3. Leapp

Leapp is my go-to tool for accessing my cloud accounts. I was fed up with manually managing local development and operations credentials, so I automated everything most securely. I can’t count how much time it saved me over the years.

It integrates with nearly any development tool (Terraform, CDK, etc.) and some excellent additions like connecting directly to SSM. Like Insomnia, it’s completely open-source, and the repository and slack channel are very active if you want to come around. Also, the tool is getting better at each release, and there is an excellent roadmap to keep up with what’s new.

4. Gitea

If you don’t live under a rock, you already know about mainstream git repository software like GitHub and GitLab. But what if a highly lightweight, community-driven, and self-hosted alternative exists? That’s gitea.io. It’s not as full-fledged as its counterparts, but it’s highly promising and painless to install and use and has a very active and welcoming community.

5. Hubot

Hubot is kind of old (at least in software terms!), but I’m amazed at how few people know about the ChatOps model. From 10000 thousand feet perspective, it’s just automation through your go-to chat software (discord, slack, rocketchat, mattermost, etc.), and Hubot paved the way to other similar tools. Maybe it’s less relevant today, but I find an interesting concept nonetheless (I love automation in any form it can take) that can solve a few issues, especially for less technical people.

6. Mkdocs-material

I love markdown, and MkDocs is my go-to tool for writing documentation, but it’s pretty bare. mkdocs-material is a template that you can put mkdocs on… except it packs a whole world of additional features: versioning your documentation, a native cookie consent solution, rich search previews, and a truckload of other things… seriously give the guy 10$ a month to get supporters-only features, and you’ll get the only documentation tool you will ever need. (for instance, the AWS Copilot CLI docs were written with this, see for yourself, that’s amazing).

7. Podman

I’ve actively used Docker for almost all my container needs, but I’m always happy to take a look at new and emerging technologies. What got me into Podman was the daemonless architectural pattern, and nowadays, I like where the project is going. However, it’s still rough around the edges (especially the docs). Still, when I tried it out the first time, I was able to drop it in the middle of my Dockerfile, and everything just worked.

8. Sshuttle

Sshuttle is an excellent tool that acts as a “poor man” VPN, allowing you to create a VPN connection from your machine to any remote server you can connect to via ssh. The exciting part is that it is not precisely a VPN and not exactly port forwarding. Internally it assembles the TCP stream locally, multiplexes it statefully over an ssh session, and disassembles it back into packets at the other end to achieve data-over-TCP, which is safe. Useful if your VPN breaks down.

9. Infracost

If you’re a Terraform fan, you’ll love this one. What if I tell you you can couple your infrastructure as code with bill forecasting? Sounds fantastic, huh? That’s what infracost.io is all about: it will scan through your Terraform files when you commit some changes to git and estimate the resulting billing of your changes! Pretty handy to have before getting unpleasant surprises.

10. Checkov

Ok, here’s the final pearl. Checkov.io is a static code analysis tool for checking infrastructure as code misconfiguration. I cannot express how much this one can help find and fix the most basic (and advanced) security problems in your cloud infrastructure, and it comes with support to a whole world of different IaC tools. I also love it because you can run it from the command line.

Final thoughts

I hope this was useful to delve a bit more into the technical tools of the DevOps and cloud world. I honestly think that newcomers and experts would benefit as there is always something more to experiment on and learn. Do you think something is missing? Write a comment or send me a DM and I’ll be sure to check them out and include them on the next post, see you next time!

How to access AWS and Azure in Terraform

Nicolò Marchesi — Thu, 04 Nov 2021 10:23:55 +0000

Intro

Being a cloud developer can be hard but there are tons of tools and resources to help smooth the hard edges. One of the tools at our disposal is Terraform, an open-source infrastructure as a code software tool that provides a consistent CLI workflow to manage hundreds of cloud services. But how exactly Terraform is able to connect to those services and create our infrastructure? And can we find the best way to manage our resources? Let's see it.

What you shouldn't do!

Specifying credentials in provider's block

For the love of everything that is good, please don't ever do this if you don't want to quickly find out a new world of grief and pain:

In AWS and Azure documentation, you will find some warning with mild words and a funny little warning block saying this is not recommended; this doesn't even start to convey how much bad practice is hardcoding credentials, so I'll tell you more bluntly:

🔥 DON'T. NEVER. EVER. HARDCODE. CREDENTIALS! 🔥

The internet is full of horror stories of people lazy or careless enough to find themselves with big surprises in their next cloud provider bill. In the most recent one, a guy found out a 65k bill because he "handled source code with hardcoded credentials to other devs".

Do you want to become part of this group? There are better ways to connect your Terraform with your cloud accounts, so let's follow best practices, shall we?

What you should handle with care

Putting credentials in environment variables

The first thing you can do to connect Terraform with your providers is to leverage environment variables. You need to export all the data needed to connect into the default environment variables and you're good to go. Just declare an empty provider and Terraform will automatically pick them up and run.

export ARM_SUBSCRIPTION_ID="00000000-0000-0000-0000-000000000000"
export ARM_TENANT_ID="00000000-0000-0000-0000-000000000000"
export ARM_CLIENT_ID="00000000-0000-0000-0000-000000000000"
export ARM_CLIENT_SECRET="00000000-0000-0000-0000-000000000000"

export AWS_ACCESS_KEY_ID="0000000000000"
export AWS_SECRET_ACCESS_KEY="0000000000000000000000000000000"
export AWS_DEFAULT_REGION="us-east-1"

provider "azurerm" {
  features {}
}

provider "aws" {}

Only downside of this approach is that managing this environment variables manually is a real pain. How cool would be to have an automated way to handling this? ;)

🔥 Refrain from exporting directly in your .bash_rc, .profile or any other automatically sourced configuration files. Those file are not encrypted you it's a very similar situation as the hardcoded credentials. Forewarned is forearmed.

What you should do! (finally)

Here things will split a bit for AWS and Azure, but I'll try to keep them as consistent as possible. We'll mostly look at interactive ways so keep that in mind.

On Azure

Logging through the CLI

This is the default and most secure method for interactively working with Terraform. By running the az login command on the Azure CLI it will export the parameters needed to work into environment variables. The only problem you can incur is by having multiple tenants; in this case, you need to specify in the Terraform Azure provider the one you will use.

provider "azuread" {
  tenant_id = "00000000-0000-1111-1111-111111111111"
}

Using service principals

Service Principal authentication is usually more suitable for automated workflows like for CI/CD or running application scenarios. This type of authentication decouples the authentication process from any specific user login and allows for managed access control.

We can see them as the Azure counterpart of IAM Roles in AWS, and one trivial advantage would be that it can be shared between people without having to resort to groups or tie directly to an identity. In essence, by using a Service Principal, you avoid creating fake users in Azure AD to manage authentication when you need to access resources.

On AWS

Assuming Roles

Assuming a role is the single most simple and secure way to interact with your cloud resources through Terraform. It comes easy as defining the assume_role parameter in the provider and filling out the role you want to assume, the name of the session (it will be handy if you need to trace calls through monitoring systems such as CloudTrail), and an optional external_id that serves as a secret.

provider "aws" {
  assume_role {
    role_arn     = "arn:aws:iam::ACCOUNT_ID:role/ROLE_NAME"
    session_name = "SESSION_NAME"
    external_id  = "EXTERNAL_ID"
  }
}

Overall is a good method but it suffers the same problem as the hardcoded credentials in the sense that when you commit it into your repository anyone can put their hands on this and manage your account.

Shared Credentials file

You can also make all those information external by telling terraform to look up a shared credential file. The default location is $HOME/.aws/credentials on Linux and macOS, or "%USERPROFILE%\.aws\credentials" on Windows. This way the only information you need to exchange with Terraform is the name of the named profile and the CLI will automatically perform the assume role for you.

provider "aws" {
  region                  = "us-west-1"
  shared_credentials_file = "/Users/tf_user/.aws/creds"
  profile                 = "customprofile"
}

This can be further enhanced by using multiple named profiles.

No MFA support, Leapp to the rescue!

Currently, Terraform doesn't support the mfa_serial while assuming roles according to this issue:

Doesn't ask MFA token code when using assume_role with MFA required #2420

jsi-p posted on Nov 23, 2017

When using multiple AWS accounts it's good practice to only allow access via AssumeRole from a master account. This can be done with or without requiring MFA. Terraform supports assume_role with s3 state file and aws provider configurations, but doesn't seem to ask the MFA token code when one is required. This prevents using AssumeRole for credentials when MFA is required.

AWS documentation describing MFA with cross account AssumeRole: http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_mfa_configure-api-require.html#MFAProtectedAPI-cross-account-delegation

Terraform Version

$ terraform --version
Terraform v0.11.0

Affected Resource(s)

Both of these support assume_role, so they should also support asking for MFA token code:

S3 backend configuration
AWS provider configuration

Terraform Configuration Files

terraform {
  backend "s3" {
    bucket = "terraform-state-bucket"
    key = "tf-state"
    region = "eu-west-1"
    role_arn = "arn:aws:iam::916005212345:role/OrganizationAccountAccessRole"
  }
}
provider "aws" {
  region = "eu-west-1"
  assume_role {
    role_arn = "arn:aws:iam::916005212345:role/OrganizationAccountAccessRole"
    session_name = "terraform-session"
  }
}

Actual Behavior (with DEBUG)

$ TF_LOG=debug terraform init
2017/11/23 13:36:12 [INFO] Terraform version: 0.11.0  
2017/11/23 13:36:12 [INFO] Go runtime version: go1.9.2
2017/11/23 13:36:12 [INFO] CLI args: []string{"/usr/local/Cellar/terraform/0.11.0/bin/terraform", "init"}
2017/11/23 13:36:12 [DEBUG] Attempting to open CLI config file: /Users/***/.terraformrc
2017/11/23 13:36:12 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2017/11/23 13:36:12 [INFO] CLI command args: []string{"init"}
2017/11/23 13:36:12 [DEBUG] command: loading backend config file: /Users/***

Initializing the backend...
2017/11/23 13:36:12 [WARN] command: backend config change! saved: 16375281725947963338, new: 2383462577283113429
Backend configuration changed!

Terraform has detected that the configuration specified for the backend
has changed. Terraform will now reconfigure for this backend. If you didn't
intend to reconfigure your backend please undo any changes to the "backend"
section in your Terraform configuration.


2017/11/23 13:36:12 [INFO] Building AWS region structure
2017/11/23 13:36:12 [INFO] Building AWS auth structure
2017/11/23 13:36:12 [INFO] Setting AWS metadata API timeout to 100ms
2017/11/23 13:36:13 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2017/11/23 13:36:13 [INFO] Attempting to AssumeRole arn:aws:iam::***:role/OrganizationAccountAccessRole (SessionName: "", ExternalId: "", Policy: "")
2017/11/23 13:36:13 [INFO] AWS Auth provider used: "SharedCredentialsProvider"
2017/11/23 13:36:13 [DEBUG] plugin: waiting for all plugin processes to complete...
Error initializing new backend: 
Error configuring the backend "s3": The role "arn:aws:iam::***:role/OrganizationAccountAccessRole" cannot be assumed.

  There are a number of possible causes of this - the most common are:
    * The credentials used in order to assume the role are invalid
    * The credentials do not have appropriate permission to assume the role
    * The role ARN is not valid

Please update the configuration in your Terraform files to fix this error
then run this command again.

View on GitHub

Leapp can work around this specific scenario by performing the assume role, prompting for the MFA token, and injecting into the credentials file the temporary credentials and tokens to let you use Terraform without further interaction. Nice, isn't it?

Conclusions

Noovolari / leapp

Leapp is the DevTool to access your cloud

Leapp

Website: leapp.cloud
Documentation Website: docs.leapp.cloud
Roadmap: Roadmap
Chat with us: Slack

Leapp is a Cross-Platform Cloud access App, built on top of Electron.

The App is designed to manage and secure Cloud Access in multi-account environments.

Key features

We Strongly believe that access information to Cloud in ~/.aws or ~/.azure files are not safe, and we prefer to store that information in an encrypted file managed by the system. Credentials will be hourly rotated and accessible in those files only when they are needed, so only when Leapp is active.

Switch Cloud Profile in a click
Secure repository for your access data
Multiple Cloud-Access strategies
No long-lived credentials
Generate and use sessions directly from your aws Organization
Connect EC2 instances straight away

All the covered access methods can be found here.

Installation

Get here the latest release.

Contributing

Please read through our contributing guidelines and code of…

View on GitHub

As we've seen there are a lot of ways to interact with Terraform for both AWS and Azure. When choosing our access methods we should always think about the security concerns and avoid at all costs manual interactions with the configurations file. For all of this, Leapp is the best tool to achieve both security and automation, abstracting away all underlying interactions and letting cloud developers focus on their tasks.

If you found this interesting, have any questions or you just want to drop a DM and connect you can find me on twitter or join our slack community. See you soon!

Road to Noovolari

Nicolò Marchesi — Wed, 30 Jun 2021 08:35:19 +0000

The journey to tech startup

What drives a person, team, or company to reach its potential? How do you discover the right approach to achieve it? And why it's critical to find motivation and define aspiration? Those are questions that our team, Noovolari, asked several times in the wandering of finding our path.

I bet that many of you will think that they are abstract, convoluted, and probably meaningless, only useful in some third-rate marketing strategy. Trust me; they are not. Taking the time to understand who you are and why you are doing what you are doing is central to achieve complete fulfillment, both personal and in business.

Don't get me wrong; it's an incredibly challenging ride with many blurred answers. Still, it's a trail we've been following for the last seven years, and today we will shed some light on what we did to achieve this formidable goal (spoiler: yes, we did it!); from how everything started and how we changed along the way up to discovering and embracing our true vision.

A safe haven to return at when in doubt.

The first steps

It's impossible to talk about Noovolari without starting from beSharp, one of the first companies in Italy to focus and train in cloud computing expertise.

BeSharp started as a university spinoff. Everyone shared the passion for solving problems with a deep technical approach, expanding and polishing the skills needed in every technology aspect involving the cloud.

Since day one, most people in beSharp were focused on becoming advanced AWS consulting partners for business, architecting and developing cloud infrastructures, and hand-crafted solutions for customers.

Instead, my team focused on research and development to learn and bring innovation to the whole company. We always had freedom in aiding our consulting associates and their customers as we saw fit; that's why we never focused on a single cloud provider. Instead, we explored the whole cloud landscape to create new and more broad solutions. Our projects ranged in vastly different cloud areas, from adaptive pipelines for genomic high-performance computing to application consistent backup and disaster recovery. Our team's name, Noovolari, was a pun involving the word "cloud" in Italian (e.d. nuvola), and the famous racing driver Tazio Nuvolari; meant that we wanted to go fast in the cloud.

We explored a little bit of everything, but we soon realized that the best results came from improving our internal processes. Our knowledge and understanding of the underlying platforms gave us an unfair advantage in creating new developer tools. Despite our small team and endless stream of work, we were able to free many of our colleagues' burdens to make them focus on what really mattered to them.

Our mission was to put to good use our skills at scale.

Growing together

It was by chance that we started showing our work to people on the outside. In our chase to give our colleagues the best tools they could have, we started putting a bit more effort into the user experience. As a result, they started using our product during meetings in front of customers... and the feedback was astounding. People started asking to use our software, and we realized that we weren't solving problems just for us but also for many people. That was a perfect opportunity to put our expertise to good use.

One of the most crucial moments was our first database rollback in production. One of our customers had a major disruption on their workloads. Since they were using our software to manage their entire backup lifecycle, it felt natural to support them. It was our first test drive, and we couldn't stop thinking about what would happen if something went wrong. And in a few minutes, the application was working as if no problem ever came up.

We were able to solve a real problem for a real customer.

Keep calm and stop

Things were going well at that moment. We were getting acknowledged and building more and more awesome features. We started making arrangements to offer the software as a service, subscribed our software to several marketplaces, and plan our commercial offering. So we felt on the edge of starting our journey as a product company. But we never took off.

Aside from few customers directed to us by our colleagues, very few people came after our solution through other channels. As this situation persisted until the beginning of 2020, we started wondering what was wrong with our project despite all our work. We were confident about our solution, but still, customers didn't flock at us as we expected. Why?

We thought the best way was to gather the courage and contact someone who already walked the path we were trailing on. We asked more successful entrepreneurs to give us some feedback on our strategy, but it was painstaking. We were scared of what they could tell us and how that could affect our morale. After some weeks of attempts, we finally found a person willing to help. And just a few minutes into the meeting, he said:

Let me guess... are you all engineers?

That was it.

At that moment, it flashed the idea that our situation was more common than we thought. The problem with engineers is that we think only in terms of features and development cycles. We thought that making the best technical product was enough for people to come by themselves, and we just had confirmation that this wasn't the case. We were really naive.

This brief call opened pandora's box, and we started to see the fallacies in our process. From focusing all our efforts on the next big feature to the grave lack of communications with our potential customers and the small and biased customer feedback we collected. We just weren't aware of the several things needed to make a successful product. And diving deeper into the hole just popped out more and more questions... it seems as if there was no end to it.

That was one of our hardest moments. We felt overwhelmed by the things we took for granted, but we couldn't go on without a plan or answers.

We decided to stop everything and go back to the fundamentals.

Time for questions

The first thing on the agenda was a retrospective of the last year of development. We focused on the creative process and feature proposal up to the customer feedback because we felt that something was off in the decision-making and feature selection. So we first examined the backlog and roadmap to find out why we chose to develop some specific features.

Up to that point, we used a scoring system based on reach, impact, confidence, and effort to value all our potential features. Each element of the scoring system had some internal weights that could tweak the overall score in favor of our current goal. For example, lower technical debt, catch up to competitors or develop new features in the market. So far, so good, we even thought that we did a pretty good job!

But the most affected phases were validation, feedback, and custom development. We assumed that our choices were objective because of the multiple steps involved, but we missed the point in disregarding the work at the top or the bottom of the process. We prioritized things based on what our colleagues doing consultancy needed at that moment, and their necessities swayed their feedback. Moreover, we offered to integrate custom features for our current customers. Trying to fit those features as generally available increased the technical debt. In the long run, it slowed us down to the point where new features required complete code refactoring.

After focusing inside, we shifted the perspective on the outside. We already had a market and competitor analysis, but after a review, we found out its main goal was to find features to bridge the gap or find features that could set us apart from the competition. We didn't make a broader analysis to find out what was setting us apart from the competition. So we started from scratch and tried to build a clear understanding of the whole market and, more importantly, where we could belong.

First, we drew a competitive landscape, a visual representation of each market segment, and where all our competitors belonged in that space. Next, we went into an in-depth analysis of each competitor's features to determine the gap between us and discover their strengths and weaknesses. It was a lengthy process that needed few iterations, and it proved invaluable to have a visual aid for discussion since the amount of data to deal with was huge, and not everyone was focused on this task. Putting everything together required a lot of effort and brainstorming, but we emerged with some answers, and most of our fears and doubts materialized.

We always thought we could manage to find our niche by creating the best possible technical solution, but we had difficulty explaining how exactly that could find its place in the landscape. It seemed that we lacked a proper area that set us aside from the competition. Moreover, the gap with our competitors was too wide. As a team of barely four people, there was no way we could catch up to their latest features.

Last but not least, we went into a thorough analysis of our marketing strategy, and we realized that we needed to take the matter into our hands. We relied too much on internal feedback and word of mouth; we needed to build an online presence. But based on what? We still didn't have a unique identity. Things were looking pretty grim, but still, we didn't want to give in.

We just needed to tackle the problem from a different perspective.

Back to why

Despite all the analysis and work we did, we were still far from finding our place. Again the help came in the unexpected form of a simple yet powerful phrase that's been my guiding principle since then:

When in doubt, be yourself

It may seem trivial, but it's easy to wander and lose focus when you're trying to do something very complex. What works really well is to have somewhere to return to, so you don't need to waste precious resources thinking about what's wrong. We focused on the process, analytics, markets, and business development, but we weren't sure what were the reasons that brought us together and why. We lacked a safe haven.

What helped us get going was the famous golden circle:

The method is super inspiring and challenges the status quo at its core for identifying your purpose for what you want to do in business and life. So many people think about this only in marketing terms, but we've relied on it as a framework to find our deepest meaning, and it has really served us well.

We had many brainstorming sessions filling on post-it notes what were the reasons we started, what we wanted to achieve, and what was driving us forward. In other words, the reasons we were doing what we were doing.

As you can guess, the "what" and "how" were pretty straightforward and slightly deviated from the average answer. Still, it was important to start with setting some common ground. Unfortunately, the "why" part was countless times more difficult. Although I know this sounds like an easy task, we got a different answer to every question, and they started piling up pretty fast.

It was important to use physical support to have a quick overview, so we put every note on a wall. We then started clustering our notes and give a general meaning to each cluster. Finally, each reason was put in a thorough examination and distilled into something we all agreed upon. I'll spare you the details about this, but it required a lot of meetings and talking.

In the end, this is what emerged:

Imagine a future where the public cloud will be the key to opening the doors of innovation to your business. What could we do if the complexity of the workloads in multi-cloud environments did not allow our users to focus on their core business? And how extraordinary would it be to accomplish all of this day after day, thanks to a work that inspires us, makes us happy, and makes us feel fulfilled?

We create products that revolutionize the way we work in complex multi-cloud environments, allowing our customers to achieve their business goals through the unification, centralization, and facilitation of cloud processes following best practices. We want to be a beacon for companies seeking a standard for complex processes in multi-cloud environments.

We believe in a world where everyone can contribute to evolve and innovate faster; we believe that all digital products should be open to contributions. We want to grow as people and be aware of the extraordinary goals that we can achieve, thanks to mutual trust and teamwork.

Get to the how and what

We had done it. 3 months had passed, but we finally had our compass.

Now it was about fitting our vision into the market... and it came unexpectedly easy. One of the few things that came up during brainstorming was that we believe in a world where everyone can contribute to evolve and innovate faster. Of course, that ringed a bell with open-source.

As software engineers, we had previous experience in that area, but we never really thought about it in business terms. So we were again back at square one, but with a viable option in mind, that could really set us apart. So in the next few weeks, we dedicated ourselves to learning everything we could about making open-source products and make them sustainable through commercial offerings. And we loved what we saw.

It was the perfect union of our beliefs and how to reach them.

Nowadays

This brings us more or less to one year ago, so now you're probably wondering what we've been up to since today? We've stopped the projects that didn't perfectly align with our vision; the effort to change them on the run was too high to be reasonable. But the good news is that we are not short of ideas.

We had a developer's tool to grant easy and secure access to our large numbers of accounts during all this time. We created it out of the necessity of increasing the number of our cloud accounts to comply with the best practice to segregate cloud workloads. It was nothing fancy, but we realized that it became a standard for our whole company in no time. Also, like before, a few of our consultancy associates' customers are starting to ask about it in detail.

But now we are prepared... well... at least more prepared than before!

We spent the rest of last year making it an open-source tool, built for cloud developers like us. And now it goes by the name of Leapp.

I'd really like to delve into the full details about how we did it, including how we structured our internal process, how we built the product roadmap, the development methodology, the go-to market strategy, how we have structured the commercial offering, and so on... but this will require at least a couple more articles.

We're still early on our journey, but things are moving as never seen before. We're about to reach 450 stars on Github repository, about 750 stable downloads at each release, and offer our first commercial support plan. We have even a few surprises waiting for you in the next months. As a little spoiler, we are planning another software specifically built for teams on top of Leapp. But we'll make a big reveal in due time!

Meanwhile, if you'd like to join us to have a chat, discuss some new features, or keep up with the news, join our community on Slack and Github.

We are Noovolari and we'll do wonderful things together.

Enhanced Serverless GitHub Metrics

Nicolò Marchesi — Thu, 11 Mar 2021 13:37:53 +0000

Since we started Leapp a few months ago, we wanted to steadily improve our open-source project to engage better all people interested in working fast and safely in the cloud.

But to improve on something, you need to be aware of what you want to improve. So the first thing we noticed was the Insight page inside our GitHub repository. You can find a ton of useful information there, but it's limited to 1 month of data, so my thought was: let's fix that!

Meanwhile...

What better chance to learn something new? Since the project was simple enough, I decided to take a little extra time to study and research new technologies that could come in handy for future projects. I like to learn new things consistently to keep up with our ever-changing world... and while having fun too!

I wanted to share the whole process and my takeout at the end, so if you're searching for inspiration, I hope you're in the right place!

Requisites

So, before writing even a few code lines, I want to have a clear overview of how I would build my project. Here's my thought process.

Choosing the framework and language

My drive here is to do some research to evaluate technologies for other projects to use in production. Since this is an easy goal, it's perfect to avoid incurring major technical issues while having the opportunity to dive deep enough to discover any significant flaws I can encounter in the future.

For a few months, I wanted to try out TypeScript and check if the Twitter hype was right for once, while for infrastructure as code (IaC), I never had the chance to research Terraform. In the end, this was an excellent use-case to finally do some serious research and development with a clear goal.

As for deployment and data persistence, I want to use something that I'm confident in; since I'm already adding uncertainty with a new programming language and deployment frameworks, it's better to stop here with the novelty and avoid delays.

Choosing the data-persistence layer

For this, I need something inexpensive and with good filtering capabilities since I'm going to query for dates. Moreover, I don't want to pay for idle, so the choice is between DynamoDB and Aurora Serverless. Since I don't forecast to access this data a lot, I think DynamoDB is a better fit. For the filtering part, I learned the hard way that it's always better to devise your queries first; I took some time to forecast what kind of questions I would run to retrieve my data, and i's something like:

How many GitHub stars did we have on that date?
How many Contributors did we have on that date?
Plot all Stargazers that were present between those dates

It's pretty clear that I wanted all data divided by date, so I designed the table to have the date set as my primary key. Unfortunately, the date range query was applicable only in two scenarios: secondary indexes or scans. But the data I'm expecting to store and search for is tiny, so I can safely ignore the mantra "never use scans" and use the date as my primary key.

Other considerations

Since I want complete automation and daily granularity, the best thing to do is fire off my Lambda on a daily cron job. To maintain the serverless spirit, CloudWatch Alarms is a perfect fit for that.

Final choice

So, with the final choice, I went with:

Platform: AWS Lambda
Language: TypeScript
Infrastructure as Code: Terraform
Data Persistence: DynamoDB
Events: CloudWatch Alarm

Developing

Once settled with the whole technology stack, it was time to do some testing and development.

Extracting data

The first thing I did was use the GitHub SDK with Octokit to query for data and start defining the data structure. I will later save this to DynamoDB.

This thing would be as fast as lightning to implement, but I made an encounter with the most dreaded thing of the whole JavaScript engine... the Promise! Octokit only provides a client that could be interacted only with asynchronous interfaces.

Thanks to MDN for the clear explanation.

After that, it was pretty easy. Put await in front of each promise to make them synchronous and extract a bunch of data to store in an Object:

async scrape(): Metrics {
        const s = await this.get_social()
        const cic = await this.get_issues_count(IssueState.Closed)
        const opc = await this.get_pulls_count(IssueState.Open)
        const cpc = await this.get_pulls_count(IssueState.Closed)
        const r = await this.get_releases()

        const g: Globals = {
            closed_issues_count: cic,
            closed_pulls_count: cpc,
            forks_count: s.forks_count,
            open_issues_count: s.open_issues_count,
            open_pulls_count: opc,
            stargazers_count: s.stargazers_count,
            subscribers_count: s.subscribers_count,
            total_issues_count: cic + s.open_issues_count,
            total_pulls_count: cpc + opc,
            watchers_count: s.watchers_count
        }

        const m: Metrics = {
            globals: g,
            releases: r
        }

        return m
    }

As you can see, I divided the metrics into globals and releases. The first contains all data tied to the repository (Stars, Contributors, Traffic, etc.), while the latter will store all releases versions and the number of times each file has been downloaded.

Writing data

The writing part was the most tricky... guess why? Again the Promise!

Since I already created the Lambda handler, it took me few attempts to return the Promise to be resolved.

import { DynamoDB } from 'aws-sdk';
import { Metrics } from "./scrape/dto";

export async function write(table: string, metrics: Metrics): Promise<any> {
    const today = new Date()
    const calendar = { date: today.toISOString().substring(0,10) }

    const item = { ...calendar, ...metrics.globals }
    const dynamo = new DynamoDB.DocumentClient({apiVersion: '2012-08-10', region: 'eu-west-1'});
    return dynamo.put({
        TableName: table,
        Item: item
    }).promise();
}

The lambda handler

And here, for the sake of completeness, the handler:

import {Scraper} from "./scrape/scrape";
import {write} from "./writer";

export const handler = async (event: any = {}): Promise<any> => {
    const scraper = new Scraper(process.env.OWNER, process.env.REPO, process.env.AUTH);
    const metrics = await scraper.scrape();
    return write(process.env.TABLE, metrics);
}

Since the handler is a Promise, and the Lambda runtime environment has a wrapper that resolves it, you don't need to wait for the write function to complete before returning it. I'm returning the Promise directly from the DynamoDB put operation.

I know it's pretty dry, but I like to have slim handlers to focus on the business logic without worrying about how they will run in the Lambda environment.

Building the infrastructure

After making everything work on the development side, I went to build and create the infrastructure. I set up everything in a folder named terraform to generate a file for each infrastructure element. Something like:

Terraform
- cloudwatch
- dynamo
- iam
- lambda
- Etc.

The variables

Since I want to consistently work between the local and cloud environments and always prefer to generalize and reuse my code when possible, I always start by forecasting the variables I need to pass to the function. Since I'm using CloudWatch Alarms to fire off Lambda those variables will be later transformed into environment variables.

variable "aws_region" {
  type    = string
  default = "eu-west-1"
}

variable "aws_profile" {
  type    = string
  default = "default"
}

variable "vertical" { }
variable "stack" { }
variable "project" { }

variable "owner" { }
variable "repo" {}
variable "auth" { }

data "aws_region" "current" {}
data "aws_caller_identity" "current" {}]

I would need the owner and repo to configure the data source and the auth token to establish the connection. Since I'm using Leapp to manage access to the cloud account, I've configured the aws_profile to pick it up from default.

The .env

As we have previously seen, I want to populate the variables for Terraform automatically. For this, I think the dotenv project is helpful. Set up a .env file to store your variables and, if there isn't an integration or you're a purist (like me), you can source the file and run any command you want like this: "source .env && terraform plan." It will export all variables, and Terraform will pick them up and assign them to the elements defined in the variables file.

The only downside here was to export manually with TF_VAR prepended to each variable. But you know, it should take about 10 seconds to do that. I think I can live with it. **

VERTICAL={my-vertical}
STACK={my-stack}
PROJECT={my-project}
OWNER=noovolari
REPO=leapp
AUTH={my-github-token}
TABLE={eddie-metrics-global-table}
AWS_SDK_LOAD_CONFIG=true

export TF_VAR_vertical=$VERTICAL
export TF_VAR_stack=$STACK
export TF_VAR_project=$PROJECT
export TF_VAR_owner=$OWNER
export TF_VAR_repo=$REPO
export TF_VAR_auth=$AUTH
export TF_VAR_table=$TABLE

The locals

Here I put the local variable I will consistently use on my whole stack to avoid repeating them and clog the configuration files.

The vertical, stack and project variables are a 3-tier set of constants used to group and identify projects by name. I have defined a composite variable named full to avoid composing the full project name in the locals file.

Moreover setting the tags like a dictionary let me use them directly as a variable and forget completely about them, but still have them consistent.

locals {
  lambda_memory = 128
  full        = "${var.vertical}-${var.stack}-${var.project}"

  tags = {
    Vertical    = var.vertical
    Stack       = var.stack
    Project     = var.project
    Name        = local.full
    ManagedBy   = "terraform"
  }
}

The DynamoDB table

I'm cheating a bit because I had already set up the DynamoDB table with Terraform... but it's really tiny so let's just ignore that. As you can see, the tags variables it is really handy!

resource "aws_dynamodb_table" "dynamodb_table" {
  name           = "${local.full}-table"
  billing_mode   = "PROVISIONED"
  read_capacity  = 5
  write_capacity = 5
  hash_key       = "date"

  attribute {
    name = "date"
    type = "S"
  }

  tags = local.tags
}

The Lambda

Here I'll break down things a bit to better explain.

Archive Files

The archive files are just zip files that contain the function and the node_modules folders. The main things to notice here are:

While there is a convenient function in Terraform that does the packaging for you, I needed a more custom approach, and thus there are two commands in the package.json for building the Lambda and the layer. They copy and zip a folder.
Never forget to add the source_code_hash and the sha256 fingerprint! It allows Terraform to put a fingerprint on the package and automatically create new versions of the layer only if something is changed.

data "archive_file" "function_archive" {
  type        = "zip"
  source_dir  = "${path.module}/../dist/lambda"
  output_path = "${path.module}/../dist/lambda/lambda.zip"
}

resource "aws_lambda_layer_version" "layer" {
  filename            = "${path.module}/../dist/layers/layers.zip"
  layer_name          = "${local.full}-layer"
  compatible_runtimes = ["nodejs12.x"]
  source_code_hash    = filebase64sha256("${path.module}/../dist/layers/layers.zip")
}

The function

The function again is pretty primary. As you can see, I have referenced the data archives containing the Lambda and the lambda layer resource name. I added the environmental variables to let me configure the function from Terraform. If I need to deploy this to track another GitHub repository, I only need to change the variables in the .env file, and everything will run the same.

resource "aws_lambda_function" "lambda" {
  filename      = data.archive_file.function_archive.output_path
  function_name = "${local.full}-lambda"
  role          = aws_iam_role.lambda_role.arn
  handler       = "index.handler"

  # Lambda Runtimes can be found here: https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
  layers = [aws_lambda_layer_version.layer.arn]
  runtime     = "nodejs12.x"
  timeout     = "30"
  memory_size = local.lambda_memory

  environment {
    variables = {
      OWNER = var.owner
      REPO = var.repo
      AUTH = var.auth
      TABLE = aws_dynamodb_table.dynamodb_table.name
    }
  }
}

R&D conclusions

Here are my conclusion of my R&D on TypeScript and Terraform

TypeScript is a potent language. I've always had some problems with JavaScript and its lack of types, and with TypeScript, it feels like a complete experience. I still need to wrap my head around the async/await paradigm and Promise class on the downsides. It's not that they don't work, but, in my opinion, it adds unnecessary overhead while programming. Since TypeScript was an extension of JavaScript, which was designed for asynchronous operations and frontend, I think it's normal to shine in these environments. For backend, synchronous operations, and scripting, I think there are better choices.

Terraform was mind-blowing. What I conservatively forecast to take at least a day to get up and running was done in about an hour. The tooling was great, the documentation was clear, many examples around the internet and on the git repository... what more could I have to ask for?

There were some minor inconsistencies with the state management, though; just once something went wrong and generating the plan threw an unrecoverable error (probably due to my inexperience). In the end, I resolved pretty quickly by tearing all down and re-creating it, so no-big-deal.

And what about metrics?

After some research around the best metrics to track for open-source projects, I set everything up to save:

Open/Closed issues and pull requests
Forks, Stargazers, Watchers, and Subscribers
Total issues and pull requests

The idea was to get an overall feeling if we are sparking interest in other people and contributing with code or just with issues and enhancements. Those metrics proved very useful and gave us some insight, but after few months of running and reviewing them, we feel that something is missing along the lines.

Nonetheless, I'm happy we started collecting those metrics and weekly review them.

It's the drive to do better and improve our community and project.

Thoughts on Ruby 3?

Nicolò Marchesi — Sat, 26 Dec 2020 10:42:11 +0000

Ruby 3 was released yesterday on Christmas 2020.

What do you think of the new features?

AWS multi-account structure with AWS Organization

Nicolò Marchesi — Wed, 23 Dec 2020 14:25:48 +0000

Welcome back to the second part of how to access your AWS accounts. Noticed the final "s" on accounts? Yeah, today I will write about multi-account strategies. We laid the foundations with the previous article on how to access your AWS account by seeing the various strategies at our disposal and when they're a good fit. The good thing is that everything still applies, but we must look at this from the perspective of multiple accounts and organizations.

Because sooner or later, one AWS account will not be enough to contain all your resources in an organized manner, and nowadays, that moment will come pretty fast too!

The evolution of an AWS account

Looks familiar? This is more or less the evolution of the AWS account when I started using AWS. The thing is that when you begin extensively using a public cloud provider for development and production workloads, everything will escalate pretty quickly. There are just too many services and resources to keep track of while proceeding at a fast pace. More than ever, if you cramp them in a single AWS account.

Multi-account and consolidated billing

By the time I started working with AWS in 2014, there was already the well-established pattern of using consolidated billing to group multiple AWS accounts under a single credit card. That solved a slightly annoying process of keeping track in your AWS bill's central place, but there wasn't any way to centralize the governance.

We did solve this kind of problem by creating a root account with billing information where only a few people had privileged access. In the root account, only one person had permission to view billing, and few other ones could create roles and manage permissions for everyone else.

But we still needed a way to tie the corporate identities to the roles to enable single sign-on to the web console and APIs for our developers. We set up the federation process between the root account roles and the users in our user directory. On which, by the way, I've published another thorough tutorial here on how to SAML federate your AWS account with G Suite. Check it out if you're curious about the process.

The nice thing was that we solved the governance problem at the first step. We could define who could access what. The "downside" was that we had to set up a trust relationship with the root account to let people assume the other account's role from the role in the root one. And there wasn't a way to centrally restrict permissions.

AWS Organization to the rescue

In 2017 AWS announced AWS Organization, which was a big step in standardizing how to govern your AWS accounts. With AWS Organization, we can create and manage AWS accounts via Console and APIs and restrict AWS services with Service Control Policies (SCP).

It even enabled further organizing the accounts under the organization in Organizational Units (OU) that can share different configurations. On the account management notice, it still lacks the account deletion API, so the process of creating and deleting an account can't be entirely automated.

The other main feature of the AWS Organization is Service Control Policies. With these, you can enable or disable APIs on an account or OU basis, and I think it's the most underused and essential feature of the whole AWS Organization. Coupled with Permission Boundaries let other people than super-admins manage IAM, which is by definition the most sensitive service in an entire AWS account, without worrying too much that they can cause problems.

With this graph, you have a quick summary of how much you can restrict your AWS account access. Correctly identifying the permissions that it's safe to give by default and the ones that need some governance escalation to get can increase teams' velocity.

Multi-account access

Once you have gone to the length of set up your AWS Organization, you need to let other people access the accounts. And at this point, it's usually tough to only rely on IAM users and roles.

With an AWS Organization, you should prefer a federated access method, and that leaves us with AWS SSO or a SAML based approach. Which one to pick largely depends on how you usually set up accounts and organizational units, so let's take an overview of the two with this perspective.

AWS SSO

This is the default way suggested by AWS, and it comes with a lot of outstanding features like auditing SSO activity across applications and AWS accounts and multi-factor authentication.

For AWS SSO to work, you need to enable it with your AWS Organization's root account. Once enabled, you can manage all directory users from the AWS SSO portal and give access to single persons or groups through roles. You can both use the internal user directory of AWS SSO or link an external directory with some third-party identity provider.

If you have "relatively" few accounts and users, AWS SSO can be the most straightforward way to access your multi-account environment. It's easy to use, it's managed, takes minutes to set up, and has a neat integration with the AWS CLI v2 to impersonate your roles directly from your local environment.

But it's when you reach hundreds of users and accounts that begin to have some difficulties in management, and it becomes more and more difficult can't keep up with that amount of resources. In my opinion, there's a point at which AWS SSO can't keep up with everything and need a dedicated external solution.

SAML Approach

The SAML approach is the manual way of doing AWS SSO. This approach's downside is that you need to set up the federation process and sync between your user directory and roles. You have strict control over your identity provider and AWS accounts and less magic on the advantage side*.* And you don't need access to your AWS Organization root account because all setup can be done in IAM.

But you can still divide your whole organization, as I've previously explained: into multiple federated root accounts where you can govern the other chaining directly from IAM. You lose the single-centralized place for user and permission management of AWS SSO, but in fact, you gain multiple areas better to segregate your users, roles, and permission. It's just a matter of getting the segregation right.

And if we change the organization?

Right now, we have two strategies at our disposal with our AWS Organization:

Centralize everything in AWS SSO
Split into multiple root accounts

But there is some variable that we can change? We are keeping fixed just one thing, and that is using a single AWS Organization. What happens if we start adding more AWS Organizations instead of relying on only one?

Consolidated billing

Off we go with the first apparent downside. We will no longer have a single place to check how much our AWS bill amounts, and instead, we are again on tracking on multiple places.

But is it a downside? Indeed, we have now split our bills in multiple places. Still, if we segregated the numerous AWS Organizations with some sense (e.g., business units that span across the company or entire departments), we now can have a better view of the single organization's spending. It depends on the use case, but I think it's both a downside and an improvement depending on the perspective.

Multiple AWS SSO

Remember we had to have access to the AWS Organization root account to set up AWS SSO? It's a shame not to leverage its features because we have too many accounts and users. But wait, having multiple AWS Organizations should mean having fewer accounts and resources to manage from a single root account and having an overall better management experience on AWS SSO. To me, this is a significant improvement.

SAML federation

It doesn't change much on this aspect since it's a manual approach to accessing your accounts. You still need to manually configure your identity provider and account, only with fewer people. I think it's safe to say that it's a draw on this.

SCPs

Like having multiple AWS SSO, we can now better tailor our SCPs to the need of that single AWS Organization without having to worry about making some disruptive changes to workloads not related to them. This can lead to some inconsistencies and replication on governance policies if not managed correctly, but overall I think it is an improvement!

Reduced blast-radius

In the scenario where some malicious person takes over an AWS Organization root account, if everything is inside just one organization, the blast-radius will be merely devastating. Being in control of the AWS Organization root account lets you access all accounts linked to it and delete resources and backups, not something I'd like to see even in my worst nightmares.

Having multiple AWS Organizations increases the surface attack, but it shouldn't change much if we're following security best-practices. What is changing is that our other AWS Organizations are safe in the unlikely event of a security breach. It's a bit of a tradeoff, but I think it's again an improvement.

To sum up

We covered a little more ground and presented a few ways to handle things at the organization level with multiple AWS Organizations.

Using a single Organization is the default way to go, but I think it will need dedicated solutions to manage a vast amount of resources in the long run.

As an alternative, we can shift to a multi-organization environment where each AWS Organization has extensive power and works in very segregated silos. This is a good thing but needs some aggregation tool to view and manage everything centralized.

Only remember there is no right or wrong since you decide based on your use-case and how your company is structured.

How to access your AWS account

Nicolò Marchesi — Fri, 04 Dec 2020 14:16:03 +0000

Hey there. You just created your first AWS account, and you can't wait to dive into all the exciting services and technologies that AWS has to offer you to start to build the next big thing. But, wait for a second... are you going to log-in with your root account credentials? Or it's better to generate a new user? Or maybe use a role?

Slow down a minute, and by the end of the blog post, I promise you will be able to choose the best way to access your AWS account securely.

Beware, though, we will consider access to a single aws account as we will cover multi-account strategies the next time. We need to walk before we can run, right?

Accounts and Resources

Each incredible journey with AWS starts with accounts and resources, and an account is just like an empty box.

The things you can put inside your account can be anything related to AWS's services, for example, networks, virtual machines, containers, and so on.

But the most crucial part is that by default, no account shares any resource so that nothing can go in or out between two different AWS account. And this is great to divide your workloads and be sure that only the right people access everything.

IAM — Identity & Access Management

Within each account, IAM is there to protect the resources inside your account by managing which entities can perform some actions on those resources:

WHO — which identities
WHAT — can do some actions
WHERE — on some resources
HOW — and met some conditions

I define these as the 4W, and for now, we will need to focus only on the first 3.

In the beginning — the root user

When you logged-in after you created the AWS account, you used that account root user.

That is different from the other ones you can create because:

Created on the account registration
You can sign in using the email address and password that you used to create the account
You can access the billing information
It has unrestricted access to all resources in your account

This user is super important, and you should protect it in the best way you can. It's a super-administrator on steroids since it's able to view the billing information along with being able to do everything with all your resources.

So, the first thing to do should be:

Enable multi-factor authentication (MFA) and save the QR code in a secure place
Delete all access keys related to this user to disable programmatic access (you wouldn't need this anyway!)

And the very next thing should be to decide how you will access this account from now on.

From inside AWS

Let's focus for now on what AWS gives us without relying on 3rd parties. This is especially useful if we focus on this single account because we need to master the three IAM building blocks: user, groups, and roles.

Users

Remember the root user? IAM users are the same type of entity but differently from the root user. They don't have default billing permissions and administrator access (unless you give them... which you shouldn't).

With a user, we can create an entity that can log-in to the AWS web console or with programmatic access through a set of credentials (an access key and a secret access key) that are fixed and will not change over time. Along with SSH/HTTPS keys for AWS CodeCommit, or Credentials for Amazon Keyspaces (for Apache Cassandra)

Sign-in credentials for the web console
Access and secret key credentials for programmatic access for APIs, CLI, and SDK
SSH/HTTPS keys for AWS CodeCommit
Credentials for Amazon Keyspaces (for Apache Cassandra)

Granular access

It's entirely possible with an IAM user to enable one or more access keys depending on how you want that user to behave. For example:

A data-analyst that needs to check only the analytics on the web console but without needing programmatic access
A serverless DevOps which need to perform actions on the console and use the CLI to provide some resources will need both of these access
A developer working on a single repository and doesn't need to interact with AWS resources will have SSH/HTTPS keys for AWS CodeCommit.

Be aware that sharing IAM user credentials is a big no-no, as it's the first step to lose credentials and governance and lay yourself open to security threats.

You don't want your production account completely erased in favor of a full set of bitcoin miners, do you?

Enforce security

Another cool feature of AWS IAM users is to be able to enforce password complexity (which is by default pretty high with a length between 8-128 characters, a minimum of 3 special characters, and to not be identical to your AWS account name or email address) and password expiration time to ensure that after a certain time your users will change the password.

On top of that, you can enable MFA with various devices:

A virtual MFA device with an authenticator app installed on your mobile device or computer
A U2F security key like a YubiKey or any other compliant U2F device
Another hardware MFA device like the Gemalto token

In general, there is a granular approach as you can have specific means for each IAM user.

When to use

If you have very few users (between 1-5) with precise permissions tied to each one, IAM users can save some time on the initial configuration and console access. Beware of static credentials because they are substantial security issues, so handle with extreme care. Or use the AWS CLI get-session-token to generate temporary credentials out of static ones.

Groups

Not an entity per se, but IAM Groups are there to bundle IAM users together and give the same set of permissions to a group of users. This smooth a little the rough edges of IAM users, but overall the same things we have seen for IAM users apply.

A nice thing to keep in mind is that an IAM user can still have personal policies while associated with a group, with the resulting permission equals the merge of the group and personal ones.

This should be more the exception than the typical workflow. With too many individual permissions, we create many unmaintainable policies and lose the overall overview of the permissions in the account.

When to use

If you have few users (between 5-15), IAM Groups can save some of the hassles of managing and configuring each IAM user. But the same consideration as IAM users applies, so if possible, avoid them unless you are just working on a single account.

Roles

And here we are to AWS roles. IAM roles can have permissions (just like users) and log to the console (again like users...). Instead of being uniquely associated with one person, anyone who needs it can assume a role.

So a person wears the cap of the role which he wants to assume, and

The IAM role logo just gained a lot more sense, right?

Temporary Credentials

Another key difference from IAM users is that IAM roles don't have static credentials. When someone assumes the roles, a set of temporary credentials is generated and given. It's possible to customize how long those credentials will last and further restrict access if needed.

Time can be set from 900 seconds (15 minutes) up to 12 hours, but in my opinion, setting one-hour-long expiration is the best tradeoff between security and performance.

CloudTrail monitoring

I'll mention this because some people don't use roles because they think that they can't audit what roles can do. The truth is that you can, and you should. By enabling Cloudtrail monitoring, you can see all actions performed inside your account, so you can always know what's going on.

Cross Account

This will be discussed in more detail in another blog post, as we are focusing on single accounts, but for now, be aware that IAM roles can be used to give access to other IAM roles or IAM users.

And that's incredibly useful.

Entry-point required

And now for the downsides. Unfortunately, you can't give direct access to the console and temporary credentials to an IAM role. First, you need to have set up a user or a federated role (more on this in a few paragraphs) to give access to an IAM role.

When to use

Always. The sole fact that IAM roles are not tied with static credentials makes them the best way to access your cloud. Combine them with the ability to monitor IAM roles actions and enforce security even with MFA. The only downside is that it a little more configuration, but it's a low price to pay in my opinion.

From outside AWS

And now, let's see what we can do when we go into the field of corporate identities. I'm referring to use an Identity Provider as an Identities source and log-in in with our corporate email and password.

Federation

But what do we mean by federation? On one side, we have the identity provider. These systems contain a directory of users and some software that can communicate through specific federation protocols. All the data of our users reside as Identities.

On the other side, there is your AWS Account as an Identity Consumer. It maintains a reference to the identities without saving them. On this side, we generally have much more granular authorization levels (thanks, IAM!).

So the federation is the relationship of trust that makes the Identity Consumer able to reference the Identities in the Identity Provider.

SAML Federation

The most common definition for Security Assertion Markup Language (SAML) is an open standard for exchanging authentication and authorization data between parties. This means that we can use our Identities to log in to AWS and even get temporary credentials by assuming roles. It's a generalistic approach you can apply to nearly all Identity Providers.

The trust relationship

To set up the federation between the two parties, we need to create and manage an IAM Identity Provider. This is necessary because it represents the real Identity Provider and holds the shared secret that lets the two parties communicate. Moreover, inside each role that we want our users to assume, we will define a trust policy to allows entities related to the IAM Identity Provider to assume the role:

Custom management

The thing is that managing the federation by yourself can be challenging and time-consuming at first. There are some concepts to understand well, and you need to create a bunch of entities, relations, and custom things to make this happen. To get an idea, you can find a complete tutorial example here.

When to use

In general, this is an excellent and battle-tested approach that will work in nearly all cases. Still, the custom management of identities can be difficult and time consuming for inexperienced people.

Typical use-cases are if you want to manage this yourself for security or compliance purposes. You don't have access to your AWS Organization, or AWS SSO does not yet support your Identity Provider.

AWS SSO

This is the cloud-based and managed SSO service that lets you connect your Identity Provider with your AWS account.

AWS Organization

To set up AWS SSO, you will need to configure AWS Organizations, and for now, you can think of it to manage multiple AWS accounts. To keep things brief, you need to know that setting up an AWS Organization needs extensive permissions (namely the root user).

Internal Identity Storage

By default, AWS SSO provides a directory to store your user information and credentials to serve as Identity Provider directly. It's great if you don't already have an Identity Provider or don't want to go to the trouble of setup one.

External Identity Storage

But suppose your organization already uses some User directory, like Microsoft Active Directory. In that case, you can connect it to AWS SSO, eliminating the need to maintain two distinct directories with automatic provisioning.

Beware, though, that this feature is supported only on a limited set of Identity Providers:

Multi-factor authentication

With AWS SSO, you can configure an MFA device for your users inside the service. Right now, it supports Authenticator apps like Google Authenticator and security keys like the Yubikey. The internal solution is exciting, but it lacks MFA support if set up on an external Identity Provider side.

When to use

It's the standard suggested by AWS, and it can be the default way to go if unsure how to proceed. Still, there are at least a couple of requirements that you need to tick make the most out of this solution:

you've already set up your AWS Organization, and you have access to the root account. You want to keep your Identities within AWS SSO or use an identity provider supported by automatic provisioning.

Final Notes

So, we have laid out the foundations to access a single AWS account, and now you should have the elements to choose the best method for your specific use-case. We will use this to cover the ground to multi-account management and AWS Organizations in the next blog post.

Leapp

I advise developers to use some tool that helps them store all the data needed to connect to their AWS account. My team and I are developing an open-source tool that supports all these use-cases, so feel free to check out:

Noovolari / leapp

Leapp is the tool to access your cloud; It securely stores your access information and generates temporary credential sets to access your cloud ecosystem from your local machine.

Leapp

Website: https://www.leapp.cloud/
Roadmap: Roadmap
Tutorials: Tutorials
Chat with us: Discord

Leapp is your everyday companion to access your cloud; designed to work with Cloud Providers APIs, CLIs, and SDKs It's a software that securely stores your access information and generates temporary credential sets to access your cloud ecosystem from your local machine.

For example, while using the AWS CLI it may become annoying to switch to a different profile or use the --profile argument before issuing every command.

Leapp lets you have a new set of credentials and give access to that account with a click.

Leapp also will manage Federated access through Identity Provides.

Key features

Switch account with a click: collect all your cloud accounts access data in a single place and connect straight away. Leverage cloud RBAC to impersonate your roles in a click, and don’t waste time manually manage or edit your credentials file.
…

View on GitHub

How to update in bulk G Suite users custom attributes with Google Admin SDK

Nicolò Marchesi — Mon, 09 Nov 2020 09:53:37 +0000

A scenario to use Google APIs to enable continuous management of AWS SAML federation in Python

In my previous blog post, we discussed in-depth how to set-up a federation between AWS and G Suite to enable developers to log-in to the AWS Console through single sign-on (SSO) and generate short-lived local credentials with Leapp.

While discussing in the Reddit forum, one exciting question came out:

What if I add a new AWS IAM role that I want lots of users to assume from G Suite – would I have to add them to each user manually, or is there a better way?

Thing is... there is! And I'll show you how to do that.

Attention please!

You can scroll down to the TL;DR section for the busiest people who don't need to dive deep into how the script works and prerequisites. I suggest you at least take a look at the configuration part.

Be aware that we will programmatically change data with extensive permissions, but there is no other way to accomplish the same thing so be careful. I suggest you try this on a test environment before changing anything on your real users.

Don't make the sysadmin team angry!

Overview

Create a Google Cloud Platform Project
Enable the Admin SDK
Create the OAuth Consent Screen
Create the OAuth Client
Configure the script
Run the script

Requirements

A G Suite subscription with an admin account
A Google Cloud Platform account
The custom schema definition
A list of users we want to update
Account number, role names and provide names to give access to

⚠️ — if you don't have already, you can follow this guide to set up the federation between AWS and G Suite

1. Create a Google Cloud Platform Project

First of all, you need to create a GCP Project, so head to https://console.cloud.google.com/projectcreate and:

Give a name to the project — use a meaningful name. For example test-update-user-ca
Select the organization — choose the one on which your users resides

Just wait a few seconds for GCP to provision your project and once it's ready, select it from the drop-down menu and proceed to the next step.

2. Enable the Admin SDK

G Suite offers in the Google Admin User Console a way to bulk update the users, but unfortunately, it does not allow to change their custom SAML attribute values.

For this, you have to use the Google Admin SDK and programmatically update users. But in order to do so you first need to enable the Admin APIs for this project. From the menu, select API & Services, Library.

From here, type in the search box admin and click on the first search result: Admin SDK. Just click on the ENABLE button, and you're good to go.

3. Create the OAuth consent screen

For the next steps, we will use the OAuth client authentication and authorization method. For security purposes, service accounts and credentials can't be used to access the Admin SDK, so we are forced to use the OAuth client flow.

But before we can create the OAuth client, we need to set up the OAuth consent screen. From the menu, select API & Services, OAuth consent screen.

Google will prompt you to select the application type since we use it only for internal purposes (we don't intend to make available the application to everyone). Select internal and click create.

Give any name you want to the consent screen, fill in the email needed for contact information, and then set the scopes.

To correctly set up the application and don't get some warnings during the script execution, we have to tell the application the scope we are requesting. Basically our script will require access to the Admin APIs, and here we will set that we are granting that access.

Click on add or remove scopes and, at the end of the page, manually add a scope to https://www.googleapis.com/auth/admin.directory.user

Click on add to table and update to commit your changes.

Review your work and proceed to the next step.

4. Create the OAuth client

Before we can dive into the code, the last step is to enable our code to interact with the Google Admin APIs. For this, we need to create an OAuth client with secrets shared between our script and Google.

From the menu, select API & Services, Credentials.

Click on create credentials and select OAuth client ID.

Fill in the information about the client and click on create.

After you have created the client, you will be shown the client secret and client id. We don't need to copy them from here but download the .json file from the credentials dashboard. So close the pop-up and click on the download button.

After you have downloaded the file, rename it credentials.json.

5. Configure the script

Let's review the most important part of the script and how do they work, but before going deep, let's quickly review how the script is set up and works:

Uses poetry (https://python-poetry.org/) as package and project management
credentials.json — place the file we downloaded before at the root of the project
custom-schema.yaml — contains how you configured the custom schema, change it accordingly to your settings
federation.yaml — contains the list of users, together with a list of federations to apply to the custom schema fields

⚠️ — the script supports only a multi-value field for the federated role; check the previous guide for more info about it

a. Clone the repository

I have already set up the whole project on a Github repo:

pethron / gsuite-custom-schema-update

A scenario to use Google APIs to enable continuous management of AWS SAML federation in Python.

View on GitHub

You need to clone or head to the repository and download and unzip the archive:

git clone https://github.com/pethron/gsuite-custom-schema-update

b. Install dependencies

I use poetry (https://python-poetry.org/) as package and project management, so you will be able to install all dependencies in a managed virtual environment by just running:

poetry install

If you prefer other methods, the requirements.txt file is present, so you can still run:

pip install

c. The custom schema file

The custom-schema.yaml file tells the script how to correctly create the custom attribute structure to be pushed on the user; the default attributes are the same used in the previous guide, so change it according to your environment:

name — the name you give to the custom schema
session — the name you give to the attribute representing the session duration
role — the name you give to the attribute representing the federated role

name: AWS SAML
session: SessionDuration
role: FederationRole

d. The federation file

The federation.yaml file tells the script the values that need to be pushed on the user (I choose to use the YAML notation as I think it's a lot more readable than JSON).

At the document root, we have a list of users with their emails. For each user, there is a federations field containing a list of the data used to create the string to update the custom schema:

email — the email of the user to change the custom schema attributes
account — the account number where the IAM Identity Provider and IAM Role resides
role — the federated role name (the script will derive the ARN)
provider — the provider name (again, the script will derive the ARN)

- email: john.doe@mydomain.net
  federations:
    - account: 123456789123
      role: fooRole
      provider: fooProvider
    - account: 123456789123
      role: barRole
      provider: fooProvider
    - account: 345678912345
      role: foobarRole
      provider: barProvider

6. Run the script

To run the script, change directory to the project root folder and type:

python cli.py

a. Get Credentials

This is the first thing that will run in our script (after loading the custom-schema and federations files). It will obtain a set of credentials to call the Google Admin Directory APIs.

On the first run, the browser will pop-up asking you to authenticate and accept the consent screen we configured before. Next, it will store the token needed to authenticate the API calls in a file named token.pickle, and every time the token expires, the browser will pop-up to let you authenticate again.

Notice that we are requesting exactly access to the same API we configured earlier in the consent screen: https://www.googleapis.com/auth/admin.directory.user.

def get_credentials():
    creds = None

    if os.path.exists('token.pickle'):
        with open('token.pickle', 'rb') as token:
            creds = pickle.load(token)

    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file(
                'credentials.json', 
        ['https://www.googleapis.com/auth/admin.directory.user'])
            creds = flow.run_local_server(port=0)

        # Save the credentials for the next run
        with open('token.pickle', 'wb') as token:
            pickle.dump(creds, token)

    return creds

b. Build the service client and retrieve users

Here is the first step of our main script. It builds the service client and fires a query to list the users in our organization. We can further filter the users using our Organizational Units through the query parameter in case.

⚠️ — the maximum number of user retrieved is 5, and the API can retrieve at most 500 users. Change the number to your needs once tested.

def main():
    creds = get_credentials()
    service = build('admin', 'directory_v1', credentials=creds)

    # Call the Admin SDK Directory API
    # If need change like "orgUnitPath='/<my organizational unit>'"
    orgPath = "orgUnitPath='/'"
    results = service.users().list(customer='my_customer',
                                   projection="full",
                                   query=orgPath,
                                   maxResults=5, # max value is 500
                                   orderBy='email').execute()
    users = results.get('users', [])

c. Iterate and find users to change

This small snippet of code is just a quick and dirty way to compare the users we retrieved with the ones defined in the federation.yaml file, and apply changes only to them.

for user in users:
    # A user can have multiple defined emails 
    for email in user['emails']:
        # Let's iterate over all users defined in federation.yaml
        for federation in federations:
            # Apply changes only if current user is found in federation.yaml
            if federation['email'] == email['address']:
                print('Update user {0} with the following custom schemas'
                    .format(email['address'])
                userUpdated = update_saml_attributes(
                    service, user, schema_config, federation['federations'])
                print(u'{0} {1} {2}'.format(
                    user['primaryEmail'], user['id'], userUpdated))

d. Update the SAML attributes and preserve the others

And finally to the function that generates the new custom schema. As you can see, we will generate an array of custom schema roles, so it will work only if you choose a multi-value field for the federated role field.

The neat thing is that by copying the current version of all the custom schemas, we can preserve all other custom schemas we defined and apply changes only to the one defined in the custom-schema.yaml file.

def update_saml_attributes(service, user, schema_config, federations, session_duration=28800):
    custom_schema_roles = []
    for federation in federations:
        custom_schema_roles.append(
            {
                'type': 'work',
                'value': "arn:aws:iam::{0}:role/{1},arn:aws:iam::{0}:saml-provider/{2}".format(
                    federation['account'], federation['role'], federation['provider'])
            }
        )

    current_schemas = user['customSchemas']
    user['customSchemas'][schema_config['name']] = {
        schema_config['session']: session_duration,
        schema_config['role']: custom_schema_roles
    }

    user.update({'customSchemas': current_schemas})
    ret = service.users().update(userKey=user['id'], body=user).execute()

    return ret['customSchemas']

I set a default value for session duration of 28800; which means that the session will expire after 8 hours. Change to fit your use-case if needed.

e. Manually update the roles on the AWS accounts

After you made changes, you need to add the IAM roles' trust relationship to allow the user to assume the role after logging in with G Suite. Again you can find extensive information on how to do that in my previous blog post.

[PRO TIP] Configure Leapp

After those two tutorials, you have deepened your knowledge on how to set up SSO federation for your G Suite users to AWS accounts role, and update more easily the cloud accounts on which your developers, sysadmins, and DevOps can access.

A good next step is to ease the process of accessing your cloud accounts.

We have developed an open-source tool for easily storing your cloud accounts access data and generating temporary credentials to access your cloud accounts from the local environment.

Head to the repository to find everything on how to configure and use it.

Noovolari / leapp

Leapp is the DevTool to access your cloud

Leapp

Website | Roadmap | Blog | TOPS community | Documentation | Troubleshooting

⚡ Lightning Fast, Safe, Desktop App for Cloud credentials managing and generation

Leapp is a Cross-Platform Cloud access App, built on top of Electron.

The App is designed to manage and secure Cloud Access in multi-account environments, and it is available for MacOS, Windows, and Linux.

For more information about features go to our documentation.

✨ Features

Cloud credentials generation in 1 click
Data stored locally encrypted in the OS System Vault
Multiple Cloud-Access supported strategies
Automatic short-lived credentials rotation
Automatic provisioning of Sessions from AWS Single Sign-on
Open multiple AWS console from different AWS accounts in Firefox and Chrome web extensions!
Connect to EC2 instances straight away
Managing Leapp with its CLI
Create your own Leapp plugin to customize the App functionalities from the template

All the covered access methods can be found here…

View on GitHub

TL;DR

Configure Google Project
Clone the repository and install dependencies
Change the custom-schema.yaml according to the template:
- name — the name you give to the custom schema
- session — the name you give to the attribute representing the session duration
- role — the name you give to the attribute representing the federated role
Change the federation.yaml according to the template:
- email — the email of the user to change the custom schema attributes
- account — the account number where the IAM Identity Provider and IAM Role resides
- role — the federated role name (the script will derive the ARN)
- provider — the provider name (the script will derive the ARN)
Run the script

Wrap-up

At this point, you will be able to change the federation between your AWS accounts and G Suite users through this script by just changing the federation.yaml file. While it's convenient, it somewhat limited to some scenarios:

The script works for only one custom schema at a time. If you need to federate multiple accounts and go with the multiple custom schema route because you won't compromise about security, you need to modify the script accordingly.

It will not take into account the complete removal of the custom schema from a user. Means that if you leave the user in the federation.yaml file but remove everything in the federations field; the script will break. That's because Google will not understand an empty custom schema.

You still have to add the trust relationship to the role. So after you made changes, you need to log in to your AWS account and manually change the role trust relationship to allow the user to assume the role after logging in through SSO.

The user list call can retrieve 500 users at max. You need to modify the source code and make a recursive method to retrieve all the users if you want to apply to more than 500 users.

Many improvements can be made, like adding the CLI configuration or make-up for the non-supported scenarios. But apart from that, I think it's a great foundation to work upon, and you will surely find it handy.

I hope you found this guide useful, and as always, let me know what you think about it, and feel free to ask anything. Cheers!

How to SAML federate your AWS account with G Suite

Nicolò Marchesi — Tue, 20 Oct 2020 10:23:28 +0000

The SAML Federation

Isn't a bother always to remember or securely store access data to log into your AWS console or run some local command from the CLI?

AWS has many means to authenticate to their accounts, and they range from federating users with identity providers to give direct access through credentials.

This blog post will run you through all the steps to federate your G Suite account with an AWS account so that you will be able to access the AWS console and generate security credentials with your G Suite identity.

You will never have to remember your access credentials again!

Why not use AWS Single Sign-on?

AWS Single Sign-on (AWS SSO from now on) is a great way to achieve the same results, but it comes with some prerequisites that not always can be met;

Have set up AWS Organizations.
Have enabled all features in the AWS Organizations (this should always be true!).
Sign-in with the AWS Organizations master account. You cannot set up AWS SSO while signed in with credentials from an Organization’s member account.
It only works with AWS CLI v2.

Long story short... to set-up AWS SSO, you need a lot of permissions.

This is actually a good thing if coupled with all security best practices that come with AWS SSO, but it can't be implemented in some cases.

Mind that, while in the context of AWS and G Suite, this is a generic approach that leverages SAML. It will be different for other Identity Providers and Service Providers, but the overall process is the same. Know one to master them all.

Bonus

As a bonus track, I'll show you how to configure our open-source tool Leapp to leverage SAML Federation to generate AWS credentials through SSO, to use in your local development environment.

Noovolari / leapp

Leapp is the DevTool to access your cloud

Spoiler: we'll be adding AWS SSO support to Leapp very soon.

Summary

Here's the rundown of the actions needed to get your federation up and running

Overview

Create a custom user attribute category
Create a SAML application
1. IdP Metadata
2. Service provider details
3. Attribute mapping
Create an AWS SAML IdP in AWS IAM
Create an IAM federated role
Fill custom SAML attributes for a user
Enable the SAML app

Bonus

Get SSO link
Configure Leapp

Requirements

A G Suite subscription with an admin account
An AWS account with IAM permissions

Tutorial

So let's dive into the tutorial to set everything in place and get your shiny new federation on.

For the sake of simplicity, we will consider that you already have a suitable user on your G Suite subscription on which to enable the federation.

This schema simplifies the flow of the authentication process. You will federate AWS with G Suite to call the AssumeRoleWithSAML call and retrieve a set of credentials associated with the IAM role that we set up during the process.

Legend

You will find some callout to highlight some aspects better; here is the meaning behind them:

✋ - ACTION REQUIRED — Act for later use.

💡 - INFO — Further insights or customization for experienced people.

💣 - WARNING — Be aware of the security implications.

1. Create a custom user attribute category

First of all, we have to add custom attributes to our users. When our Identity Provider (Google) will communicate the user's identity during login, it will transmit additional information to the Service Provider (AWS) in the form of custom attributes.

The custom attributes are FederationRole and SessionDuration. After everything is set up, we can fill in each user a set of these attributes. These will be passed to enable AWS to automatically generate a set of credentials with an AssumeRoleWithSAML API call of the duration of the SessionDuration parameter.

So from the admin dashboard, head to the user directory by choosing Users. From the top right corner in the user dashboard, choose Manage User Attributes and Add Custom Category.

Type AWS SAML in the category name with and proceed to fill the attributes data with:

Attribute name — FederationRole, Text, Visible to users and admin, Multi-Value
Attribute name — SessionDuration, Whole number, Visible to users and admin, Single-Value

💡 — You can use any name you want for the category and attribute names. When we later map each attribute name to an attribute recognized by AWS to enable the federation, so feel free to use the names that better suits you and your environment.

💡 — We are using a multi-value field for FederationRole to federate multiple accounts simultaneously. If you choose a single-value, you will need to create a new category of custom attributes for each AWS account that you want to federate. Feel free to do it if you know what you are doing.

2. Create a SAML application

Now, to set up your SAML SSO, we need to create a custom application in the G Suite management console. The application will represent your AWS accounts in the G Suite environment and will hold all security data to make AWS and G Suite establish a secure connection. We will obtain this by sharing the secret generated during the creation of the custom application with the AWS account.

Again, from your G Suite admin dashboard, choose Apps, SAML Apps. From here, choose the big yellow plus icon (+) and select SETUP MY OWN CUSTOM APP.

2.a. IdP Metadata

This will present you with the google identity provider information and some download options. Download the IDP Metadata (option 2) and keep it close; we will need this file soon.

This file contains an XML that contains some configuration parameters and the most important X.509 certificate. The certificate is the entity on which the trust relationship between IdP and SP is based on.

💣 — Keep your X.509 certificate in a secure place and don't disclose it's contents for any reason. The security of the entire solution relies on keeping the content of this file confidential.

2.b. Service provider details

We continue the configuration by mapping the SAML entity known as (that is, the user will be presented to the AWS console with their email address as their unique identifier).

ACS URL — https://signin.aws.amazon.com/saml
Entity ID — https://signin.aws.amazon.com/saml
Signed Response — leave unchecked
Name ID — Primary Email
Name ID Format — Email

2.c. Attribute mapping

Now it's time to make AWS understand our custom attributes. This is done by mapping our G Suite custom attributes with specifically designed parameters on AWS:

Application attribute — https://aws.amazon.com/SAML/Attributes/RoleSessionName, Basic Information, Primary Email
Application attribute — https://aws.amazon.com/SAML/Attributes/Role, AWS SAML, FederationRole
Application attribute — https://aws.amazon.com/SAML/Attributes/SessionDuration, AWS SAML, SessionDuration

💡 — If you used another name for the custom category, choose accordingly.

💡 — You can still change the mapping after creating the SAML application. From the admin dashboard, go to the Apps, SAML Apps, and choose the application that wants to modify. Here select Choose Attribute Mapping to edit.

3. Create an AWS SAML IdP in AWS IAM

We need to create some way to make AWS aware that we are using G Suite as an identity provider. This is pretty simple, but it needs the X.509 certificate that we saved earlier.

Log in to the AWS console, open the IAM section, and from the left-side column, select Identity Providers. Here click on Create Provider.

Provider Type — SAML
Name — GSuiteSAML (if you don't like it, change as you wish)
Metadata Document — choose the file downloaded when creating the SAML app

✋ — Select your newly created identity provider ARN, copy it, and set it aside.

4. Create an IAM federated role

We need to create a role that will be automatically assumed when our corporate user will access the console. In IAM, go to Roles, Creare role. Here choose the SAML 2.0 federation and select from the dropdown the SAML provider created earlier; the other fields will be automatically selected. Next, assign the correct set of permission, and you're good to go!

✋ — Select your newly created user ARN, copy it, and set it aside.

This procedure will create a trust-relationship with a condition. This basically means "If someone is authenticated through the G Suite Identity Provider, and G Suite says that they can assume this role, then they are allowed to do so".

This policy can be further tweaked to let only specific users access the role. We can add more conditions to reach our goal; for example, this role will be accessed only by this user. You can mix and match all conditions to achieve your desired logic.

5. Fill custom SAML attributes for a user

Now, remember the custom attribute category that we created at the very start of the tutorial? Now it's time to fill in the info for our users. From your G Suite dashboard, choose Users, select the user you want to federate, and choose ** User Information** in the User details page.

Scroll down until you see the category with AWS SAML (or whatever name you used) and click on it. You will see the two values of the custom category and fill them with: ****

FederationRole — This is a comma-separated string with the IAM role ARN and IdP ARN in the following format: <Role-ARN>,<IDP-ARN>.
SessionDuration — This is the maximum time in seconds that the session will be active. Fill in 28800.

💣 — As we have enabled MFA for all our users, we usually choose to have 1 working day-long sessions (8 hours) or 28800 seconds. Feel free to reduce the time or increase it, but mind that it cannot be longer than 12 hours and shorter than 1 hour.

💣 — As we have used a multi-part value for the FederationRole, we can add more values for roles and Identity Providers in different accounts so that you can manage everything from your G Suite user. Mind that you need to use the same IdP metadata to create the IAM Identity Provider in AWS. This is a tradeoff between security and ease of management.

💡 — Alternatively, you can create multiple SAML apps, one for each account you want to federate. Like this, the metadata file will be different for each federation process. In case of a security breach and disclosure of the metadata file, the only account affected will be the one that gets disclosed. While this process is more secure, it also increases the management overhead and needs to keep track of everything.

6. Enable the SAML app

We need to enable the SAML app to make everything work and leverage our G Suite SSO to the AWS console. Choose the SAML application from the Apps dashboard. On the details page, choose the one that fits you best:

ON for everyone to turn on the app for every user in your G Suite account
On for some to turn on the app for a selective organization under your G Suite account

7. Test the SAML app

Now you can test the federation by choosing the SAML apps from the Google Apps menu and clicking on it. You may have to choose More to see the SAML app.

INFO — We named our SAML app Amazon Web Services, but here you will see the apps with the names you have chosen earlier. Select accordingly.

Bonus

We use the same configuration at our company, and during the years, we developed an internal tool that we open-sourced under the name of Leapp. As we have many accounts from different cloud providers, we collect all our cloud access data and switch between accounts. It supports both federated and simpler access credentials.

I'll show you how to configure Leapp for this use-case.

1. Get SSO link

Right-click on the SAML all on your Google Apps menu and Copy Link Location.

2. Configure Leapp

Open Leapp. Here's the link to the latest release.

Click on the top-left menu, select Options, and fill in the URL you just copied.

3. Create a Federated Account

On the home page, click on the big green (+) icon and select AWS and Federated.

Here you will fill in the information you got before:

Account Alias — A name to recognize this set of accounts and roles.
Account Number — The account number of your federated role AWS account. You can get it from the role ARN or log in to the AWS console and click on your name.
Role — Your role name. Not the ARN, just the last part after the backslash.
IdP ARN — Your AWS IAM Identity Provider ARN.

4. Start your session

Now you can click on the newly created entry and start your session so you can use both programmatic and console access through SSO.

Two Cents on Security

There are still a couple of things that you can do to ensure the highest level of security.

You may want to consider enforcing MFA on your Identity Provider (G Suite). This lowers the risk of someone’s account being compromised and an attacker gaining access to multiple systems.

The other thing is that AWS has a neat service called CloudTrail, where you can log, monitor, and retain account activity related to actions and events across your AWS infrastructure. While we are using roles for gaining access to our AWS account, the user identity is brought along with every API call, meaning that you can still monitor what actions the federated user is performing on the account.

Final Notes

In this blog post, we covered all the steps necessary to SAML federate your G Suite and AWS accounts and shared some technical insights. It may seem like a lot to get going, and it is a fair amount of work to get set up, but if you don't want to commit to AWS SSO or want to understand better what is involved in the process of SAML federation, it's totally worth your time.

If you are interested in those topics, follow us on our medium publication for the third article of this series, coming next week.