DEV Community: Teresa N. Fontanella De Santis

Authorization on FastAPI with Casbin

Teresa N. Fontanella De Santis — Sat, 23 Apr 2022 01:47:38 +0000

Nowadays, tons of APIs (both external and internal) are created and used every day. With methods like authentication/firewall restriction, we can identify who can invoke the methods, or restrict from where is trying to access. But, can we identify and authorize given users to invoke some methods rather than others? In the following tutorial we will cover how to authorize different users to execute certain REST API methods in an easy and straightforward way.

Situation

In this case, we have an Items REST API implemented on Python 3.10 with FastAPI framework. It allows to list all items and get, create and delete an item. All of these operations must be performed by authenticated users. For sake of simplicity, the following users can be used for authentication:

User	Password	Role
alice	secret2	Admin
johndoe	secret	User

The application consists of the files: main.py, utils.py and requirements.txt, with the following code.
main.py

utils.py

requirements.txt

You can create a conda environment, install required packages and run the api with:



conda create -n itemsapi pip
conda activate itemsapi
pip install -r requirements.txt
python3 -m uvicorn main:app --reload

After the API is up and running, let's follow these steps to test it:

Open http://127.0.0.0:8000/docs url in browser.
Click on "Authorize" and login with username and password (as per Users table shown before).
To get all items list, select on /items GET API method. Then, click on "Try out" button and on "Execute" button.
To delete the item with id = 1, select on /item DELETE API method, click on "Try out" and execute the method with item_id = 1. The response is 204 and item is deleted successfully.

Goal

Although only registered users (johndoe and alice in this case) can perform items actions, all of them are able to delete items. As per their roles, alice should be able to delete items, but johndoe shouldn't. To achieve this we will implement authorization at REST API method level, in an easy and extensible way with Casbin.

Implementation

Overview

Casbin is an open source authorization library with support for many models (like Access Control Lists or ACLs, Role Based Access Control or RBAC, Restful, etc) and with implementations on several programming languages (ie: Python, Go, Java, Rust, Ruby, etc).
It consists of two configuration files:

A model file: a CONF file (with .conf extension) which specifies the authorization model to be applied (in this case we will use Restful one)
A policy file: a CSV file that list API methods permissions for each user.

Steps

1) Install casbin python package with pip
pip install casbin
2) Define Conf policy
Create new file called model.conf with the following content:

You can find more details about Casbin model syntax on the documentation

3) Define Policy file

Create a new CSV file called policy.csv and paste the following:

Each row is an allowed permissions with the following values: the second column is the user, the third is the API resource or url, and the last one is a set of allowed methods. In this case, alice will have access to list create and delete items, while johndoe may list and create items but not delete them.

4) Update Python API code to enforce authorization.
In the main.py file, with following lines:



import casbin
...
async def get_current_user_authorization(req: Request, curr_user: User = Depends(get_current_active_user)):
    e = casbin.Enforcer("model.conf", "policy.csv")
    sub = curr_user.username
    obj = req.url.path
    act = req.method
    if not(e.enforce(sub, obj, act)):
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Method not authorized for this user")
    return curr_user

It imports the casbin module and create a new authorization function that read the configuration files with casbin.Enforcer and enforce the user has the required permission.

Then, on the defined API methods, change the old method get_current_active_user with the new get_current_user_authorization



@app.get("/items/{item_id}")

async def read_item(item_id: int, req: Request, curr_user: User = Depends(get_current_user_authorization)):

    return items_dao.get_item(item_id)

@app.post("/items/")

async def create_item(item: Item, req: Request, curr_user: User = Depends(get_current_user_authorization)):

    answer = items_dao.create_item(item)

    if not(answer):

        raise HTTPException(

            status_code=status.HTTP_400_BAD_REQUEST,

            detail="Item with given id already exists")

    else:

        return answer

@app.delete("/items/{item_id}", status_code=status.HTTP_204_NO_CONTENT)

async def delete_item(item_id: int, req: Request, curr_user: User = Depends(get_current_user_authorization)):

    items_dao.delete_item(item_id)

    return Response(status_code=status.HTTP_204_NO_CONTENT)

Test

Start the updated API
Open http://127.0.0.0:8000/docs url in browser.
Click on "Authorize" and login with "johndoe".
Try to delete item with id=1. It will be rejected with a 401 Unauthorized error.
Logout from that user. Then login with "alice".
Try to delete item with id=1. The request works fine, returns 204 and item is deleted.

Conclusion

On this post we saw how to use Casbin to implement authorization on REST APIs. Keep in consideration that this example can be extended combining with other authorization models like RBAC, and only changing the model and policy configuration files.

Preventing higher costs on EC2

Teresa N. Fontanella De Santis — Tue, 15 Mar 2022 02:43:08 +0000

In this post we will discuss an example of how to prevent running unwanted expensive EC2 instances.

Situation

We currently have several AWS accounts with consolidated billing in one AWS account using AWS Organizations. Hundreds of users are using those AWS Accounts daily to complete their daily work.

Issue

We want to prevent all users to launch expensive EC2 instances with higher prices (like the GPU Accelerated instances types), to do not have a heart attack when looking at the bill in the following month. We can use IAM custom policies to define those conditions, but the restriction must be applied also in those who had full administration access.

Solution

We can take advantage of AWS Organizations and create Service Control Policies (or SCPs) to define IAM guardrails. These policies will help us to define a set of Denied actions on an AWS Account, no matter IAM user's permissions. Additionally, this won’t required any additional cost. The following steps can be run on a CloudShell terminal.

Enable organization Secure Control Policies (SCPs).

rootId=$(aws organizations list-roots | jq .Roots[].Id | sed 's/"//g')
aws organizations enable-policy-type --root-id  $rootId  --policy-type SERVICE_CONTROL_POLICY`

Create SCP to restrict expensive instance types.
Save a SCP policy in a json file and create the policy. Then, obtain the policyId to be used in next step.

cat > scp_ec2_policy.json <<'END'
{
"Version": "2012-10-17",
"Statement": [
    {
        "Sid": "VisualEditor0",
        "Effect": "Deny",
        "Action": "ec2:RunInstances",
        "Resource": "*",
        "Condition": {
            "StringLike": {
                "ec2:InstanceType": "p*.*"
            }
        }
    }
]
}
END
policyName="DenyEC2InstanceTypes"
description="Denies launch of expensive  EC2 instances"
file="file://scp_ec2_policy.json"
policy=$(aws organizations create-policy --content $file --name $policyName --type SERVICE_CONTROL_POLICY --description "$description")
policyId=$(echo $policy  | jq ".Policy.PolicySummary.Id"  | sed 's/"//g')

Attach it on AWS Account on AWS Organizations.
Apply the following command per each AWS Account invited into the AWS Organization, except the management account. Replace <aws_account_id> by AWS Account's ID.
```
aws organizations attach-policy --policy-id $policyId --target-id <aws_account_id>
```

Finally, when trying to launch a new EC2 instance with p3 instance type, we'll get an error no matter we have EC2 Full Access. This is great to prevent unexpected costs for launching expensive EC2 instances!

Accessing S3 Buckets from CloudShell

Teresa N. Fontanella De Santis — Sun, 26 Dec 2021 23:24:09 +0000

After one year since AWS CloudShell was released, it worth to comment about a connection issue between this technology and S3 Buckets.

Issue

We have a S3 bucket (in this case, named mytestbucket0123) that we need to access through AWS CloudShell.
But when trying to list all objects on a bucket from CloudShell, executing aws s3 ls s3://mytestbucket0123 we’re getting the following error

"An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied"

Verification steps

As a starting point of this situation, we’ll need to analyze the scene and do the following verification questions:

Has the user got the required permissions to list objects of our S3 Bucket? To answer this we have several ways: first check on IAM that the user has assigned those permissions.

The user has attached the AmazonS3ReadOnlyAccess Policy, so it has ListObjects required permission. So let’s verify that the user can already list the s3 bucket objects (from the AWS console for example). Listing objects on the bucket seems to work fine from the AWS Console with the same user (from server machine) as per screenshot below.

So, it doesn’t seem to be user IAM permissions at least. Which leads us to the following step.

Has the S3 Bucket got any Bucket policy enabled? We need to search, on the AWS console, the S3 bucket and look the “Bucket policy” section in the “Permissions” tab.
In this case, the S3 Bucket has the following Bucket Policy



{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::mytestbucket0123",
            "Condition": {
                "NotIpAddress": {
                    "aws:SourceIp": "<Public IP Address>"
                },
                "StringEqualsIfExists": {
                    "aws:SourceVpc": "<VPC-Id>"
                }
            }
        }
    ]
}

    This bucket policy denies access to all users (no matter they have the required IAM permissions), except they access from a specific IP Address or connect from our VPC (which, in this case is the AWS Account’s default VPC). That means the CloudShell is not accessing to the S3 Bucket from the VPC… So let’s ask the next question.

3. **Does CloudShell terminal connect to the S3 Bucket through a public IP Address?** To answer this, we need to execute the following command into the CloudShell terminal: `curl ifconfig.me`![The result of executing  curl ifconfig.me is 18.224.171.123, which means Public IP Address](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ycs8gjesegoplzoizvlg.png) It seems that CloudShell is trying to access to the S3 Bucket from a non whitelisted Public Address (which belongs to AWS Reserved IP Addresses).

### Cause explanation
After the verification steps, we can realize this is a CloudShell’s current limitation: it cannot connect to AWS Resources using VPC Endpoints or with VPC restricted access. Besides, the IP Address allocated per each CloudShell terminal may change over the time. 

### Solution 
To solve this issue we need to whitelist the Cloudshell’s user agent. Although this may not be a 100% secure resolution, it can be a temporary fix until AWS CloudShell can improve (or no CloudShell access is required). At the time of this writing, the user agent is "[aws-cli/2.4.5 Python/3.8.8 Linux/4.14.248-189.473.amzn2.x86_64 exec-env/CloudShell exe/x86_64.amzn.2 prompt/off command/s3.ls]”. Then edit the S3 Bucket Policy to the following (excluding also the CloudShell user agent).

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Deny",
"Principal": "",
"Action": "s3:",
"Resource": "arn:aws:s3:::mytestbucket0123",
"Condition": {
"StringEquals": {
"aws:UserAgent": "[aws-cli/2.4.5 Python/3.8.8 Linux/4.14.248-189.473.amzn2.x86_64 exec-env/CloudShell exe/x86_64.amzn.2 prompt/off command/s3.ls]"
},
"NotIpAddress": {
"aws:SourceIp": ""
},
"StringEqualsIfExists": {
"aws:SourceVpc": ""
}
}
}
]
}



Finally, when trying to perform on the terminal the command  `aws s3 ls s3://mytestbucket0123`  finally works!

![aws s3 ls working correctly from CloudShell after S3 Bucket Policy update](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1eylou7y1pq6kr50ckd7.png)

Working on AWS console during outage

Teresa N. Fontanella De Santis — Wed, 08 Dec 2021 20:35:01 +0000

Situation

As yesterday December 7th a new AWS outage made headlines, it seems opportune to talk about some AWS Services that are global (and not only tied to one region endpoint) like IAM and S3. For some Cloud Operations teams, to face an AWS outage in some regions is to have blockers on their daily jobs. The purpose of this post is to show how to continue doing admin jobs during a partial outage.

Issue

AWS reported issues with their APIs on North Virginia (us-east-1) region. And let’s suppose we are on a Cloud Operations team and we need to perform some tasks in AWS Console, like for example, create a S3 Bucket and create a new user that have access to the given bucket (all in region US East 1)… How will be supposed to perform those actions with AWS issue without losing all day?

Steps

Login into the AWS Console with your user and password. If you are redirected by us-east-1 region, go to your browser url and change it with:
https://<region>.console.aws.amazon.com/
where <region> is a valid AWS region (for example, in this case we’ll use the Ohio us-east-2 region).
So now we are logged into the AWS console successfully! We can then go to Services->IAM.

Clicking on Users -> Add users we can create a new user and assign the AmazonS3FullAccess AWS Managed Policy (just an example, it can be any other permission).
After creating our user, let’s create a new S3 Bucket from where we can read/write data (keeping logged in a different region). So we can go to Services-> S3.

Then we create a new bucket. Provide a unique name for that S3 Bucket and then on the region select the US East 1. Then, go to the end of the page and click on Create.

Conclusion

Voilà! We did our task but located on a different region! Although it is better to have AWS working 100% on all the regions, and that this approach cannot be applied on several services. This can be a temporal workaround to avoid getting stuck when performing some Cloud Operations tasks.

Curl issue: SSL certificate problem: certificate has expired

Teresa N. Fontanella De Santis — Sun, 17 Oct 2021 23:58:54 +0000

In the following article we'll cover a common certificate issue faced with cURL application. curl is a command line client URL, which provides us the response of a given request for any HTTP(S) method. After this introduction, let's go deep into our issue...

Issue

When trying to execute a curl command to a specific site, like curl https://airlabs.co/api/v9/ping.json it is giving the following error:

“curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not establish a secure connection to it. To learn more about this situation and how to fix it, please visit the web page mentioned above.”

The same url is working fine on any browser, and we have the openssl library installed on our server,

Root cause explanation

CURL certificate stored on the server has expired. So we need to obtain the updated certificate for the site and replace it in the certificates’ system folder.

Resolution steps

First make sure you have wget installed on your server.
You can install it on Mac using brew install wget.
For Ubuntu, you can use apt install wget.
For CentOS/RHEL, you can use yum install wget.
Download the updated curl’s SSL certificate (from site curl.se), doing: wget https://curl.se/ca/cacert.pem
The certificate will be downloaded as cacert.pem file. Then, you can execute the curl command with the flag --cacert <path_to_cacert.pem_file>.
For example: curl --cacert ./cacert.pem https://airlabs.co/api/v9/ping.json
If the certificate file is a valid one, the error should have disappeared. As we don’t want to add the --cacert flag for every curl command, we’ll go to the next step.
Replace the updated certificate on the certificates’ system folder. To get the folder path, execute the openssl version -a on your terminal. You’ll see something similar to this (it may vary according to the OS configuration).

The OPENSSLDIR folder is the folder where the certificates are stored by default; so copy it to the clipboard.

Then, copy (or move) the certificate into that folder. In our example, it can be:
cp cacert.pem <OPENSSL_DIR>

After that, if we execute our curl command again, it will work as expected!