DEV Community: farzana-juthi

How to use a single base URL for multiple services in big serverless applications — SAM, API gateway

farzana-juthi — Sat, 23 Sep 2023 06:18:37 +0000

When you are working on a big application, you may have to use multiple backend services like authentication, payment, access management, and so on. If your architectural design is based on microservices, then all these services will have their own URL link to connect. But think, what happens when you use those separate links in your frontend code? Then if you want to add one more service to your application, you have to change your frontend routing again. It will complicate your system and add extra hassle.

To solve this problem, you can add an Amazon API gateway proxy which helps you to connect all separate URLs using a single base URL.

Lets think you decide to make an e-commerce application where you will have product service, authentication service, cart management, payment service.

Backend Services:

Product service: https://product-service-url
Authentication service: https://auth-service-url
Cart management: https://cart-service-url
Payment service: https://payment-service-url

Now, you do not need to know or manage multiple URLs. API gateway will create a single base URL like https://single-base-url/prod and it will route other services with proxy. Create methods and resources for each services and create path like /product, /auth, /cart, /payment. When you use /product after base URL, it will route you to product service, same as for other services.

I created a github repo for this. Follow the steps in README section to see how functionality in product services can be used through a single base URL. You can create other services like product service.

Here I describe what does mean each section of template.yaml. To create a proxy for each service you need four section:

i) Serverless application: To check all syntax and properties of AWS::Serverless::Application, please check this link.

  ProductStack:
    Type: AWS::Serverless::Application
    Properties:
      Location: product-stack/template.yaml
      Parameters:
        StageName: !Ref StageName

Here in this section,

1. ProductStack is logical id of this resource. If you want to create other 
   service you have to give a name of that service. Lets say, you will create 
   an application for auth service, you can name it Authstack. 
2. Location: It will indicate the nested application code path. 
   You will give the location of your code.
3. Parameters: If you want to pass any parameters to your cloudformation 
   parameter, then you can pass it here. Here, StageName used for making 
   different state of your project like developement, tesing, production etc. 
   You can pass more parameteres through this section to your nested stack.

ii) API gateway resource for path: To check all syntax and properties of AWS::ApiGateway::Resource, check this link.

  ProductStackApiResource:
    Type: AWS::ApiGateway::Resource
    Properties:
      ParentId: !GetAtt RootDefaultApi.RootResourceId
      RestApiId: !Ref RootDefaultApi
      PathPart: 'product'

Here in this section,

1. Type:AWS::ApiGateway::Resource mean it will create a resource in an API.
2. ParentId: It will be the parent resource. As you want to use a single URL, 
   this will be that URL's resource ID. You have to use in all your 
   nexted stack to get signle URL.
3. RestApiId: It will be identifier of associated RestApi
4. PathPart: It is the last path segment of this resource. Here use "product" 
   for productStack. You can use your own name for your stack like "auth" for 
   AuthStack
5. So, for your own nested stack, you have to change the value of ParentId,
   RestApiId and PathPart in this section

iii) API gateway resource for proxy: It is same as previous section. Just use proxy as PathPart.

  ProductStackProxyApiResource:
    Type: AWS::ApiGateway::Resource
    Properties:
      ParentId: !Ref ProductStackApiResource
      RestApiId: !Ref RootDefaultApi
      PathPart: '{proxy+}'

Here in this section,

1. Here {proxy+} is used for routing dynamically. Lets say you want to use 
   base_url/product/{id}, base_url/product/users/{id} under same product rest 
   API. This proxy part allows you to make flexible API endpoint.
2. To make your own nested stack, you have to change only these two
   ParentId, RestApiId in this section. Here your nested stack API gateway 
   resource will be ParentId and your base URL resource will be RestApiId.

iv) API gateway method: To check all syntax and property details, check this link.

  ProductStackProxyMethod:
    Type: AWS::ApiGateway::Method
    Properties:
      HttpMethod: ANY
      ResourceId: !Ref ProductStackProxyApiResource
      RestApiId: !Ref RootDefaultApi
      AuthorizationType: NONE
      RequestParameters:
        method.request.path.proxy: true
      Integration:
        CacheKeyParameters:
          - 'method.request.path.proxy'
        RequestParameters:
          integration.request.path.proxy: 'method.request.path.proxy'
        IntegrationHttpMethod: ANY
        Type: HTTP_PROXY
        Uri: !Sub "https://${ProductStack.Outputs.ProductStackApiId}.execute-api.${AWS::Region}.amazonaws.com/${StageName}/{proxy}"
        PassthroughBehavior: WHEN_NO_MATCH
        IntegrationResponses:
          - StatusCode: 200

Here in this section,

1. You have to change the value of ResourceId, RestApiId and Uri. Keep all the 
   value as it is.
2. Value of Uri will be made of the output of your nested stack. So make sure 
   that you export those values as output in nested stack. Otherwise you will 
   not get that value in root stack.

I have added my git link in this post and README describes how you can run this project. Hope you enjoy it. If you have any question, ask me.

If you find this post helpful, please follow me to get more tutorials like this.

How to create virtual environment from terminal (command line)

farzana-juthi — Sat, 19 Aug 2023 12:22:10 +0000

There are many reasons for creating virtual environment. I have mentioned two of them.

In our system, there are different type of projects and they need different versions of packages or libraries. If we install all dependencies in our system directly, then there will be conflict in different versions. To avoid this we need a virtual environment where we can install specific versioned library for specific projects.
Other benefit of creating virtual environment is to protect your system from polluting with different libraries. If you use virtual environment, you can get a clean and organized environment.

To create a successful environment, follow the steps —

To open command line based on your operating system choose one command:

    For windows:
    Short cut key:       Win + R    
                         Then write "cmd" and press "Enter" button

    For Linux:
    Short cut key:       Ctrl + Alt + T


    For Mac:
    Short cut key:       Command + Space
                         Then write "Terminal" and press "Enter" button

    Here "+" sign meaning - press all these keys at a time.

If you want to go to specific folder, then run following command

    Command:     cd <path of your destiantion>
    Example:     cd Documents/

Then make a folder.

    Command:     mkdir <folder_name>
    Example:     mkdir video_processor

Then go into this folder by running following command:

    Command:     cd <folder_name>
    Example:     cd video_processor

If you are using any environment previously deactivate it.

    deactivate

If you are using Conda previously deactivate it using following command.

    conda deactivate

If you are using python version 3.3 or later use following command to create a virtual environment. Here I named it as myenv.

    python -m venv <your_env_name>
    Example: python -m venv myenv

If you are using python version before 3.3 then use following command one after another.

    1. pip install virtualenv
    2. virtualenv myenv

After creating your virtual environment, you have to activate this. To activate it, you have to run following commands one after another.

    Based on your operating system, run the command. 
    Here myenv is the name of my virtual envirnment.

    For Windows 
            command:       
                         <your_env_name>\Scripts\activate
            Example:     myenv\Scripts\activate

    For Mac/Linux 
            command:     
                         source <your_env_name>/bin/activate
            Example:     source myenv/bin/activate

Now you successfully create your virtual environment. What do you want next? Do you want to make a file to store your library name and run all these dependencies from there? Follow the steps outlined in rest of this post.

To create a file named “requirement.txt”, you have to run one of the commands based on your operating system and it will create an empty file:

    For Windows:
            Command:     type nul > filename.ext
            Example:     type nul > requirement.txt
    For Mac/linux:
            Command:     touch filename.ext
            Example:     touch requirement.txt

    Here,    
            filename = you will give your file name 
            ext = extension of your file like py, txt, pdf etc

Now you have to give library name into this newly created file. You can do this in following way. Here I give 2 library name numpy and matplotlib.

    Command:     echo "<library_name>" >> <file_name>

    Example:     echo "numpy" >> requirements.txt
                 echo "matplotlib" >> requirements.txt

Then run following command to install these libraries into your virtual environment:

    Command:     pip install -r <your_file_name>

    Example:     pip install -r requirements.txt

How to setup HTTPS users to upload code into codecommit using git credential

farzana-juthi — Sat, 19 Aug 2023 12:09:33 +0000

CodeCommit is a fully managed source control service that makes it easy to host secure and highly scalable private Git repositories. CodeCommit eliminates the need to operate your own source control system or worry about scaling its infrastructure. You can use CodeCommit to securely store anything from source code to binaries, and it works seamlessly with your existing Git tools.

Prerequisites:

Install git: To install Git, we recommend websites such as Git Downloads .
Create IAM user and give permission: If you don’t know how to create IAM user and give permission for aws services, then please check this reference post. You need to follow the steps from 1–9.

Create Git credentials for HTTPS connection to CodeCommit:

If you have an IAM user, please go that “user” page from left menu and select the checkbox of the user where you want to configure “HTTPS Git credentials for AWS CodeCommit”. Then select “Security credentials” tab of this page.

Then click on “Generate credentials” button.

It will generate username and password like the following image. Download the credentials for further use. This credential will be used in the gitlab mirroring repository section.

Create CodeCommit repository and link it to gitlab code:

Search CodeCommit in aws console and click on it. You will find an option to create repository like following image:

Then click on “Create repository” button and fill up the following fields:

Here I am giving tutorial-app as repository name. You have to give your repository name here and click on “Create” button.
Then you will see newly created repository into your repositories list like following:

Then click on the radio button of “tutorial-app” repository (your repository name) and then from that page you will get following “clone url” option and select that button and choose “clone HTTPS”:

When you click on “clone HTTPS” button, you will see a link is copied like following image:

You have to use this link into gitlab.

Setup mirroring gitlab code to CodeCommit:

First browse the link https://about.gitlab.com/ and click on the Login button and then login into the system by giving our gitlab credentials.
Then search your gitlab project and click on that project. Here I will use my previously created “tutorial-backend” project.
Then select Settings -> Repository from tutorial-backend:

Then click on “Expand” from “Mirroring repositories” like following image:

Then take the previously copied url (step no 6 from Setup mirroring gitlab code to CodeCommit section) and paste it like following image:

Then you have to use the IAM codeCommit username after “https:// ” and then “@” sign after codeCommit username like following image and also copy the IAM codecommit password into the password field. Then click on the “Mirror repository” button. (Get your codecommit username and password from step no 3 from Create Git credentials for HTTPS connection to CodeCommit section)

After that click on “sync” icon in gitlab mirroring section like following:

After that please go to CodeCommit page from aws console and see that your code is synced to codeCommit like following image:

How to make an application using serverless application model (SAM) and python

farzana-juthi — Fri, 18 Aug 2023 13:00:13 +0000

When you want to organize your aws resources and components from a single stack, you can use SAM. It makes easy to share configuration between resources and deploy all things together.

Here in this article, I will show you that how to create API gateway, and lambda function in a single template.

Prerequisites:

Creating an AWS account: Open the Amazon Web Services (AWS) home page. Choose Create an AWS Account. If you signed in to AWS recently, choose Sign in to the Console. If Create a new AWS account isn’t visible, first choose Sign in to a different account, and then choose Create a new AWS account.
Configuring IAM: Configuring AWS Identity and Access Management (IAM) permissions is another prerequisite. You can not access aws from your local pc if you don’t have permission. If you don’t know how to create an IAM user and give permission on that user, please check this post
https://medium.com/@farzanajuthi08/how-to-create-user-and-give-permission-of-aws-services-using-iam-credential-42644f61fea5
Install Docker (Optional): If you want to test your application locally, then you have to install docker. Otherwise there is no need to install docker.
Install Homebrew: If you are a Linux or MacOS user, then you have to install Homebrew.
Install the AWS command line interface (CLI): If you don’t have aws CLI installed on your local machine, please follow the installation process from this link https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html.
Install latest version. Check the version by running the command
sam --version
Configure aws CLI into your local machine: You need access key and secret access key to configure AWS CLI into your machine. You have to download your keys when go through “Configuring IAM” step.
Open your terminal and run following code one after another to configure credentials into your pc:
```
   aws configure 
   AWS Access Key ID [None]: <your_access_key_id> 
   AWS Secret Access Key [None]:<your_secret_access_key>
   Default region name [None]:  <Give your region name here>
   Default output format [None]:
```
Install SAM CLI: If you don’t have SAM CLI installed into your local machine, please follow step 4 of this link https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install-windows.html
Install Python: If you want to use python as your application language, you have to install python in your local pc.

Steps to download a SAM project and deploy:

Open terminal on your pc and type the following command
```
                  sam -- init
```

and get following information like as this image

2. Here you have to use option 1 from the above image and you will get following options to choose:

3. You can use either 1 or 2. If you want to keep your artifact as a zip uploaded to s3, you have to choose option 1. Otherwise you have to choose option 2 if you want to keep it as image uploaded into ECR image repository.
Here I will choose option 1 and get an option to choose coding language like following image:

4. Then you have to give your required python version. Here I want to use python3.8 that’s why I have used number 9. And give my project name as tutorial-backend and give an Enter.
You can rename your application name after downloading it from the cloud.

5. After giving an Enter, *you will get some *options to choose the template you want to download from cloud:

6. Here I will choose option **1 **and download the **Hello World Example **template like following image:

7. Then change our current directory by typing command in the terminal:

                    cd <project_name>
          Example:  cd tutorial-backend

8. Then type following command into terminal to build your project:

                     sam build

9. Then you have to run following command for the first time to deploy this project into cloud:

                  sam deploy — guided

10. You will get following options to configure:
Stack name: Give your project name here. You will see this name in cloudformation.
AWS region: Choose your required region.
Other options will remain same as like the following image:

11. Then you will be asked to choose one option for deploying your code:

12. Then give y as value like Deploy this changeset? [y/N]: y

13. After successful deployment you can check your deployed project in cloudformation.
Go to aws console and search cloudformation and see your project:

14. After one successful deployment, if you change anything into your code, you don’t need to run sam deploy --guided again. Just remember you have to build it and deploy it by following commands one after another.

                    sam build
                    sam deploy

N.B: You may face problem if your code deployed region and your aws CLI configuration region is not same. So, please keep your cli configuration region and code deployment region same.

How to integrate gmail configured Backend APIs into frontend

farzana-juthi — Tue, 14 Mar 2023 08:03:02 +0000

If you are new and don't know how to integrate gmail with cognito, then follow this tutorial.
Now you can use cognito and created APIs on your frontend side.

To start implementing it into code, check the flow from the diagram:

First make the url which will be used in Sign In with gmail button.

    <domain_url>/oauth2/authorize?identity_provider=Google&redirect_uri=<frontend_redirect_url>&response_type=CODE&client_id=<cognito_client_id>&scope=email openid profile aws.cognito.signin.user.admin

Change curly bracket variables value like following
- domain_url: https://learning-auth-auth-stack-dev.auth.us-west-2.amazoncognito.com
- frontend_redirect_url: Remember this url is very important. It is used in cognito side as UserPoolClientCallBackUrlDevAndProd. If this url not match then you will get redirect mismatch error. Here my redirect url is http://localhost:4200/dashboard/
- cognito_client_id: Just check nested stack output from aws console and variable name is NestedCognitoUserPoolClientId. Here my client id is 7vfokrcbq7tc7s3u35qhj2054h

    Example: https://learning-auth-auth-stack-dev.auth.us-west-2.amazoncognito.com/oauth2/authorize?identity_provider=Google&redirect_uri=http://localhost:4200/dashboard/&response_type=CODE&client_id=7vfokrcbq7tc7s3u35qhj2054h&scope=email openid profile aws.cognito.signin.user.admin

Then this url will take you to google sign in page. Here you have to give your sign in email and password. If google verifies that credentials are right then it will redirect your to your application with a code.

    Example: http://localhost:4200/dashboard?code=09266419-d8b9-4005-8830-3bc57255c802

After that you have to call internally another url with following information to get tokens information.
- This will be a post method. Parameters need to pass along with the url are:
  - Method: POST
  - URL: /oauth2/token
  - Example: https://learning-auth-auth-stack-dev.auth.us-west-2.amazoncognito.com/oauth2/token
  - Parameters: {
    - grant_type: "authorization_code",
    - client_id: "Here you have to pass cognito_client_id what you have given into first step",
    - code: "Code you have got into the last step",
    - redirect_uri: "Here you have to give redirected url what you have given into first step" }
  - Example: {
    - grant_type: "authorization_code",
    - client_id: "7vfokrcbq7tc7s3u35qhj2054h",
    - code: "09266419-d8b9-4005-8830-3bc57255c802",
    - redirect_uri: "http://localhost:4200/dashboard/" }
Then you can get social sign in related data by calling following API
- Method: POST
- URL: https://
- Example: https://joc4u21r8e.execute-api.us-west-2.amazonaws.com/dev/auth/authentication/social-signin
  - API base url: https://joc4u21r8e.execute-api.us-west-2.amazonaws.com/dev
  - Path: /auth/authentication/social-signin
  - Parameter: { "id_token": "", "access_token": "", "refresh_token": "", "expires_in": "", "token_type": "" }

  {
    "id_token": "",
    "access_token": "",
    "refresh_token": "",
    "expires_in": "",
    "token_type": ""
  }

If you want to log-out, then you have to call logout API
- Method: POST
- URL: <base_url><path>
- Example: https://joc4u21r8e.execute-api.us-west-2.amazonaws.com/dev/auth/authentication/social-signin
- API base url: https://joc4u21r8e.execute-api.us-west-2.amazonaws.com/dev
- Path: /auth/authentication/global-logout
- Parameter: {"access_token": ""}

  {
   "access_token":"Here you have to pass access token "
  }

How to integrate gmail with cognito as social authentication (using SAM, python)

farzana-juthi — Tue, 14 Mar 2023 06:48:14 +0000

In this application, we will learn how to configure google app into AWS cognito.

Installation

First, you have to install aws sam cli into your machine.
Then configure aws cli.

If you don't have CLI installed and configured into your local machine please follow prerequisite steps from this link

Local Development

After installation is done, you have to pull the code from git repository (HTTPS link)
Then go to the project directory by using the following command:

    cd <your folder name>
    example: cd authentication

Then open template.yaml file and change following (15-20) lines with appropriate data:
- In line 15 (variable CallBackUrlUserPoolClient), you have to provide the url where google will be redirected after verifying user credential into google side. You can add multiple urls by comma separation.
- In line 16 (variable LogOutUrlUserPoolClient), you have to provide the url where cognito will be redirected after logout. You can add multiple url by comma separation.
- In line 17 (variable FIDGoogleClientId), you have to give client id of google app. See How to configure google app
- In line 18 (variable FIDGoogleClientSecret), you have to give secret client id of google app.

  Example:
    "CallBackUrlUserPoolClient": "http://localhost:4200/dashboard/" 
    "LogOutUrlUserPoolClient": "http://localhost:4200/login/" 
    "FIDGoogleClientId": "816187719480-26d52o4pnfe789ivkjqdm0jqakk19m38.apps.googleusercontent.com" 
    "FIDGoogleClientSecret": "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"

Then give a project name into line 27. Here you have to set the value of the ProjectName parameter. Remember this data will be used to make domain name and domain name need to be unique.
You must change this value
Then open terminal in root folder of this project and run following command:

    sam build

If you deploy first time, then you have to run following command:

    sam deploy --guided --capabilities CAPABILITY_IAM CAPABILITY_AUTO_EXPAND

If you want to run this command with your predefined profile then command will be

    sam deploy --guided --profile <your_profile_name_without_this_bracket> --capabilities CAPABILITY_IAM CAPABILITY_AUTO_EXPAND

After running the previous command, you will see that you have to set a stack name. Please give a unique stack name.

For example: I am giving stack name as auth-app.

Then click enter one after another until SAM configuration environment variable set to default value.

Then wait for the successful creation of cloudformation stack.
If you want to deploy after changes, then you need to build it first and run only deploy command like following:

    sam build
    sam deploy

After successful deployment you will get some output. Save those for further implementation:

DomainURL: It's value will be used as domain_url in frontend side
CognitoAppClientID: It's value will be used as cognito_client_id in frontend side
RootAPI: It's value will be used as base_url in frontend side

Update Google APP - Authorized JavaScript origins and Authorized redirect URIs

You must have to follow step 11 of this link

How to SAM accelerate work with lambda layers

farzana-juthi — Thu, 09 Mar 2023 07:33:32 +0000

If you are a developer, you know testing is very important for application development. But integration testing in cloud is really a concern. As deployment time plays a vital role in development, many of us try to do mock test before deploy it into cloud. There are some frameworks for doing that. But in that case integration testing is not possible. If you are a SAM user, you can speed up your development with SAM accelerate. In this blog post, you will see an example of lambda layers and its implementation in sam accelerate.

Prerequisite

First, you have to install aws sam cli into your machine.
Then configure aws cli.

If you don't have CLI installed and configured into your local machine please follow the prerequisite steps from this link

Local Development

After installation done, you have to pull the code from git repository (HTTPS link)
Then go to project directory by using following command:

    cd <your folder name>
    example: cd sam-accelerate-example

Then you will see following structure:

Lambda Layers code details

In template.yaml file, you will get following code for creating lambda layers:

In this image,

Type: This is used to make resource you want to create. AWS::Serverless::LayerVersion means it will create lambda layer.

Properties:

LayerName: This is for layer name you want to give. Here you can give any name. In this demo it is global-utils .

ContentUri: This will be your code location. In this demo it is lambda_layers/global_utils/

CompatibleRuntimes:

python3.9 Here you have to give your choosen language. In this demo, I used python. There are specific folder structure for lambda layers according to language. See this link for other languages.

Metadata:

BuildMethod: makefile If you want to create custom layer you can do it easily through SAM. In this make file you have to give commands to run your configuration or settings. See this link for documentation.

If you need share some libraries through layers, you can add those into requirements.txt file in layer's python folder like following image:

In make file within global_utils folder, you have to give path to $(ARTIFACTS_DIR)/python, otherwise libraries in requirements.txt file will not found into .aws-sam folder and when calling through layers it will not work. So this line is very important.

  build-GlobalUtils:
      echo "Global layers build started"
      pip install --default-timeout=100 -r ./python/requirements.txt -t $(ARTIFACTS_DIR)/python
      echo "Global layers build completed"
      cp -r ./python $(ARTIFACTS_DIR)/

If you want to pass some code through layers, you can also do that. In global_utils.py file you can see example of it.

Here three libraries (json, os, boto3) and also dynamo client, resource and table name are passed through this layer. For table name, you have to pass it through function's Environment parameter. You will see it in a while.

Now this layer has to be attached with lambda function where it will be used. In this post you will see global_utils layer is passed to HelloWorldFunction . In case of sharing resources with !Ref key, you have to use logical id of that resource. Here logical id for layer is GlobalUtils and for dynamodb table it is TestTable .

How to run sam sync

You have to run following command into your terminal

    sam sync --watch --stack-name <give your stack name>
    example: sam sync --watch --stack-name sam-accelerate-example

After running this command, you will see like following image:

Now you have to press Enter .
Now open your aws console from browser, you will see your stack is created into cloud like following image.

After sync, you will see an API link in your device terminal like following image:

After clicking on this API, you will see the table name what is passed through layer.

If you change anything in layers or anything into your template file, you will see that change into cloud. So you can test your application with real aws resources. If you have any question please let me know in comment.