DEV Community: Shivang Chauhan

How I Built a Video Transcoder using AWS MediaConvert

Shivang Chauhan — Mon, 15 Aug 2022 16:50:27 +0000

In this post, you are going to see how I created a Serverless Video Transcoding Pipeline using AWS MediaConvert using Nodejs.

There will be two parts to this project this is the first part where I will be showing you how I built the backend for this using AWS Serverless.

Let’s get started by creating a blank folder with serverless.yml file which will be the core file to deploy our Serverless stack to AWS.

Creating Serverless.yml file

service: video-transcoding-pipeline

provider:
  name: aws
  region: ${file(./env.yml):${opt:stage}.REGION}
  runtime: nodejs14.x
  versionFunctions: false
  tracing:
    lambda: true

functions:
  - ${file(./lambdaFunctions.yml)}

resources:
  - ${file(./permissions.yml)}
  - ${file(./db.yml)}
  - ${file(./s3.yml)}

As you can see here that we are importing a bunch of yml files which we will be creating next, we are also setting the region which is imported from the env file of the project.

To know more about serverless.yml file check out “What is a serverless.yml file?” section here.

Creating S3 buckets

Resources:
  MediaInputBucket:
    Type: AWS::S3::Bucket
    Properties:
      CorsConfiguration:
        CorsRules:
          - AllowedHeaders: ["*"]
            AllowedMethods: [GET, PUT, POST]
            AllowedOrigins: ["*"]

  MediaOutputBucket:
    Type: AWS::S3::Bucket
    Properties:
      CorsConfiguration:
        CorsRules:
          - AllowedHeaders: ["*"]
            AllowedMethods: [GET, PUT, POST]
            AllowedOrigins: ["*"]
      AccessControl: PublicRead

Now we will be creating the s3.yml file which will be responsible for creating the S3 buckets, we are creating two buckets here.

The MediaInputBucket is the input bucket where the video file will be uploaded to get transcoded.

The MediaOutputBucket is the output bucket where the transcoded video will be saved by AWS MediaConvert.

CorsRules : This config is used to set the Cors for the buckets so we can interact with the buckets through the client side (these can be changed according to the need).
AccessContro l: This is giving the public access to the bucket so transcoded videos can be played publicly.

To check more configurations provided for S3 bucket creation, check out the official documentation.

Creating a DynamoDB Table

Resources:
  VideoStatus:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: ${file(./env.yml):${opt:stage}.VIDEO_STATUS_TABLE}
      PointInTimeRecoverySpecification:
        PointInTimeRecoveryEnabled: true
      AttributeDefinitions:
        - AttributeName: id
          AttributeType: S
      KeySchema:
        - AttributeName: id
          KeyType: HASH
      BillingMode: PAY_PER_REQUEST

Here a DynamoDB table is being created, this table will be used to store the AWS MediaConvert Job status (more on this later).

Also, you can see that the table name is also being imported from the env file, so let’s create this file now.

Creating the env.yml file

prod:
  MEDIA_INPUT_BUCKET: !Ref MediaInputBucket
  MEDIA_OUTPUT_BUCKET: !Ref MediaOutputBucket
  REGION: us-east-2
  VIDEO_STATUS_TABLE: VideoStatusTable
  ACCOUNT_ID: [REPLACE_THIS_WITH_YOUR_ACCOUNT_ID]
  MEDIA_ENDPOINT: [REPLACE_THIS_WITH_YOUR_ENDPOINT]
  MEDIA_CONVERT_ROLE: !GetAtt MediaConvertRole.Arn

Here we are creating a bunch of env variables under the prod stage name.

MEDIA_ENDPOINT : This is the endpoint for MediaConvert which you can get from your AWS Console by going under the Account section in the MediaConvert dashboard.
MEDIA_CONVERT_ROLE : This is the IAM role for AWS MediaConvert.

Creating permissions.yml file

Now it’s time to create the permissions.yml file, there will be two roles created in this file, one will be used by all the Lambda functions and another one will be used by AWS MediaConvert.

Let’s break down this file as it is a bit long.

Creating policy for interacting with DynamoDB

Resources:
  LambdaRole:
    Type: "AWS::IAM::Role"
    Properties:
      RoleName: "LambdaRole-${opt:stage}"
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: "sts:AssumeRole"
      Policies:
        - PolicyName: "LambdaRolePolicy-${opt:stage}"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Action:
                  - "dynamodb:PutItem"
                  - "dynamodb:UpdateItem"
                  - "mediaconvert:*"
                Effect: Allow
                Resource: "*"

This policy will allow the lambda functions to interact with the DynamoDB table.

Creating policy for interacting with AWS MediaConvert

Policies:
  - PolicyName: 'MediaConvertLambdaPolicy-${opt:stage}'
    PolicyDocument:
      Version: '2012-10-17'
      Statement:
        - Sid: PassRole
          Effect: Allow
          Action:
            - 'iam:PassRole'
          Resource: !GetAtt MediaConvertRole.Arn
        - Sid: MediaConvertService
          Effect: Allow
          Action:
            - 'mediaconvert:*'
          Resource:
            - '*'
        - Sid: MediaInputBucket
          Effect: Allow
          Action:
            - 's3:*'
          Resource:
            - '*'

This policy will allow the Lambda functions to interact with AWS MediaConvert, to read more about how these permissions work check out this official documentation by AWS.

Creating policy to write CloudWatch Log streams

Policies:
  - PolicyName: 'CloudWatchLogsPolicy-${opt:stage}'
    PolicyDocument:
      Version: '2012-10-17'
      Statement:
        - Action:
            - 'logs:CreateLogGroup'
            - 'logs:CreateLogStream'
            - 'logs:PutLogEvents'
          Effect: Allow
          Resource:
            - >-
              arn:aws:logs:${file(./env.yml):${opt:stage}.REGION}:${file(./env.yml):${opt:stage}.ACCOUNT_ID}:*

This is straightforward as we are allowing the lambda log to be created in the same region and AWS account where we are deploying the stacks.

Now we will create the second role which will be attached to the MediaConvert.

Creating IAM role for AWS MediaConvert

MediaConvertRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: "MediaConvertRole-${opt:stage}"
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - "mediaconvert.amazonaws.com"
                - "mediaconvert.us-east-2.amazonaws.com"
            Action:
              - sts:AssumeRole
      Policies:
        - PolicyName: "MediaConvertPolicy"
          PolicyDocument:
            Statement:
              - Effect: "Allow"
                Action:
                  - "s3:*"
                Resource:
                  - "*"
              - Effect: "Allow"
                Action:
                  - "cloudwatch:*"
                  - "logs:*"
                Resource:
                  - "*"

This role will allow AWS MediaConvert to interact with S3 and also be able to write AWS CloudWatch logs to the AWS account.

That was a lot to take in but now you are done with creating core yml files, now there is only one yml file left that will create all the lambda functions which are needed, so let’s start with that.

Creating lambdaFunctions.yml file

startJob:
  handler: resolvers/job/startJob.handler
  name: ${opt:stage}-startJob
  timeout: 600
  role: LambdaRole
  description: Lambda function to start the media convert job
  environment:
    VIDEO_STATUS_TABLE: ${file(./env.yml):${opt:stage}.VIDEO_STATUS_TABLE}
    MEDIA_INPUT_BUCKET: ${file(./env.yml):${opt:stage}.MEDIA_INPUT_BUCKET}
    MEDIA_OUTPUT_BUCKET: ${file(./env.yml):${opt:stage}.MEDIA_OUTPUT_BUCKET}
    MEDIA_ENDPOINT: ${file(./env.yml):${opt:stage}.MEDIA_ENDPOINT}
    REGION: ${file(./env.yml):${opt:stage}.REGION}
    MEDIA_CONVERT_ROLE: ${file(./env.yml):${opt:stage}.MEDIA_CONVERT_ROLE}
  events:
      - s3:
          bucket: ${file(./env.yml):${opt:stage}.MEDIA_INPUT_BUCKET}
          event: s3:ObjectCreated:*
          existing: true

getSignedUrl:
  handler: resolvers/getSignedUrl.handler
  name: ${opt:stage}-getSignedUrl
  timeout: 600
  role: LambdaRole
  description: Lambda function to get the signed url to upload the video
  environment:
    MEDIA_INPUT_BUCKET: ${file(./env.yml):${opt:stage}.MEDIA_INPUT_BUCKET}
  events:
      - http:
          path: getSignedUrl
          method: post
          cors: true

updateJobStatus:
  handler: resolvers/job/updateJobStatus.handler
  name: ${opt:stage}-updateJobStatus
  timeout: 600
  role: LambdaRole
  description: Lambda function to update the media convert job status in the DB
  environment:
    VIDEO_STATUS_TABLE: ${file(./env.yml):${opt:stage}.VIDEO_STATUS_TABLE}
    REGION: ${file(./env.yml):${opt:stage}.REGION}
  events:
      - cloudwatchEvent:
          event:
            source:
              - 'aws.mediaconvert'
            detail-type:
              - 'MediaConvert Job State Change'

There are three lambda functions that are being created here.

startJob : This lambda function will be responsible to start the AWS MediaConvert job and it will be called whenever any file will be uploaded to the input S3 bucket which you created earlier.
getSignedUrl : This lambda function will return the signed URL to upload the video file to the input bucket from the client side.
updateJobStatus : This lambda function will be updating the MediaConvert job status to the DynanmoDB table and it will be called whenever the job status gets changed in MediaConvert.

Now you are done with creating all the required yml files, let’s move on to creating the resolvers for the lambda functions.

getSignedUrl Lambda Resolver

This lambda function will be called first to get the signed URL back and then that signed URL will be used to upload the video file to S3 from the client side so we are uploading the video from the backend.

Adding validations

const {
    fileName,
    metaData
} = JSON.parse(event.body)

if (!fileName || !fileName.trim()) {
    return sendResponse(400, {
        message: 'Bad Request'
    })
}

Here you are getting the file name and the metadata from the client side and you are also checking that the file name must exist otherwise 400 Status code is being returned.

The sendResponse is a utility function that is just sending the response to the API request, you can find it in the source code.

Creating the signed URL

const params = {
    Bucket: process.env.MEDIA_INPUT_BUCKET,
    Key: fileName,
    Expires: 3600,
    ContentType: 'video/*',
    Metadata: {
        ...metaData
    }
}

const response = s3.getSignedUrl('putObject', params)

Here parameters are being created and getSignedUrl API call is made to get the signed URL, ContentType is set to video/* because only videos will be uploaded to the S3 bucket from the client side.

Now when the file will get uploaded to the S3 bucket by the client application using this signed URL, startJob lambda function will be triggered which will start the AWS MediaConvert job, let’s see what this lambda function looks like.

startJob Lambda Resolver

The first thing that I wanna show you is what are the imports that are added in this lambda resolver.

Imports

const {
    sendResponse
} = require('../../utilities/index')
const AWS = require('aws-sdk')
AWS.config.mediaconvert = {
    endpoint: `https://${process.env.MEDIA_ENDPOINT}.mediaconvert.${process.env.REGION}.amazonaws.com`
}
const MediaConvert = new AWS.MediaConvert({
    apiVersion: '2017-08-29'
})
const s3 = new AWS.S3()
const params = require('./mediaParams.js')
const dbClient = new AWS.DynamoDB.DocumentClient()

Notice here that I am updating the endpoint for the MediaConvert config, also there is a file named mediaParams.js which is being imported here.

This file will hold the configuration for starting the MediaConvert job, so we will now create this file first.

Creating mediaParams.js config file

module.exports = {
  Settings: {
    TimecodeConfig: {
      Source: 'ZEROBASED'
    },
    OutputGroups: [
      {
        Name: 'Apple HLS',
        Outputs: [
          {
            ContainerSettings: {
              Container: 'M3U8',
              M3u8Settings: {}
            },
            VideoDescription: {
              Width: '',
              Height: '',
              CodecSettings: {
                Codec: 'H_264',
                H264Settings: {
                  MaxBitrate: '',
                  RateControlMode: 'QVBR',
                  SceneChangeDetect: 'TRANSITION_DETECTION'
                }
              }
            },
            AudioDescriptions: [
              {
                CodecSettings: {
                  Codec: 'AAC',
                  AacSettings: {
                    Bitrate: 96000,
                    CodingMode: 'CODING_MODE_2_0',
                    SampleRate: 48000
                  }
                }
              }
            ],
            OutputSettings: {
              HlsSettings: {}
            },
            NameModifier: 'hgh'
          }
        ],
        OutputGroupSettings: {
          Type: 'HLS_GROUP_SETTINGS',
          HlsGroupSettings: {
            SegmentLength: 10,
            MinSegmentLength: 0,
            DestinationSettings: {
              S3Settings: {
                AccessControl: {
                  CannedAcl: 'PUBLIC_READ'
                }
              }
            }
          }
        }
      },
      {
        CustomName: 'Thumbnail Creation Group',
        Name: 'File Group',
        Outputs: [
          {
            ContainerSettings: {
              Container: 'RAW'
            },
            VideoDescription: {
              Width: 1280,
              Height: 720,
              CodecSettings: {
                Codec: 'FRAME_CAPTURE',
                FrameCaptureSettings: {
                  FramerateNumerator: 1,
                  FramerateDenominator: 5,
                  MaxCaptures: 5,
                  Quality: 80
                }
              }
            }
          }
        ],
        OutputGroupSettings: {
          Type: 'FILE_GROUP_SETTINGS',
          FileGroupSettings: {
            DestinationSettings: {
              S3Settings: {
                AccessControl: {
                  CannedAcl: 'PUBLIC_READ'
                }
              }
            }
          }
        }
      }
    ],
    Inputs: [
      {
        AudioSelectors: {
          'Audio Selector 1': {
            DefaultSelection: 'DEFAULT'
          }
        },
        VideoSelector: {},
        TimecodeSource: 'ZEROBASED'
      }
    ]
  },
  AccelerationSettings: {
    Mode: 'DISABLED'
  },
  StatusUpdateInterval: 'SECONDS_60',
  Priority: 0
}

As you can see that there are a lot of parameters added here but most of these values are static in this project, you will only be modifying transcoded video width/height and bitrate (many many more configurations can be made dynamic according to the requirement).

Fetching the metadata from the uploaded file

const fileKey = event.Records[0].s3.object.key
const {
    metaData
} = await fetchMetaData(fileKey)

Here you are getting the uploaded file key (which will be received in the lambda trigger attached to the S3 bucket) and calling fetchFromS3 function.

Creating fetchFromS3 function

async function fetchMetaData (key) {
  try {
    const params = {
      Bucket: MEDIA_INPUT_BUCKET,
      Key: key
    }

    const response = await s3.headObject(params).promise()
    return { metaData: response.Metadata }
  } catch (err) {
    throw new Error(err)
  }
}

Creating the params for starting the MediaConvert job

const input = `s3://${MEDIA_INPUT_BUCKET}/${fileKey}`
const output = `s3://${MEDIA_OUTPUT_BUCKET}/`

params.Role = MEDIA_CONVERT_ROLE
params.Settings.OutputGroups[0].OutputGroupSettings.HlsGroupSettings.Destination = output
params.Settings.OutputGroups[1].OutputGroupSettings.FileGroupSettings.Destination = output
params.Settings.Inputs[0].FileInput = input
params.Settings.OutputGroups[0].Outputs[0].VideoDescription.Width = metaData.videowidth || 1920
params.Settings.OutputGroups[0].Outputs[0].VideoDescription.Height = metaData.videoheight || 1080
params.Settings.OutputGroups[0].Outputs[0].VideoDescription.CodecSettings.H264Settings.MaxBitrate = metaData.videobitrate || 6000000

const response= await MediaConvert.createJob(params).promise()

We are setting the IAM role for MediaConvert and other settings with metadata as we discussed previously.

Creating initial entry for the created job in the DB

const vodObj = {
    TableName: VIDEO_STATUS_TABLE,
    Item: {
        id: response.Job.Id,
        createdAt: new Date().toISOString(),
        vodStatus: 'SUBMITTED'
    },
    ConditionExpression: 'attribute_not_exists(id)'
}

await dbClient.put(vodObj).promise()

We are taking the created job id and making it a Sort key in the DynamoDB table and we are also setting the initial job status to SUBMITTED.

Now it is time to work on the last lambda function resolver.

updateJobStatus Lambda Resolver

try {
    const { VIDEO_STATUS_TABLE, REGION } = process.env
    const { jobId, status, outputGroupDetails } = event.detail

    const params = {
      TableName: VIDEO_STATUS_TABLE,
      Key: {
        id: jobId
      },
      ExpressionAttributeValues: {
        ':vodStatus': status
      },
      UpdateExpression: 'SET vodStatus = :vodStatus',
      ReturnValues: 'ALL_NEW'
    }

    if (status !== 'INPUT_INFORMATION') {
      if (status === 'COMPLETE') {
        const splitOutput = outputGroupDetails[0].outputDetails[0].outputFilePaths[0].split('/')
        params.ExpressionAttributeValues[':outputPath'] = `https://${splitOutput[2]}.s3.${REGION}.amazonaws.com/${splitOutput[3]}`
        params.UpdateExpression += ', outputPath = :outputPath'
      }

      await dbClient.update(params).promise()
    }
  } catch (err) {
    return sendResponse(500, { message: 'Internal Server Error' })
  }

This will be the final lambda function resolver you will need, this lambda will be called whenever the status of the MediaConvert job changes and it will update the new status to the DynamoDB table using the job id we stored earlier.

There are three main stages of a job progression –

SUBMITTED : This is the initial job status when it gets started and this is being stored by startJob lambda function.
PROGRESSING : This is the status when the job is going on and it will be set through this lambda function.
COMPLETE : This is the final status when the job gets successfully completed.

If you want to read more about the different stages of a job, you can check here.

AND we are done, pat yourself on the back if you have reached this far, there are a lot of improvements that can be done in this project.

Improvements

MediaConvert endpoint can be fetched using describeEndpoints API, read more here.
More configuration can be added to the AWS MediaConvert startJob parameters.
Multi-part upload can be implemented to upload larger video files.
Job status can be pushed to SNS topic to use in other places.
AWS CloudFront distribution can be used to distribute the transcoded video.

Conclusion

Today you saw how we can create a video transcoder using AWS MediaConvert with Serverless and Nodejs, you can play around with it and have fun adding new things, there will be a part 2 to this series where i will be showing how to make the Frontend for this

Find the full source code here.

Check more posts:

What is AWS Athena?

AWS Cognito Pricing

DynamoDB VS MongoDB: Detailed Comparison

AWS DynamoDB Pricing Model and Features Explained

The post How I Built a Video Transcoder using AWS MediaConvert appeared first on DevsWisdom.

AWS Athena: All You Need To Know

Shivang Chauhan — Tue, 05 Jul 2022 17:08:08 +0000

What is AWS Athena?

AWS Athena is a serverless service that allows you to search through the data in AWS S3 buckets by using SQL queries on the data, you can create different kinds of schemas to query on the data and run SQL queries on that data.

One of the best things about AWS Athena is that it is _ serverless _ so you only pay for what you use, meaning pricing depends upon the amount of data scanned.

Sometimes we have to analyze large sets of data to find any statistics, for example in the case of finance reports it could be profits earned, top-selling products, fastest-selling products, etc.

If we try to find all this information manually then it will be nearly impossible, then in situations like these, we can use AWS Athena to run all types of SQL queries on the data and find any statistics we want.

AWS Athena Features

There are a lot of great features which this service offers, let’s see some of them.

Serverless

The best feature about AWS Athena is that it is a serverless service which means that you don’t pay anything extra, you don’t need to set up servers and maintain them.

Everything is managed by AWS itself and you only pay for the amount of data you are scanning using the queries.

SQL Support

This is the main feature offered by AWS Athena as you can basically write all types of SQL queries so anyone having an experience with SQL will be able to use this service write away for querying any kind of data they want by putting it to Amazon S3 bucket.

User Friendly

The Amazon dashboard to use AWS Athena is very user-friendly as it allows you to directly write queries from the editor and see all the tables, and schemas that you created for the dataset.

You can even save the queries you want to use later on and export the results obtained through the query.

Step Functions Support

This is a newly added feature by Amazon which allows you to create different workflows to analyze different data sets in a sequence by using Step functions state machines.

Step functions allow you to execute multiple queries at once or in a sequence and provide a way to use the previous queried data to execute the next one.

To know more about AWS Step Functions, read here.

Machine Learning

There is also an option to include machine learning in the SQL queries in AWS Athena, which allows you to perform tasks like making predictions using the data, etc.

Scalable Performance

As you know that AWS Athena is a serverless service so you don’t need to worry about scaling the servers for performing heavy CPU-intensive queries, you can perform all sizes of queries on all sizes of data and AWS will make sure that performance is scaled by running the queries in parallel and distributing the query load.

To read about more features in detail, check out the official documentation by Amazon.

AWS Athena Pricing

Pricing for this service is pretty straightforward, let’s see with an example.

As of currently writing this post you are charged according to the amount of data scanned, which is $5 per TB of data, whenever you run any query, and if any data gets scanned you are charged for that size of data scanned.

AWS automatically rounds the size of scanned data to the nearest megabyte, cancelled queries are also charged if any data was scanned before canceling the request.

To know about the detailed pricing model for AWS Athena, check out the official documentation here.

Conclusion

In this article we saw how AWS Athena is a great tool to query through any kind of data set by using SQL queries and find out the important statistics, we discussed some of the features which it offers in brief and how the pricing is calculated, to know more about this service check out the official documentation here.

The post AWS Athena: All You Need To Know appeared first on DevsWisdom.

How to remove an element from an array?

Shivang Chauhan — Sat, 18 Dec 2021 16:24:31 +0000

Introduction

In this post I am going to discuss multiple ways which can be used to remove an element from an array, there are couple of ways which can be used to achieve this. Some are splice, slice, filter etc, depending on the requirement we use different method to remove the element from an array.

Let's get started.

Using Array splice()

splice() can be used to do many things with an array like adding elements, removing elements, replacing the current element on any position, but in our case we are going to see how to remove an element from an array with splice().

const numbers = [1, 2, 3, 4, 5];
numbers.splice(1,1);
console.log(numbers) // [1, 3, 4, 5];

const deletedElement = numbers.splice(0,1);
console.log(deletedElement ) // [1];

Explanation

Let's try to see what is going on in this example step by step, so basically splice() accepts two main arguments to remove an element from an array, first is the starting position which means from where in the array it needs to start modifying the elements (in our case removing them).

Then there is a second optional argument which is basically describing the number of elements to be removed from the starting position.

Using Array Filter()

This is another way to remove an element from an array, but in my opinion this is not necessary if we have a very simple array, like array of integers, letters, etc, this filter method is generally used to do complex filters on an array, anyways let's see how it works with an example.

const numbers = [1, 2, 3, 4, 5];
const modifiedArr = numbers.filter((el) => el !== 2);
console.log(modifiedArr); // [1, 3, 4, 5]

Explanation

Let's try to understand this, filter() takes a callback function as an argument, this callback functions gets executed for each element in the array on which filter was called, this callback functions executes a condition for each element in an array which returns either true or false, if it returns true that elements gets pushed to the new array and if it returns false for any element then that elements gets skipped and not gets pushed to the new array.

Using Array Slice()

This is the final method which we are going to discuss, this method is mostly used when we want to extract a part of an array and leave previous and next elements, this is not our typical method to remove any element from any position in an array, so let's see how this method works and understand it's important arguments.

const numbers = [1, 2, 3, 4, 5];
const modifiedArr = numbers.slice(1,4);
console.log(modifiedArr); // [2, 3, 4]

Explanation

As you can see in above example this method is used for very specific use case, so there are two main arguments which this method accepts, first is the starting position which is the index number from where we need to start taking the elements into our new array, in our example it is '1' which means second element, elements will be picked up to the second argument but slice() will not include that element at the index passed as the second argument.

Conclusion

In this post we saw different ways which can be used to remove an element from an array, each of these methods are used according to different use cases.

Check out more:

CRUD with DynamoDB, Nodejs and Serverless

DynamoDB VS MongoDB

What is AWS Artifact?

Javascript Math floor(), Math ceil() and Math round()

Shivang Chauhan — Sun, 21 Nov 2021 13:01:30 +0000

Introduction

In javascript there are many ways to round a number, one of the methods is to use a built-in object called Math which gives us three methods to use, Javascript Math.floor(), Math.ceil() and Math.round(), each of these methods serve their purpose and should not be confused with each other or alternatives for each other, in this post, we will try to see how each method works with different examples and which one is suitable for which case.

Math.floor()

This method returns us the largest integer less than or equal to the number which we pass as the input to the method.

Examples

console.log(Math.floor(5.95)); // output: 5

console.log(Math.floor(-11.23)); // output: -12

console.log(Math.floor(9.78)); // output: 9

Math.ceil()

This method returns us the smallest integer greater than or equal to the number which we pass as the input to the method, which means it rounds up the number to the next greater or equal integer.

Examples

console.log(Math.ceil(5.95)); // output: 6

console.log(Math.ceil(-11.23)); // output: -11

console.log(Math.ceil(9.78)); // output: 10

Math.round()

This method returns us the number rounded to the nearest integer, the question may arise that is this going to round up or round down to get to the nearest integer, this depends on the fractional part of the number, so if the fractional part is greater than 0.5 then the number is rounded up and if the fractional part is less than 0.5 then this method rounds down the number and if it is equal to the number than it also rounds up the number.

Examples

console.log(Math.round(5.95)); // output: 6

console.log(Math.round(5.23)); // output: 5

console.log(Math.round(-15.5)); // output: -15

Conclusion

So now it is clear that these methods are different and cannot be used as an alternative for each other, we need to use each one according to our use case.

Check out some more posts from DevsWisdom

What is AWS Artifact?

How to extract text from an image with AWS Textract?

Most Common Methods Used In Javascript and FAQ

AWS Cognito Authentication With Serverless and NodeJS

The post Javascript Math floor(), Math ceil() and Math round() appeared first on DevsWisdom.

Dev.to posts with canonical url idexing on google, why?

Shivang Chauhan — Sun, 21 Nov 2021 04:36:58 +0000

Hello guys, i have a blog https://devswisdom.com/ whenever i create a post on my blog i repost it here and add the canonical URL pointing to the original content so google knows that which page is original and to only index that page and not the post which is here.

But all my reposts are getting indexed on google in spite of added canonical tags, here is an example.

If you type "AWS Artifact" on Google, you will see my post on Dev.to
https://dev.to/shivangchauhan7/what-is-aws-artifact-2641

But canonical tag is added on this post and it should not show up on google, the original post is here
https://devswisdom.com/what-is-aws-artifact/

Please help me with this, admins/moderators.

What is AWS Artifact?

Shivang Chauhan — Fri, 19 Nov 2021 12:58:55 +0000

Introduction

AWS Artifact is a self service which anyone with AWS account can access and use it for free, it provides AWS customers to access compliance documentation and agreements on demand for free. It is like a portfolio for AWS reports which we can see and read about it in details. Even all IAM users with admin permissions can access this service and get to know more about AWS compliance and agreements.

All the agreements found here will be of two types

Active – Agreements which are signed by users and are currently going on.

Inactive – These are agreements which are not accepted by the user or user has terminated these agreements.

AWS root account has all the permissions to access any kind of document through this service, for IAM users we have an option to use IAM policies to give access to certain documents to certain IAM users according to their authority and requirement.

For example if there is an audit needed and our auditors need access to certain documents needed for the audit, we can create IAM user and give it to access certain types of documents only.

How to access AWS Artifact?

We simply need to go to our AWS account and type “artifact” in the AWS service search option and select Artifact from the list.

How to download reports?

On the current AWS service console, we will see these two buttons, one is to view agreements and one is to view reports, click on “View reports”.

We need to select the document which we want to download and then click on “Download report” button.

How to accept agreements?

Select “View agreements” which is on the home page of AWS artifact service then select the agreement which you want to accept and click on “Accept agreement” button on the page.

Conclusion

Overall AWS Artifact is a great service provided by AWS so their users can get to know them more and evaluate them.

Check more posts from DevsWisdom

AWS Cognito Pricing and Features – 2021
AWS Step Functions Most Used Nodejs SDK APIs
AWS Step Functions Cheatsheet
How to extract text from an image using AWS Textract

The post What is AWS Artifact? appeared first on DevsWisdom.

AWS Cognito Pricing and Features – 2021

Shivang Chauhan — Thu, 18 Nov 2021 17:45:08 +0000

What Is Amazon Cognito?

It is a service introduced by AWS (Amazon Web Services) which allows users to add authentication to projects, it allows users to sign in with different social identity providers, and scales to millions of users and requests.

It supports access management standards like OAuth and SAML, users do not have to worry about setting up a UI for authentication as it provides a built-in UI that can be integrated with any application very quickly, this UI can also be customized according to specific requirements of the company.

AWS Cognito Pricing

**DISCLAIMER**

As of today when writing this post, which is Thursday, November 18, 2021, GMT+0530 (India Standard Time), this is the pricing model for Amazon Cognito.

For Cognito User Pools Amazon charges on the basis of how many users are active within a month, they call those users MAU (Monthly Active Users), a user is counted as MAU if there is any authentication operation done through that user in a particular month, authentication operations like SignIn, SignUp, Password Change, Refreshing the auth token, any user attribute gets updated in the user pool.

For users who use enterprise directory through SAML federation to SignIn, there is a different pricing model set by Amazon.

Free Tier Pricing

50,000 MAU’s for Cognito User Pool users and 50 MAU for users federated through SAML 2.0 based identity providers.

Pricing For Users Who Use User Pool Credentials or social Identity Providers To SignIn

This pricing data is for US-EAST (Ohio) AWS region.

Pricing For Users Who Use SAML or OIDC Federation To SignIn

For these types of users, pricing is set to $0.015 above the 50 MAU free tier.

Check the official AWS Pricing Documentation for more information or for current accurate numbers related to AWS Cognito Pricing.

Features

Password Security

When we are using Amazon Cognito, we as a developer don’t need to worry about doing the password management for our users, meaning we don’t need to Hash, Encrypt/Decrypt passwords, store them because all this is handled by Cognito itself. It stores your passwords following all the top standards for authentication.

Built-in UI

It offers users a built-in UI to quickly start with, we can use hosted web UI to quickly launch an authentication-based frontend application which will give us options to SignUp, SignIn, Password Change, and much more, read this AWS official documentation to know more.

Role-Based Access Control

This is one of the greatest features of AWS Cognito, it provides us to do role-based access control for users, it gives us an option to assign different roles with customized privileges, AWS IAM is used to control these roles, different roles can be given to users to access a particular AWS resource, it also provides an option to set default permissions for not authenticated users, read this AWS official documentation to know more.

Conclusion

All this information mentioned in this post can be outdated at the time someone is reading this, so please regarding these sensitive topics it is always a good idea to refer to the official documentation, this post is just meant to give a brief introduction to the different topics related to Amazon Cognito, finally, this service is an amazing way to delegate the process of authentication to AWS and focus on other parts of the application without having to worry about things like scalability and authentication security.

The post AWS Cognito Pricing and Features – 2021 appeared first on DevsWisdom.

AWS Step Functions Most Used Nodejs SDK APIs

Shivang Chauhan — Thu, 18 Nov 2021 09:24:05 +0000

Introduction

In this post, I will be listing some of the most important AWS Nodejs SDK APIs for AWS Step Functions, we will go through the syntax and brief explanation of each of these APIs.

You might want to check out this AWS Step Functions Cheatsheet to know about the most used AWS CLI Commands used with explanations.

Most Important APIs

APIs syntax will be given in a format of Lambda function so it can be directly used.

Create State Machine

exports.createStateMachine = async (event) => {
  try {
    const params = {
      definition: '<String>',
      name: '<String>',
      roleArn: '<String>'
    };
    const data = await stepfunctions.createStateMachine(params).promise();
    console.log(data);
  }
  catch (error) {
    console.log(error)
    // Do something with the error
  }
};

Explanation

createStateMachine API is used to create a state machine.

name – This is the name of the state machine.

definition – This is the stringified JSON which will contain all the configuration and state machine steps with their respective configuration.

role-arn – This is the ARN of the role which you need to create, role policies will depend on what actions you want to perform in your state machine.

Know more

Describe The Execution

exports.describeExecution = async (event) => {
  try {
    const params = {
      executionArn: '<String>'
    };
    const data = await stepfunctions.describeExecution(params).promise();
    console.log(data);
  }
  catch (error) {
    console.log(error)
    // Do something with the error
  }
};

Explanation

describeExecution API returns the execution details of any executed state machine, it returns important data like inputs passed while starting the execution, the current status of the execution, start date and end date of the execution, and other data as well.

Know more

Start The State Machine Execution

exports.startExecution = async (event) => {
  try {
    const params = {
      stateMachineArn: '<string>',
      input: '<string>',
      name: '<string>'

    };
    const data = await stepfunctions.startExecution(params).promise();
    console.log(data);
  }
  catch (error) {
    console.log(error)
    // Do something with the error
  }
};

Explanation

startExecution API is used to start the execution of any state machine which will run our AWS Step Functions, it has some of the important parameters

state-machine-arn - This is our ARN for the created state machine.

input - This is the initial stringified JSON that is passed to our state machine as input data.

Know more

Stop The State Machine Execution

exports.stopExecution = async (event) => {
  try {
    const params = {
      executionArn: '<string>',
      cause: '<string>'
    };
    const data = await stepfunctions.stopExecution(params).promise();
    console.log(data);
  }
  catch (error) {
    console.log(error)
    // Do something with the error
  }
};

Explanation

stopExecution API is used to stop any running state machine.

execution-arn – This is our ARN of the execution of the state machine, when we start any state machine this ARN gets returned to us.

Know more

Conclusion

These are some of the most important and most used AWS Step Functions Nodejs SDK APIs, I tried to keep it as simple as I could, to know more about these APIs please visit the official AWS documentation links mentioned in this post.

To learn about a practical application of AWS Step Functions using Nodejs and AWS Serverless Framework read this guide

Extract text with AWS Textract using AWS Step functions

Check out more posts like AWS Cognito Pricing and Features – 2021

The post AWS Step Functions Most Used Nodejs SDK APIs appeared first on DevsWisdom.

Edit npm package and persist the changes on npm install

Shivang Chauhan — Wed, 21 Apr 2021 14:28:45 +0000

Have you ever wondered how you can edit any npm package and have those changes persist if npm i gets run?

You can use patch-package to do this, all you need to do is follow these steps -

Make changes to the npm package.
After making changes to the npm run this command npx patch-package (package name).
This command will create a patches folder inside your project directory and that directory will hold all the changes which you did to the npm package.
Now whenever you do npm i it will install the packages, to get those changes which you did in the npm package just run this command npx patch-package to apply the changes.

Now you are done, all your changes will be applied.

Making this process automatic

Just add the command as postinstall like this in package.json file -

"postinstall": "npx patch-package"

Check out more:

What is AWS Artifact?

How to extract text from an image using AWS Step Functions and Serverless

Image Recognition using AWS Serverless

Understand Math functions in Javascript

DynamoDB VS MongoDB