DEV Community

loading...
Cover image for CI/CD For CDN Invalidation Using AWS Lambda Function And Gitlab Pipeline

CI/CD For CDN Invalidation Using AWS Lambda Function And Gitlab Pipeline

Vu Dao
Awesome Devops || AWS SA || CloudOpz
・4 min read

- CloudFront can speed up the delivery of your static content (for example, images, style sheets, JavaScript, and so on) to viewers across the globe. By using CloudFront, you can take advantage of the AWS backbone network and CloudFront edge servers to give your viewers a fast, safe, and reliable experience when they visit your website.

- A simple approach for storing and delivering static content is to use an Amazon S3 bucket. Using S3 together with CloudFront has a number of advantages, including the option to use Origin Access Identity (OAI) to easily restrict access to your S3 content.

- When developers want to update the static files, they just need to push the commit of changes, everything else leave for Gitlab pipeline job

- General Flow: Gitlab piple job sync files to S3 -> S3 notification event triggers lambda function -> Lambda function create invalidation request to cloudfront



What’s In This Document


πŸš€ Create Gitlab pipeline job for sync file changes to S3

⚑ $ cat .gitlab-ci.yaml 
before_script:
  - echo "Deploy CDN"

deploy_cdn:
  stage: deploy
  timeout: 5m
  script:
    - aws s3 sync static/src s3://static-demo/src/
  only:
    refs:
      - master
    changes:
      - static/src/**/*
  tags:
    - gitlab-runner
Enter fullscreen mode Exit fullscreen mode

πŸš€ Create lambda Function associate with the S3 event using AWS Chalice


Alt-Text

1. Create aws chalice new-project cdn-invalidation

⚑ $ chalice new-project cdn-invalidation
⚑ $ tree
.
β”œβ”€β”€ app.py
β”œβ”€β”€ invalidation-cdn.json
β”œβ”€β”€ __pycache__
β”‚Β Β  └── app.cpython-38.pyc
β”œβ”€β”€ README.md
└── requirements.txt

1 directory, 6 files
Enter fullscreen mode Exit fullscreen mode

2. Define which region to create lambda function instead of the default in local aws configuration

⚑ $ export AWS_DEFAULT_REGION=us-east-1
Enter fullscreen mode Exit fullscreen mode

3. Create lamdba function handler which create invalidation of object files input

  • The handler listen to s3:ObjectCreated:Put event so any changes in s3://mybucket/static/src will trigger the lambda function with input of only changed object files

https://github.com/vumdao/cicd-invalidation-cdn/blob/master/cdn-invalidation/app.py

from chalice import Chalice
import boto3
import time


app_name = 'cdn-invalidation'
app = Chalice(app_name=app_name)
app.debug = True


class InvalidateCDN:
    """ Invalidate CDN """
    def __init__(self):
        self.distribution_id = 'A1AA1AA11A11AA'
        self.client = boto3.client('cloudfront')

    def create_invalidation(self, file_change):
        try:
            res = self.client.create_invalidation(
                DistributionId=self.distribution_id,
                InvalidationBatch={
                    'Paths': {
                        'Quantity': 1,
                        'Items': ["/{}".format(file_change)]
                    },
                    'CallerReference': str(time.time()).replace(".", "")
                }
            )
            invalidation_id = res['Invalidation']['Id']
            return invalidation_id
        except Exception as err:
            print(f"Failed to create invalidation, error {err}")
            exit(1)

    def get_invalidation_status(self, inval_id):
        try:
            res = self.client.get_invalidation(
                DistributionId=self.distribution_id,
                Id=inval_id
            )
            return res['Invalidation']['Status']
        except Exception as err:
            print(f"Failed to get invalidation status ID {inval_id}, error {err}")
            exit(1)

    def run(self, key):
        print(f"Deploying CDN file: {key}")
        the_id = self.create_invalidation(key)
        count = 0
        while True:
            status = self.get_invalidation_status(the_id)
            if status == 'Completed':
                print(f"Completed, id: {the_id}")
                break
            elif count < 10:
                count += 1
                time.sleep(30)
            else:
                print("Timeout, please check CDN")
                break


@app.on_s3_event(bucket='mybucket',
                 prefix='static/src/',
                 events=['s3:ObjectCreated:Put'])
def handle_s3_event(event):
    cdn = InvalidateCDN()
    cdn.run(event.key)
Enter fullscreen mode Exit fullscreen mode

4. Updaterequirements.txt to include boto3 in lambda fuction

⚑ $ cat requirements.txt
boto3
Enter fullscreen mode Exit fullscreen mode

5. Chalice Deploy to create the lambda function

⚑ $ chalice deploy
Enter fullscreen mode Exit fullscreen mode

6. Result

  • S3 event notifications

Alt-Text

  • Lambda Function with s3 event layer

Alt-Text

πŸš€ Conclusion

  • Create CI/CD of CDN Invalidation will boost-up the deployment and clear edged location cache of your static files, especially only the changed one
  • The combine of S3 notification event and lambda function will secure your flow better than executing in gitlab runner or aws cli commands
  • Thank you for reading this blog, hope you learned some new thing.

🌠 Blog · Github · stackoverflow · Linkedin · Group · Page · Twitter 🌠

Discussion (0)