DEV Community: GilHope

DynamoDB and Lambda Triggers

GilHope — Wed, 22 Nov 2023 18:04:34 +0000

Understanding DynamoDB Streams and Lambda Triggers

Introduction to DynamoDB Streams

DynamoDB Streams are a feature of AWS DynamoDB that provide a 24-hour rolling window of time-ordered changes to items in a DynamoDB table. These streams are essential for capturing and acting upon data modifications like inserts, updates, and deletes. They are configurable on a per-table basis and come with four distinct view types:

Keys Only: Captures only the partition key and, if applicable, the sort key value of the changed item.
New Image: Stores the complete state of the item as it appears after the change.
Old Image: Retains the item's state before the change, enabling a comparison with its current state.
New and Old Images: Provides a comprehensive view of the item's state both before and after the modification.

Integration with Lambda for Trigger Functionality

Lambda functions can be integrated with DynamoDB Streams to create a powerful, event-driven architecture. This setup allows for automated actions in response to data changes in DynamoDB tables, a concept known as database triggers. This approach is particularly effective in serverless architectures, where it can be utilized for various purposes, including reporting, analytics, aggregation, messaging, or notifications.

How the Trigger Architecture Works

Item Change: An item change occurs in a DynamoDB table with streams enabled.
Stream Record: This change generates a stream record that is added to the DynamoDB Stream.
Lambda Invocation: The Lambda function is then automatically invoked in response to the stream event. The function receives the data change as an event input, which can be based on any of the configured stream view types.

The Power of DynamoDB Streams and Lambda Triggers

The combination of DynamoDB Streams and AWS Lambda forms the backbone of a robust trigger architecture in DynamoDB. This setup exemplifies the event-driven paradigm, allowing for real-time, automated responses to data changes. It's particularly valuable for applications that require immediate action based on data modifications, such as real-time analytics, user notifications, or data synchronization across systems.

Exam Perspective

From an examination standpoint, it's crucial to understand the relationship between DynamoDB Streams and Lambda triggers. These components work together to implement a dynamic and responsive trigger architecture in DynamoDB, offering a versatile solution for various real-world scenarios.

Conclusion

DynamoDB Streams, combined with Lambda triggers, offer a powerful way to build responsive, event-driven architectures. This technology allows for efficient monitoring and handling of data changes, making it an invaluable tool in the AWS ecosystem. Whether for analytics, reporting, or real-time data processing, understanding and implementing this setup can significantly enhance the capabilities of your DynamoDB-based applications.

Introduction to Lambda

GilHope — Wed, 22 Nov 2023 16:38:09 +0000

Introduction to AWS Lambda

AWS Lambda is a powerful and versatile service offered by Amazon Web Services (AWS). In this comprehensive guide, we will explore various aspects of AWS Lambda, from its core concepts to advanced features.

Understanding AWS Lambda

Lambda as a FaaS Product

AWS Lambda falls under the category of Function-as-a-Service (FaaS) products. In simple terms, it allows you to provide specialized, short-running, and focused code to Lambda, and it takes care of running it while billing you only for the resources you consume.

Lambda Functions and Runtimes

A Lambda Function is a piece of code that Lambda runs. Every Lambda function is associated with a specific runtime. For example, you might use Python 3.8 as the runtime for your Lambda function. It's essential to define the runtime when creating a Lambda function.

Resource Allocation

When you provide code to Lambda, it's loaded into and executed within a runtime environment. You need to define the amount of resources allocated to this environment, including memory and CPU. AWS Lambda charges you based on the duration that a function runs, making it a cost-effective choice for many use cases.

Common Uses of AWS Lambda

AWS Lambda is a versatile service that can be applied in various scenarios:

Serverless Applications: Lambda forms a core part of serverless applications in AWS, enabling you to build scalable and cost-effective solutions.
File Processing: Lambda can be used for processing files, making it suitable for tasks like data transformation or image processing.
Database Triggers: You can use Lambda functions as triggers for your databases, responding to changes in data.
Serverless CRON Jobs: Lambda can execute scheduled tasks, providing serverless alternatives to traditional CRON jobs.
Real-time Stream Data Processing: AWS Lambda is ideal for processing real-time streaming data from sources like Amazon Kinesis.

AWS Lambda Part 2: Networking

Lambda offers two networking modes: public and VPC (Virtual Private Cloud) Networking. Public networking allows Lambda to access public AWS services and internet-based services, such as IMDb. It offers optimal performance but lacks access to VPC-based services without specific configurations.

Private Lambda

Lambda VPC Networking enables access to resources within your VPC but requires networking configurations for external access. Lambda functions running in a VPC adhere to VPC networking rules and cannot access public space services or the public internet directly. VPC endpoints and Nat Gateways can be used to provide necessary connectivity.

Security

Lambda function security is managed through execution roles, which are IAM (Identity and Access Management) roles assumed by Lambda functions. Execution roles grant permissions to interact with other AWS services. Additionally, resource policies can be used to allow external accounts or services to invoke Lambda functions securely.

Logging and Monitoring

AWS Lambda leverages CloudWatch, CloudWatch Logs, and X-Ray for logging and monitoring:

CloudWatch Logs: Logs generated during Lambda executions, including messages, errors, and execution duration, are stored here.
CloudWatch Metrics: Metrics like invocations, success/failure counts, retries, and latency-related data are available in CloudWatch.
X-Ray: Lambda can use X-Ray to add distributed tracing capabilities for improved monitoring.

Access to CloudWatch Logs requires appropriate permissions via the execution role.

AWS Lambda Part 3: Invocation

AWS Lambda functions can be invoked in three ways:

Synchronous Invocation: Triggered directly through CLI commands or APIs, with results returned during the request. This method requires client-side handling of errors and retries.
Asynchronous Invocation: Typically used when AWS services invoke Lambda functions on your behalf. Events are sent to Lambda for processing, and retries are managed by AWS services.
Event Source Mappings: Applied to streams or queues that don't generate events (e.g., Kinesis, DynamoDB, SQS). Lambda polls these sources for new data and processes batches of events.

Versions and Aliases

Lambda functions have versions that encompass both code and configuration. Versions are immutable and have their own Amazon Resource Names (ARNs). You can create aliases that point to specific function versions, providing flexibility and control over which version is invoked.

In-Depth Insights

Execution Context: Lambda functions run within an execution context. A "cold start" occurs during the initial creation and configuration of this context, which may take time. Subsequent invocations within a short time benefit from a "warm start," significantly reducing launch time.
Provisioned Concurrency: To address the cold start issue when multiple invocations are needed simultaneously, AWS offers Provisioned Concurrency. It allows you to keep execution contexts "warm" for faster starts.

Conclusion

AWS Lambda is a fundamental component of serverless computing in AWS. Understanding its core concepts, networking modes, security, invocation methods, and advanced features is essential for building efficient and cost-effective applications.

Stay tuned for more in-depth articles on specific Lambda topics! AWS Lambda offers a world of possibilities for developers and organizations looking to leverage the power of serverless computing.

The Orchestration of The Ring

GilHope — Thu, 29 Jun 2023 15:07:08 +0000

In the land of Mordor,

In the fires of Mount Volume...

The Dark Lord Sau..-Cron.. forged in secret a...

uh...

A container orchestration platform!!!

Is he really going to do this again?

Yes.

In this blog, i'm going to go over a little bit about the basics of Kubernetes. If you haven't seen my previous Lord of the Rings flavored post on Docker then you can also check that out HERE

What is Kubernetes?

Kubernetes is an open-source container orchestration platform and is currently the leading container orchestration tool so this is an important one to know. In essence, Kubernetes is designed to efficiently automate the deployment, scaling, and operation of containerized applications. It is also a vendor-neutral tool so it can run on all cloud providers.

Just as the One Ring has the ability to wield power and influence over other ring bearers for Sauron, Kubernetes has the power to wield and orchestrate container systems, such as Docker, for you!

Kubernetes, also known as K8s because of the number of letters between ‘K’ and ‘s’, is derived from the Greek word κυβερνήτης (kubernḗtēs), which means “pilot”. You can see this clearly illustrated in their logo design; and that is the essence of what Kubernetes does.

What are K8s key features?

Service Discovery: Kubernetes offers a built-in service discovery mechanism by which different services within a cluster can discover each other dynamically with the assignment of DNS names.
Storage Orchestration: With Kubernetes, users are provided robust storage orchestration capabilities which allow them to manage and mount regardless of whether it is local, network-attached, or cloud-specific storage.
Automated Rollouts and Rollbacks: With K8s, you can define the desired state of deployments while the ability to make seamless updates without downtime. And in the case of issues or failures, you can easily roll back to the previous desired state.
Automate Bin Packing: K8s can efficiently utilize resources by maximizing CPU utilization and optimizing resource allocation across nodes in your clusters.
Self-Healing: In order to maintain your desired state, K8s automatically can detect any failed containers that don't respond to health-checks and then either replace, kill, or restart them.
Secret and Configuration Management: K8s allows you to store secrets and configurations securely and while doing so also maintaining the ability to update them without the need to rebuild them your container images.

How is K8s architected from a bird’s eye view?

Kubernetes utilizes what is known as a ‘cluster architecture’. For K8s, this is a highly available ‘cluster’ of compute resources which can be organized to work as one single unit. Within these clusters run nodes and within the nodes are pods.

Control Plane: The cluster is captained by the cluster Control Plane, aka the Master Node, and this is responsible for maintaining the desired state of the cluster. It manages things such as the scheduling, scaling, deploying, and the applications within the cluster.

Worker Nodes: The Control Plane’s crew are the Worker Nodes, aka the Data Plane. Worker nodes are the virtual machines or physical servers that run your applications and workflows. Each worker nodes contains the the services necessary to manage and run what are known as ‘Pods’.

Pods: Pods are the smallest and simplest units within the K8s architecture. A pod encapsulates an application container(s), storage resources, a unique network IP, and the options that govern how the container should run.

Deployments: Deployments in K8s allows users to define the desired state of their applications, including the number replicas, container images, and also features, such as rolling updates, rollbacks, scaling, and self-healing. Deployments are apart of the Control Plane and specifically are managed by the Deployment Controller which monitors and manages the deployments to ensure the desired and current state actually match.

The K8s architecture in more detail.

Control Plane

The control plane is made up of a few key components, including: the API Server, Etcd, Scheduler, and Controllers.

API Server (kube-apiserver): This is the front-end for the control plane and is what the user, management devices, and command-line interfaces all communicate with.

Etcd: This provides a highly available key value store within the cluster for storing the cluster state and configuration.

Scheduler (kube-scheduler): The scheduler identifies any newly created pods without an assigned node and then selects a node for them to run on based on on resource requirements, deadlines, affinity/anti-affinity, data locality, and any constraints.

Controller Manager (kube-controller-manager): This manages the cluster controller processes which include the Node Controller, Replication Controller, Endpoints Controller, and Service Account & Token Controllers.

Node controller: Monitors and responds to node outages. If a nodes fails to respond then it is marked ‘unhealthy’ and no longer schedules it work.
Job/Cronjob controller: The job controller tracks Job objects (one-off tasks/jobs) and the cronjob controller manages jobs set to run at specific times or intervals.
Endpoint controller: Populates the endpoint object used for network communications between services within the cluster.
Service Account and Token Controller: These create default accounts and API access tokens for new namespaces.
Replication Controller: Ensures the desired number of pod replicas are running at any given time.
Deployment Controller: Manages the lifecycle of deployments.

Cloud Controller Manager (cloud-controller-manager): This manages the cloud-specific control logic which enables the communication with the underlying cloud services through their APIs.

Worker Nodes

Within the Worker Nodes are components, such as the Kubelet, Kube Proxy, and Container Runtime.

The Kubelet: Is an agent that runs on each worker node in the cluster and ensures that the containers are running within a pod. These utilize ‘PodSpecs’, a YAML or JSOn object, which describe a pod and makes sure that the described containers in those PodSpecs are running healthily.

Kube Proxy (kube-proxy): Is a network proxy that runs on each worker node and maintains network rules which allow communication to pods from inside or outside the cluster.

Container Runtime: This is the software that is responsible for running the containers. K8s supports several runtimes, such as Docker, Containerd, CRI-), and any implementation of the Kubernetes CRI.

Let's get started using K8s

1) Install prerequisites

We will be running through a quick tutorial for how we can work with K8s locally. Before we can get started doing anything you will need to have a few things installed.

Be sure to have the latest versions of minikube, kubectl, and docker.

2) Creating nodes with Minikube

Let's first try creating nodes using Minikube. For this, try the following command:

minikube start --nodes=2

In this command, we tell minikube to start up and to create 2 nodes with the trailing flag.

Note that if this is your first time using minikube then this process will take about 5 minutes or so to complete.

Now, lets check the status of our nodes out by running:

minikube status

You can see the control plane up top and about the worker at the bottom.

If you were to run a docker ps then you should see them running as 2 containers there as well.

We can run kubectl get nodes to view our nodes.

Try running kubectl get pods -A and we can view all pods in all namespaces.

If you notice, these pods above are what make up the control plane.

3) Deployment

To create a K8s deployment, the command syntax follows: `kubectl create deployment --image=.

We will use kubectl create deployment nginx-depl --image=nginx for this tutorial.

After you have run that then run kubectl get deployment and you should see something like the following indicating that we now have a ready deployment:

Then try running kubectl get pod and that should look similar as below:

If you would like to see if the desired state is matched with the actual current state then we can view the Replica Set with kubectl get replicaset.

Conclusion

I hope you all enjoyed this article. I had quite a bit of fun writing this article and I hope the illustration may help some of the information stick more effectively. Understanding of Kubernetes is a pretty important knowledge to have for anybody working in the Cloud and it would do you very well to get accustomed to working with it.

PS - I am actively looking for new work in the Cloud so please feel free to start a conversation if you are looking for a new addition to your team.

Check me out here:
Cloud Resume
GitHub
LinkedIn

The Containerization of The Ring

GilHope — Mon, 19 Jun 2023 13:39:23 +0000

The world is changing. I feel it in my local host. I feel it in my operating system. I smell it in the virtual machine.

It began with the forging of the Great Docker Images.

Three were given to the DevOps Engineers, immortal, wisest, and most difficult to explain to their friends and relatives.

Seven to the Developers, great coders and application craftsmen.

And nine, nine rings were gifted to the race of Operations, who, above all else, sought control and orchestration.

But they were, all of them, deceived, for a new image update was made.

In the land of DockerHub, in the fires of Mt Volume, the Dark Lord Sauron pushed in secret a new image, to create all new containers. And into its Dockerfile he set his base image, his run instructions, and his ENTRYPOINT commands to be executed when the containers were created. One Docker Image to rule them all!

One by one, the admins of Middle Earth pulled the new image
And there were none of them who said “it works on my machine”!

In this blog I am going to attempt to force as many LOTR references as possible. I can't promise that they will all be good. If you have any better ones then please comment below as I would love to hear them! I hope you all enjoy! PS I am actively looking for new work in the Cloud so please feel free to start a conversation if you are looking for a new addition to your team.

Check me out here:
Cloud Resume Challenge Site
GitHub
LinkedIn

Middle Earth is fantastical world full of many peoples, beasts, and fell creatures, and the local environments between them are not always the same. This is where Docker comes in. Docker allows your Fellowship to package your applications and its dependencies into an self-contained, isolated unit (called a Docker Container) that can run consistently across different environments, be it the Forrests of Fangorn, the rolling plains of Rohan, or the firey pits of Mordor.

But the power of Docker doesn't stop at portability and consistent environments. Docker is an amazing tool that encompasses features for version control, CI/CD integration, security, isolation, and an extensive ecosystem which I will touch on as we move along.

Virtual Machines vs Containers

Imagine if the one ring were actually a virtual machine and Mr Frodo was tasked to carry it all the way across Middle Earth to chuck it in another developer's local machine instead of Mount Doom. That thang would be heeeeavy! Weighed down by the immense crushing weight of the operating system, Mr Frodo would probably never make it out of the Shire!

With the help of Docker containers, we can make Mr Frodo's burden so much less arduous! Containers, unlike VMs, are lightweight and isolated units that package an application and its dependencies, without the need for an entire OS. Especially if using the powers of Docker Compose, Frodo and Sam could carry many containers to Mordor's local host with ease, but we'll get to that later.

Containers provide strong isolation between applications and their dependencies, ensuring you can "keep it secret and keep it safe" from the powers of any malicious Nazgûl that might seek to directly access or exploit vulnerabilities of their container's security as according to Gandalf's security instructions.

How does Docker work?

At it's core Docker's architecture is its client-server model which is constituted of the Docker Client, Docker Daemon, and Docker Engine. The rest of its key features include images, containers, and registries, but I will get more into these later.

The Docker Client is where your interaction as a user primarily takes place, be it through the CLI or Docker Desktop. When a command is executed, the Client translates this command into a REST API call for the Daemon.

The Docker Daemon, is not a Balrog. Full stop. Disappointing, I know, but however, it is comparably powerful. The Daemon, as the server component of this architecture, processes requests sent by the Client. It builds, runs, and manages Docker containers, coordinating their lifecycles based on the command received.

Sitting above the Client and Daemon, and encapsulating both, is the Docker Engine. The Docker Engine is an application installed on the host machine that provides the runtime capabilities to build and manage your Docker containers. It also is what allows the Daemon to interact with operating system's underlying resources, acting as the intermediary between your commands (via the Docker Client) and the execution of these commands (by the Docker Daemon).

Dockerfiles

Think of Dockerfiles as the DNA of Docker Images... Or maybe in this context, how the cruelty, hate, and malice form the essence of the One Ring. When we issue the docker build command, Docker reads these instructions, step by step, from the Dockerfile and builds your Docker Image. Each instruction represents a specific action, such as installing packages, copying files, or running commands.

The syntax of Dockerfiles is quite simple and straightforward. Conventionally, instructions are made in ALL CAPS so as to be distinguished from arguments. While specific Dockerfile commands will depend on your application's requirements, some essential and commonly used commands and some of which i'll be using in a quick tutorial later on in this blog:
FROM: This sets the base image on which your Docker Image will be built and is typically the first instruction in a Dockerfile.
WORKDIR: Sets the working directory inside the container where subsequent commands will be executed.
RUN: Executes commands inside the container. Often used for dependency install or application config.
COPY: Copies new files/folders from client machine to the container's file system. This is preferred for simple file copying.
ADD: Is similar to COPY except it allows features like URL downloads and extraction of compressed files.
CMD: Set the default executable of a container and arguments. Can be overridden at runtime by run parameters.
ENTRYPOINT: Similiar to CMD but it cannot be overridden.
EXPOSE: Informs Docker what ports the container application is running on. It should be noted that this does publish the ports but instead only documents it as metadata.

Images and Containers

Docker Images, or container images, are used to run containers. It is important to think of these as immutable templates which are read only. Docker Images are essentially snapshots of a Docker container's file system. When you create a Docker Image, they do not change. If you think you "changed" them then what you have actually done is created a completely new image entirely.

When you launch a container using an image, a writable layer (sometimes referred to as the "container layer") is added. This writable layer is attached to the container and allows for data writes to occur within the container. Instead of altering the image itself, any changes or data writes happen within this writable layer. This is what makes the container mutable - changes are isolated to the running container and do not affect the underlying image. It also enables containers to have their own local storage and system data.

DockerHub and Registries

DockerHub is a cloud-based registry provided by Docker which serves as a centralized storage, sharing, and management platform for Docker Images. It's kind of like GitHub, but for Docker. There are other container registries, such as Amazon's Elastic Container Registry (ECR), Google's Container Registry (GCR), Azure's Container Registry (ACR), and open-source projects like Harbor. Fundamentally, they all do the same thing; give or take some additional features and pricing structures.

Like GitHub, you split registries into repositories. Docker provides several commands to interact with these repositories. You can use the docker pull command to retrieve images from a repo and the docker push command to upload your own images. The naming convention used to specify a particular image within a repo goes like this <username>/<repository>:<tag>, for example, lsauron/containerofrings:latest.

How does Docker streamline the development pipeline?

In the realm of Middle-earth, Docker proves to be an invaluable tool in streamlining the development pipeline for Frodo and his companions. Here's how Docker facilitates a smooth and efficient development process throughout their quest:

Continuous Integration and Deployment (CI/CD): Docker integrates smoothly with popular CI/CD tools, enabling your Fellowship team to automate the build, test, and deployment processes. Leveraging Docker's containerization to package their applications and dependencies, your team can ensure consistency across the different stages of the pipeline. Automating these processes can save time, improve efficiency, and achieve fast iteration cycles.

Versioning and Rollbacks: Docker's versioning capabilities enable your Fellowship to keep track of your application's changes. That way, when you all get stuck by an avalanche along the Path of Caradhras, you can take a step back, realize that you may need to rollback to a previous state, and consider that perhaps taking the shortcut through Moria will be better (probably not).

Scalability and Orchestration: As the darkness of Sauron grows in power and complexity, so too might your applications. Docker allows you to scale your applications horizontally, adding more containers as your load increases. Docker works seamlessly with orchestration tools like Kubernetes and Docker Swarm, making it easier to manage multiple containers and services, which can be a significant advantage for large-scale projects.

How do you manage multiple containers?

So what if Sam and Frodo needed to bring many Containers of Power for the local hosts of Mordor? Well, that's where Docker Compose comes in! Docker Compose is used for creating, managing, and cleaning up multi container applications and particularly in development. It uses a special YAML file called 'compose.yml', or the legacy naming 'docker-compose.yml', to specify the configuration and dependencies of multiple services making up your unified application.

However, for larger production deployments spanning across multiple hosts or for cases where you need high availability, tools like Docker Swarm or Kubernetes are often more commonly used. These tools provide additional capabilities like load balancing, service discovery and scaling services across multiple Docker hosts.

DIY Docker Image

Prerequisites

Docker installed on your machine
DockerHub account
Python3 installed on your machine

If you would just like to try running a custom made image I created for this blog then all you have to do is open your terminal and run docker pull ghope94/repo-of-the-ring:latest. Here's what you should see returned:

What just happened is that Docker first checked the local host and realized the files were not present. By default, it in these events it will then try to pull the information from DockerHub.

Moving on, now try entering docker run ghope94/repo-of-the-ring:latest "Hello, Middle Earth" and you should see a cool ASCII art depiction of Gollum!

If you would like to know how I was able to create this Docker Image from scratch then keep reading (don't worry, it's easy)...

1. Create a repo in DockerHub

2. Open your favorite code editor and create a new folder.

3. For this tutorial, we are going to have to create three files. A text file for our custom ASCII art, our Dockerfile, and a Python script which will read our ASCII file, format it with a custom message, and then print to the console.

4. Source or create your own ASCII image and add it to your text file and save.

5. Copy this python code and replace everything with your own corresponding information.

Here's what is happening in this script:

The smeagolsay function reads the ASCII art from smeagol.txt and formats it with the input text to display.
In the main execution block, the script first imports the sys module to access command-line arguments.
If arguments were passed to the script, they are combined into a single string and passed to the smeagolsay function. If no arguments were provided, a default string is used instead.
The return value of smeagolsay, which includes the ASCII art and the input text, is printed to the console.

6. Create your first Dockerfile!

Here's what going on in this Dockerfile:

FROM python:3.8-slim-buster: This line is telling Docker to use a pre-made Python image from Docker Hub. The version of Python is 3.8 and it is based on a "slim-buster" image, which is a minimal Debian-based image. This is the base image for your Docker container.

WORKDIR /app: This line is setting the working directory in your Docker container to /app. This means that all subsequent actions (such as copying files or running commands) will be performed in this directory.

COPY ./gollum.py /app/gollum.py: This line is copying the local file gollum.py into the Docker container, placing it in the /app directory and keeping its name as gollum.py.

COPY ./smeagol.txt /app/smeagol.txt: This line is doing the same thing as the previous line, but for the smeagol.txt file.

ENTRYPOINT ["python", "/app/gollum.py"]: This line is specifying the command to be executed when the Docker container is run. In this case, it's running your Python script.

You've now finished this phase of the setup. We'll move over to the terminal next.

7. Open up your terminal and cd into to the appropriate directory where you are storing the files we just created.

8. In the commandline, run docker build -t <your-dockerhub-user>/<your-repo>:latest .

9. Then docker run <your-dockerhub-user>/<your-repo>:latest "Hello, Middle Earth"

10. If you would like you can push your new image to your DockerHub repo using docker push <your-dockerhub-user>/<your-repo>:latest.

And that's it! You now have created your very own personal Gollum! Good luck with that..

Conclusion

Docker has emerged as a powerful tool in the software development landscape, offering portability, consistency, and efficient resource utilization. It streamlines the development pipeline, enabling continuous integration, version control, and scalable deployments. With Docker, developers can easily manage multiple containers, leverage container orchestration platforms, and benefit from a thriving ecosystem. Embracing Docker allows us to navigate the complexities of software development while harnessing the magic of containerization to create robust and resilient applications.

I hope you you found this atleast as mildly amusing to read as I did to write it up. There were a lot of even more forced LOTR references - I decided to spare you all.

Conquering the Cloud: My Adventure with the Cloud Resume Challenge

GilHope — Wed, 07 Jun 2023 19:56:30 +0000

Late last night...

... At approximately 02:00 hours, while many slept, a colossal breakthrough was made ...

... After many long nights and countless eons ...

I finally completed my Cloud Resume Challenge Project !!!!!!!1!!!1!1!

You can find my Cloud Resume Project HERE!!
GitHub
LinkedIn

Also, I am actively looking for an opportunity to prove myself and develop my skills with my first job in the cloud! If you could please like and share I would immensely appreciate any and all help!

Contents

The Challenge
Certification
Front End
Back End
IaC / CICD / Version Control
Testing
Challenges and Future Improvements

The Challenge

The Cloud Resume Challenge is a 16-step challenge designed for beginners to get hands-on experience with some of the tools and applications that DevOps Engineers and Cloud Architects use in their day-to-day tasks.

The steps outlined in the CRC were intentionally high-level and incredibly vague, providing a fun and sometimes challenging learning opportunity.

Certification

The first steps of the CRC recommended preparing to take the AWS Cloud Practitioner Exam. However, fancying myself an overachiever, I decided to pursue the AWS Solutions Architect Associate Course (SAA-C03) instead.

So, I signed up for Adrian Cantrill's AWS Certified Solutions Architect - Associate Course and Jon Bonzo’s AWS Certified Solutions Architect Associate Practice Exams and I am glad I did.

In March of '23, I passed the Cloud Practitioner exam, and finally, the Solutions Architect Associate.

I was quite nervous leading up to my exam day for SAA-C03, but when it finally came time to sit down for it... I immediately realized how well Cantrill's and Bonzo's services had prepared me.

Front End

Alright, so getting into the real meat ‘n potatoes, this is what I did…

The challenge required writing up my resume in HTML and CSS… Which I had some light experience with, but I did a some speedruns through a few free courses, found an example of a resume I liked, and mixed it all up and got that working locally in VSCode…

Next, I needed to create a domain using Route53 and setup a Cloudfront Distribution with a S3 Bucket as the origin… I found these sections to be pretty simple and straightforward… Again, I believe Adrian Cantrill’s course exceptionally prepared me for this area with the practical knowledge covered between his lectures and tutorials.

Back End

For this section, I needed to create a visitor counter for my site using Lambda, DynamoDB, and API Gateway, and some Javascript…

Figuring out the Javascript was by far the most difficult part of this challenge for me… I especially struggled trying to figure out how to make sure I could fetch my API URL dynamically.

IaC / CICD / Version Control

When first approaching these chunks I stalled for some time to do as much reading up as I could to understand. I had heard of these terms in passing but they were always quite vague to me as to what exactly they were and how they worked.

Setting up Version Control and my CICD pipelines seemed like a pretty daunting order at first, but after skimming through the GitHub Documentations I found these to be some of the easier steps to complete.

Getting to the Infrastructure-as-Code, I initially debated between using AWS SAM or Terraform as I would be learning about either completely fresh anyways. Ultimately, I went with SAM for its simplicity of use with serverless.

As I started to figure it out I had a surprising amount of fun working with SAM… And it was without a doubt my favorite part of the challenge.

This article, One Artifact, One Pipeline, One Path to Production, was particularly beneficial for helping get me started in this section.

Testing

This was another especially challenging portion of the challenge for me and it took a significant amount of tie for me to get this hammered out, but boy did it feel good when I got those 'PASSED' outputs.

Challenges and Future Improvements

1.
Improve the API functionality of the visitor counter to be more efficient in distinguishing repeat visitors.
2.
I would probably like to add a little bit more testing for my Lambda code.
3.
I may consider reworking the IaC with Terraform. Initially, I had explored the idea of using Terraform but decided I’d be better off going forward with SAM as it seemed more intuitive for me… Also, Terraform just has a cooler sci-fi sounding name!

Ok, now...

Crab rave

Amazon Macie: A Comprehensive Data Security and Privacy Solution

GilHope — Sat, 01 Apr 2023 03:03:31 +0000

Wrote a quick blog post on Amazon Macie tonight. You can find it at Arcadian Cloud

DEV Community: GilHope

DynamoDB and Lambda Triggers

Understanding DynamoDB Streams and Lambda Triggers

Introduction to DynamoDB Streams

Integration with Lambda for Trigger Functionality

How the Trigger Architecture Works

The Power of DynamoDB Streams and Lambda Triggers

Exam Perspective

Conclusion

Introduction to Lambda

Introduction to AWS Lambda

Understanding AWS Lambda

Lambda as a FaaS Product

Lambda Functions and Runtimes

Resource Allocation

Common Uses of AWS Lambda

AWS Lambda Part 2: Networking

Private Lambda

Security

Logging and Monitoring

AWS Lambda Part 3: Invocation

Versions and Aliases

In-Depth Insights

Conclusion

The Orchestration of The Ring

Table of Contents:

What is Kubernetes?

What are K8s key features?

How is K8s architected from a bird’s eye view?

The K8s architecture in more detail.

Control Plane

Worker Nodes

Let's get started using K8s

1) Install prerequisites

2) Creating nodes with Minikube

3) Deployment

Conclusion

The Containerization of The Ring

Table of Contents:

Why should you use Docker?

Virtual Machines vs Containers

How does Docker work?

Dockerfiles

Images and Containers

DockerHub and Registries

How does Docker streamline the development pipeline?

How do you manage multiple containers?

DIY Docker Image

Conclusion

Conquering the Cloud: My Adventure with the Cloud Resume Challenge

The Challenge

Certification

Front End

Back End

IaC / CICD / Version Control

Testing

Challenges and Future Improvements

Amazon Macie: A Comprehensive Data Security and Privacy Solution