DEV Community: OMKAR KADAM

AI-Hackathon ft AWS Community Builders

OMKAR KADAM — Sat, 01 Jul 2023 05:04:35 +0000

Something's brewing in the AWS Community Builders network! Join me as I take you on an exhilarating journey of the first ever AWS AI-Hackathon experience featuring AWS Community Builders, where creativity, passion, and collaboration converged to create innovative solutions. Buckle up and let's dive into the captivating story of our project - AWS Cost Advisor!

The Spark of Inspiration:
As an active member of the AWS Community Builder program, I was constantly surrounded by the spirit of innovation and the drive to push boundaries. It was during one of our vibrant discussions that the idea of AWS Cost Advisor came to life. Inspired by the immense potential of AI and the need for effective cost optimization on AWS, I embarked on a mission to create something truly impactful.

The Hackathon Begins:
The AI-Hackathon marked the start of an intense and thrilling journey. Fuelled by curiosity, I immersed myself in extensive research, exploring the realms of cost optimization strategies and AI algorithms. The more I delved into the subject, the clearer my vision became.

Collaboration and Challenges:
Throughout the hackathon, I had the privilege of collaborating with talented minds like @bismillahkani @md__mostafa and Manoj P. from such diverse backgrounds. Together, we tackled challenges head-on, brainstorming ideas and finding innovative solutions. The exchange of knowledge, ideas, and perspectives within the AWS Community Builder Slack Channel was truly invaluable.

Building AWS Cost Advisor:
Armed with a wealth of knowledge and the support of the AWS Community Builders, we set out to build AWS Cost Advisor — an intelligent solution that would revolutionize cost optimization on AWS. We dived into the AWS ecosystem, leveraging powerful services and machine learning capabilities to develop a user-friendly interface, real-time insights, and proactive recommendations tailored to individual business needs.

The Moments before final submission deadline:
As the hackathon drew to a close, I couldn't help but feel an overwhelming happiness for the kind of collaboration and camaraderie that we had throughout the duration of the hackathon. AWS Cost Advisor(Trademark pending, just kidding) evolved into a comprehensive tool that empowers businesses to make data-driven decisions and optimize their costs effectively. The immense value it brings to organizations is truly remarkable.

Sharing the Journey:
Now, it's time to share this incredible journey with the world! I invite you to read our official submission entry

(https://dev.to/aws-builders/aws-cost-advisor-cb-ai-hackathon-44ck)

You can also read my LinkedIn post which highlights this journey over here:

(https://www.linkedin.com/feed/update/urn:li:activity:7080764559908573184/)

Things to note:

Infinite Growth Potential:
This hackathon side-project is merely the tip of the iceberg. AWS Advisor has the potential to transform the landscape of Q&A and knowledge sharing. With a robust data pipeline, this tool can outperform human capabilities, providing accurate, personalized, and efficient solutions to complex queries. The possibilities are endless, and the impact on businesses and individuals alike is immeasurable.

Join the Revolution:
Now, it's time to join the revolution. Reading the above post will surely give you an idea about AWS Cost Advisor and the limitless potential it holds. Like, share, and engage with the article to amplify the message and inspire others to embark on their own transformative projects.

Conclusion:
As we reflect on this incredible AI-Hackathon journey, we realize that this is just the beginning.Together, we can unlock its full potential and create a future where AI-driven tools empower us beyond what we ever imagined.

I extend my gratitude to the AWS Community Builders and all those who have supported this endeavor, and built amazing products that solve real-life problems. Let's continue pushing boundaries, fueling innovation, and embracing the infinite growth potential that lies ahead.

https://dev.to/aws-builders/aws-cost-advisor-cb-ai-hackathon-44ck

AIHackathon #AWSCommunityBuilders #Innovation #AWS #TechCommunity

Infraception (Infrastructure + Inception) with CDK Pipelines

OMKAR KADAM — Tue, 04 Apr 2023 06:24:22 +0000

Are you ready to be amazed with some magic?

Well, I am not a magician, per se, but as Arthur Clarke says,

“Any sufficiently advanced technology is indistinguishable from magic”

I totally agree.

One of my favorite movies, The Prestige states that

“Every great magic trick consists of three parts or acts”

let us try incorporating these parts into our story.

The Pledge: (This is the part where I walk through the problem statement: Something ordinary)

We previously had around 50 static websites hosted on EC2+ELB, this was a bit cost intensive, and this architecture could be improved upon with some effort.

That is when I challenged myself to re-architect this setup so that it is secure, reliable, supports automated deployments and saves us some costs as compared to the existing Architecture.

The Turn: (This is the part where I briefly describe my approach and convert the ordinary problem statement into an extraordinary solution)

In this stage we identified the tasks to be worked on for moving from current architecture to the desired state.

We have chosen S3 Static Website connected to CloudFront Distribution for each website/partner added to the Monorepo for this project.

This is achieved using AWS CDK (Cloud Development Kit) to create CloudFormation Stacks such that every new website gets unique resources such as:

S3 Bucket

CloudFront Distribution

ACM Certificate

For automating deployments, i.e., Bucket level deployments to these sites based on changes, we have set up a CodeBuild Project which runs the steps mentioned in buildspec.yml file of the Mono repo

When any push activity/SHA is returned via Webhook to CodeBuild, it triggers the above Build Project, and deploys the changesets to CloudFormation via CDK deploy

301 Redirection support is implemented via a Lambda@Edge Function shared across stacks, to avoid maintenance hassles and multiple copies of same Lambda code.

The Prestige: (If you are still reading, your hypothetical EC2 servers and Load balancers have disappeared, and now you have corresponding websites powered by S3 and CloudFront, secured using ACM SSL Certificates.)

Highlights of this solution:

Infrastructure as Code/Infrastructure is Code: As the entire project along with the code necessary to spin up necessary infrastructure resides in the same repository
Adding a new site to the repository automatically creates corresponding resources on the Cloud as the CDK Pipeline detects any changes to the repository
Stack is CI-CD ready; we can add CI steps easily into the CDK pipeline. Currently it only checks/validates CDK syntax.
*Cost Savings compared to EC2 and ELB based hosting.

In less than 500 lines of code, I have provisioned more than 200 distinct cloud resources in two AWS accounts which include but not limited to (1 S3 Bucket, 1 Lambda Edge, 1 CloudFront Distribution, 1 SSL Certificate, 1 S3 Logging Bucket) * 50 Stacks * 2 Environments (Stage & Prod)

As seen above, CDK when combined with CDK Pipelines, can be a beast as it can repeatably deploy/provision thousands of similar stacks with customized Stack Parameters, across multiple environments, and multiple AWS Accounts.

This phenomenon is what I call Infraception (Infrastructure + Inception)

**Pricing comparison:*

EC2+ELB estimated costs (Traditional Stack):

~ 200$/mo.

S3+CloudFront (Serverless Stack):

~ 25-30$/mo.

Over 85% cost reduction if moved from EC2+ELB to the proposed solution.

Hope that this piece has shed some light on the optimizations you can use to transform your cloud landscape, by using CDK Pipelines as a CI-CD tool for deploying cloud resources at scale.

Further Reading:
Amazon S3 + Amazon CloudFront: A Match Made in the Cloud | Amazon Web Services
Getting started with a secure static website
Parallel and dynamic SaaS deployments with AWS CDK Pipelines | Amazon Web Services

This article was originally published on My Medium