DEV Community: Alejandro Lazaro

Weekend Build: Real-Time Raffle System with AWS AppSync for Community Events

Alejandro Lazaro — Mon, 02 Mar 2026 07:58:42 +0000

This is a submission for the DEV Weekend Challenge: Community

The Community

I'm an AWS User Group Leader and an AWS Community Builder in Zaragoza, Spain. I also co-organize the AWS Community Day España, our annual flagship event with 100-200 attendees. Between the Community Day and our regular user group meetups (6–10 events per year, 30–50 attendees each), every single event means juggling the same mess: one tool for RSVPs (Meetup, Eventbrite), spreadsheets and docs to track speakers/sponsors/venues, WhatsApp threads for last-minute changes, and absolutely nothing to keep people engaged during the event itself.

That's the problem I wanted to solve: tool sprawl. One platform that covers before, during, and after an event, for both organizers and attendees. Maybe not everything, but a few things done in a cool way.

In the short term, this app is for my own user group. In the medium term, I'd love other user groups to use it too.

What I Built

Community Organizer Hub is a multi-tenant community event platform with a backoffice for organizers and public event pages for attendees.

Organizer backoffice (login required)

Full CRUD for Events, Speakers, Sponsors, and Venues (these are the 4 most important domains for an event)
Event-scoped permissions: organizers only see and manage the events they're assigned to
Real-time raffle admin:
- Live participant list with total count
- Animated "roulette" wheel when drawing a winner
- Optional prize name per draw
- Draw history with timestamps
- Exclude previous winners from future draws
- Full raffle reset when needed
- "Draw winner" pushes results instantly to all attendee devices
A first simple version of Q&A and Polls (MVP-grade and intentionally simple for now)

Attendee public pages (no login required)

Public event landing page
Real-time raffle join:
- Join and receive a sequential number
- Your number persists on page refresh (device-based)
- Winner notification is pushed instantly to all open attendee devices
Q&A: view and upvote questions in real time
Polls: participate in live single-question polls during the event

The UI supports English and Spanish from day one (i18n), and is ready to expand to more languages.

Demo

🚀 Live app: https://d342vpi9y53cq7.cloudfront.net/admin/live

📹 Demo video: Live Demo

In the video I follow a simple on-screen script:

Public event landing page
Organizer flow: open backoffice and view assigned events, speakers, sponsors, venues, settings
Access to Live Event, the main area, and prepare the Raffle
Attendee flow: the attendees (6) join the user group page and access the raffle to receive a number (using different windows)
Real-time raffle in action: organizer draws winner, 6 attendees receive instant notifications
Stop raffle and disconnect all the participants

Code

The repository is intentionally private during this phase. This is a real product I'm building for a real community, and keeping it private while I stabilize the architecture and the solution is the responsible approach. I'm not hiding the work. I'm protecting the people who will use it.

To keep this submission transparent, I'm sharing:

A short code walkthrough video showing the monorepo structure and key modules: Code Walkthrough
The AWS architecture diagram below

How I Built It

This was a weekend build. Around an hour scoping a realistic MVP, then two intense days implementing it end to end. Yes, very intense...

Built with Kiro AI

I built this entire project using Kiro, an AI-powered IDE that helped me move from idea to deployed infrastructure in a weekend. Kiro assisted with code generation, AWS CDK infrastructure setup, and rapid iteration across the full stack.

Tech stack

Frontend: React 18 + Vite + TypeScript + Tailwind CSS
Routing: React Router
i18n: react-i18next (English default + Spanish, from the beginning... bad experiences in the past)
Hosting: S3 + CloudFront (I always do it this way for my side projects)
Auth: Amazon Cognito (embedded auth UI, no hosted UI)
Backend: AWS Lambda + API Gateway (REST) - traditional serverless approach
Real-time: AWS AppSync (GraphQL subscriptions) for raffle, Q&A, and polls
Database: DynamoDB
IaC: AWS CDK (TypeScript)

Technical highlights

💰 100% serverless, cost-optimized architecture: DynamoDB on-demand pricing, Lambda pay-per-invocation, AppSync pay-per-request, and S3+CloudFront for static hosting means the platform costs nearly nothing at rest and scales automatically under load. Designed specifically for community budgets.

🎯 Real-time event engagement: AppSync GraphQL subscriptions push updates instantly to all connected attendee devices: raffle results, Q&A updates, poll updates. Bidirectional communication with zero polling overhead.

Multi-tenant isolation by design: every item includes groupId in its partition key, ensuring logical data isolation within shared DynamoDB tables. Authorization checks at the application layer prevent cross-tenant access. Other user groups can use this platform without any code changes. They just get their own isolated space.

Event-scoped authorization: organizers cannot access other events in the same group unless they're explicitly assigned to them.

🎲 The raffle is more thoughtful than it looks: attendees get a sequential number (not random) that persists across page refreshes via a device fingerprint. The organizer can name each prize, exclude previous winners from future draws, and keep a full draw history (persisted in DynamoDB for audit trail). All within the same live session. The winner is pushed to every open attendee device the moment it's drawn.

Basic anti-abuse controls: one raffle entry per device, plus an optional join code per event.

What I'd improve next

Overall UI/UX polish, especially on attendee pages
Raffle animation polish and smoother state transitions
Q&A and polls: moderation, richer poll types, better admin controls
Asset upload per event (logos, images, PDFs) via S3 + presigned URLs
Test and improve the multi-tenant isolation
Operational hardening: stronger rate limiting, monitoring dashboards, and more tests
Higher personalization of the group public page

AI-assisted coding: what worked for me and what didn’t (after 6 months)

Alejandro Lazaro — Wed, 21 Jan 2026 07:00:00 +0000

1. Introduction

Over the last six months, AI-assisted coding has become a core part of how I build software.

I’ve been using it almost daily across my AWS projects. One of them is my personal project https://playingpadel.es{:target="_blank"}, a web app for padel players that keeps evolving.

During this time I’ve tried different tools (Amazon Kiro, Copilot, Antigravity, Cursor, Windsurf). They all helped me iterate faster. And with all of them, I eventually made mistakes for the same underlying reasons.

That’s when I realized something important: the tool mattered much less than how I was using it. The same mistakes appeared across all of them whenever I skipped validation or gave them too much freedom.

For something simple, or even for a proof of concept (PoC), you can use these assistants in many ways and almost everything seems to work. But once you apply them to production code, the story changes.

You need more discipline and more control: tighter scope, clear constraints, tests, and verification. Otherwise, speed becomes misleading: you move fast today, but you pay for it later with technical debt, regressions, and production surprises.

This article is not about hype or tools. It’s a practical summary of what didn’t work for me, what did, and the workflow that has kept me safe while building and evolving real systems.

2. My default working loop

Before getting into what didn’t work, this is the loop I try to follow every day when I’m coding with AI assistance:

Plan a small change
Implement
Test
Review the changes
Commit
Repeat

3. Mapping: from my mistakes to my fixes

This is a quick map to help you navigate. The value is in the details: examples, nuances, and how the pieces connect.

What didn’t work (section 4)	What actually helped (section 5)
Asking for too many things at once	Keep changes small and scoped (5.1)
Blindly trusting the output	Review the plan and the diff first (5.2)
Not asking for validation	Ask for verification, not just code (5.5)
Losing control of the code	Refactor after correctness (5.7)
Not setting clear constraints	Define intent and limits clearly (5.3)
Taking shortcuts without thinking long-term	Checklist and continuous validation (5.8)
Not committing frequently	Commit small and often (5.6)
Too many tests without strategy	Treat tests as the contract (5.4)
Using weak models for complex tasks	Use the right model for the job (5.9)
Constantly switching tools	Master one tool and configure it well (5.9)

4. What didn’t work for me

This section is intentionally long.

Not because things went particularly wrong, but because most of the problems I ran into kept repeating themselves in different forms.

The important part is that many of them share the same root cause: once you lose control of the loop (validation, review, and feedback), everything becomes fragile very quickly.

4.1. Asking for too many things at once

The most common mistake I made was asking the AI to implement too many changes in a single request.

Even when the result looks “correct”, the changes become hard to review and validation slows down. That’s where bugs start to sneak in.

What I learned

Fast generation only helps if the validation loop is also fast.
If I can’t validate something easily, I shouldn’t generate it in bulk.
Small changes reduce stress and errors.

4.2. Blindly trusting generated code

Sometimes the code looks clean and “well done”. That doesn’t mean it’s correct, safe, or consistent with your system.

The problem is that many issues aren’t obvious. They don’t break the build, they don’t fail locally, and yet they introduce behavior changes: missing validations, runaway retries, wrong timeouts, or unhandled edge cases.

In cloud environments this becomes even more visible, because those changes can translate into latency, unexpected costs, or degraded behavior under real load.

What I learned

Never assume that “it compiles” means “it’s fine”.
Review generated code like you would review a teammate’s PR.
If I can’t explain a piece of code, I don’t want it in production.

4.3. Not asking the AI to validate its own work

Many assistants make changes and stop. They won’t validate anything unless you explicitly ask.

If you assume the output works, you’ll often end up testing it yourself, watching it fail, and wasting time rebuilding context.

What I learned

Always ask for implementation plus validation.
“Implement + validate with tests + explain why it works” is a solid default.
Verification is not an extra phase, it’s part of the work.

4.4. Losing control of the code

This happens faster than it seems. You keep asking for changes and suddenly you’re no longer sure what you have or how it works.

The AI can generate working code, but you are still the owner and responsible for maintaining it.

What I learned

If something isn’t clear, ask for explanations before moving on.
Prefer simpler code, even if it’s slightly longer.
Don’t move forward without understanding the current state.

4.5. Not setting clear constraints

Without constraints, the assistant will make decisions for you: changing structure, adding dependencies, or “improving” things you didn’t ask for.

The problem is not that it does this, but that it does it without context.

What I learned

Constraints are not optional.
Being explicit about what I do NOT want prevents surprises.
Rules like “minimal diff”, “no new services”, or “don’t touch IAM” save a lot of trouble.

4.6. Choosing the fast path without thinking long-term

I accepted shortcuts because I wanted the solution quickly. Later, undoing them was slow and expensive.

What felt like a pragmatic decision turned into hard-to-pay technical debt.

What I learned

Think about trade-offs before accepting shortcuts.
Document temporary decisions with an exit plan.
Be especially careful when shortcuts affect security, data, or cost.

4.7. Not committing frequently

Accumulating working changes without committing is accumulating risk.

Without commits you lose traceability, rollback points, and confidence to keep iterating.

What I learned

Small commits reduce fear.
Checkpoints make experimentation cheaper.
Rolling back should always be easy.

4.8. Having too many tests without a clear strategy

It’s very easy to generate tests today. It’s also very easy to end up with a slow and redundant test suite.

In my padel project, I ended up with hundreds of E2E tests, many of them duplicated, and I didn't want to run them because they took too long.

What I learned

Tests need a strategy, not just volume.
Slow tests are deferred cost.
Not every change needs the same level of testing.

4.9. Using weaker models for complex tasks

I tried to “save tokens” by using smaller models for tasks that weren’t simple. The result was more time spent fixing and reworking.

What I learned

If this option is available in your assistant, let it choose which model to use. This is usually the best option, unless you know what you are doing.
If you choose, use the right model for the complexity of the task.
- Small models for boilerplate.
- Strong models for design, debugging, and risky refactors.

4.10. Switching tools too often

Jumping between IDEs, chats, and assistants made me lose context and consistency.

Each tool came with different assumptions and rules.

What I learned

Do not change tools in the middle of a task.
If I switch tools, explicitly copy the plan and constraints.
Close my working loop before moving on.

5. What worked for me

If the previous section lists the mistakes, this is the practical part: the practices that, together, helped me use coding assistants without losing control.

They’re not tricks or shortcuts. They’re repeatable habits that let me keep momentum without sacrificing quality.

Most of these practices are boring.

That’s exactly why they work.

5.1. Keep control: request a change, validate, continue

If I had to pick one change that made everything else easier, this would be it.

I used to ask for too much in a single request. Now I go one change at a time: request a small change, validate it, move on.

Instead of asking for “refactor the whole module”, I ask for one concrete improvement.

Small changes are easier to review, validation stays fast, and rollbacks stop being scary.

5.2. Review the plan before implementing

When a change is complex or high-impact, I review what will be done before any code is written.

My pattern is simple:

First, ask for the plan (options, risks, steps).
Once I’m happy, ask to implement only what was agreed.

That avoids surprises and keeps scope under control.

I treat the assistant like a teammate.

If I can’t review the changes quickly, the problem isn’t the assistant. It’s the size of the change.

Quick rule: if changes are big, split them. I prefer three small changes over one huge one.

5.3. Define what I want, in as much detail as needed

When the prompt is vague, the output is vague.

If you’re not clear, the assistant will fill the gaps for you.

I don’t follow a rigid template, but I usually include:

what I want
what I don’t want (clear limits)
how I’ll know it’s done (acceptance criteria)
relevant context for this change

It may feel excessive, but it reduces misunderstandings and improves output quality.

Amazon Kiro has a spec-driven development mode that forces you to define intent first, validate how it will be built, and only then generate code. It’s uncomfortable at first, but it reduces errors.

5.4. Have a clear testing strategy

Tests are not just there to “make the pipeline green”. They’re a control mechanism: confidence to change things and fast feedback when regressions appear.

What works for me is making three decisions explicitly:

Which test levels I have (and what each level is for).
When I add new tests (new feature, new bug, new edge case).
When I run them (what runs on every PR vs what I run at specific times).

My levels, ordered by criticality, are:

Level 1: Smoke tests
Level 2: Unit tests
Level 3: Integration tests
Level 4: Contract / API tests
Level 5: E2E tests

Be careful with E2E tests that trigger real side effects (emails, data writes, AWS events, etc.). If you add E2E tests, also add cleanup and isolation.

When a generated test fails, the assistant may “fix” it by weakening the test itself. That can keep the pipeline green while the bug is still there.
{: .prompt-warning }

A prompt line that helps me:

Do not change tests just to make them pass. Fix production code first.
If you change a test, explain why the original expectation was wrong.

5.5. Ask for verification, not just code

Many times I asked for changes and got code that simply didn’t work.

The good news is that this usually has a simple fix: explicitly ask for verification.

By default, I ask for:

confirmation that the code works
a short verification plan
tests to add or update

If something fails, my rule is simple: stop, review, and iterate again with smaller scope and more context.

For delicate changes, I add a quick risk pass (security, reliability, cost, maintainability). In a minute, obvious issues usually surface.

5.6. Commit frequently

Small commits give me confidence.

I commit after every working step, before risky changes, and whenever I finish a refactor or test improvement.

If the assistant goes in a weird direction, rolling back is trivial.

Without frequent commits, the speed you gain coding turns into fear of breaking things.

5.7. Refactor with clean code, after it works

The AI is great at rewriting code, but I want correctness first.

The workflow that works for me:

Get correct behavior
Automate checks
Refactor with clean code principles

When the output looks messy, I explicitly ask:

"apply clean code principles and justify any refactor you propose."

Clean Code principles aim for readable, maintainable, and efficient code: clear and descriptive names, small functions that do one thing (aligned with SOLID), DRY (Don’t Repeat Yourself), consistent formatting rules, proper error handling, and solid unit tests. The Boy Scout Rule (“leave the code cleaner than you found it”) matters a lot, together with SOLID principles for class design and good dependency management.

5.8. A lightweight end-of-loop checklist

Before merging (or even before stopping for the day), I go through this checklist:

Did I review the changes?
Does my code compile?
Do tests cover critical paths?
Do my tests work?
Does my code follow the lint rules I have defined?
Did I keep changes scoped?
Did I commit?
Did I verify assumptions?

It sounds basic, but it prevents most of tomorrow’s problems.

5.9. Master one tool and configure it well

Choosing one main tool and configuring it properly completely changed my experience.

Constantly switching tools made me lose context and, more importantly, my rules: style, limits, and how I ask for changes.

Once I left one tool well configured (persistent rules, templates, shortcuts), quality jumped and friction dropped: less re-explaining context, fewer surprises.

I’ll write a dedicated post about Amazon Kiro and its SPEC mode later, because that’s where this really shines.

If your IDE supports persistent rules (project rules, custom instructions, workspace guidelines), put your defaults there so every request starts from the same baseline.

5.10. Keep the right mental state when working with AI

When things don’t go as expected, it’s easy to get frustrated and respond impulsively. That almost always makes things worse.

I’ve learned that output quality depends heavily on my mental state.

When I’m tired, frustrated, or in a rush, it shows immediately in my prompts and results.

What I do now

Stop when I notice frustration.
Rewrite the prompt calmly with more context.
Treat the assistant as a predictable tool, not something that “understands” me.

It sounds minor, but it breaks many bad loops before they escalate.

6. Final thoughts

AI-assisted coding changes speed dramatically.

But speed without control only amplifies existing problems.

When you work with real systems, especially in the cloud, small mistakes can quickly turn into reliability, security, or cost issues.

Much of what I share here was refined as I iterated on my personal project https://playingpadel.es.

The most important lesson I’ve learned over these months is simple: the tool matters less than the method.

If I had to summarize the whole article in one idea, it would be this: keep the loop healthy. Small changes, constant validation, conscious decisions.

Developers still build systems. AI changes the speed.
Used well, it doesn’t replace good engineering. It amplifies it.

Quick Wins That Matter: Improve Your AWS Architecture Now

Alejandro Lazaro — Sun, 16 Feb 2025 18:13:51 +0000

I know, I know, this isn’t exactly a new topic. And yes, everyone is talking about Generative AI these days. But let’s be honest: the AWS Well-Architected Framework is still one of the most valuable resources for building in the cloud.

I’ll admit it. I hadn’t given it much thought before. But now? Its value is crystal clear. If you're building on AWS, doesn't it make sense to follow AWS's own best practices and recommendations to create scalable, secure, and cost-efficient systems?

Seems too obvious, right?

Also, the AWS Well-Architected Framework is extended and complex because it contains much information, but if you can apply a few things in your real projects, aren't you interested?

I first wrote a similar version of this article in my blog, playingaws.com, where I share hands-on AWS insights. But since quick wins are always useful, I thought it made sense to bring it here too!

1. Introduction

In this article, we’re focusing on quick wins: practical, actionable steps you can take right now to improve your AWS architecture by following the AWS Well-Architected Framework.

Each of the Six Pillars is packed with best practices, and while implementing the entire framework takes time, there are some small changes you can make today that will have an immediate impact.

This guide breaks down pillar-by-pillar quick wins that are easy to implement and can immediately improve your AWS workloads in terms of operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability.

2. Take action: Quick Wins You Can Apply Today

Once you understand the six pillars (more info here, you can jump straight into these actionable insights.

Quick wins are a great way to start with the AWS Well-Architected Framework because they:

offer immediate improvements without requiring a complete revision of your cloud environment.
enhance performance, security, and cost-efficiency quickly.
help teams build momentum before tackling more complex optimizations.

By implementing these small but impactful changes first, you can see real benefits fast and implement more complex strategies after that.

2.1. Operational Excellence

Main Recommendation: Automate everything you can to minimize human error and improve operational efficiency.

Quick-Wins:
- Enable AWS Config: Start tracking configuration changes for auditing.
- Set up CloudWatch alerts: Get notified about issues before they escalate.
- Use AWS Trusted Advisor: Identify operational improvements instantly.
- Document key processes: A simple shared document can prevent chaos.
Other important recommendations:
- Continuously improve operations through feedback and automation.
- Build a culture of accountability and iteration.

2.2. Security

Main Recommendation: Apply security from the beginning and at every layer.

Quick-Wins:
- Enable AWS Security Hub: Get a centralized view of security alerts.
- Enable AWS Security Hub best practice standards: Configure built-in best practice standards to assess compliance and detect potential security gaps.
- Enable GuardDuty: Detect threats and unauthorized activities in real-time.
- Enable MFA: Multi-Factor Authentication is a must-have.
- Turn on EBS default encryption: Protect your storage automatically.
Other important recommendations:
- Build layered security across identity, encryption, and networking.
- Regularly assess and audit your security posture with automated compliance checks.

2.3. Reliability

Main Recommendation: Build for failure. Assume services will fail and design for recovery.

Quick-Wins:
- Set up CloudWatch alarms: Detect failures before they cause downtime.
- Automate backups: Use AWS Backup to centralize backup management.
- Document recovery procedures: Ensure your team knows what to do when things go wrong.
- Test resilience: Simulate failures with AWS Fault Injection Simulator.
Other important recommendations:
- Ensure redundancy and fault tolerance at all levels of your architecture.
- Implement proactive monitoring and automated recovery for key services.

2.4. Performance Efficiency

Main Recommendation: Continuously review your resource usage to ensure you are not over-provisioned.

Quick-Wins:
- Use AWS Compute Optimizer: Get recommendations for right-sizing resources.
- Optimize EC2 instances: Adjust sizes for better performance and cost savings.
- Clean up unused resources: Identify and terminate unused EC2 instances, EBS volumes, and other resources.
Other important recommendations:
- Regularly optimize and scale resources based on demand.
- Leverage serverless or managed services to minimize infrastructure management and focus on performance.

2.5. Cost Optimization

Main Recommendation: Always optimize by rightsizing and using savings plans.

Quick-Wins:
- Review AWS Trusted Advisor: Identify cost-saving opportunities instantly.
- Set budget alerts: Use AWS Budgets to create cost thresholds and receive alerts if you are exceeding your budget.
- Use AWS Cost Explorer: Analyze your AWS costs and usage patterns for better financial efficiency.
- Delete unused resources: Stop paying for things you don’t use.
- Minimize data transfer costs by keeping workloads and resources within the same region.
Other important recommendations:
- Adopt a pay-for-what-you-use mindset.

2.6. Sustainability

Main Recommendation: Optimize infrastructure usage by embracing serverless and managed services.

Quick-Wins:
- Use AWS Managed Services: Benefit from AWS’s energy-efficient infrastructure.
- Turn Off Idle Resources: Automate shutdown of non-critical resources outside business hours.
- Optimize storage: Enable S3 Intelligent-Tiering to optimize storage costs based on access patterns.
- Track your AWS carbon footprint with the AWS Customer Carbon Footprint Tool.
Other important recommendations:
- Reduce environmental impact by optimizing resource usage.
- Automate infrastructure management to minimize waste.

2.7. Visualizing the Quick-Wins

3. Conclusion

These quick wins give you a fast, high-impact way to start** applying AWS Well-Architected best practices. Even small tweaks can *significantly improve* your cloud efficiency, security, and cost-effectiveness.

But remember, quick wins are just the beginning. To fully optimize your workloads and ensure you're following best practices across all pillars, a deeper review is necessary. This is where the AWS Well-Architected Tool comes into play.

Remember, the Well-Architected Framework isn’t a one-time exercise. It’s a continuous improvement journey. Start small, iterate often, and keep refining your AWS architecture.

4. Next steps

If you want to go beyond quick wins, I wrote a 4-article series diving deeper into the Well-Architected Framework on my blog:

Overview of the AWS Well-Architected Framework: Why It’s Essential for Every Cloud Professional
Deep Dive: Six Pillars
AWS Well-Architected Tool: How the AWS Well-Architected Tool Can Transform Your Cloud Architecture

Also, you can explore AWS’s comprehensive resources:

Amazon Q Developer: The AI-Powered Code Companion You’ve Been Waiting For

Alejandro Lazaro — Tue, 17 Dec 2024 09:30:45 +0000

I have published this article first on my blog here.

For months, I tried different generative AI tools in my IDE, but none felt AWS-oriented or fully integrated. I tested five other generative AI tools in my IDE, but none felt truly AWS-oriented or integrated, until now.

In this post, I’ll show you how Amazon Q can enhance your productivity, from chatting about code and inline suggestions to advanced code transformations and automated test generation. I’ll also tackle the big question: what happens to your code when using a generative AI service like this? And yes, I’ll show you actual examples using one of my own public GitHub repositories, appsync-website-mutation, so you can see how effortless it is to get started.

Introduction

This post focuses on Amazon Q Developer, but first, let me provide some context.

Amazon Q one year ago

Amazon Q was announced one year ago, at re:Invent 2023 (last week of November 2023), as an AI-powered assistant designed to help developers and IT professionals build, deploy, and manage applications and workloads on AWS.

While its potential was evident, my initial experience left me feeling that it was released prematurely. It felt incomplete and not yet suitable for daily workflows. It seemed like AWS had rushed to join the generative AI race, and the tool wasn’t quite ready to deliver on its promise.

Now, one year later, at re:Invent 2024 (first week of December 2024), AWS presented Amazon Q renewed and ready to be used. I’ve tested it, and now I can confidently say it’s now a valuable addition to any developer’s toolkit. This is the main reason for this post.

Amazon Q: Amazon Q Developer + Amazon Q Business

Built on Amazon Bedrock and leveraging 17 years of AWS expertise, Amazon Q encompasses both Amazon Q Developer for software development tasks and Amazon Q Business for enhancing enterprise productivity. It draws from AWS documentation, best practices, and real-world examples, making it a reliable and comprehensive AI assistant.

Amazon Q Developer

Specifically designed for software developers
Focuses on coding and development tasks
Key features include:
- Code chat and assistance
- Inline code completions
- New code generation
- Security vulnerability scanning
- Code upgrades and improvements
- Debugging and optimization suggestions
- AWS architecture and resource guidance
Available through IDEs and development environments

Amazon Q Business

Designed for enterprise use across organizations
Focuses on business operations and productivity
Key features include:
- Provides answers based on enterprise data
- Content summarization
- Document analysis
- Task automation
- Custom application creation (Q Apps)
- Integration with business tools (40+ integrations)
- Connectivity to data sources like SharePoint, S3, and Salesforce
Includes security features like:
- Role-based access control
- Data privacy controls
- Content filtering
- Integration with existing security permissions

The main differences

Purpose: Amazon Q Developer focuses on development tasks, while Amazon Q Business is designed for enterprise productivity.
Target Users: Developers and technical teams for Q Developer vs. business employees across various departments for Q Business.
Data Sources: Q Developer uses code and AWS documentation, while Q Business leverages enterprise data and documents.

Amazon Q Developer: Generative AI in Your IDE

On April 30, 2024, Amazon CodeWhisperer became part of Amazon Q Developer, bringing its generative AI-powered inline code suggestions, security vulnerability scanning, and remediation into the new platform. At re:Invent 2024, Amazon Q Developer introduced an expanded feature set, transforming it into much more than just a suggestion tool. It now helps you generate unit tests, review code for best practices, and even produce documentation on demand. This evolution makes Amazon Q Developer feel like a real AI partner, rather than just an autocomplete tool.

Where You Can Use Amazon Q Developer

Amazon Q Developer is available across several environments, making it easy to integrate into your workflow:

on AWS: more info
in your IDE: more info
on your command line: more info
in GitLab (in preview): more info

Generative AI at Your Fingertips

When integrated with your IDE, Amazon Q Developer acts as a powerful assistant, offering real-time development guidance. With its chat-based interface and slash commands, you can interact directly with the model to request suggestions or commands in plain English. Whether you need guidance on a new feature or help refactoring and improving your code, it's just a command away.

DISCLAIMER! Amazon Q Developer is not only about code. You can ask questions about AWS architecture, your AWS resources, best practices, documentation, support, and more. Here you have the full list of features of Amazon Q Developer.

Is it really free?

Amazon Q Developer offers two plans:

Amazon Q Developer Free Tier: Free (with limits)
Amazon Q Developer Pro Tier: $19/month per user

Important: In the Free Tier, when you reach 50 interactions in the IDE, you will receive the following notification:

More information: Pricing and plan comparative

What Happens With Your Code?

A common concern is whether Amazon Q Developer uses your private code for model training or sharing. Here’s what you need to know:

Free Tier: Unless you opt out, your content may be used for model enhancement. You can opt out by following the instructions in the documentation.
Pro Tier: Your content is not used for service improvement or to train foundation models.

AWS also clarifies in the documentation:

Amazon Q stores your questions, its responses, and additional context, such as console metadata and code in your IDE, to generate responses to your questions. Your code is also stored for features like code transformation and software development in the IDE. This data is stored for up to 90 days to provide the service, and then is permanently deleted.

How to Opt Out

If you're using the Free Tier and prefer not to share your content, you can opt out easily. Here’s how:

The important one is this "Amazon Q: Share Content". More information

Hands-on with Amazon Q Developer

Before getting started, if you haven't installed the plugin in your IDE yet, follow this guide.

Then, you have to access to the plugin of AWS / Amazon Q, and you can start writing or just type /:

1. Chatting About Code or Something Else (AWS-related)

You can use the chat with Amazon Q in the IDE to ask any AWS-related question. I will had some concerns about Amazon Q sharing my data, so I will ask to Amazon Q directly instead of retrieve it from the documentation (as I did for the previous section):

![q-question(https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lklfh7lg86k0goedrum5.jpg)

As you can see, you can simply open the AWS Amazon Q chat extension and type your question directly.

2. Generating Inline Suggestions

As you type, Q Developer suggests completions right in your editor. For instance, if you're writing Terraform code and need to add a new DynamoDB table, Amazon Q can suggest completions:

resource "aws_dynamodb_table" "feedback" {
  name         = var.dynamodb_table_name
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "userId"
  range_key    = "notificationId"

  a
}

Before you finish typing, Q Developer suggests the next part of the code, helping you code faster without breaking your flow.

3. Generate documentation (/doc)

/doc to task Amazon Q with writing API, technical design, and onboarding documentation.

In my project, I wanted to generate a README file based on my code. To do this, I typed:

/doc

To open Q-Doc window. And there you don't have to type anything else, just select the options:

Of course, you can customize the message and ask for customized instructions.

However, I have found a few limitations in this /doc feature:

If your workspace contains multiple projects,

selecting the correct one can be unintuitive

if you select a different project than the first one, the README might still be generated in the first project folder

If you have a workspace with more than one project:

the selection of the project can be improved (is not very user-friendly)

if you select one project different to the first one, the content is created in the first project in one folder. This is wrong but I am sure they will fix it.

This feature only will create README files and only in the root of the project. What about if I want to create the README of only a part of a project? You can do it with custom instructions, but the file will be created in the root folder.

The markdown generated doesn’t completely follow the linting styles (markdownlint).

4. Developing Features (/dev)

/dev to task Amazon Q with generating new code across your entire project and implement features.

You can use the /dev to refactor current code:

5. Generating Unit Tests (/test)

/test to ask Amazon Q to generate unit tests and add them to your project, helping you improve code quality, fast.

Tired of writing boilerplate unit tests? Just highlight a function or class and run:

/test create the unit test for main.ts file

Now, only Java and Python are supported, as you can view in the image. If you try to generate unit tests for other languages, Amazon Q will likely provide suggestions in the chat instead of generating actual test files. It will fall back to providing guidance rather than generating the actual test code.

6. Reviewing Code (/review)

/review to ask Amazon Q to perform code reviews, flagging suspicious code patterns and assessing deployment risk.

Need a second pair of eyes? Q Developer can review your code and highlight potential issues, security concerns, or performance bottlenecks. After highlighting a section and type:

/review

You get back actionable suggestions: maybe you’re missing error handling, or you’re not using pagination for large DynamoDB queries. It’s like having a senior engineer at your side 24/7.

7. Transforming Code (/transform)

/transform to upgrade your Java applications in minutes, not weeks.

My code is not a Java code so I can't show you this feature but you can try yourself.

8. Select code and right-click

You have another option. You can just select the code you want, right-click and then you can select one of the options:

Conclusions

Amazon Q Developer feels like the next generation of coding assistance, purpose-built for AWS developers. With chat-based guidance, inline suggestions, powerful transformations, automated test generation, code review capabilities, and easy documentation, it streamlines many of the mundane coding tasks that eat away at our productivity.

That being said, if you plan to rely on Amazon Q Developer extensively, the Free Tier may not be sufficient due to its interaction limits. If you find yourself needing more, the Pro Tier offers additional features and a higher usage limit.

If you’ve been thinking about using a generative AI coding assistant for your AWS projects, Amazon Q Developer might be the tool that makes everything click. After trying it out in some personal repositories and seeing the immediate gains in speed and quality, I can say this is one AWS tool you don’t want to overlook.

How to add CI/CD to my SAM project

Alejandro Lazaro — Fri, 26 Jan 2024 06:28:00 +0000

TLDR

In this article we will use sam pipeline to deploy one SAM application. However, we are not going to use the standard templates.

I had to create one custom template to be able to deploy the solution using only 1 stage. Using the default templates you have to use 2 (test/prod?), and I don't want to do it for my personal projects, I don't need it, so I just decide create what I need and then customize it and do it public. This is the GitHub code of my custom template that you can use if you want!

I don't know why AWS don't provide it. If you know it, please leave a comment with this information. I will appreciate it!

Introduction

We will add CI/CD to our SAM application through the pipeline integration of the AWS SAM CLI. And as we want to add automation to our deployment process and integrate it with the AWS ecosystem, we will use the AWS Developer tools (CodePipeline and CodeBuild). For the Git repository, we will use GitHub.

This is the SAM project code on GitHub that we will use in the article. In the commit history you can find the evolution of the application through the steps explained.

Add CI/CD to a SAM project

We will create the CI/CD pipeline to implement continuous deployment, so when we push new code, the pipeline deploys our resources automatically.

From AWS documentation: AWS SAM provides a set of default pipeline templates for multiple CI/CD systems that encapsulate AWS's deployment best practices. These default pipeline templates use standard JSON/YAML pipeline configuration formats, and the built-in best practices help perform multi-account and multi-region deployments and verify that pipelines cannot make unintended changes to infrastructure.

We want to create a new pipeline in the AWS CodePipeline resource using the SAM templates.

To generate the pipeline for AWS CodePipeline, we have to perform the following tasks in this order:

Create infrastructure resources
Generate the pipeline configuration
Commit the pipeline configuration to the Git repository
Deploy the pipeline
Connect the Git repository with the CI/CD system

After you've generated the pipeline and committed it to your Git repository, whenever someone commits a code change to that repository your pipeline will be triggered to deploy the new changes automatically.

Step 1: Create infrastructure resources (bootstrap)

The Pipelines that use AWS SAM require certain AWS resources, like an IAM user and roles with necessary permissions, an Amazon S3 bucket, and optionally an Amazon ECR repository. You must have a set of infrastructure resources for each deployment stage of the pipeline.

Then, for each stage we need (dev, test, prod...), we have to run sam pipeline bootstrap. This command will create a CloudFormation stack with the name aws-sam-cli-managed-${stage}-pipeline-resources which the necessary resources that SAM needs.

We will create a single stage with the name test:

> sam pipeline bootstrap

sam pipeline bootstrap generates the required AWS infrastructure resources to connect
to your CI/CD system. This step must be run for each deployment stage in your pipeline,
prior to running the sam pipeline init command.

We will ask for [1] stage definition, [2] account details, and
[3] references to existing resources in order to bootstrap these pipeline resources.

[1] Stage definition
Enter a configuration name for this stage. This will be referenced later when you use the sam pipeline init command:
Stage configuration name: > test

[2] Account details
The following AWS credential sources are available to use.
To know more about configuration AWS credentials, visit the link below:
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html
    1 - Environment variables
    2 - default (named profile)
    3 - localstack (named profile)
    q - Quit and configure AWS credentials
Select a credential source to associate with this stage: > 1
Associated account 00000000000 with configuration test.

Enter the region in which you want these resources to be created [eu-west-1]:
Select a user permissions provider:
    1 - IAM (default)
    2 - OpenID Connect (OIDC)
Choice (1, 2): 1
Enter the pipeline IAM user ARN if you have previously created one, or we will create one for you []:

[3] Reference application build resources
Enter the pipeline execution role ARN if you have previously created one, or we will create one for you []:
Enter the CloudFormation execution role ARN if you have previously created one, or we will create one for you []:
Please enter the artifact bucket ARN for your Lambda function. If you do not have a bucket, we will create one for you []:
Does your application contain any IMAGE type Lambda functions? [y/N]:

[4] Summary
Below is the summary of the answers:
    1 - Account: 00000000000
    2 - Stage configuration name: test
    3 - Region: eu-west-1
    4 - Pipeline user: [to be created]
    5 - Pipeline execution role: [to be created]
    6 - CloudFormation execution role: [to be created]
    7 - Artifacts bucket: [to be created]
    8 - ECR image repository: [skipped]
Press enter to confirm the values above, or select an item to edit the value:

This will create the following required resources for the 'test' configuration:
    - Pipeline IAM user
    - Pipeline execution role
    - CloudFormation execution role
    - Artifact bucket
Should we proceed with the creation? [y/N]: y
    Creating the required resources...
    Successfully created!
The following resources were created in your account:
    - Pipeline execution role
    - CloudFormation execution role
    - Artifact bucket
    - Pipeline IAM user
Pipeline IAM user credential:
  AWS_ACCESS_KEY_ID: xxxxxxxxxx
  AWS_SECRET_ACCESS_KEY: xxxxxxxxxx
View the definition in .aws-sam/pipeline/pipelineconfig.toml,
run sam pipeline bootstrap to generate another set of resources, or proceed to
sam pipeline init to create your pipeline configuration file.

Before running sam pipeline init, we recommend first setting up AWS credentials
in your CI/CD account. Read more about how to do so with your provider in
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-generating-example-ci-cd-others.html.

A new stack is created for our new stage test.

If you want more than 1 stage you should repeat the sam pipeline bootstrap for the new stage (prod?). In this example I only want 1 stage to simplify.

In our SAM project now we have 1 new file containing our stage information:

Step 2: Generate the pipeline configuration

To generate the pipeline configuration, run the command sam pipeline init:

> sam pipeline init

sam pipeline init generates a pipeline configuration file that your CI/CD system
can use to deploy serverless applications using AWS SAM.
We will guide you through the process to bootstrap resources for each stage,
then walk through the details necessary for creating the pipeline config file.

Please ensure you are in the root folder of your SAM application before you begin.

Select a pipeline template to get started:
    1 - AWS Quick Start Pipeline Templates
    2 - Custom Pipeline Template Location
Choice: > 1

Cloning from https://github.com/aws/aws-sam-cli-pipeline-init-templates.git (process may take a moment)
Select CI/CD system
    1 - Jenkins
    2 - GitLab CI/CD
    3 - GitHub Actions
    4 - Bitbucket Pipelines
    5 - AWS CodePipeline
Choice: 5
Which pipeline template would you like to use?
    1 - Two-stage pipeline
    2 - Two-stage pipeline with monorepo
Choice []: 1
You are using the 2-stage pipeline template.
 _________    _________
|         |  |         |
| Stage 1 |->| Stage 2 |
|_________|  |_________|

Checking for existing stages...

Only 1 stage(s) were detected, fewer than what the template requires: 2. If these are incorrect, delete .aws-sam/pipeline/pipelineconfig.toml and rerun

To set up stage(s), please quit the process using Ctrl+C and use one of the following commands:
sam pipeline init --bootstrap       To be guided through the stage and config file creation process.
sam pipeline bootstrap              To specify details for an individual stage.

To reference stage resources bootstrapped in a different account, press enter to proceed []:
2 stage(s) were detected, matching the template requirements. If these are incorrect, delete .aws-sam/pipeline/pipelineconfig.toml and rerun
What is the Git provider?
    1 - Bitbucket
    2 - CodeCommit
    3 - GitHub
    4 - GitHubEnterpriseServer
Choice []: > 3
What is the full repository id (Example: some-user/my-repo)?: > alazaroc/aws-sam-app
What is the Git branch used for production deployments? [main]:
What is the template file path? [template.yaml]:
We use the stage configuration name to automatically retrieve the bootstrapped resources created when you ran `sam pipeline bootstrap`.

Here are the stage configuration names detected in .aws-sam/pipeline/pipelineconfig.toml:
    1 - test
Select an index or enter the stage 1's configuration name (as provided during the bootstrapping): 1
What is the sam application stack name for stage 1? [sam-app]:
Stage 1 configured successfully, configuring stage 2.

Here are the stage configuration names detected in .aws-sam/pipeline/pipelineconfig.toml:
    1 - test
Select an index or enter the stage 2's configuration name (as provided during the bootstrapping):
Select an index or enter the stage 2's configuration name (as provided during the bootstrapping):

I have to stop the execution here.

In the console log below, the following is displayed:

You are using the 2-stage pipeline template
Only 1 stage(s) were detected, fewer than what the template requires: 2.

In any real project you should have at least two stages, so you could use the default AWS template.

However, I don't want to create two stages in my CI/CD pipeline, I am testing a simple SAM project, and I only want ONE.

Unfortunately, you can't do that with the AWS Quick Start Pipeline Templates so I forked the main AWS project and I created a custom template with only ONE stage.

This is my forked project: https://github.com/alazaroc/aws-sam-cli-pipeline-init-templates.git

I had to put my custom template in the root folder because otherwise, the AWS SAM CLI doesn't work.

In the next execution, I will select the option Custom Pipeline Template Location and use my updated forked repository to create only one stage in the AWS CodePipeline service.

You will be asked if you already have an S3 bucket and you want to reuse it. If you say yes, and you set it:
What is the S3 bucket name used for artifacts of SAM deployments? Not the ARN, the name. >[aws-sam-cli-managed-test-pipeline--artifactsbucket-gro48levpwla]: aws-sam-cli-managed-test-pipeline--artifactsbucket-gro48levpwla
You have to do one manual change in your code (to be able to deploy it in a later phase).

> sam pipeline init

sam pipeline init generates a pipeline configuration file that your CI/CD system
can use to deploy serverless applications using AWS SAM.
We will guide you through the process to bootstrap resources for each stage,
then walk through the details necessary for creating the pipeline config file.

Please ensure you are in the root folder of your SAM application before you begin.

Select a pipeline template to get started:
    1 - AWS Quick Start Pipeline Templates
    2 - Custom Pipeline Template Location
Choice: 2
Template Git location: https://github.com/alazaroc/aws-sam-cli-pipeline-init-templates.git

Cloning from https://github.com/alazaroc/aws-sam-cli-pipeline-init-templates.git (process may take a
moment)
You are using the 1-stage pipeline template.
 _________
|         |
| Stage 1 |
|_________|

Checking for existing stages...

1 stage(s) were detected, matching the template requirements. If these are incorrect, delete .aws-sam/pipeline/pipelineconfig.toml and rerun
What is the Git provider?
    1 - Bitbucket
    2 - CodeCommit
    3 - GitHub
    4 - GitHubEnterpriseServer
Choice []: 3
What is the full repository id (Example: some-user/my-repo)?: alazaroc/aws-sam-app
What is the Git branch used for production deployments? [main]:
What is the template file path? [template.yaml]:
We use the stage name to automatically retrieve the bootstrapped resources created when you ran `sam pipeline bootstrap`.

Here are the stage configuration names detected in .aws-sam/pipeline/pipelineconfig.toml:
    1 - test
What is the name of stage (as provided during the bootstrapping)?
Select an index or enter the stage name: 1
What is your sam application stack name? [sam-app]:
What is the S3 bucket name used for artifacts of SAM deployments? Not the ARN, the name. [aws-sam-cli-managed-test-pipeline--artifactsbucket-gro48levpwla]: aws-sam-cli-managed-test-pipeline--artifactsbucket-gro48levpwla
What is the prefix of the S3 bucket used for artifacts of SAM deployments? []:
Stage configured successfully (you only have one stage).

To deploy this template and connect to the main git branch, run this against the leading account:
`sam deploy -t codepipeline.yaml --stack-name <stack-name> --capabilities=CAPABILITY_IAM`.
SUMMARY
We will generate a pipeline config file based on the following information:
    What is the Git provider?: GitHub
    What is the full repository id (Example: some-user/my-repo)?: alazaroc/aws-sam-app
    What is the Git branch used for production deployments?: main
    What is the template file path?: template.yaml
    What is the name of stage (as provided during the bootstrapping)?
Select an index or enter the stage name: 1
    What is your sam application stack name?: sam-app
    What is the pipeline execution role ARN for this stage?: arn:aws:iam::00000000000:role/aws-sam-cli-managed-test-pipe-PipelineExecutionRole-Fv4wReTqFrHy
    What is the CloudFormation execution role ARN for this stage?: arn:aws:iam::00000000000ole/aws-sam-cli-managed-test--CloudFormationExecutionRo-4iXZtj3Xzch9
    What is the S3 bucket name used for artifacts of SAM deployments? Not the ARN, the name.: aws-sam-cli-managed-test-pipeline--artifactsbucket-gro48levpwla
    What is the prefix of the S3 bucket used for artifacts of SAM deployments?:
    What is the ECR repository URI for this stage?:
    What is the AWS region?: eu-west-1
Successfully created the pipeline configuration file(s):
    - codepipeline.yaml
    - assume-role.sh
    - pipeline/buildspec_unit_test.yml
    - pipeline/buildspec_build_package.yml
    - pipeline/buildspec_integration_test.yml
    - pipeline/buildspec_feature.yml
    - pipeline/buildspec_deploy.yml

Now we have the new files in our project that CodePipeline will use to deploy our code:

Step 3: Commit the pipeline configuration to Git

This step ensures that your CI/CD system recognizes your pipeline configuration and triggers deployments upon code commits.

Step 4: Deploy the pipeline

We have selected before that we will use AWS CodePipeline like our CI/CD system.

Now, we have to deploy the pipeline resources we just created in the previous step:

sam deploy -t codepipeline.yaml --stack-name <pipeline-stack-name> --capabilities=CAPABILITY_IAM --region <region-X>

Don't set the same stack name as your SAM application because doing so will overwrite your application's stack (and delete your application resources).

> sam deploy -t codepipeline.yaml --stack-name pipeline-sam-app --capabilities=CAPABILITY_IAM

        Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-k2z3x0eqvuxq
        A different default S3 bucket can be set in samconfig.toml
        Or by specifying --s3-bucket explicitly.

    Deploying with following values
    ===============================
    Stack name                   : pipeline-sam-app
    Region                       : eu-west-1
    Confirm changeset            : True
    Disable rollback             : False
    Deployment s3 bucket         : aws-sam-cli-managed-default-samclisourcebucket-k2z3x0eqvuxq
    Capabilities                 : ["CAPABILITY_IAM"]
    Parameter overrides          : {}
    Signing Profiles             : {}

Initiating deployment
=====================

    Uploading to sam-app/1ac2fdc178c63ba988480068fe211880.template  15696 / 15696  (100.00%)

Waiting for changeset to be created..

CloudFormation stack changeset
-----------------------------------------------------------------------------------------------------
Operation                 LogicalResourceId         ResourceType              Replacement
-----------------------------------------------------------------------------------------------------
+ Add                     CodeBuildProjectBuildAn   AWS::CodeBuild::Project   N/A
                          dPackage
+ Add                     CodeBuildProjectDeploy    AWS::CodeBuild::Project   N/A
+ Add                     CodeBuildServiceRole      AWS::IAM::Role            N/A
+ Add                     CodePipelineExecutionRo   AWS::IAM::Role            N/A
                          le
+ Add                     CodeStarConnection        AWS::CodeStarConnection   N/A
                                                    s::Connection
+ Add                     PipelineStackCloudForma   AWS::IAM::Role            N/A
                          tionExecutionRole
+ Add                     Pipeline                  AWS::CodePipeline::Pipe   N/A
                                                    line
-----------------------------------------------------------------------------------------------------

Changeset created successfully. arn:aws:cloudformation:eu-west-1:00000000000:changeSet/samcli-deploy1706135069/11d2764f-90e7-4495-b70c-513533ce61d2

Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: > y

2024-01-24 23:26:04 - Waiting for stack create/update to complete

CloudFormation events from stack operations (refresh every 5.0 seconds)
-----------------------------------------------------------------------------------------------------
ResourceStatus            ResourceType              LogicalResourceId         ResourceStatusReason
-----------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS        AWS::CloudFormation::St   pipeline-sam-app          User Initiated
                          ack
CREATE_IN_PROGRESS        AWS::IAM::Role            CodeBuildServiceRole      -
CREATE_IN_PROGRESS        AWS::CodeStarConnection   CodeStarConnection        -
                          s::Connection
CREATE_IN_PROGRESS        AWS::IAM::Role            PipelineStackCloudForma   -
                                                    tionExecutionRole
CREATE_IN_PROGRESS        AWS::CodeStarConnection   CodeStarConnection        Resource creation
                          s::Connection                                       Initiated
CREATE_IN_PROGRESS        AWS::IAM::Role            PipelineStackCloudForma   Resource creation
                                                    tionExecutionRole         Initiated
CREATE_COMPLETE           AWS::CodeStarConnection   CodeStarConnection        -
                          s::Connection
CREATE_IN_PROGRESS        AWS::IAM::Role            CodeBuildServiceRole      Resource creation
                                                                              Initiated
CREATE_COMPLETE           AWS::IAM::Role            PipelineStackCloudForma   -
                                                    tionExecutionRole
CREATE_COMPLETE           AWS::IAM::Role            CodeBuildServiceRole      -
CREATE_IN_PROGRESS        AWS::CodeBuild::Project   CodeBuildProjectDeploy    -
CREATE_IN_PROGRESS        AWS::CodeBuild::Project   CodeBuildProjectBuildAn   -
                                                    dPackage
CREATE_IN_PROGRESS        AWS::CodeBuild::Project   CodeBuildProjectDeploy    Resource creation
                                                                              Initiated
CREATE_IN_PROGRESS        AWS::CodeBuild::Project   CodeBuildProjectBuildAn   Resource creation
                                                    dPackage                  Initiated
CREATE_COMPLETE           AWS::CodeBuild::Project   CodeBuildProjectDeploy    -
CREATE_COMPLETE           AWS::CodeBuild::Project   CodeBuildProjectBuildAn   -
                                                    dPackage
CREATE_IN_PROGRESS        AWS::IAM::Role            CodePipelineExecutionRo   -
                                                    le
CREATE_IN_PROGRESS        AWS::IAM::Role            CodePipelineExecutionRo   Resource creation
                                                    le                        Initiated
CREATE_COMPLETE           AWS::IAM::Role            CodePipelineExecutionRo   -
                                                    le
CREATE_IN_PROGRESS        AWS::CodePipeline::Pipe   Pipeline                  -
                          line
CREATE_IN_PROGRESS        AWS::CodePipeline::Pipe   Pipeline                  Resource creation
                          line                                                Initiated
CREATE_COMPLETE           AWS::CodePipeline::Pipe   Pipeline                  -
                          line
CREATE_COMPLETE           AWS::CloudFormation::St   pipeline-sam-app          -
                          ack
-----------------------------------------------------------------------------------------------------

CloudFormation outputs from deployed stack
-------------------------------------------------------------------------------------------------------
Outputs
-------------------------------------------------------------------------------------------------------
Key                 CodeStarConnectionArn
Description         The Arn of AWS CodeStar Connection used to connect to external code repositories.
Value               arn:aws:codestar-connections:eu-
west-1:00000000000:connection/0b2a540b-8af9-490f-91e7-24e26e16a313
-------------------------------------------------------------------------------------------------------

Successfully created/updated stack - pipeline-sam-app in eu-west-1

A new stack has been created to deploy our AWS CodePipeline:

Now, we need to access AWS CodePipeline to check the execution.

However, the first execution has failed as we can see in the following image:

We can see in the CodePipeline flow that all steps have been created:

Source: integrated with GitHub as we indicated before
UpdatePipeline: the pipeline can update itself
BuildAndPackage: the SAM application is built, packaged, and uploaded (with AWS CodeBuild service)
DeployTest: the SAM application is deployed (with AWS CodeBuild service)

The cause of the error was that the connection between GitHub and AWS must be confirmed after being created:

Step 5: Connect the Git repository with the CI/CD system

If you are using GitHub or Bitbucket, after executing the sam deploy command for your pipeline, you need to complete the pending connection in the Settings/Connection section of the Developer Tools.

In addition, you could store a copy of the CodeStarConnectionArn from the output of the sam deploy command, because you will need it if you want to use AWS CodePipeline with another branch than main.

By accessing the Settings/Connections in the Developer Tools, you can validate that the connection is pending approval:

After activating it, we run the pipeline again (by clicking on the Release change button) and now the pipeline ends correctly.

Update the CI/CD steps in the SAM project

Note that now we have a pipeline that first checks for changes in the pipeline itself and then checks the code and deploys the resources.

Step 1: Update the steps in the pipeline automatically

With this pipeline configuration (with UpdatePipeline step) all the changes that we make in the pipeline will be updated automatically.

We want to test the automatic update of the pipeline if we make some changes.

So, let's change the pipeline steps to:

delete the UpdatePipeline step
add the UnitTest step

To do this, we have to update the codepipeline.yaml file with the necessary changes:

comment all the UpdatePipeline step code
uncomment all the UnitTest step code

And after that, commit the changes and push them to the repository:

When we push the changes, the UpdatePipeline step executes an update on the pipeline-sam-app stack (in the CloudFormation service) and it will update the pipeline definition.

As we expected, the pipeline has updated itself and now we have UnitTest step but not UpdatePipeline.

Step 2: Update the steps in the pipeline manually

If you remove the UpdatePipeline step, when you push a change to the repository the pipeline won't be updated, so you have to run manually the update of the pipeline.

To update the pipeline we have to run again the command sam deploy -t codepipeline.yaml --stack-name pipeline-sam-app --capabilities=CAPABILITY_IAM.

Now, we will remove the UnitTest stage.

> sam deploy -t codepipeline.yaml --stack-name pipeline-sam-app --capabilities=CAPABILITY_IAM

    Deploying with following values
    ===============================
    Stack name                   : pipeline-sam-app
    Region                       : eu-west-1
    Confirm changeset            : True
    Disable rollback             : False
    Deployment s3 bucket         : None
    Capabilities                 : ["CAPABILITY_IAM"]
    Parameter overrides          : {}
    Signing Profiles             : {}

Initiating deployment
=====================

Waiting for changeset to be created..

CloudFormation stack changeset
-----------------------------------------------------------------------------------------------------
Operation                 LogicalResourceId         ResourceType              Replacement
-----------------------------------------------------------------------------------------------------
* Modify                  CodeBuildProjectBuildAn   AWS::CodeBuild::Project   Conditional
                          dPackage
* Modify                  CodeBuildProjectDeploy    AWS::CodeBuild::Project   Conditional
* Modify                  CodeBuildServiceRole      AWS::IAM::Role            False
* Modify                  CodePipelineExecutionRo   AWS::IAM::Role            False
                          le
* Modify                  CodeStarConnection        AWS::CodeStarConnection   False
                                                    s::Connection
* Modify                  PipelineStackCloudForma   AWS::IAM::Role            False
                          tionExecutionRole
* Modify                  Pipeline                  AWS::CodePipeline::Pipe   False
                                                    line
- Delete                  CodeBuildProjectUnitTes   AWS::CodeBuild::Project   N/A
                          t
-----------------------------------------------------------------------------------------------------

Changeset created successfully. arn:aws:cloudformation:eu-west-1:00000000000:changeSet/samcli-deploy1706140589/27c840fe-922c-4232-861f-51c03d45471c

Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: > y

2024-01-25 00:57:26 - Waiting for stack create/update to complete

CloudFormation events from stack operations (refresh every 5.0 seconds)
-----------------------------------------------------------------------------------------------------
ResourceStatus            ResourceType              LogicalResourceId         ResourceStatusReason
-----------------------------------------------------------------------------------------------------
UPDATE_IN_PROGRESS        AWS::CloudFormation::St   pipeline-sam-app          User Initiated
                          ack
UPDATE_COMPLETE           AWS::IAM::Role            PipelineStackCloudForma   -
                                                    tionExecutionRole
UPDATE_COMPLETE           AWS::IAM::Role            CodeBuildServiceRole      -
UPDATE_COMPLETE           AWS::CodeStarConnection   CodeStarConnection        -
                          s::Connection
UPDATE_COMPLETE           AWS::CodeBuild::Project   CodeBuildProjectBuildAn   -
                                                    dPackage
UPDATE_COMPLETE           AWS::CodeBuild::Project   CodeBuildProjectDeploy    -
UPDATE_IN_PROGRESS        AWS::IAM::Role            CodePipelineExecutionRo   -
                                                    le
UPDATE_COMPLETE           AWS::IAM::Role            CodePipelineExecutionRo   -
                                                    le
UPDATE_IN_PROGRESS        AWS::CodePipeline::Pipe   Pipeline                  -
                          line
UPDATE_COMPLETE           AWS::CodePipeline::Pipe   Pipeline                  -
                          line
UPDATE_COMPLETE_CLEANUP   AWS::CloudFormation::St   pipeline-sam-app          -
_IN_PROGRESS              ack
DELETE_IN_PROGRESS        AWS::CodeBuild::Project   CodeBuildProjectUnitTes   -
                                                    t
DELETE_COMPLETE           AWS::CodeBuild::Project   CodeBuildProjectUnitTes   -
                                                    t
UPDATE_COMPLETE           AWS::CloudFormation::St   pipeline-sam-app          -
                          ack
-----------------------------------------------------------------------------------------------------

CloudFormation outputs from deployed stack
-------------------------------------------------------------------------------------------------------
Outputs
-------------------------------------------------------------------------------------------------------
Key                 CodeStarConnectionArn
Description         The Arn of AWS CodeStar Connection used to connect to external code repositories.
Value               arn:aws:codestar-connections:eu-
west-1:00000000000:connection/0b2a540b-8af9-490f-91e7-24e26e16a313
-------------------------------------------------------------------------------------------------------

Successfully created/updated stack - pipeline-sam-app in eu-west-1

And the pipeline will be updated:

After all the changes that we made:

Now, we have 3 steps in our CodePipeline:
- Source: integrated with GitHub as we indicated before
- BuildAndPackage: the SAM application is built, packaged, and uploaded (with AWS CodeBuild service)
- DeployTest: the SAM application is deployed (with AWS CodeBuild service)
To summarize the current state:
- If we change the SAM application and update the code in our GitHub repository, the pipeline will update the resources in our AWS account.
- If we want to update the pipeline itself, we have to update the specific pipeline files in our SAM application, and then run manually the command
- What happens if you want to update the pipeline itself? Then, with the recent changes we did, you have to:
- modify the codepipeline.yaml file in our SAM application
- manually execute the command sam deploy -t codepipeline.yaml --stack-name pipeline-sam-app --capabilities=CAPABILITY_IAM.

Clean up

We have created 3 or 4 stacks in CloudFormation related to SAM, depending if you already had used AWS SAM before (then the aws-sam-cli-managed-default already existed):

aws-sam-cli-managed-default: general SAM resources
sam-app: application code
pipeline-sam-app: CI/CD of the sam-app
aws-sam-cli-managed-test-pipeline-resources: test stage resources

If you want to use AWS SAM in the future you could keep the aws-sam-cli-managed-default stack.

You have several ways to delete your resources:

AWS CloudFormation service
AWS CLI
AWS SAM CLI

If you execute the command sam delete, it only will delete the main stack (sam-app) but not the CI/CD pipeline or the stage resources stack, so you have to specify the pipeline stack in the second execution.

sam delete --stack-name pipeline-sam-app

And don't forget to remove the environment stack resources:

sam delete --stack-name aws-sam-cli-managed-test-pipeline-resources

Wrapping It Up

That brings us to the end of our journey on integrating CI/CD into AWS SAM projects. Let's recap the highlights:

Ease of Deployment with SAM CLI: We utilized the sam pipeline command for a streamlined deployment process. It's efficient and simplifies the deployment of SAM applications.
Customizing the Pipeline: Demonstrating AWS SAM's adaptability, we successfully used a custom template for a single-stage pipeline, allowing for tailored deployment strategies.
Deployment Automation: Highlighting the power of AWS tools, we've automated our deployments, which simplifies application management and ensures our apps are always current.

What's next?

How to create serverless applications with AWS SAM (Serverless Application Model)

Alejandro Lazaro — Wed, 24 Jan 2024 11:24:00 +0000

This is the second article of a serie about AWS SAM. The next will be one related with the deployment.

To keep it simple, we will create a SAM application from a quick start template using the standalone function. However you could try a different template, the steps to follow should be the same.

The code example is available in GitHub repository here. If you want to see the step by step you can check the commit history, where you can find the evolution of the application through the steps explained in the following lines.

These are all the steps that I want to show you in this article:

Step 1: Create a SAM application
Step 2 (Optional): Test your application locally
Step 3 (Optional): Unit test
Step 4: Build your application
Step 5: Deploy manually your application with the CLI
Step 6 (Optional): AWS SAM Accelerate (Preview) - Sync

Step 1: Create a sample SAM application

The first step is to create our application through a quick start template: Standalone function.

To create a new application from a template we run the sam init command.

> sam init
Which template source would you like to use?
1 - AWS Quick Start Templates
2 - Custom Template Location
Choice: > 1

Choose an AWS Quick Start application template
  1 - Hello World Example
  2 - Data processing
  3 - Hello World Example with Powertools for AWS Lambda
  4 - Multi-step workflow
  5 - Scheduled task
  6 - Standalone function
  7 - Serverless API
  8 - Infrastructure event management
  9 - Lambda Response Streaming
  10 - Serverless Connector Hello World Example
  11 - Multi-step workflow with Connectors
  12 - GraphQLApi Hello World Example
  13 - Full Stack
  14 - Lambda EFS example
  15 - Hello World Example With Powertools for AWS Lambda
  16 - DynamoDB Example
  17 - Machine Learning
Template: > 6

Which runtime would you like to use?
  1 - dotnet6
  2 - nodejs20.x
  3 - nodejs18.x
  4 - nodejs16.x
Runtime: > 2

Based on your selections, the only Package type available is Zip.
We will proceed to selecting the Package type as Zip.

Based on your selections, the only dependency manager available is npm.
We will proceed copying the template using npm.

Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: > N

Would you like to enable monitoring using CloudWatch Application Insights?
For more info, please view https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-application-insights.html [y/N]: > N

Would you like to set Structured Logging in JSON format on your Lambda functions?  [y/N]:

Project name [sam-app]: > sam-app

    -----------------------
    Generating application:
    -----------------------
    Name: sam-app
    Runtime: nodejs20.x
    Architectures: x86_64
    Dependency Manager: npm
    Application Template: quick-start-from-scratch
    Output Directory: .
    Configuration file: sam-app/samconfig.toml

    Next steps can be found in the README file at sam-app/README.md


Commands you can use next
=========================
[*] Create pipeline: cd sam-app && sam pipeline init --bootstrap
[*] Validate SAM template: cd sam-app && sam validate
[*] Test Function in the Cl

At the end of the command line messages appears Commands you can use next, where other SAM CLI commands are suggested as the next steps to execute.

This is the basic application that has been created (only with one lambda function for easy understanding):

Note that we have 4 of the 5 files that we reviewed before:

src/handlers/hello-from-lambda.js
_test_/unit/handlers/hello-from-lambda.test.js
template.yaml
package.json

We don't have the folder events because we only create one simple Lambda Function with no event integrations

Step 2 (Optional): Test your application locally

The AWS SAM CLI provides the sam local command to run your application using Docker containers that simulate the execution environment of Lambda.

Invoke your Lambda function running sam local invoke:

> sam local invoke
Invoking src/handlers/hello-from-lambda.helloFromLambdaHandler (nodejs20.x)
Local image was not found.
Removing rapid images for repo public.ecr.aws/sam/emulation-nodejs20.x
Building image.............................................................................................................................................................................................................................................................................
Using local image: public.ecr.aws/lambda/nodejs:20-rapid-x86_64.

Mounting /Users/alazaroc/Documents/MyProjects/github/aws/sam/sam-app as /var/task:ro,delegated, inside
runtime container
START RequestId: ba22ff8c-02f8-42a7-980d-0f0248ee63bf Version: $LATEST
2024-01-22T22:11:35.245Z    ae4b9347-e347-4ec0-a7a1-f34a3533ec8b    INFO    Hello from Lambda!
END RequestId: ae4b9347-e347-4ec0-a7a1-f34a3533ec8b
REPORT RequestId: ae4b9347-e347-4ec0-a7a1-f34a3533ec8b  Init Duration: 0.06 ms  Duration: 181.07 ms Billed Duration: 182 ms Memory Size: 128 MB Max Memory Used: 128 MB
"Hello from Lambda!"

We received the response "Hello from Lambda!" and more useful information (Duration, Billed Duration, Memory Size, or Max Memory Used).

If you have more than one Lambda Function, you must add the name which appears in the template.yaml file.

sam local invoke "helloFromLambdaFunction"
Invoking src/handlers/hello-from-lambda.helloFromLambdaHandler (nodejs20.x)
Local image is up-to-date
Using local image: public.ecr.aws/lambda/nodejs:20-rapid-x86_64.

Mounting /Users/alazaroc/Documents/MyProjects/github/aws/sam/sam-app as /var/task:ro,delegated, inside
runtime container
START RequestId: 323bbea4-2d41-407d-97b6-8605cbe454c3 Version: $LATEST
2024-01-22T22:20:25.541Z    9fadbf3c-0c9f-4728-b1a7-4ab96e715172    INFO    Hello from Lambda!
END RequestId: 9fadbf3c-0c9f-4728-b1a7-4ab96e715172
REPORT RequestId: 9fadbf3c-0c9f-4728-b1a7-4ab96e715172  Init Duration: 0.05 ms  Duration: 142.91 ms Billed Duration: 143 ms Memory Size: 128 MB Max Memory Used: 128 MB
"Hello from Lambda!"

You also can test an API locally if your SAM project includes it.

You should run sam local start-api command, which starts up a local endpoint that replicates your REST API endpoint.

Step 3 (Optional): Unit test

Tests are defined in the _tests_ folder in this project. Use npm to install the Jest test framework and run unit tests.

npm install
...
npm run test
> replaced-by-user-input@0.0.1 test
> node --experimental-vm-modules node_modules/jest/bin/jest.js

  console.info
    Hello from Lambda!

      at helloFromLambdaHandler (src/handlers/hello-from-lambda.mjs:9:13)

(node:31412) ExperimentalWarning: VM Modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
PASS  __tests__/unit/handlers/hello-from-lambda.test.mjs
  Test for hello-from-lambda
    ✓ Verifies successful response (30 ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        0.553 s
Ran all test suites.

Step 4: Build your application

The sam build command builds any dependencies that your application has, and copies your application source code to folders under .aws-sam/build to be zipped and uploaded to Lambda.

> sam build
Starting Build use cache
Manifest file is changed (new hash: 57574ed173cdc6a98b283bd7b00b44ca) or dependency folder
(.aws-sam/deps/ee51225c-b9b8-433f-ab8b-3821871f820a) is missing for (helloFromLambdaFunction), downloading
dependencies and copying/building source
Building codeuri: /Users/alazaroc/Documents/MyProjects/github/aws/sam/sam-app runtime: nodejs20.x metadata:
{} architecture: x86_64 functions: helloFromLambdaFunction
Running NodejsNpmBuilder:NpmPack
Running NodejsNpmBuilder:CopyNpmrcAndLockfile
Running NodejsNpmBuilder:CopySource
Running NodejsNpmBuilder:NpmInstall
Running NodejsNpmBuilder:CleanUp
Running NodejsNpmBuilder:CopyDependencies
Running NodejsNpmBuilder:CleanUpNpmrc
Running NodejsNpmBuilder:LockfileCleanUp
Running NodejsNpmBuilder:LockfileCleanUp

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Validate SAM template: sam validate
[*] Invoke Function: sam local invoke
[*] Test Function in the Cloud: sam sync --stack-name {{stack-name}} --watch
[*] Deploy: sam deploy --guided

These are the new files of our SAM project:

Step 5: Deploy manually your application with the CLI

Now we want to deploy our application and We will do it manually using the CLI, although in this other article. I will explain how to do it with a pipeline (automatically).

Remember that AWS SAM uses AWS CloudFormation as the underlying deployment mechanism.

As we don't have a configuration file containing all the values, we are going to create one. We run the sam deploy -- guided command which will search as a first step if a samconfig.toml file exists and if not the AWS SAM CLI will ask us about the necessary information to deploy our application.

The sam deploy command will package and upload the application artifacts to the S3 bucket, and deploys the application using AWS CloudFormation

> sam deploy --guided
Configuring SAM deploy
======================

  Looking for config file [samconfig.toml] :  Found
  Reading default arguments  :  Success

  Setting default arguments for 'sam deploy'
  =========================================
  Stack Name [sam-app]:
  AWS Region [eu-west-1]:
  #Shows you resources changes to be deployed and require a 'Y' to initiate deploy
  Confirm changes before deploy [Y/n]:
  #SAM needs permission to be able to create roles to connect to the resources in your template
  Allow SAM CLI IAM role creation [Y/n]:
  #Preserves the state of previously provisioned resources when an operation fails
  Disable rollback [y/N]:
  Save arguments to configuration file [Y/n]:
  SAM configuration file [samconfig.toml]:
  SAM configuration environment [default]:

  Looking for resources needed for deployment:

  Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-k2z3x0eqvuxq
  A different default S3 bucket can be set in samconfig.toml and auto resolution of buckets turned off by setting resolve_s3=False

        Parameter "stack_name=sam-app" in [default.deploy.parameters] is defined as a global parameter
[default.global.parameters].
        This parameter will be only saved under [default.global.parameters] in
/Users/alazaroc/Documents/MyProjects/github/aws/sam/sam-app/samconfig.toml.

  Saved arguments to config file
  Running 'sam deploy' for future deployments will use the parameters saved above.
  The above parameters can be changed by modifying samconfig.toml
  Learn more about samconfig.toml syntax at
  https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html

  Uploading to sam-app/c20f3d4923fb213486783dd25f269a52  38478 / 38478  (100.00%)

  Deploying with following values
  ===============================
  Stack name                   : sam-app
  Region                       : eu-west-1
  Confirm changeset            : True
  Disable rollback             : False
  Deployment s3 bucket         : aws-sam-cli-managed-default-samclisourcebucket-k2z3x0eqvuxq
  Capabilities                 : ["CAPABILITY_IAM"]
  Parameter overrides          : {}
  Signing Profiles             : {}

Initiating deployment
=====================

  Uploading to sam-app/3752a1ab023524618cd64cc55081484c.template  734 / 734  (100.00%)


Waiting for changeset to be created..

CloudFormation stack changeset
-------------------------------------------------------------------------------------------------------------
Operation                   LogicalResourceId           ResourceType                Replacement
-------------------------------------------------------------------------------------------------------------
+ Add                       helloFromLambdaFunctionRo   AWS::IAM::Role              N/A
                            le
+ Add                       helloFromLambdaFunction     AWS::Lambda::Function       N/A
-------------------------------------------------------------------------------------------------------------


Changeset created successfully. arn:aws:cloudformation:eu-west-1:000000000000:changeSet/samcli-deploy1705962828/ee61a1a2-9083-4029-9ee1-126661b04541


Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y

2024-01-22 23:34:17 - Waiting for stack create/update to complete

CloudFormation events from stack operations (refresh every 5.0 seconds)
-------------------------------------------------------------------------------------------------------------
ResourceStatus              ResourceType                LogicalResourceId           ResourceStatusReason
-------------------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS          AWS::CloudFormation::Stac   sam-app                     User Initiated
                            k
CREATE_IN_PROGRESS          AWS::IAM::Role              helloFromLambdaFunctionRo   -
                                                        le
CREATE_IN_PROGRESS          AWS::IAM::Role              helloFromLambdaFunctionRo   Resource creation
                                                        le                          Initiated
CREATE_COMPLETE             AWS::IAM::Role              helloFromLambdaFunctionRo   -
                                                        le
CREATE_IN_PROGRESS          AWS::Lambda::Function       helloFromLambdaFunction     -
CREATE_IN_PROGRESS          AWS::Lambda::Function       helloFromLambdaFunction     Resource creation
                                                                                    Initiated
CREATE_COMPLETE             AWS::Lambda::Function       helloFromLambdaFunction     -
CREATE_COMPLETE             AWS::CloudFormation::Stac   sam-app                     -
                            k
-------------------------------------------------------------------------------------------------------------

Successfully created/updated stack - sam-app in eu-west-1

Remember that the executed command will create the samconfig.toml file in our project to save the deployment configuration and be able to repeat it without configuration.

From now on, to deploy our SAM project we just need to run the sam deploy command, so we run it but if we have no changes, the deployment will fail:

> sam deploy
Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-k2z3x0eqvuxq
    A different default S3 bucket can be set in samconfig.toml
    Or by specifying --s3-bucket explicitly.
File with same data already exists at sam-app/c20f3d4923fb213486783dd25f269a52, skipping upload

  Deploying with following values
  ===============================
  Stack name                   : sam-app
  Region                       : eu-west-1
  Confirm changeset            : True
  Disable rollback             : False
  Deployment s3 bucket         : aws-sam-cli-managed-default-samclisourcebucket-k2z3x0eqvuxq
  Capabilities                 : ["CAPABILITY_IAM"]
  Parameter overrides          : {}
  Signing Profiles             : {}

Initiating deployment
=====================

File with same data already exists at sam-app/3752a1ab023524618cd64cc55081484c.template, skipping upload


Waiting for changeset to be created..

Error: No changes to deploy. Stack sam-app is up to date

Step 6 (Optional): AWS SAM Sync

We already have deployed our application in the cloud and you may want to synchronize the changes, i.e. deploy the changes in real-time when we save the changes (without running the deploy command).

The sam sync command syncs your local application changes to the AWS Cloud. Use sync to build, package, and deploy changes to your development environment as you iterate on your application. As a best practice, run sam sync after you finish iterating on your application to sync changes to your AWS CloudFormation stack.

Be careful if you use this functionality.

First it is in preview and also as you will see in the next lines in the console: "The SAM CLI will use the AWS Lambda, Amazon API Gateway, and AWS StepFunctions APIs to upload your code without performing a CloudFormation deployment. This will cause drift in your CloudFormation stack."

The sync command should only be used against a development stack.

> sam sync --stack-name sam-app --watch
The SAM CLI will use the AWS Lambda, Amazon API Gateway, and AWS StepFunctions APIs to upload your code without
performing a CloudFormation deployment. This will cause drift in your CloudFormation stack.
**The sync command should only be used against a development stack**.

Confirm that you are synchronizing a development stack.

Enter Y to proceed with the command, or enter N to cancel:
[Y/n]:
Queued infra sync. Waiting for in progress code syncs to complete...
Starting infra sync.
Manifest is not changed for (helloFromLambdaFunction), running incremental build
Building codeuri: /Users/alazaroc/Documents/MyProjects/github/aws/sam/sam-app runtime: nodejs20.x
metadata: {} architecture: x86_64 functions: helloFromLambdaFunction
Running NodejsNpmBuilder:NpmPack
Running NodejsNpmBuilder:CopyNpmrcAndLockfile
Running NodejsNpmBuilder:CopySource
Running NodejsNpmBuilder:CleanUpNpmrc
Running NodejsNpmBuilder:LockfileCleanUp
Running NodejsNpmBuilder:LockfileCleanUp

Build Succeeded

Successfully packaged artifacts and wrote output template to file /var/folders/wq/bz6xngtx5h3f5gf8py3kf28c0000gn/T/tmppcg60qga.
Execute the following command to deploy the packaged template
sam deploy --template-file /var/folders/wq/bz6xngtx5h3f5gf8py3kf28c0000gn/T/tmppcg60qga --stack-name <YOUR STACK NAME>


  Deploying with following values
  ===============================
  Stack name                   : sam-app
  Region                       : eu-west-1
  Disable rollback             : False
  Deployment s3 bucket         : aws-sam-cli-managed-default-samclisourcebucket-k2z3x0eqvuxq
  Capabilities                 : ["CAPABILITY_NAMED_IAM", "CAPABILITY_AUTO_EXPAND"]
  Parameter overrides          : {}
  Signing Profiles             : null

Initiating deployment
=====================


2024-01-22 23:43:31 - Waiting for stack create/update to complete

CloudFormation events from stack operations (refresh every 0.5 seconds)
---------------------------------------------------------------------------------------------------------
ResourceStatus             ResourceType               LogicalResourceId          ResourceStatusReason
---------------------------------------------------------------------------------------------------------
UPDATE_IN_PROGRESS         AWS::CloudFormation::Sta   sam-app                    User Initiated
                          ck
UPDATE_IN_PROGRESS         AWS::CloudFormation::Sta   sam-app                    Transformation succeeded
                          ck
CREATE_IN_PROGRESS         AWS::CloudFormation::Sta   AwsSamAutoDependencyLaye   -
                          ck                         rNestedStack
CREATE_IN_PROGRESS         AWS::CloudFormation::Sta   AwsSamAutoDependencyLaye   Resource creation
                          ck                         rNestedStack               Initiated
CREATE_COMPLETE            AWS::CloudFormation::Sta   AwsSamAutoDependencyLaye   -
                          ck                         rNestedStack
UPDATE_IN_PROGRESS         AWS::Lambda::Function      helloFromLambdaFunction    -
UPDATE_COMPLETE            AWS::Lambda::Function      helloFromLambdaFunction    -
UPDATE_COMPLETE_CLEANUP_   AWS::CloudFormation::Sta   sam-app                    -
IN_PROGRESS                ck
UPDATE_COMPLETE            AWS::CloudFormation::Sta   sam-app                    -
                          ck
---------------------------------------------------------------------------------------------------------

Stack update succeeded. Sync infra completed.

Infra sync completed.

The console still listens for changes and if we change our lambda code and save it:

The console will be updated automatically as follow:

Syncing Lambda Function helloFromLambdaFunction...
Manifest is not changed for (helloFromLambdaFunction), running incremental build
Building codeuri: /Users/alazaroc/Documents/MyProjects/github/aws/sam/sam-app runtime: nodejs20.x
metadata: {} architecture: x86_64 functions: helloFromLambdaFunction
 Running NodejsNpmBuilder:NpmPack
 Running NodejsNpmBuilder:CopyNpmrcAndLockfile
 Running NodejsNpmBuilder:CopySource
 Running NodejsNpmBuilder:CleanUpNpmrc
 Running NodejsNpmBuilder:LockfileCleanUp
 Running NodejsNpmBuilder:LockfileCleanUp
Finished syncing Lambda Function helloFromLambdaFunction.

When you stop it (control + C) in the console it will appear:

^CShutting down sync watch...
Sync watch stopped.

When you executed the sync command, a nested stack associated with your main stack (sam-app) was created:

And when the console stops being synchronized, this nested stack is NOT deleted.

How to remove the nested stack created with the sync command?

You have to run the sam deploy command again and it will be removed:

> sam deploy
Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-k2z3x0eqvuxq
      A different default S3 bucket can be set in samconfig.toml
      Or by specifying --s3-bucket explicitly.
  File with same data already exists at sam-app/c20f3d4923fb213486783dd25f269a52, skipping upload

    Deploying with following values
    ===============================
    Stack name                   : sam-app
    Region                       : eu-west-1
    Confirm changeset            : True
    Disable rollback             : False
    Deployment s3 bucket         : aws-sam-cli-managed-default-samclisourcebucket-k2z3x0eqvuxq
    Capabilities                 : ["CAPABILITY_IAM"]
    Parameter overrides          : {}
    Signing Profiles             : {}

  Initiating deployment
  =====================

  File with same data already exists at sam-app/3752a1ab023524618cd64cc55081484c.template, skipping upload


  Waiting for changeset to be created..

  CloudFormation stack changeset
  ---------------------------------------------------------------------------------------------------------
  Operation                  LogicalResourceId          ResourceType               Replacement
  ---------------------------------------------------------------------------------------------------------
  * Modify                   helloFromLambdaFunction    AWS::Lambda::Function      False
  - Delete                   AwsSamAutoDependencyLaye   AWS::CloudFormation::Sta   N/A
                            rNestedStack               ck
  ---------------------------------------------------------------------------------------------------------

  Changeset created successfully. arn:aws:cloudformation:eu-west-1:000000000000:changeSet/samcli-deploy1705963790/6b5464b9-0b8e-44c2-93c0-8e0f509c9110


  Previewing CloudFormation changeset before deployment
  ======================================================
  Deploy this changeset? [y/N]: y

  2024-01-22 23:50:02 - Waiting for stack create/update to complete

  CloudFormation events from stack operations (refresh every 5.0 seconds)
  ---------------------------------------------------------------------------------------------------------
  ResourceStatus             ResourceType               LogicalResourceId          ResourceStatusReason
  ---------------------------------------------------------------------------------------------------------
  UPDATE_IN_PROGRESS         AWS::CloudFormation::Sta   sam-app                    User Initiated
                            ck
  UPDATE_IN_PROGRESS         AWS::Lambda::Function      helloFromLambdaFunction    -
  UPDATE_COMPLETE            AWS::Lambda::Function      helloFromLambdaFunction    -
  UPDATE_COMPLETE_CLEANUP_   AWS::CloudFormation::Sta   sam-app                    -
  IN_PROGRESS                ck
  DELETE_IN_PROGRESS         AWS::CloudFormation::Sta   AwsSamAutoDependencyLaye   -
                            ck                         rNestedStack
  DELETE_COMPLETE            AWS::CloudFormation::Sta   AwsSamAutoDependencyLaye   -
                            ck                         rNestedStack
  UPDATE_COMPLETE            AWS::CloudFormation::Sta   sam-app                    -
                            ck
  ---------------------------------------------------------------------------------------------------------

  Successfully created/updated stack - sam-app in eu-west-1

Step 7: Clean up

If you are followed this tutorial you will only have one stack in our AWS Account, so you can run the sam delete command (which deletes the main stack: sam-app).

> sam delete
Are you sure you want to delete the stack sam-app in the region eu-west-1 ? [y/N]: y
Are you sure you want to delete the folder sam-app in S3 which contains the artifacts? [y/N]: y
      - Deleting S3 object with key sam-app/c20f3d4923fb213486783dd25f269a52
      - Deleting S3 object with key sam-app/3752a1ab023524618cd64cc55081484c.template
- Deleting Cloudformation stack sam-app

Deleted successfully

Next steps

If you need more information about SAM I recommend you to visit the AWS documentation here
Next post: How to add CI/CD to my SAM project
Comment this post

Introduction to AWS SAM (Serverless Application Model)

Alejandro Lazaro — Tue, 23 Jan 2024 17:57:30 +0000

I've decided to repost an article I originally published on my blog here two years ago, but this isn't just a simple repost. After rereading the original pieces, labeled as '22 min read' and '20 min read,' I realized updates were necessary. Consequently, I've thoroughly reviewed, updated, and reorganized all the content to ensure it remains relevant and up-to-date.

I will create an AWS SAM series featuring 3 articles:

Introduction to AWS SAM (Serverless Application Model)
How to create serverless applications with AWS SAM (Serverless Application Model)
How to add CI/CD to my SAM project

This will be my first series of articles here.

Introduction

The first thing to understand is the relationship between serverless applications and AWS SAM, followed by a review of the basics.

What is a serverless application?

A serverless application is more than just a Lambda Function. It is a combination of Lambda functions, event sources, APIs, databases, and other resources that work together to perform tasks.

What is AWS SAM?

The AWS Serverless Application Model (AWS SAM) is an open-source framework that you can use to build serverless applications on AWS. SAM is an extension of AWS CloudFormation but SAM is streamlined and specifically designed for Serverless resources.

SAM is the specific IaC solution of AWS for defining and deploying serverless applications.

Benefits of SAM

Local Testing and Debugging
- With the aws sam cli you can execute and test your serverless applications on your local environment (by mounting a docker image and running the code)
Extension of AWS Cloud Formation
- You get reliability on the deployment capabilities
- You can use the SAM yaml template with all of the resources that are available in CloudFormation
Single Deployment Configuration
- You can easily manage all your necessary resources in one single place that belongs to the same stack
Built-in best practices
- You can define and deploy your Infrastructure as Config
- you can enforce code reviews
- you can enable safe deployments through CodeDeploy
- you can enable tracing by using AWS X-Ray
Deep integration with development tools
- AWS Serverless Application Repository: discover new applications
- AWS Cloud9 IDE: For authoring, testing, and debugging. I have written a post about Cloud9; you can find it here
- CodeBuild, CodeDeploy, and CodePipeline: To build a deployment pipeline
- AWS CodeStar: To get started with a project structure, code repository, and a CI/CD pipeline that's automatically configured for you

Basics

We will use NodeJS as programming language. However, as AWS Says in the FAQs, you can use AWS SAM to build serverless applications that use any runtime supported by AWS Lambda.

To understand the code structure of the SAM projects, five files are particularly important:

Common to all the programming languages:

template.yaml: This file contains the AWS SAM template that defines your application's AWS resources.
events/file.json: events folder contains the invocation events that you can use to invoke the function.
- Depends on the programming language (in this case NodeJS), so it will be different in each case (but the idea is the same):
src/handlers/file.js: src folder contains the code for the application's Lambda Function.
__tests__/unit/handlers/file.test.js: test folder contains the unit tests for the application code.
package.json: This file of NodeJS contains the application dependencies and is used for the sam build. If you are using Python language instead of NodeJS, the file will be requirements.txt.

AWS SAM template anatomy

This is the structure of the template.yaml.

# The AWSTemplateFormatVersion identifies the capabilities of the template
# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/format-version-structure.html
AWSTemplateFormatVersion: '2010-09-09'
# Optional: description
Description: >-
  Any text here.
  Multi-line

# Transform section specifies one or more macros that AWS CloudFormation uses to process your template
# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/transform-section-structure.html
Transform: AWS::Serverless-2016-10-31  

# Optional: Globals section defines properties that are common to all your serverless functions and APIs
Globals:
  Function:
    Timeout: 3
    MemorySize: 128
    Tracing: Active
    Tags:
      iac: SAM

# Optional: provides additional information about the template
Metadata:
  template metadata

# Optional: Values to pass to your template at runtime (when you create or update a stack).
Parameters:
  set of parameters

# Optioal: A mapping of keys and associated values that you can use to specify conditional parameter values, similar to a lookup table
Mappings:
  set of mappings

# Optional: conditions that control whether certain resources are created or whether certain resource properties are assigned a value during stack creation or update
Conditions:
  set of conditions

# Resources declares the AWS resources that you want to include in the stack.
# Resources section can contain a combination of AWS CloudFormation resources and AWS SAM resources
# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resources-section-structure.html
Resources:
  set of resources

# Optional: The values that are returned whenever you view your stack's properties
Outputs:
  set of outputs

More information here.

Prerequisites

AWS CLI
- how to install it
- how configure it
AWS SAM CLI (here)

Example of SAM application

Let's start reviewing one example of the SAM application to show some options about what we can do here.

I will show you the part of the template.yaml file which affects the specific service.

It's important to remember that you can incorporate CloudFormation resources into our SAM template. However, AWS SAM offers specific resources that are specially tailored for creating Lambda Functions, API Gateway, AppSync, DynamoDB, Step Functions, among several other services. All the relevant information can be found here.

Example of Lambda Function

The first special type, and more important, is the AWS::Serverless::Function, which you should use to create Lambda Functions (more information here).

Resources:
  # Each Lambda function is defined by properties:
  # https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction

  # This is a Lambda function config associated with the source code: hello-from-lambda.js
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/hello-from-lambda.helloFromLambdaHandler
      Runtime: nodejs14.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: A Lambda function that returns a static string.
      Policies:
        # Give Lambda basic execution Permission to the helloFromLambda
        - AWSLambdaBasicExecutionRole

Adding an API Gateway

To add the API Gateway resource you can use the specific type of AWS::Serverless::Api (more information here).

Resources:
  ...
  BasicAWSApiGateway:
    Type: AWS::Serverless::Api
    Properties:
      Name: Basic AWS Api Gateway resource
      StageName: poc
      ...

You can also add a trigger in the Lambda Function updating the section Events:

Resources:
  HelloWorldFunction:
    Properties:
      ...
      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /
            Method: get

And you should create in events folder the json definition of the method:

{
    "httpMethod": "GET"
}

Adding an scheduled event to the Lambda Function

Including a scheduled event to the Lambda Function is quite similar to adding an API.

Resources:
  HelloWorldFunction:
    Properties:
      ...
      Events:
        CloudWatchEvent:
          Type: Schedule
          Properties:
            Schedule: cron(0 * * * ? *)

And you should create in events folder the json definition of the rule:

{
  "id": "cdc73f9d-aea9-11e3-9d5a-835b769c0d9c",
  "detail-type": "Scheduled Event",
  "source": "aws.events",
  "account": "",
  "time": "1970-01-01T00:00:00Z",
  "region": "us-west-2",
  "resources": [
    "arn:aws:events:us-west-2:xxxxxxxxxxxx:rule/ExampleRule"
  ],
  "detail": {}
}

Adding the SNS topic resource

In this case, I will include the Lambda Function as a subscriber of my SNS topic.

Resources:
  ...
  SnsTopicAsTriggerOfLambdaFunction:
    Type: AWS::SNS::Topic
    Properties:
      TopicName: !Sub '${ResourcesName}'
      DisplayName : Topic used for trigger the lambda function
      Subscription:
        - Protocol: lambda
          Endpoint: !GetAtt HelloWorldFunction.Arn
      TracingConfig: Active

Saving one property in the SSM Parameter Store

Resources:
  ...
  MySnsTopicArnParameter:
      Type: AWS::SSM::Parameter
      Properties:
        Name: /general/sns/topic-test
        Type: String
        Value: !Ref SnsTopicAsTriggerOfLambdaFunction

More examples

As you can see, you can add any resource using the usual CloudFormation code inside the template.yaml file.

I recommend that you run aws sam init and try to create different projects from the templates.

Using AWS Cloud9 to Test and Deploy Docker Applications: A Step-by-Step Guide

Alejandro Lazaro — Thu, 18 Jan 2024 07:44:04 +0000

TLDR

I first wrote this post here on my own blog. You will find much more content there.

This is the guide I would have liked to find before I started using AWS Cloud9. There are many similar articles out there, but none quite like this one, or at least, I didn't find any!

AWS Cloud9 is a cloud-based integrated development environment (IDE) that lets you write, run, and debug your code with just a browser.
It includes a code editor, debugger, and terminal.
You already have pre-installed the main tools you will need to download and create your code (including git or docker).

Introduction to AWS Cloud9

Have you ever been frustrated by the restrictions of your local development environment? If so, you're not the only one.

Enter AWS Cloud9, cloud-based Integrated Development Environment (IDE) that stands apart with its seamless Docker integration and browser-based accessibility, liberating you from the constraints of traditional setups. Imagine running code, debugging, and integrating Docker applications directly from your browser, without the usual setup headaches. That's the power of Cloud9.

In this article, we delve into how Cloud9 doesn't just offer convenience but brings a new level of power and versatility to your coding experience. Its robust code editor, efficient debugger, and integrated terminal transform coding into a streamlined, hassle-free process. This is especially crucial for projects that demand flexibility, such as serverless applications or containerized apps. With Cloud9, the world of cloud-based development is at your fingertips, allowing you to focus more on creating and less on configuring. Let's explore how Cloud9 can revolutionize your workflow and take your development projects to new heights.

Advantages and Limitations

To give you a clear picture, here are the foremost strengths and weaknesses.

Advantages:

Integrated Development Environment: Cloud9 offers a comprehensive IDE setup directly in the cloud. This means you can bypass the often tedious process of configuring your local environment, streamlining your development process significantly.
Accessibility: With its cloud-based nature, Cloud9 provides the flexibility to code from anywhere. Whether you're at home, in a café, or halfway across the world, all you need is an internet connection and you're ready to code.
AWS Integration: Cloud9's integration with AWS services is seamless, enhancing your workflow efficiency. This includes direct access to the AWS CLI, easy integration with AWS CodeCommit, and more, making it a breeze to manage AWS-related tasks within the IDE.
Collaborative Features: The platform supports real-time collaboration, allowing multiple developers to work on the same project simultaneously. This feature is invaluable for team projects and paired programming sessions.

Limitations:

Cost Factor: While Cloud9 offers an abundant set of features, it's important to note that it's not entirely free. Usage beyond the AWS free tier involves costs, such as running an EC2 instance or a server. For instance, in Ireland, a t2.micro instance costs about $0.0126 per hour.
Internet Dependency: As a cloud-based IDE, Cloud9 requires a stable and continuous internet connection. This could be a drawback in environments with limited or unreliable internet access.
Limited Customization: When compared to some local IDEs, Cloud9 might have limitations in terms of customization options. While it offers a range of functionalities, developers accustomed to highly personalized local environments might find this aspect slightly restrictive.

Starting with Cloud9

Initializing Your Cloud9 Environment

Setting up your AWS Cloud9 environment is a straightforward process. Here's how you can do it:

Access AWS Cloud9 Service: Log in to your AWS account and navigate to the Cloud9 service.
Create a New Environment: Select the Create a new environment option. Here, you'll need to provide some details:
- Instance Type: Choose the appropriate EC2 instance type based on your project needs.
- Timeout Settings: Set the desired idle time after which the instance should shut down to save resources.
- Network Settings: Configure network settings as per your requirements.
Start the Environment: Once you've configured these options, AWS will initiate the EC2 instance. To enter the Cloud9 IDE, click the Open button, and a new browser tab will launch with your environment.

Managing Your Environment Post-Use

You have to know how to stop the environment if you want to reuse it.

You have different options.

Stopping the Instance in the EC2 console: If you stop the EC2 instance, the environment will be stopped. However, if you leave the IDE open in the browser, the instance will be initialized again. Be careful.
Closing the IDE in the browser: Closing the IDE leads to the automatic stopping of the EC2 instance after the defined timeout, helping save resources.

Important Note: If you terminate the EC2 instance via the EC2 console, the Cloud9 environment remains but becomes inaccessible, leading to errors when trying to re-access it. Therefore, terminate the instance only if you intend to permanently remove the environment.

Sharing your environment

Cloud9 also allows for collaborative work. You can share your development environment with another IAM user within AWS:
In the IDE, use the Share option to grant access to another user.

These environments will then appear under 'My environments' or 'Shared with me' in the AWS Cloud9 service, facilitating easy collaboration.

Working with Cloud9

Now that your Cloud9 environment is set up, let's explore some tasks you can perform within this versatile IDE.

Integration with git repositories

Version control integration is a crucial first step in any development project, and Cloud9 simplifies this with its seamless Git integration.

With Cloud9, you can effortlessly connect with AWS CodeCommit or other Git repositories, streamlining your workflow. Here’s how you can clone a repository:

Cloning from GitHub:

I have created this public repository with 2 microservices, (one in Node.js and the other in Python). You can use it to follow this tutorial.

Use the command below to clone
```
git clone https://github.com/alazaroc/microservices.git
```
Cloning from AWS CodeCommit:

The integration is straightforward. You don’t have to manage permissions, users, or anything like that. You can directly clone the CodeCommit code:

The following is an example of how you can clone the CodeCommit repository (you have to use your own repository):
```
git clone codecommit::eu-west-1://microservices
```

After cloning, your application code is immediately accessible in the IDE, ready for the next steps.

Prepare the application for Testing

Before diving into testing, there are a couple of essential steps we need to take to ensure our application is ready and accessible.

Security Group Configuration

To ensure our application is accessible and secure, we first need to address the Security Group settings in our Cloud9 EC2 instance. Initially, it comes with no inbound rules, which means external access is restricted. This is a crucial aspect of AWS's security-first approach, minimizing vulnerabilities.

However, to test our application from outside, we'll need to modify these settings by adding specific inbound rules, allowing traffic on necessary ports while keeping our instance secure.

Third-party cookies

In your browser settings, ensure that third-party cookies or cross-site tracking is enabled. This step is crucial for certain functionalities within Cloud9 to work seamlessly. Without this setting, you might encounter issues or error messages in the preview phase.

Testing your application

Now, let's proceed to test two microservices – one built with Node.js and the other with Python. You can find the sample code for these microservices here.

These two applications are prepared to be executed locally directly (after installation of the packages/libraries), but they also contain a Dockerfile, so they are containerized and we can use docker to test them.

Docker and Cloud9 together create a robust environment for containerized application development.
Docker encapsulates the application environment, ensuring consistency across different stages of development.
This integration with Cloud9 enhances productivity and reduces the 'it works on my machine' syndrome. We'll delve into how Dockerizing our Node.js and Python applications within Cloud9 streamlines the development and deployment processes, making our applications more portable and scalable.

Node.js application

Installation

Install the required npm packages:

npm install

Running the Application

To start the Node.js microservice, use the command:

node app.js

Alternatively, you can open the app.js file and click on the Run button in the Cloud9 IDE.

The server will start and be available at http://localhost:3000.

Accessing the Application

Now that our application is running, the next step is to verify its functionality. AWS Cloud9 offers two primary methods to access your running application:

Preview running application

If you try to preview the application running in this port 3000, you will receive a message like this:

AWS says:

You aren't required run using HTTP over port 8080, 8081, or 8082. If you don't do this, you can't preview your running application from within the IDE. For more information, see Preview a running application.

In our case we are using the port 3000, so we can't use the preview functionality.

If you change the application port for 8080, then, you are able to do it:

Using the public IP of the instance

If changing the port is not an option or if you prefer external access, you can use the public IP of the EC2 instance. Here’s how to find your instance's public IP and access the application:

Dockerization

To build the Docker image of the application, run:

docker build -t ms-nodejs .

To run the application as a Docker container:

docker run -p 3000:3000 --name ms-nodejs ms-nodejs

Python application

Installation

Install the required Python packages:

pip install Flask requests

Running the Application

To run the microservice, execute the following command:

python app.py

By default, the server starts at http://localhost:5000.

Refer to the Node.js application section to know how to access this URL.

Dockerization

To build the Docker image, run:

docker build -t ms-python .

To run the application as a Docker container:

docker run -p 5000:5000 --name ms-python ms-python

Pushing Docker Images to Amazon ECR

The next step, after building and testing the Docker images of these 2 applications is upload them to Amazon Elastic Container Registry (ECR).

Create the ECR repositories

aws ecr create-repository \
    --repository-name ms-nodejs \
    --image-scanning-configuration scanOnPush=true \
    --region eu-west-1
aws ecr create-repository \
    --repository-name ms-python  \
    --image-scanning-configuration scanOnPush=true \
    --region eu-west-1

Get the local images

docker images

Login to ECR

aws ecr get-login-password --region eu-west-1 | docker login --username AWS --password-stdin xxxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com

Replace xxxxxxxxxxxx by your account_number

Push images to ECR

docker tag edSae8fef003 xxxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/ms-python:latest
docker push xxxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/ms-python:latest

docker tag fa3757c48c72 xxxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/ms-nodejs:latest
docker push xxxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/ms-nodejs:latest

Replace edSae8fef003 and fa3757c48c72 by your ms_nodejs and ms_python images ids

Conclusion

We've journeyed through setting up AWS Cloud9, integrating it with Git repositories, testing Docker applications, and finally, pushing Docker images to Amazon ECR. This walkthrough demonstrates the seamless integration and powerful capabilities of AWS Cloud9 in managing Docker-based projects.

Now, as you embark on applying these insights to your development endeavors, remember that each project is a unique mosaic of challenges and learning opportunities. I encourage you to not just use Cloud9, but to experiment with it, mold it into your workflow, and uncover the myriad ways it can amplify your productivity and creativity within the AWS ecosystem.

But don't let the journey end here. Your experiences, insights, and innovations are invaluable – and sharing them creates a rich tapestry of knowledge that benefits the entire development community. I invite you to use the comments section below as a platform to share your own stories with Cloud9. How has it transformed your development process? What unique challenges have you overcome using it? Your shared experiences are not just stories; they're the waypoints for others in their journey of cloud-based development.

Together, let's continue to explore, innovate, and elevate the art of development with AWS Cloud9. Dive in, share your story, and let's all grow as a community of forward-thinking developers.

If you want to find much more content like this you can check my aws blog: https://www.playingaws.com

Happy coding!

Challenge: 'AWS Certified Data Analytics Specialty' in 30 days [before to be retired - April 9, 2024]

Alejandro Lazaro — Sun, 12 Nov 2023 14:57:20 +0000

TLDR

I wrote this article in first place in my own blog, https://www.playingaws.com/posts/challenge-data-certification-in-30-days/, but I want to post it here to expose it to all interested people!

The AWS Certified Data Analytics Specialty is going to be retired on April 9, 2024. You still have time to do it. However, you have another option, AWS presented a new Data certification of the Associate level, AWS Certified Data Engineer, you have more information here.

If you are here, it's likely because you're eager to obtain the certification, or perhaps you're curious about the insights I've shared in this guide!

In either case, rest assured that this guide is a firsthand account of my successful journey to achieve the AWS Certified Data Analytics Specialty certification. I conquered the exam on October 15th.

If I could do it, you certainly can too.

I didn't have experience in data analytics (not enough) and neither specific knowledge, but I wanted to prepare the certification because I wanted to get more knowledge in this area. I am a Cloud & Solutions Architect who wanted to know how to design, build, secure, and maintain analytics solutions using AWS tools and services. However, this certification is not easy and I had to dedicate a lot of hours to be prepared to pass the exam.

I will use in this article [30 days] to prepare the certification's journey. However, remember, everyone's learning pace is different. Adjust the suggested plan based on your comfort level and progress. You could need more or less than 30 days.

Introduction

Welcome to the challenge of achieving the AWS Certified Data Analytics Specialty certification in just 30 days!

In this article, we will embark on a journey together to prepare for and conquer this certification. Whether you are an AWS Solutions Architect or a data enthusiast looking to expand your knowledge and credentials, this challenge will provide you with the guidance and resources needed to succeed.

Good luck with your certification journey!

Challenge: 'AWS Certified Data Analytics Specialty' in 30 days

I wrote this other article about 10 steps to prepare any AWS Certification, and I will follow those 10 steps in this challenge. This is the summary:

Find the proper certification for you
Find your WHY: Motivation is the key
Create a study plan
Take the official AWS Exam Readiness course
Choose a main training course
Practice using the AWS console to gain hands-on experience
Read the recommended whitepapers
Read the recommended FAQs
Take your own notes (and study them)
Practice with exam-style questions (tests)

Day 1: validate that the Data analytics certification is right for you and know what is your motivation (WHY)

Validate that the certification is right for you (step 1)

[x] Read exam guide

In this challenge, our goal is to obtain the AWS Certified Data Analytics Specialty certification, so it is important to carefully read the exam guide and ensure that this certification aligns with your goals and aspirations:

Data Analytics exam landing page

Data Analytics Exam guide

[x] Decide if you want to achieve the certification

I am far from being the target candidate described in the exam guide:

The target candidate should have 5 years of experience with common data analytics technologies. The target candidate also should have at least 2 years of hands-on experience and expertise working with AWS services to design, build, secure, and maintain analytics solutions.

It means that I need to intensify my efforts and commence my journey with a solid grasp of the essentials.

Motivation (step 2)

[x] WHY?

Before starting, you must have a very good reason to do it, good enough to not quit in the middle.

What are your reasons for getting this new AWS certification?

In my case, I need the Data certification because I don't have as much experience in data projects as I want, and I truly wanted to learn how to design, build, secure, and maintain analytics solutions using AWS tools and services so that I can collaborate in a helpful way on data projects. The second reason is because the data certification is the nearest one to an event driven solution, and I want to be involved in this solutions in the future.

Your motivation is your WHY. How powerful to you is your WHY to you?

If it is not good enough, probably you are going to give up...

Day 2: Target date, buy the exam and create the study plan (step 3)

[x] Purchase the exam

My target date was one month away, so I purchased the exam to do it even more real. I usually set a target date and buy the exam to motivate myself. Don't worry, you can always reschedule the exam twice for free with at least a minimum of 24 hours prior notice. I had to reschedule exams several times, but it is a good idea have a date in mind from the beginning.

I strongly recommend you request the time extension of 30 minutes if English is not your mother tongue.

[x] Create a study plan

Like I said, we will follow these 10 steps to prepare any AWS Certification.

You have to allocate time to prepare the exam. When creating your study plan, it is important to be realistic about the amount of time you can commit to studying. Be sure to factor in other commitments, such as work, family, and social obligations.

Probably your plan will change and you need to be flexible, but it is a good idea try to plan all the things you want to do to prepare the exam.

These are all the tasks that I had to do to prepare the exam, with an estimated amount of time [updated: I had to spend more hours in practically all the tasks]:

8h: Learn the basics of data analytics
4h of online content (so I will spend at least 8 hours)
Xh: Take a main training course. I will left here different options for you (there are many many more).
4h (my estimation) - Whitepapers:
5h (my estimation) - FAQs
20h (my estimation) - Practice with the main services (list above)
8h (my estimation) - Take my own notes and study it
8h (my estimation) - Practice exam questions

Day 3 - Starting with the basics

[x] Introduction to AWS Data Analytics

In my case, I had to start with the basics, so I decided to spend one full day and also writing this article about the Introduction to Data Analytics.

Initially I have included all the information in the same article but it was too long so I decided separate, also because maybe you are not interested in the introduction to the Data Analytics!

Days 4-6 - Exam readiness (step 4)

[x] Exam readiness

Take the time to go through the Exam Readiness: AWS Certified Data Analytics Specialty course provided by AWS. This course is specifically designed to help you prepare for the exam and covers important topics, including domain areas, question formats, and exam strategies.

AWS always show one message similar to this one: This is NOT training. This is to complement your training....
{: .prompt-warning }

These are the five domains for this certification. Four of them are the same that the Process of Data Analytics mentioned above + Security domain.

Collection (18%)
Storage and data management (22%)
Processing (24%)
Analysis and visualization (18%)
Security (18%)

Let me show the main AWS Services for each domain, and let me show you the more important ones.

Domain 1: Collection

Focuses on ingesting raw data from transactions, logs, and Internet of Things (IoT) devices.

Main AWS Services

Amazon Kinesis Data Streams
AWS Glue
Amazon Kinesis Data Firehose
AWS DMS
Amazon SQS
AWS IoT
AWS Snowball
Amazon MSK

Domain 2: Storage and Data Management

Main AWS Services:

Amazon RDS
Amazon Neptune
Amazon DynamoDB
Amazon Redshift
Amazon ElastiCache
Amazon S3
AWS Lake Formation
Amazon Aurora

Domain 3: Processing

The goal of the processing system is to transform data and make it more consumable by analytics and visualization tools

Main AWS Services:

Amazon EMR
Amazon Kinesis Data Analytics
AWS Glue
AWS Lambda
AWS Step Functions
AWS Data Pipeline

Domain 4: Analysis and Visualization

The analysis and visualization domain is about using the data you've collected, processed and transformed to generate actionable insights.

Amazon Athena
Amazon Kinesis Video Streams
Amazon OpenSearch
Amazon EMR
Amazon Redshift
Amazon kinesis Data Analytics
Amazon SageMaker
Amazon QuickSight

Domain 5: Security

Main AWS Services:

AWS IAM
AWS KMS
Check encryption of the following services: S3, Redshift, Kinesis, EMR and Glue
[x] Take my own notes

I take notes of all the videos/material I think it is useful so after that I can check it again. Sometimes is just do an screenshot of something important and sometimes I take my own notes.

Days 7-17 - Main training course & Practice resources (step 5 & 6)

First of all, I think you have to check this AWS resource: Ramp-up data analytics. This is a recompilation of many other courses and AWS content related with Data Analytics.

[x] Main training course

You have to select your own training course. Here, if you select one course that covers all the certification content you will save a lot of time searching for additional resources.

Some examples:

17h of content: Udemy [not free course]
42h of content: Cloud Academy [not free course]
AWS SkillBuilder courses [not free courses]

In my case, I selected SkillBuilder resources because I wanted to test this platform. Also, I don't wanted to watch training videos... I preferred choose all downloadable content and online content, and take my own notes to check them later. However, there is no one official preparation course in the SkillBuilder platform, so I had to select a few resources to try to cover all the content (and definitely I had to invest more time).

Disclaimer: these are not free courses. If you are only looking for free content, you can search yourself a free main training course or you can omit this step. However, probably you have to compensate with MANY more hours of preparation to fill all the gaps you have and to cover all the certification content

These are a few courses I did in the SkillBuilder platform:

Data Analytics on AWS (Technical), 4h. Digital content (no videos)
AWS Partner Certification Readiness: Data Analytics - Specialty (18h 27m). It includes pdf material that you can download, very useful! I didn't check the videos... only the pdf content! Also, this is a readiness course...
Data & Analytics Tech Talk (Partner Learning Plan), 10h 32m. Again, it includes pdf content that you can download, very useful! It contains 8 different courses
- Transform your data approach: Develop a modern data strategy - Technical
- Better, faster, and lower-cost storage : Optimizing Amazon S3 & FSx/EFS storage - Technical
- Analytics Readiness for BFSI - Technical
- Transactional data lakes on AWS - Technical
- Serverless Data Integration for a Modern Data Infrastructure with AWS Glue - Technical
- Analytics on SAP Data - Technical
- Redshift Migrations & POCs - Technical
- Right Data Streaming Architecture for your Streaming Analytics
AWS Cloud Quest Data Analytics. AWS Cloud Quest is the only role-playing game to help you build practical AWS Cloud skills. The challenges were easy and you have to complete many very un-useful challenges related with other services not related with the Data Analytics certification, but it was funny! I completed all the challenges and I achieved the AWS Cloud Quest Data Analytics badge
[x] Practice

Follow a training course is highly recommended but practice with the AWS console is even more useful to retain information. And much more if you don't have much experience with some of the AWS services.

I experiment by myself directly with some of the AWS services but I also get some internet examples.

I used these Skill Builder labs to practice with the main AWS resources in a provided AWS account.

Again, there is A LOT of free content you can check... Here are a few examples:

Days 18-19 - Whitepapers (step 7)

The link of the official AWS Whitepapers is included in the documentation landing page:

Days 20-21 - FAQs of the main services (step 8)

The link of the official FAQs is included (again) in the same documentation landing page:

Days 22-24 - Study my notes (step 9)

Unfortunately I cannot share my own notes, but I used this time to check everything again and again.

Days 25-29 - Practice with exam-style questions (step 10)

You have many options on internet.

The 2 AWS official test resources are the following:

10 questions - AWS Certified Data Analytics Specialty Sample Questions
20 questions - AWS Certified Data Analytics Specialty Official Practice Question Set

Day 30 - EXAM

A few advices that you already know... so, reminders!

BEFORE the exam:

Sleep well the previous night to be rest in the exam. It will be a long exam with long questions...

DURING the exam:

Identify keywords in the questions
Eliminate wrong answers
If you have doubts, answer the question, write in the comments your possible answers and everything that will help you in your next revision (example: A or B doubt in xxxxx), flag it, and go to the next one
Don't spend to much time on each question. 3 minutes if you have requested the additional 30 minutes for non-native english speakers
Review all the flagged questions and un-flag the ones that you are more confident now
Review again

With all the work already done, you only have to do the exam and pass it!

One last thing! If you followed this guide or found it helpful, let me know in the comments, or you can access to my blog to find more AWS useful content! https://www.playingaws.com/

AWS WAF (Web Application Firewall): Deep Dive

Alejandro Lazaro — Fri, 19 May 2023 18:41:17 +0000

Introduction

This article was first published in my own blog:
https://www.playingaws.com/posts/aws-waf-web-application-firewall-deep-dive/

A Web Application Firewall (WAF) is a security solution that protects web applications from malicious attacks, such as cross-site scripting, SQL injection, and malicious bot traffic. WAF is typically deployed as a reverse proxy, sitting between the internet and the web application, to inspect and filter incoming requests before they reach the web server.

Here's a visual representation of the basic architecture of AWS WAF:

How WAF Works

WAF works by analyzing incoming HTTP and HTTPS requests to a web application and allows or blocks requests based on pre-defined security rules. Security rules can be based on IP addresses, headers, parameters, and other attributes of the request. WAF can also perform Deep Packet Inspection (DPI) to inspect the contents of the request payload and determine if the request contains malicious content.

If a request violates a security rule, the WAF blocks the request and returns an error response to the client.

AWS WAF

AWS WAF is a popular choice for cloud-based WAF solutions, providing a comprehensive set of security rules to protect web applications.

AWS WAF (Web Application Firewall) is a cloud-based service that protects your web applications, defending against common web exploits that could impact availability, compromise security, or consume excessive resources. It enables you to control access to your web content and provides customizable security rules to filter traffic based on IP addresses, HTTP headers, HTTP body content, or URI strings.

Protected resources

You can protect the following resource types:

Amazon CloudFront distribution
Amazon API Gateway REST API
Application Load Balancer
AWS AppSync GraphQL API
Amazon Cognito user pool
AWS App Runner service
AWS Verified Access instance

Main components

The main components of AWS WAF include the following:

Rules: AWS WAF allows you to create rules that define the types of traffic you want to allow or block from reaching your web applications. You can create rules based on various conditions such as IP addresses, HTTP headers, URI strings, and HTTP body content.
Managed Rule Groups: AWS WAF provides pre-built managed rule groups that offer protection against common web attacks such as SQL injection, cross-site scripting (XSS), and more. These rule groups are created and maintained by AWS and updated regularly to ensure they provide up-to-date protection against the latest threats.
Web ACLs: AWS WAF uses web ACLs (Web Access Control Lists) to group together rules that you can then apply to one or more web applications. Web ACLs allow you to apply a set of rules across multiple web applications, making it easier to manage and apply security policies consistently.

Benefits and features

AWS WAF offers a wide range of benefits and features, empowering you to secure your web applications effectively. Here are some key advantages of using AWS WAF:

Agile protection against web attacks: AWS WAF rule propagation and updates take just under a minute, enabling you to react faster when you are under an attack or when security issues arise.
Ease of deployment and maintenance: AWS WAF is easy to deploy and protects application(s) deployed on either Amazon CloudFront, the Application Load Balancer, or Amazon API Gateway. There is no additional software to deploy, DNS configuration, or SSL/TLS certificate to manage.
Cost-effective: AWS WAF is a pay-as-you-go service, which means you only pay for the resources you use. This makes it a cost-effective solution for securing your web applications.
Improved web traffic visibility: AWS WAF gives near real-time visibility into your web traffic, which you can use to create new rules or alerts in Amazon CloudWatch.
Scalability: AWS WAF is designed to handle high volumes of traffic and can scale automatically to meet the demands of your web applications.
Flexibility: AWS WAF provides a wide range of options for creating custom rules to filter traffic, giving you greater flexibility in defining your security policies.

Hands-on with AWS WAF

First of all, you must know that AWS WAF is a regional service. However, it seems a global service when you access it, but you have to change between regions using this option:

Getting started with WAF is relatively straightforward, simply log in to the AWS Management Console and navigate to the AWS WAF service. From there, you can create a new WAF policy and configure the policy with the desired security rules.

Step 1: Describe web ACL and associate it to AWS resources
Step 2: Add rules and rule groups
Step 3:Set rule priority
Step 4: Configure metrics
Step 5: Review and create web ACL

Enabling logs

Logs are disabled by default, so we have to enable it. We have to enter the Web ACL created and access to Logging and Metrics tab:

As you can check in the following image We will have 3 options as Logging Destination (CloudWatch Logs Groups, Kinesis Data Firehose stream and S3 bucket):

After enabling CloudWatch logs, we can access the CloudWatch Log Insights tab and execute sample queries or create custom queries to analyze the logs. We can see the results as logs or in a diagram (Visualization):

Metrics

AWS WAF reports metrics to CloudWatch in one-minute intervals by default, and the metrics are retained for 2 weeks.

The metrics monitored are the following:

AllowedRequests
BlockedRequests
CountedRequests
PassedRequests

These metrics provide a SUM count of web requests that hit a specific Rule or Web ACL.

Create custom alarms

Usually is a good idea to create some alarms to be notified if some block requests appear on our web, so we can configure a CloudWatch Alarm.

The easy way to create the alarm is through the AWS WAF Metric for BlockeRequests. There we have a direct link to create one alarm with this associated metric if we access Graphed Metrics` and select the indicated icon in red color:

After the creation, we will have something like that:

Estimating Costs

The cost of AWS WAF can vary depending on the scale of your deployment, ranging from a few dollars per month for small deployments to several thousand dollars per month for large-scale deployments. AWS WAF pricing is based on the number of web requests processed and the number of security rules that are used.

Example of cost for our example (1 Web ACL with a few managed rules):

$5.00 per web ACL per month (prorated hourly) * 1 web ACL = $5.00
$1.00 per rule per month (prorated hourly) * 5 rules = $1.00
$0.60 per million requests processed * 1 (we will assume 1 million request) = $0.60
$0.10 per alarm metric * 1 alarm = $0.10

Total: $6.70 per month

More Information about AWS WAF

For more information about AWS WAF, you can visit the following links

AWS WAF (Developer Guide): https://docs.aws.amazon.com/waf/latest/developerguide/waf-chapter.html
Security Automations for AWS WAF: https://aws.amazon.com/solutions/implementations/security-automations-for-aws-waf/
WAF FAQs: https://aws.amazon.com/waf/faqs/

How to deploy a serverless website with Terraform

Alejandro Lazaro — Sat, 06 May 2023 10:18:48 +0000

Introduction

This article was first published in my own blog: https://www.playingaws.com/posts/how-to-deploy-serverless-website-with-terraform/

As you already know, creating and managing infrastructure can be a complex and time-consuming process, and fortunately, tools like Terraform can simplify this process by allowing you to define your Infrastructure as Code (IaC). In this article, we will explore the basics of Terraform and walk through how to create infrastructure on AWS using it.

However, this is not my first article about Infrastructure as Code tools.

If you want to know how to create infrastructure using familiar programming languages you can review this article about CDK: How to create infrastructure with CDK
If you want to create serverless applications on AWS, you can review this other article about SAM: How to create serverless applications with SAM

I have uploaded the source code used in this article in the following GitHub repository https://github.com/alazaroc/aws-terraform-serverless-website

What is Terraform?

Terraform is an open-source Infrastructure as Code (IaC) software tool created by HashiCorp. It allows you to define and manage your infrastructure using human-readable configuration files that you can version, reuse, and share. You can then use a consistent workflow to provision and manage all of your infrastructure throughout its lifecycle. Terraform supports a wide range of cloud providers, including AWS, Microsoft Azure, and Google Cloud Platform.

How Terraform Works

Terraform uses a declarative configuration language called HashiCorp Configuration Language (HCL).

In a nutshell:

Write: You define your infrastructure in a series of .tf files, which contain the configuration for your resources
Plan: Terraform then uses this configuration to generate an execution plan, which outlines the changes that will be made to your infrastructure.
Apply: Once you have reviewed the execution plan, you can apply it to your infrastructure using the "terraform apply" command. Terraform will then provision your resources according to the configuration you defined.

Getting Started with Terraform

To get started with Terraform, you will need to install it on your machine. You can download the latest version of Terraform from the official website here, and if you want to deploy on AWS, you also need AWS CLI.

Once you have installed Terraform, you can start creating your infrastructure.

In this section, we will review how Terraform works creating a first example to deploy a simple S3 bucket:

Create a new directory for your Terraform configuration file:

  mkdir my-serverless-website
  cd my-serverless-website

Create a new file called "main.tf" and there we are going to include the AWS provider information with the "us-east-1" region, and one resource of type aws_s3_bucket to create an S3 bucket with all the default configuration. Remember that the S3 bucket name must be unique globally.

  provider "aws" {
    region = "us-east-1"
  }

  resource "aws_s3_bucket" "website_bucket" {
    bucket = "my-unique-bucket-name-12y398y13489148h"
  }

Initialize your Terraform configuration.

  terraform init

When you run terraform init, Terraform downloads and installs the necessary provider plugins and modules required for your configuration. It also initializes the backend, which is the storage location for your Terraform state file. The state file is used to store the current state of your infrastructure, and it is used by Terraform to plan and apply changes to your infrastructure.

Generate an execution plan (optional):

  terraform plan

This command generates an execution plan based on the configuration in your main.tf file, and let you check what is going to be deployed.

Apply the execution plan to deploy on AWS:

  terraform apply

When you run terraform apply, Terraform compares the current state of your infrastructure with the desired state defined in your configuration. It then creates, modifies, or deletes resources as necessary to bring your infrastructure into the desired state. In our example, a new S3 bucket will be deployed.

Now, the AWS resources have been created. Let's check it accessing the AWS S3 service to access the new resource created by Terraform:

We already know how to deploy AWS resources with Terraform, so in the following section, we will evolve the initial example to deploy three serverless websites using S3 bucket.

How to deploy a serverless website with Terraform

We have an AWS S3 bucket deployed, and now we are going to use it to host a serverless website. There are different ways to do it, and we are going to evolve the S3 bucket to create a serverless static website.

This is what we are going to build:

v1.1: public S3 bucket
- Advantage: easy to implement
- Disadvantages: no custom domain, no aligned with security best practices (public bucket), no cache for static files
v1.2: public S3 as Static website hosting
- Advantages: easy to implement, index document and error page, redirection rules
- Disadvantages: not aligned with security best practices (public bucket), no cache for static files, Amazon S3 website endpoints do not support HTTPS (if you want to use HTTPS, you can use Amazon CloudFront to serve a static website hosted on Amazon S3)
v2: CloudFront distribution + private S3 bucket
- Advantage: easy to implement, private s3 bucket, cache for static files
- Disadvantages: auto-generated domain name
v3: Route53 + ACM + CloudFront Distribution + private S3 bucket + optionally Lambda Edge
- Advantages: custom domain name using AWS managed certificates, private s3 bucket, cache for static files
- Disadvantages: more complex to implement

In these examples, we are going to deploy a static website based on an index.html file with this content:

This is my serverless website

v1.1: public S3 bucket

In this version 1.1 we are going to expose the S3 bucket publicly so any person can access the index.html file using the public S3 endpoint.

Advantage:
- easy to implement
Disadvantages:
- no custom domain
- no aligned with security best practices (public bucket)
- no cache for static files

This solution is not aligned with security best practices because it expose an S3 bucket publicly.

Update the file "main.tf" with the following content:

provider "aws" {
  region = "us-east-1"
  default_tags {
    tags = {
      project = "serverless-demo",
      type    = "web",
      iac     = "terraform"
    }
  }
}

resource "aws_s3_bucket" "website_bucket" {
  bucket = "my-unique-bucket-name-12y398y13489148h"
}

resource "aws_s3_object" "website_bucket" {
  bucket       = aws_s3_bucket.website_bucket.id
  key          = "index.html"
  source       = "index.html"
  content_type = "text/html"
}

resource "aws_s3_account_public_access_block" "website_bucket" {
  block_public_acls   = false
  block_public_policy = false
}

resource "aws_s3_bucket_public_access_block" "website_bucket" {
  bucket = aws_s3_bucket.website_bucket.id
  block_public_acls       = false
  block_public_policy     = false
  ignore_public_acls      = false
  restrict_public_buckets = false
}

resource "aws_s3_bucket_policy" "website_bucket" {
  bucket = aws_s3_bucket.website_bucket.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid = "PublicReadGetObject"
        Effect = "Allow"
        Principal = "*"
        Action = [
          "s3:GetObject",
          "s3:ListBucket",
        ]
        Resource = [
          "${aws_s3_bucket.website_bucket.arn}",
          "${aws_s3_bucket.website_bucket.arn}/*"
        ]
      }
    ]
  })
}

Now, execute the terraform apply command: terraform apply --auto-approve

Then, open a private window in your browser and access the index.html content using the public endpoint of our S3 bucket. If you have executed the code before, it will be the following URL: https://my-unique-bucket-name-12y398y13489148h.s3.amazonaws.com/index.html

v1.2: Static website hosting using S3

In this version, similar to the previous one, we still have exposed the S3 bucket publicly but we will enable the static website hosting feature of S3.

Advantages:
- easy to implement
- static website includes the configuration of an index document, an error page and also redirection rules
Disadvantages:
- not aligned with security best practices (public bucket)
- no cache for static files
- Amazon S3 website endpoints do not support HTTPS (if you want to use HTTPS, you can use Amazon CloudFront to serve a static website hosted on Amazon S3)

This solution is not aligned with security best practices because it expose an S3 bucket publicly.

Using the content of the "main.tf" showed in the previous v1.1 example, add at the end of the document the following lines:

resource "aws_s3_bucket_website_configuration" "website_bucket" {
  bucket = aws_s3_bucket.website_bucket.id
  index_document {
    suffix = "index.html"
  }
}

Now, execute the terraform apply command: terraform apply --auto-approve

Then, open a private window in your browser and access through the static website to the index.html content: http://my-unique-bucket-name-12y398y13489148h.s3-website-us-east-1.amazonaws.com/

If you didn't realize before, look that we are using HTTP, not HTTPS. S3 static website don't support HTTPS.

v2: CloudFront Distribution + private S3 bucket

In this version, we are going to change the bucket to private again (and also, we are going to enable again the Block Public Access settings for this account feature of S3), and we are going to create a CloudFront Distribution connected with the private S3 bucket. So, we are going to access the S3 bucket using the CloudFront distribution.

Advantage:
- easy to implement
- private s3 bucket (aligned with the security best practices)
- includes cache for static files
Disadvantages:
- auto-generated domain name (by CloudFront)

Replace the "main.tf" content with the following lines:

provider "aws" {
  region = "us-east-1"
}

resource "aws_s3_bucket" "website_bucket" {
  bucket = "my-unique-bucket-name-12y398y13489148h"
}

resource "aws_s3_account_public_access_block" "website_bucket" {
  block_public_acls   = true
  block_public_policy = true
  ignore_public_acls = true
  restrict_public_buckets = true
}

resource "aws_s3_object" "website_bucket" {
  bucket       = aws_s3_bucket.website_bucket.id
  key          = "index.html"
  source       = "index.html"
  content_type = "text/html"
}

resource "aws_cloudfront_distribution" "cdn_static_site" {
  enabled             = true
  is_ipv6_enabled     = true
  default_root_object = "index.html"
  comment             = "my cloudfront in front of the s3 bucket"

  origin {
    domain_name              = aws_s3_bucket.website_bucket.bucket_regional_domain_name
    origin_id                = "my-s3-origin"
    origin_access_control_id = aws_cloudfront_origin_access_control.default.id
  }

  default_cache_behavior {
    min_ttl                = 0
    default_ttl            = 0
    max_ttl                = 0
    viewer_protocol_policy = "redirect-to-https"

    allowed_methods  = ["GET", "HEAD", "OPTIONS"]
    cached_methods   = ["GET", "HEAD"]
    target_origin_id = "my-s3-origin"

    forwarded_values {
      query_string = false
      cookies {
        forward = "none"
      }
    }
  }

  restrictions {
    geo_restriction {
      locations        = []
      restriction_type = "none"
    }
  }

  viewer_certificate {
    cloudfront_default_certificate = true 
  }
}

resource "aws_cloudfront_origin_access_control" "default" {
  name                              = "cloudfront OAC"
  description                       = "description of OAC"
  origin_access_control_origin_type = "s3"
  signing_behavior                  = "always"
  signing_protocol                  = "sigv4"
}

output "cloudfront_url" {
  value = aws_cloudfront_distribution.cdn_static_site.domain_name
}

data "aws_iam_policy_document" "website_bucket" {
  statement {
    actions   = ["s3:GetObject"]
    resources = ["${aws_s3_bucket.website_bucket.arn}/*"]
    principals {
      type        = "Service"
      identifiers = ["cloudfront.amazonaws.com"]
    }
    condition {
      test     = "StringEquals"
      variable = "aws:SourceArn"
      values   = [aws_cloudfront_distribution.cdn_static_site.arn]
    }
  }
}

resource "aws_s3_bucket_policy" "website_bucket_policy" {
  bucket = aws_s3_bucket.website_bucket.id
  policy = data.aws_iam_policy_document.website_bucket.json
}

Now, execute the terraform apply command: terraform apply --auto-approve

Then, open a private window in your browser and access the index.html content using the CloudFront DNS. The execution of the terraform template has as output the URL of the CloudFront Distribution.

Now, the S3 bucket is private, so if you access to the public endpoint of S3 trying to get the index.html file you will receive an error

v3: Route53 + ACM + CloudFront Distribution + private S3 bucket

In the last example, we will use our domain (registered in Route53) and we will create a certificate using ACM.

Requirement:
- Required a custom domain
Advantages:
- custom domain name using AWS managed certificates
- private s3 bucket (aligned with the security best practices)
- includes cache for static files
Disadvantages:
- more complex to implement

To be able to run this example you need your own domain (example.com) and you have to register it in Route53 service. If you don't have it you can buy a new domain in the Route53 server but it will cost you about 10 dollars per year.

These are the changes that you have to do in the previous "main.tf" file:

Update in your CloudFront Distribution resource aws_cloudfront_distribution the following, (where ${var.domain_name_simple} is for example example.com):

    viewer_certificate {
      cloudfront_default_certificate = true 
    }

replacing it for this:

    viewer_certificate {
      acm_certificate_arn      = aws_acm_certificate.cert.arn
      ssl_support_method       = "sni-only"
      minimum_protocol_version = "TLSv1.2_2021"
    }

    aliases = [
      var.domain_name_simple,
      var.domain_name
    ]

Next, add the following lines:

  resource "aws_acm_certificate" "cert" {
    provider                  = aws.use_default_region
    domain_name               = "*.${var.domain_name_simple}"
    validation_method         = "DNS"
    subject_alternative_names = [var.domain_name_simple]

    lifecycle {
      create_before_destroy = true
    }
  }

  data "aws_route53_zone" "zone" {
    provider     = aws.use_default_region
    name         = var.domain_name_simple
    private_zone = false
  }

  resource "aws_route53_record" "cert_validation" {
    provider = aws.use_default_region
    for_each = {
      for dvo in aws_acm_certificate.cert.domain_validation_options : dvo.domain_name => {
        name   = dvo.resource_record_name
        record = dvo.resource_record_value
        type   = dvo.resource_record_type
      }
    }

    allow_overwrite = true
    name            = each.value.name
    records         = [each.value.record]
    type            = each.value.type
    zone_id         = data.aws_route53_zone.zone.zone_id
    ttl             = 60
  }

  resource "aws_acm_certificate_validation" "cert" {
    provider                = aws.use_default_region
    certificate_arn         = aws_acm_certificate.cert.arn
    validation_record_fqdns = [for record in aws_route53_record.cert_validation : record.fqdn]
  }

  resource "aws_route53_record" "www" {
    zone_id = data.aws_route53_zone.zone.id
    name    = "www.${var.domain_name_simple}"
    type    = "A"

    alias {
      name                   = aws_cloudfront_distribution.cdn_static_site.domain_name
      zone_id                = aws_cloudfront_distribution.cdn_static_site.hosted_zone_id
      evaluate_target_health = false
    }
  }

  resource "aws_route53_record" "apex" {
    zone_id = data.aws_route53_zone.zone.id
    name    = var.domain_name_simple
    type    = "A"

    alias {
      name                   = aws_cloudfront_distribution.cdn_static_site.domain_name
      zone_id                = aws_cloudfront_distribution.cdn_static_site.hosted_zone_id
      evaluate_target_health = false
    }
  }

Now, execute the terraform apply command: terraform apply --auto-approve
Finally, open a private window in your browser and access your registered domain: https://example.com

Conclusion

In this article, we have explored the basics of Terraform and walked through how to create infrastructure with different examples. With Terraform, you can define your infrastructure as code and automate the process of creating and updating your resources.

This example is based on the code I have created to deploy my own blog https://playingaws.com using Route53 + ACM + CloudFront Distribution + private S3 bucket.

My first year with my AWS blog

Alejandro Lazaro — Thu, 16 Feb 2023 07:30:53 +0000

Introduction

Hello! I am Alejandro Lazaro, AWS community builder, and I want to share with you all my experience after 1 year with my own blog about AWS.

It has not been easy, writing articles takes a lot of time, and in this year I have published 17 articles about different topics (the blog itself; IaC & DevOps with SAM & CDK; Security; Machine Learning; Multi-account approach; Open-source tools...), but let's start at the beginning.

Why I decided to create my personal AWS blog?

I know, there is a lot of content on the Internet, and the official AWS content is awesome, but I had two main reasons to do it:

The main goal was to create periodic AWS-related content and share it publicly. Learning something is good, but sharing it is the next level, and I wanted to do that.
The secondary goal was to have an excuse to keep learning and practicing with AWS resources. I like to use the expression "play with AWS"... and so, my blog is https://playingaws.com

Lessons learned

There are a lot of lessons learn but the 3 more important for me are the following:

Write articles requires A LOT of time. If you are thinking whether create or not your own blog... my recommendation is YES! It will not be easy of course but you could learn a few things in the way. Take the opportunity and do it!
Keep it simple: it's important to keep things as simple as possible, and to explain concepts in plain language that anyone can understand because this helps to make the content more accessible and useful to a wider audience.
Feedback is a treasure: I always want feedback to know what I should improve or what I am doing wrong, but feedback is a treasure, is very hard to get it! However, when you have it, you have to take it very seriously. I had a few very good comments in the legacy comment system of my blog but unfortunately I changed it (the comment system) and I am starting from scratch again.

Content of the blog

About the blog and technologies used

This is the architecture of my blog:

Deciding how to create the blog took me a while, and in this article I explain how I decided on the technology behind the blog https://www.playingaws.com/posts/the-technology-behind-this-blog/

My first technical article about one AWS service was about Amplify Hosting. I used it to deploy my blog (I use Jekyll as static site generator) and wanted to share the experience. This AWS service really simplifies the deployment of a website and manages the entire application lifecycle for you. Here's the article: How to deploy a website with Amplify Hosting https://www.playingaws.com/posts/how-to-deploy-a-web-with-amplify-hosting/

IaC (CDK and SAM)

I published articles about CDK and SAM to create infrastructure and serverless applications.

Personally I love CDK and I think you should try it and give it a try.

How to create infrastructure with CDK: https://www.playingaws.com/posts/how-to-create-infrastructure-with-cdk/
How to add CI/CD to my CDK project: https://www.playingaws.com/posts/how-to-add-ci-cd-to-my-cdk-project/

And if you don't know SAM (Serverless Application Model) you should check it out:

How to create serverless applications with SAM: https://www.playingaws.com/posts/how-to-create-serverless-applications-with-sam/
How to add CI/CD to my SAM project: https://www.playingaws.com/posts/how-to-add-ci-cd-to-my-sam-project/

Finally, with CDK + SAM you can combine the best of both solutions:

How to create serverless applications with CDK and SAM: https://www.playingaws.com/posts/how-to-create-serverless-applications-with-cdk-and-sam/

Security

These articles will help you think about and improve the security of your AWS accounts.

Getting Started with AWS Security: https://www.playingaws.com/posts/getting-started-with-aws-security/
How to improve your account security: https://www.playingaws.com/posts/how-to-improve-your-account-security/
Amazon GuardDuty: Deep dive: https://www.playingaws.com/posts/aws-security-hub-deep-dive/
AWS Security Hub: Deep dive: https://www.playingaws.com/posts/amazon-guardduty-deep-dive/

Multi-account approach

Then, I published these 2 articles about the multi-account approach. When do you need a multi-account solution? Why do you need a multi-account solution? AWS multi-account approach in detail: AWS Organizations and AWS Control Tower:

Getting started with AWS Multi-account approach https://www.playingaws.com/posts/multi-account-approach/
AWS Control Tower Deep Dive https://www.playingaws.com/posts/aws-control-tower-deep-dive/

Machine Learning

This article about machine learning is specific of AWS DeepRacer. If you are not familiar with this AWS service I recommend you to check it out. It is a fun way to learn machine learning!

DeepRacer - First steps with Machine Learning: https://www.playingaws.com/posts/deepracer-first-steps-with-machine-learning/

Open-source tools

Lastly, I have posted 3 articles about open source tools that can help you with your AWS account and your AWS code.

Getting started with AWS open-source tools: https://www.playingaws.com/posts/aws-open-source-tools/
Open source tools to analyze your AWS environment: https://www.playingaws.com/posts/aws-open-source-tools-environment/
How open source tools can help you with your code: https://www.playingaws.com/posts/aws-open-source-tools-code/

What's next?

I will follow publishing regularly about some AWS topics. I have a lot of topics and I just need time to do it!

And as I said, feedback is like a treasure and I would appreciate it very much, here or on any of my blog posts!

Feel free to comment, share and enjoy. Thanks for reading to the end!