DEV Community: michal salanci

A small guide how to start AWS Community Day from scratch

michal salanci — Tue, 10 Jun 2025 18:51:02 +0000

AWS Community Day is a one day, community led conference, totally organized by AWS community. It is a great way to bringing AWS conference into your town or country...

This type of event is organized by AWS Community, from the biggest one as AWS Community Day DACH, organized by multiple AWS User Groups from multiple countries, to the smallest one organized by a single AWS User Group like AWS Community Day Slovakia.

I created this article is based on how we prepared the AWS Community Day Slovakia for the first time, what we have to deal with and how it did go at the end.

Web page

This is one of the first things you are going to need. It's up to you whether you create your own or use some template. We used a hugo template, which was created by AWS User Group Nederland and is available for other AWS Community Day organizers. 🙏👏
This is our page.

Registation
There are plenty of tools you can use for registration, such as: Eventbrite, Konfhub, Google forms and many of others. We decided to go with Eventbrite.

Call for speakers
This is same as with meetups, most people use Sessionize, or Google forms

AWS support

AWS Community Day page
Make sure to to over this page, where you can find basic information about AWS Community Day concept, FAQs, etc...

Downloadable content
AWS provide some downloadable content, which can be very helpful with planing and organizing your community day:
UG_toolkit.zip is very handy content of files containing templates, fonts, etc..

Slack channel
Make sure to follow the Slack channel community-day-organizers, where above many other stuff you can find a list of other community days, so you all got coordinated like not to schedule the community day in the same region on the same day, etc...

Also, in the same channel you can find information how to ask for funding - yes, AWS can provide some 💵 for you.😉

The event

Attendees estimation
This is pretty tricky, especially if you are doing it for the first time.

Try to look at:

- How big your community(s) is.

- How many people attend the meetup(s).

- How are much and how far are people willing to travel.

- How good your marketing was (will talk about that later).

Please be realistic and rather expect less and be surprised, than expect "summit style attendance" and be disappointed.

An example from us: Our Community Day was organized only by a single User Group having 200+ members and the meetups attendance is between 40 and 80.
The willing to travel is not that high.

So we started low, and thought that if highest meetup attendance was 80 out of 200, for a community day we can aim for 120 - 150 attendees (at the end we got 166).

This is almost pure alchemy 🤯 as there are other variables that comes into play like weather (during the storm you should expect less, during the super nice sunny weather probably as well, etc...), but some guesses can be done.

...and don't be surprised, if you see a registration boom on the last day(s) before the event starts. 😀

The venue
The venue should be selected based on the number of attendees you expect and have to choose the venue that can dynamically work with number of attendees.Let's say you estimated it to 150, so they (or you) must be capable to adapt the venue for 100 people and same for 200 people, by different type of seating.

Count at least +2 rooms more. You gonna need one room for storage which can be also used as your '3 minutes quiet&chill out room' (thank me later), another room should be reserved for the speakers.

Also make sure the expo won't be isolated too much from where people are gathered. This is not what you want - You want the people to interact with the sponsors. That said, it's not the best idea to have expo on the other floor than the sessions are. Ideally when people get out of the session, or going from one room to another they should cross the expo area. Good plan is to get the food and drink tables directly to the expo as well.

The catering
This is a full day conference, where people expect some refreshment but don't overthink it. Of course it depends on the eating habits in particular country, we did snack, lunch, snack.
Make sure to also put some refreshment to speakers room.

The tracks
Don't be the overthinker here - less is more. The more tracks or rooms you create, the less people you have in each. It's tempting to have 4-5 tracks in the same time, but really think about it before you do.
I must admit, we did a bad job in that. Expecting 150 people, we created 4 tracks which was not the best idea. Yes, venue can make them look that even with 40 people the 100-chair room looks almost full, but the people were complaining they had to do a hard decision to choose between the sessions they really wanted to attend.

This may lead you to another double edged sword - to stream or record the sessions. We decided not to do it, even if recording seems like a good idea for those who had to choose between the sessions. Maybe I am wrong, but if the sessions are recorded, what would make people to come?

What about the track format? It's up to you, but usually what I saw on previous community days or summits I attended, we choose 1 hour format per speaker

- 30 minutes session

- 15 minutes for Q/A after session

- 15 minutes break for another speaker to prepare and for attendees to walk the expo and have something to drink

It may seem like too generous time, but don't forget you have the sponsors out there at the expo, and they are expecting people to come.

With all the snack and lunch breaks, this is how our whole day looked like:

08:00: Start of the registrations

09:00 - 09:15: Organizers intro speech

09:15 - 10:00: Keynote

10:00 - 10:30: Snack break at the expo

10:30 - 11:15: Sessions slot 1

11:30 - 12:15: Sessions slot 2

12:15 - 13:00: Lunch at the Expo

13:00 - 13:45: Sessions slot 3

14:00 - 14:45: Sessions slot 4

14:45 - 15:15: Snack break at the expo

15:15 - 16:00: Sessions slot 5

16:20 - 16:30: Thank you from organizers

Planned start
This is very much dependent on when people used to start to work and how punctual they are. In Slovakia people usually start to work between 8am and 9am, and we are pretty punctual. But I can imagine in some countries 9am is pretty soon, so I would not plan keynote there.

We opened a registration at 8:00am, at 9:00 started a short welcome speech from the organizers, followed by the keynote at 9:15am When keynote started, more than 2/3 of the attendees were already there. Having a different habits, I would think about starting with one or two sessions, and then kick a keynote.

The Speakers
We believe in equal opportunities, so we tried to create a good mix between AWS employees, kickass experienced speakers from community and new speakers (everyone started somehow, and this is good opportunity). Also we tried to find balance between international and domestic speakers.
Make sure to communicate with speakers about their preferred time of their presentation (morning/afternoon).

Free or paid
The community day organizers are always dealing with this one... and there is no right or wrong way. Both have pros and cons.

Paid Event - Even symbolic price can reduce the no-shows (ratio between registered and the ones that actually showed-up) and increase the budget you get. But there is a chance you have to pay taxes, as you are creating the profit.

Free Event - Prepare yourself for a no-shows... 😬 It's frustrating, but it is what is is.

We decided to go free and we experienced about 40% no-shows.

Marketing

This is probably something we underestimated a lot. I think having proper marketing, would end up in more attendees. We received a lot of feedback that people knew about the even only by coincidence or from 'friend of a friend...'
Creating a linkedin group and meetup.com page is apparently not enough. Next year we will get more focus on that topic.

This is also something you can ask your sponsors to help you with.

Things you thought you never deal with, but you will 😂

How to get the money
You can't get the sponsorship money just like this (I wish I could🤣). For that you need some company, or civic association, or something similar. It's up to you, everything have pros and cons.

Organization team
It's up to you, but I would say for small community day 2-3 people may be enough. We started 2 people team, then we asked another friend to join us.

Volunteers
Volunteers are very helpful, at least for registering and other stuff too. Try to ask the sponsors if they can allocate some people for you, maybe for additional benefit or so.

Event manager
Same goes for event manager. If you can afford event manager, or sponsor is able to allocate one for you, by all means take it. Having an event manager, you don't have to deal with things like (which we had to deal with):

Badges: pre-printed or stickers?
We did not want to go the way to pre-print the badges with names. We rather ordered empty badges, and printed the stickers ourselves. The reason for that was that we were expecting some no-shows and also the emopty badges can be used next year. So we ordered the empty ones and just pre-printed the stickers with names of the attendees.

Printers
We had many discussions if to buy or borrow and at the end we decided to buy one, which we can use next years. The one that we voted for was Brother QL-820NWBc, because this is the one multiple computers can share.

Earlier I mentioned the speakers' room. Having a printer can solve the problem who should be allowed into the speakers' room. Marking speakers and organizers on their badges will make it easier, as on picture above.

Lanyards
This is also something you can get from the sponsor, but we didn't want to go that way. We wanted to distinguish between Speakers, Sponsors, Attendees and Organizers - and we did it with different lanyard colors: Red for organizers, Orange for Sponsors, Black for attendees and speakers. Same lanyards can be used next year if you have some left.

Some more advices at the end

Communication channel
This is a must have. For official announcements before the event, we used Slack with closed channel only for speakers and organizers.

We also created WhatsApp channel between speakers and organizers for quick updates during the day.

Sepparate WhatsApp channel between organizers and volunteers is also good idea.

Speakers' slides
Surprisingly (or maybe not 🤣), many of the attendees asked for a slides. Communicate that with speakers, and if they are ok with providing them, put them on the website after the event.

Speakers' dinner
Either sponsored, or paid by your budget - I definitely vote for yes. This is a great way to know your speakers, also they can meet each other before and have some food, drinks and a good time.

People being people 🫣
There is always someone not ok with something, requesting something, need something... Prepare for that. Even is you think you prepared everything, there is always something.😅

All being said, organizing AWS Community Day is a lot of fun, but also a hard work to do. It took us 6 months of work, from idea that we are doing that, to the actual event.

If you are still thinking if to do it or not - by all means we say Yes, go for it! 😉

I migrated my private Github repo to AWS CodeCommit

michal salanci — Sun, 25 Feb 2024 15:42:09 +0000

I am using GitHub a lot as my private and public repositories. Especially those private ones are used only as an "archive" of my files, with version control. So why not have it in AWS CodeCommit?

AWS CodeCommit
AWS CodeCommit is fully managed, highly available source control service that hosts private git repositories. Just like Github, data is encrypted in transit using SSH or HTTPS. There is also encryption at rest using AWS Key Management Service (AWS KMS). There is an option to use an AWS managed key for this encryption (by default), or to create and use your own customer managed key.
Behind the scene, AWS CodeCommit stores your repositories in Amazon S3 and Amazon DynamoDB and the data data is redundantly stored across multiple facilities.
To migrate the data from Github (or any other git service) to AWS CodeCommit, all you need is AWS Account.
Migrating to AWS CodeCommit keeps all your previous commits and branches.

Part 1 - GitHub repository
In this section, I will create the Github repo from scratch.
If you already have a GitHub repo, just skip this section and continue to Part 2.
Let's create some GitHub repo, do some commits and a new branch.

In your GitHUb account, navigate to Repositories and hit New.

Choose a name whatever you like, I chose 'myfilesbackup' and make sure the repo is private.

Once the Github repo is created, we can push our files there.
For start I created this simple file structure:

Let's initialize git:

Add the Github repository as a remote to your local repository.

Now you should finally add, commit and push your files to master branch.

Let's do some more commits. For start create another folder with some dummy file.

Another commit and push will do the job.

Let's make it more fun and create another branch, called development and switch to it.

Now let's create another file

I want this file to be pushed to branch development

So to summarize, we did 3 commits and 1 additional branch.
This is how it looks like in the Github repo:

Part 2 - AWS CodeCommit repository

You have to have an AWS account. If you don't, create one
https://aws.amazon.com/resources/create-account/

Once you have an AWS account, you need to create 2 (3) things:

AWS CodeCommit repo
AWS IAM user with CodeCommit credentials (or access key)
This is optional, but once you create AWS account, you can sign in as a root user. That approach is not the best way, thus you should creatale an IAM User with admin rights you can use to sign in to the console.

Let's presume you already have AWS account and can log in either as root or IAM User (this is more suggested), so let's create AWS CodeCommit repo and IAM User with CodeCommit credentials.

Create AWS CodeCommit repo
In the AWS account navigate to Developer Tools > CodeCommit > Repositories and hit Create repository

Fill in:

Name of the repo
Description (optional)
Choose AWS KMS key for encryption (AWS managed, or your own if you have it and want to use it). If you with to create your own AWS KMS key, this comes with additional cost. AWS Managed KMS key is provided for free.
Optinaly you can also enable Amazon CodeGuru reviewer for Java and Python, which is machine learning powered code reviewer. This may also come with additional cost.

Once the repository is created, you have 2 options how to clone it:

HTTPS
SSH

If you are signed as a root user, you only can use HTTPS, not SSH. Me personally prefer HTTPS, so I will choose this one.

Before we clone this repo, we need IAM user we will use to connect to AWS CodeCommit.

Navigate to IAM > Users > Create user and let's create IAM User we will use exclusively to connect to AWS CodeCommit.

Give it a name, click Next and then choose Attach policies directly.
From the filter menu, find AWSCodeCommitPowerUser policy, mark it and click Next > Creat User

This will give the IAM User enough permissions to pull, push, etc...

Once the user is created, we need to assign a credentials. Go inside the user, tab Security Credentials, where you have 2 options:

You can assign SSH key or HTTPS credentials valid only for AWS CodeCommit.
You can assign Security Credentials.

The difference is, that with AWS CodeCommit SSH key or HTTPS credentials, the user is only able to connect to AWS CodeCommit service, while user with Security Credentials can potentially connect to the AWS console, or CLI.
The less priviledge the better I say, so I choose AWS CodeCommit credentials.
As mentioned before, I personally prefer HTTPS over SSH, therefore I scroll down to HTTPS Git credentials for AWS CodeCommit and hit Generate credentials

This wil transfer you to a new window, where you can see those credentials.

I suggest you download them and store securely, because this is the only time you can see your password. Of course if you loose it, you can generate it again, or just reset the password.

Ok, so now that we have everything set up, let's push the repo to AWS CodeCommit cloned by HTTPS.

As first, pull the repo to make sure you are up to date.

Copy the repo link from HTTPS tab,:

and modify the git origin to that value:

You will be asked for username and password - that's the AWS CodeCommit HTTPS credentials you set up in AWS Console.

Once you add the credentials, the value of remote repo is modified to AWS CodeCommit.

We are now ready to push everything into AWS CodeComit repo.

All my previous commits and branches are now part of AWS CodeCommit repo

For some reason it made development branch the default, so I will change the default branch back to master.

In repository, navigate to Settings,

and scroll to Default branch, where you can change it to master.

Now we are fully migrated from Github to AWS CodeCommid.

Let's summarize the benefits:

This is not a challenge between Github and AWS CodeCommit, as each offers different benefits, but:

By defining the IAM user with CodeCommit credentials, you have full controll who can access the repo.
The data is in your account and cannot be accessed from another account or another user, if you don't specifically allow it.
The data is encrypted at rest with KMS key.
The repo can be easily integrated with other AWS services like EventBridge and SNS (can come with addional cost), so you are notified about every change to your repo (commit, pull, etc...).
You can have unlimited number of repositories.
No Size Limits on Repositories, aw AWS CodeCommit does not impose hard limits on repository sizes (unlike GitHub).
Free tier is available (see below).

Cost
Up to 5 active users, 50 GB-month of storage, and 10,000 Git requests per month is for free. So in most cases, your repo will be free all the time.

Conclusion
Creating and migrating the repo to the AWS CodeCommit is very easy. Migrating a GitHub repo to AWS CodeCommit can offer numerous benefits, especially for those already running the AWS ecosystem for its ability of integration with AWS services, scalability, and security features present a compelling case for teams looking to streamline their development workflows within AWS.

Running forward proxy in AWS

michal salanci — Sun, 24 Dec 2023 16:05:22 +0000

Hello friends, let me introduce you to our serverless forward proxy concept in AWS, which runs on AWS Network Firewall and Squid proxy in ECS container.

There will be upcoming articles soon, where I will dive deeper into setup of the AWS NFW and Squid in ECS, Cloudwatch logs, DNS setup with Dnsmasq, testing the network performance with K9, monitoring with Telegraf, etc...

Now let's see how the basic setup of forward proxy in AWS may look like.

Introduction to forward proxy

What is forward proxy and why we need it

Imagine you are in a corporate datacenter, or at home and you want to connect to a website in the internet. You send HTTP or HTTPS request to a website. Webserver process the request and responds with the payload.

This is how it should look like in the ideal world. However, you can unintentionally access a harmful website, risking exposure to malware or other security threats? To mitigate those risks, organizations often use an outbound filtering system known as a forward proxy.

A forward proxy acts as an intermediary solution between a user's device and the internet. It helps manage and control internet traffic, ensuring security and compliance.

It examines outgoing requests and filters the traffic based on pre-set rules. This could include checking the destination URL, IP address, or type of requested content. By doing so, the proxy ensures that only safe and compliant requests reach the internet, thereby enhancing security and privacy.

For instance, in a corporate environment, a forward proxy might block access to non-work-related websites, ensuring both network security and employee productivity.

When user creates a request, if the request complies with the rules, the proxy allows it to pass through to the internet. If not, it blocks the request, effectively preventing access to potentially harmful or non-compliant content.

Forward proxies can also anonymize web requests, hiding the user's IP address from external web servers. This adds a layer of privacy and security, protecting users from potential tracking or hacking.

Some forward proxies cache frequently accessed content. This means that if multiple users request the same resource, the proxy can serve it from its cache, reducing load times and saving bandwidth.

Explicit and Transparent proxy

Proxy can handle the traffic in two ways – as an explicit proxy or transparent proxy.

Below is the brief comparison of both:

Transparent proxy being invisible to users is actually a great security advantage, because explicit proxy can be bypassed simply by not specifying its address in the request, however user can’t bypass the transparent, as the requests are routed there by default.

Serverless forward proxy in AWS

Let’s imagine that customers managing their own VPC and are connecting to the internet via Outbound VPC, as a central point of internet access.

Outbound VPC is the place where egress connections can be secured and controlled and this is also the place where forward proxy operates.

The initial design is modified by introducing an inspection subnet, where all the magic happens.

AWS offers a native solution for transparent proxy – AWS Network Firewall.

Since there is no native solution for explicit proxy, 3rd party solution, such as Squid proxy can be used. It can be placed into the container and managed by AWS Fargate.
Let’s examine the components of the Inspection subnet in more detail.

Explicit forward proxy on Squid

As mentioned before, since there is no native AWS solution for explicit proxy, it is necessary to use some of the 3rd party solutions. This article aims to use of Squid Proxy.

Squid Proxy is widely used open source proxy solution. It can terminate the TCP and that makes it a perfect candidate for explicit proxy. It can run on EC2 instance, or in ECS container.

In this architecture, Squid runs in an ECS container, managed by AWS Fargate.

AWS Fargate is a compute engine for Amazon ECS, which allows you to run containers without having to manage servers or clusters. Fargate abstracts the underlying infrastructure management tasks such as provisioning, scaling, and maintaining servers, enabling you to focus on designing and building your applications.

When creating a Docker image for squid proxy, we used 3 main components:

urlwhitelist.txt – list of allowed URLs.
ipwhitelist.txt – list of allowed IP addresses.
squid.conf – configuration file of the Squid - this is where all the behavior (what is denied, what is allowed, caching, etc..) is defined.

In this particular scenario squid proxy configured like this:

Listens for HTTP and HTTPS traffic on port 3128 and enable SSL bumping for HTTPS traffic.
Blocks access to all destinations (URLs and/or IPs), except for what is allowed in the whitelist files.
Caches the content.

When user establish a HTTP/HTTPS request via explicit proxy this is what happens:

Since Squid is configured to operate as a proxy and is listening for incoming requests on port 3128.
Request is evaluated against the rules which determine if the requested URL is permitted. This decision is based on whether the URL is listed in the whitelist_URL.txt file.
If the requested URL is not whitelisted in urlwhitelist.txt file, the request is denied.
If the requested URL is whitelisted it is allowed further.
For allowed requests, Squid checks its cache. If a cached version of the requested resource is available, Squid will serve this content directly to the client.
If the requested content is not in the cache, Squid fetches the content from the destination web server and forwards it to the original client.
To the client, it appears as if it received the response directly from the web server, even though it was routed through Squid.

Combo with AWS Network Loadbalancer

For users to be able to successfully send HTTP/HTTPS request to the Squid container, another AWS component is necessary – AWS Network Load Balancer

ECS Tasks with Squid running inside as a container are part of NLB’s target group.

The purpose of AWS Network Loadbalancer is to listen to the traffic in front of the Squid and then redistribute the traffic to its targets – ECS Tasks running Squid.

This setup has several advantages:

Performance: NLB is designed to handle millions of requests per second while maintaining low latencies. It operates at Transport Layer (L4) of the OSI model, which allows them to efficiently route TCP traffic. This is particularly beneficial for a proxy server like Squid that handles a significant amount of TCP traffic.

High Availability and Reliability: The use of a Network Load Balancer ensures that traffic is distributed efficiently across available ECS Tasks. If one instance becomes unhealthy or fails, the NLB can redirect traffic to the remaining healthy instances, maintaining service availability. With that setup, we can have as many ECS containers as we need.

Running with sidecar

Putting the Squid container into an ECS Task, has another advantage – possibility of using a sidecar container.

A sidecar container is a design pattern where a secondary container is deployed alongside a primary application container, sharing the same lifecycle and resources, but performing a supporting function that's essential to the operation or management of the primary container.

As it turned out, logs created by Squid are not visible in the Cloud Watch, so some kind of a log processor is needed to parse the logs from Squid and send them to the Cloudwatch.

There are plenty of log processors available, however AWS supports and provides the Docker image of FluentBit log processor. Except for others, it includes plugins and configurations that are optimized for sending logs to CloudWatch.

Because ECS Task allows us to run multiple containers inside, FluentBit can now run as sidecar container, to gather the logs from Squid container and to send them to CloudWatch.

But how exactly Fluentbit gets the logs created by Squid?

Let’s examine the ECS topology in more detail:

Squid container and Fluentbit as a sidecar container are both part of same ECS Task.

ECS Tasks are part of ECS service, which is part of ECS Cluster. ECS Cluster spans through multiple Fargate instances.

For squid to be able to exchange the logs with fluentbit, some kind of a storage is needed. There are multiple options here, such as using EFS, or instance store. We decided to use instance store of particular Fargate instance, as it seems to be the simplest and most cost effective solution.

When squid created the log, it sends it immediately to the instane store of the Fargate instance it runs on. Fluentbit then reads the logs from the store, parse it to the appropriate format and forwards to Cloudwatch.

Please beware, that instance store is temporary – once the container dies and is redeployed in new Fargate instance, you loose all your data. However, this should not be a big concern, because once the logs are sent to the Cloudwatch, they stay there even if the instance store is gone.

Transparent forward proxy on AWS network firewall

Transparent proxy is also necessary, in case the users do not specify any proxy in the request. AWS provides a native solution for that – AWS Network Firewall.

AWS Network Firewall, introduced in 2020, is a managed firewall that primarily provides firewall protection for VPC resources in AWS. It's designed to provide stateful inspection of network traffic, intrusion detection and prevention, and web filtering.

AWS Network Firewall is able to inspect both ingress and egress traffic.

All its features are behind the scope of this article, but let’s just focus on some which are important for transparent proxy capabilities.

Stateful Inspection: AWS Network Firewall tracks the state of active connections and makes decisions based on the context of the traffic (not just the individual packets). It is able to inspect both inbound and outbound traffic.

Web Filtering: It can also block or allow access to specific websites or categories of websites.

Those 2 features are exactly what we need for AWS Network Firewall to act as a transparent proxy.

AWS Network Firewall consists of 3 main components

Firewall rule

Basic building component of network inspection behavior.
It defines the criteria to inspect and control the traffic, such as IP addresses, ports, protocols, etc…
Rules are grouped in the Rule Group

Firewall rule group

Collection of rules, organized into single manageable unit.
Can be stateful or stateless. Stateful rule groups can track the state of network connections, while stateless Rule groups treat each packet individually and independently.
Rule groups can be applied to Firewall policy.

Firewall Policy

Collection of one or more rule groups, organized into single manageable unit.
Organizes the order in which the rule groups are being evaluated and defines a default action (what happens if no rule is hit).

More on AW Network Firewall concepts can be found here:
https://aws.amazon.com/blogs/aws/aws-network-firewall-new-managed-firewall-service-in-vpc/
https://aws.amazon.com/de/blogs/networking-and-content-delivery/deployment-models-for-aws-network-firewall/

Setting up AWS Network Firewall for transparent proxy

In Firewall policy, the default order in the stateful rule group is Strict, and the default action is Alert established + Drop all

Let’s break it down:

Drop all + Alert established:

Drop all: Any traffic that doesn't match any of the rules in the stateful rule group, will be dropped. This is kind of implicit deny at the end of the ruleset.
Alert established: While network firewall drops traffic not matching the allow rules, it will specifically log (alert) the traffic that is part of an already established connection. An established connection is part of already ongoing session, when 3-way TCP handshake is done. It does not log the TCP 3-way handshake itself, instead it logs traffic that occurs after the TCP is correctly established.

Strict rule ordering – when firewall finds a match in the rule of the rulegroup, no further evaluation is done and the action defined in the rule is taken

When user creates a HTTP/HTTPS request via transparent proxy this is what happens:

Request is evaluated against rules in the rulegroups. The decision is based on whether it finds a match in any of the rules or not.
If request matches any of the rules, appropriate action defined in that rule is taken.
If request does not match any of the rules, the default action is taken (Drop all) and request is dropped.
There are no caching possibilities in network firewall.

Routing and network flow

Once everything is set up, let’s check the routing and network flow of explicit and transparent proxy

Explicit proxy network flow

When user wants to reach www.amazon.com while usage explicit proxy is required, the proxy address must be specified in the request. In this case, the network loadbalancer DNS acts as a proxy address.

User creates request to www.amazon.com, from EC2 10.0.1.130, while specifying network loadnalncer DNS name in the request - internal-fwdproxynlb-1234567890-eu-central-1.elb.amazonaws.com and port 3128.
DNS name of the loadbalancer is translated to its IP address 192.168.3.10 – which is now the destination IP address of the packet.
Based on the default route in the user’s VPC, traffic is sent to AWS transit gateway.
In transit gateway, there is a route to 192.168.0.0/16, towards transit gateway attachment in private subnet of Outbound VPC.
From Outbound VPC private subnet, the traffic gets to network loadbalancer, based on a local route.
Network loadbalancer makes a loadbalancing decision and picks up one of the members of its target group, to send packets to. This is actually an ECS Task. NLB preserves the client's source IP, so the Squid inside the ECS Task sees the original source IP - 10.0.1.130.
In ECS Task, the packet is evaluated against the urlwhitelist.txt, and if allowed, squid terminates the initial request, and creates a new one. Now the source IP address is ECS Task IP – 192.168.2.28 and destination is www.amazon.com. There is a default route towards the NAT gateway, so the packet is sent there.
NAT gateway performs source NAT from 192.168.2.28 to its own public IP 3.48.29.55 and sends it to the internet gateway.
Internet gateway sends it to the destination.
When destination responds, and packet gets back to the internet gateway, it is sent back to NAT Gateway.
In NAT gateway the destination IP is changed back to 192.168.2.28 and on a local route the packet gets back to ECS Task and the Squid inside. Squid forwards the response back to network loadbalancer.
Network loadbalancer knows the client IP and based on the route 10.0.0.0/16 in the routing table, the packet is sent to transit gateway.
Transit gateway checks its routing tables and finds a route to 10.0.0.0/16 towards its attachment in private subnet of client VPC.
Once packet reaches private subnet of client VPC, by local route it gets back to client’s EC2.

Transparent proxy network flow

When user wants to reach www.amazon.com and no proxy is specified, it automatically goes via transparent proxy.

User creates request to www.amazon.com, from EC2 10.0.1.130.
Based on the default route in the user’s VPC, traffic is sent to AWS Transit Gateway.
From transit gateway, the packets is sent to the transit gateway attachment in private subnet of Outbound VPC.
From there, based on the default route it gets to AWS network firewall.
Traffic is inspected against the firewall rules, and if allowed, based on the default route it gets to NAT gateway.
NAT gateway performs source NAT from 10.0.1.130 to its own public IP 3.48.29.55 and sends it to the internet gateway.
Internet gateway sends it to the destination.
When destination responds, and packet gets back to the internet gateway, it is sent back to NAT Gateway.
In the NAT gateway the destination IP is changed back to 10.0.1.130. NAT gateway knows the route for 10.0.0.0/16, so response packet is sent to network firewall.
In network firewall the response packet is evaluated against the rules and if allowed, based on the routing it is sent to transit gateway.
Transit gateway checks its routing tables and finds a route to 10.0.0.0/16 towards its attachment in private subnet of client VPC.
Once packet reaches private subnet of client VPC, by local route it gets back to client’s EC2.

Conclusion

As we conclude this comprehensive exploration of forward proxies, it's clear that these tools are very important.

Forward proxies play a critical role in enhancing network security, regulating internet traffic, and ensuring compliance with organizational policies. Their ability to filter, monitor, and control access to web resources is vital in protecting against cyber threats.

Whether it's a explicit proxy running in container, or transparent proxy in AWS Network Firewall, these solutions are tailored to address a broad spectrum of security and compliance requirements.

We've seen that explicit proxies offer more control and detailed traffic inspection, making them ideal for environments requiring stringent security measures.

Transparent proxies, on the other hand, provide ease of use and maintenance, making them suitable for basic filtering and routing without needing end-user configuration.
The integration of forward proxies within the AWS VPC, such as using Squid inside the ECS container managed by Amazon Fargate, for explicit forward proxy or leveraging AWS Network Firewall for transparent forward proxy, showcases the versatility and scalability of AWS ecosystem.

How I became cloudbased from being cloudless, in 2022

michal salanci — Sun, 24 Dec 2023 14:40:12 +0000

This article was originally published on 2023/01/22 on my wix blog.
As I am shutting down the blog, all my articles are being moved here.

If you are considering shifting your career in the direction of AWS, this article may be an inspiration to you.

Having worked as an AWS DevOps engineer since December 2021 I would like to encourage all of you who are still doubtful to make a change.

This is my story of how I got from cloudless to cloudbased.
I am old school networking guy, for my whole career I worked with different kinds of networks and datacenter technologies – routers, switches, loadbalancers, and firewalls. I had built quite a successful career there and a get into the great team of colleagues. One might say it was an ideal job. Well, not quite – I felt that I was missing something. For the past years, I witnessed my customers leaving DC for AWS.

Master Shifu once said:

If you only do things you can do, you can never be more than you are.

Amen to that, bro!

Until 2021 I had no knowledge about AWS...

But how difficult can it be, right?

I said to myself…

I was wrong and I learned it the hard way.

You may ask yourself a question – why should I learn AWS?

Well, let me tell you:

AWS is one of the biggest cloud providers.
You will have the opportunity to work with the latest technology.
There is a high potential for career growth because there is a high demand for AWS professionals.
Getting an AWS job requires a set of skills and certifications that will help you a lot as well.

WLNSC is all you need

Have you heard about the WLNSC method? The shame on you if not! (Don't worry, I made it up).

WLNSC is the abbreviation for what I have started with and it worked pretty well.

Let's get step by step with the WLNSC method that has no copyright.

W for Will to make a change
This is the first step you have to make – find a will to start. Learning new technology is never easy. It costs time, stepping out of your comfort zone, and maybe a couple of dollars (you better stop that EC2 after you are done with it).

L for Lab to practice
One can't learn something without practicing. Lucky for us, AWS provides a lot of free resources. You just need to create an AWS account – don’t worry, it’s free. AWS also provides a lot of free resources to your AWS lab.

N for Networking with other professionals
There are a lot of inspiring people who can help you, without even knowing you. Networking with other AWS professionals can be a great way to learn new things and stay up-to-date with the latest developments in the platform. Just go and check the profiles of Viktoria Semaan, Linda Haviv, Madhu Kumar, Artur Schneider and many more, whose profiles are full of interesting ideas, good tips, tricks, etc…

You will also get information about AWS meetups, conferences, and other networking events that you can attend to meet even more inspiring professionals.

S for Support from people around
If you are not the lucky one with a photographic memory, there will be some sacrifices, you have to understand that. Learning something new and learning it good needs takes time.
I used to exercise in the morning before work and watch series with my wife in the evening when the kids went to bed. Instead of that for a good amount of time, I was exercising the lab and watching AWS Skill Builder, Coursera, Udemy...

But trust me every minute is worthy.

C for Certification
I found that the best way (for me) to learn AWS is by learning and practicing for AWS certifications.

AWS Certified Cloud Practitioner for starter

Checking the certification path on the AWS page and as a knower of nothing (sorry Jon Snow), I decided to start with AWS Certified Cloud Practitioner.

Lucky me, I found a great and free essentials training on Coursera, created by AWS - AWS Cloud Practitioner Essentials. AWS Instructors Morgan Willis, Blaine Sundrud and Rudy Chetty are explaining the essentials of AWS in a very understandable way – comparing AWS to a coffee shop. If you are completely new to that field, I definitely suggest this course to start with.

AWS also provides tons of free trainings. Login to AWS Skill Builder, Coursera, create a free account and start learning for free. I definitely recommend AWS Cloud Practitioner Essentials and AWS Cloud Quest: Cloud Practitioner, but there are more.

I passed this certification in April 2021 with pretty good score, and suddenly there was me thinking how good I am. If you haven’t heard about Dunning–Kruger effect, this is exactly the book example.

AWS Certified Architect Associate for main course
Feeling like Po the Dragon Warrior, I just started to prepare for AWS Certified Architect Associate and that was a real deal. I've spent evenings and evenings labing and watching the content (my wife had almost finished 6 seasons of a TV show).

This time I decided to go not just with AWS Skill Builder, but also with the learning platform Udemy. I purchased Ultimate AWS Certified Solutions Architect Associate SAA-C03 from Stéphane Maarek. The topics I found most crucial, like IAM, EC2, S3, VPC, and others I dove deeper into with specific courses on AWS Skill Builder.

Somewhere in the middle of the preparation, I found out that the AWS DevOps team within my company is hiring, I applied and was accepted. With a good attitude and a new role in my pocket, I was able to pass.

Specialties for dessert
If you’re still not full and thinking about some desserts (like lava cake right after 1kg of ribs you think to order just because your teammate ordered it too, even if you are fuller than you have ever been – ain’t that right Lydia Delyova ?), there is nothing better than Specialties.

AWS offers multiple specialties. Working for years with BGP, VPNs, and IP subnets, first logical choice for me was the AWS Advanced Networking Specialty, and I must admit this certification was pretty doable, with all my networking backround. Without that, the exam might be pretty though.

For my passion for security, I also took the AWS Security Specialty, and I can tell you this was the most challenging one for me.

End of story?
Going through all of this, I encourage you to do the same if you are still considering. Getting from classic DC networking, or any other field to the AWS is a huge change, but I can assure you it's worthy.

What will however never change, is you still being that hey, my PC is so slow, can you do something about it? and also hey, can you set up my wireless router kind of guy for the whole your family, friends, neighbors, their friends…

I wish I had a dollar for every router I have set up…

This is not the end and the story continues. Let's see what 2023 will bring.

And what should your next steps be? Make the step and start the unexpected journey to the clouds.