DEV Community: Carlos Armando Marcano Vargas

Analyzing Network Traffic With Wireshark

Carlos Armando Marcano Vargas — Wed, 03 Apr 2024 02:05:57 +0000

Previously, I published an article about analyzing network traffic with Tcpdump. In this article, we are going to analyze network traffic using Wireshark. We will learn what Wireshark is, how to install it, and how to analyze network traffic by reading a pcap file and applying filters.

The content of this article is limited by what I learned about Wireshark in the Google Cybersecurity course. So, this is not comprehensive material about Wireshark.

What is Wireshark?

According to the Wireshark official page:

Wireshark is a network protocol analyzer. It lets you capture and interactively browse the traffic running on a computer network. It has a rich and powerful feature set and is one worlds most popular tool of its kind. It runs on most computing platforms including Windows, macOS, Linux, and UNIX.

Installing Wireshark

To install Wireshark, we visit this page and select one of the options available according to our OS.

I have Windows 11, so I have to select "Windows x64 Installer".

Exploring a pcap file

After installing Wireshark, we open the application and explore a pcap file.

We can download a sample pcap file from these websites.

I downloaded a sample for this website.

If you want to use the same files I used to follow along, these are the ones I'm used for these examples:

arp_pcap.pcapng.cap

Open Network Connection.pcapng.cap

Next, we open a sample with Wireshark. I will open first the "arp_pcap.pcapng.cap" file.

As we can see in the image above, Wireshark shows seven columns or attributes.

"No." is the index number of the packet in this packet capture file. Time is the packet's timestamp. Source is the source IP address. Destination , The destination IP address. Protocol is the protocol contained in the packet. Length is the total length of the packet. Info, some information about the data in the packet (the payload) as interpreted by Wireshark.

Using filters

Now, we can use the search bar in Wireshark to apply filters.

I will search all the packets that contain the IP address "192.168.1.110" no matter if it is the source or destination. So, we need to write this text in the search bar.

ip.addr == 192.168.1.110

If we double-click on any packet, we can have more detailed information about it.

If we click on Frame , which shows details about the overall network packet, including the packet's length and arrival time.

Then, we click on Ethernet II , this subtree contains details about the packet at the Ethernet level, including the source and destination MAC addresses and the type of internal protocol the Ethernet packet contains.

Next, we click on Internet Protocol Version 4, this section contains information about the source and destination IP addresses and the Internal Protocol (for example, TCP or UDP), which is carried inside the IP packet.

In the case of the file we are analyzing, the packet shows User Datagram Protocol, UDP, and this section contains information about UDP ports. It will show Transmission Control Protocol if the file that is being analyzed contains packets using a TCP connection.

If we want to be more specific with our search, we can use the filter "ip.src" to look for an IP address as source or "ip.dst" to look for an IP address as destination.

ip.src == 153.16.31.81

ip.dst == 147.83.131.33

We can use a filter to look for a specific MAC address too.

eth.addr == 30:b5:c2:eb:4c:b0

Also, we can use a filter to examine DNS traffic.

DNS is the system used to resolve store information about domain names including IP addresses, mail servers, and other information.

I will open a different pcap file ("The open connection" file) for this task.

udp.port == 53

We can select any entry, scroll down, and double-click on Domain Name System (query).

We click on "Queries" and in the case of the file I am analyzing, we can see the queried website (cl.qustodio.com).

If we click on the "Answers" section, we can see the address associated with the website (50.17.253.86).

Finally, in this article, we will examine the traffic of a TCP port.

We use the filter:

tcp.port == 80

TCP port 80 is the default port that is associated with web traffic.

We select any entry. And double-click on "Internet Protocol Version 4". We scroll down and see the packet's Time To Live value, it indicates the maximum amount of time a packet can exist on the network before being discarded.

In this section, we can see the Header Length , the total size of the header section, in this case, 20 bytes.

Clicking on the "Frame" section, we can see the Frame Length , which is the total size of the captured network packet.

Conclusion

So far, I have tried TCPdump and Wireshark. Both are amazing, useful, and widely used tools. However, the GUI that Wireshark offers makes analyzing network data more comfortable, in my opinion. If you want to try a packet sniffer with a GUI, that is widely used in the industry, I recommend Wireshark.

Thank you for taking the time to read this article.

If you have any recommendations about other packages, architectures, how to improve my code, my English, or anything; please leave a comment or contact me through Twitter, or LinkedIn.

Resources

What is Wireshark?

Sample Captures

Get Started with TCPdump: Capture and Analyze Network Traffic

Carlos Armando Marcano Vargas — Mon, 11 Mar 2024 01:06:14 +0000

In this article,, we will explore what is Tcpdump and how to use some of its basic features like listing interfaces available, capturing data packets, and reading packet data files.

I recently started to learn how to use TCPdump as a part of the Google Cybersecurity course. And I thought it was a good idea to write and share what I learned, and also to have material to revisit in the future.

All the console screenshots are from a virtual machine I used in a lab that was part of the Cybersecurity course, this means that the information that is printed out probably will be different from yours. Also, many of the explanation about some features and flags of TCPdump comes from the course.

Requirements

A Linux distro installed.
WSL2 for Windows users.

What is Tcpdump?

Tcpdump is a Command-Line Interface or CLI to capture network traffic. It is used to identify malicious activity in a network and to identify issues in a network. We can capture network traffic and saved it to a packet capture (p-cap), a file containing data packets intercepted from an interface or network.

Identifying interfaces

In Tcpdump we use the -D flag to list all the interfaces available for packet capture.

We open a console or WSL console for Windows users, and execute the following command:

sudo tcpdump -D

After we execute the command above, the console should show the following list:

Now, we will capture the packets from the eth0 interface, but not all of them, just five packets of data.

sudo tcpdump -i eth0 -v -c5

The -i option is used to select an interface, in this case, the eth0 interface.

The -v means "Verbose" and it prints out detailed information about the packet data. And the -c option allows us to control how many packets we want tcpdump to print out.

In the example data at the start of the packet output, tcpdump reported that it was listening on the eth0 interface, and it provided information on the link type and the capture size in bytes:

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes

On the next line, the first field is the packet's timestamp, followed by the protocol type, IP:

22:24:18.910372 IP

The verbose option, -v, has provided more details about the IP packet fields, such as TOS, TTL, offset, flags, internal protocol type (in this case, TCP (6)), and the length of the outer IP packet in bytes:

(tos 0x0, ttl 64, id 5802, offset 0, flags [DF], proto TCP (6), length 134)

Now, let's see what it means in the snippet above:

The tos 0x0 represents the Type of Service (ToS) field in the IP header. In this case, 0x0 indicates the default ToS, with no specific priority or options set.

The ttl 64 , time to Live (TTL) field in the IP header. It specifies how many hops (routers) a packet can traverse before being discarded. 64 is a commonly used value for local networks.

id 5802 this is the identification number of the TCP packet. It helps differentiate different packets belonging to the same TCP connection. offset 0 indicates that the data portion of the packet starts immediately after the header, with no header options present.

The flags [DF] shows the flags set in the TCP header. The [DF] flag specifically indicates that the Don't Fragment (DF) flag is set, meaning the packet should not be fragmented if it doesn't fit the Maximum Transmission Unit (MTU) of any network device along the path. proto TCP (6) specifies the transport layer protocol used, which in this case is TCP (Transmission Control Protocol). And length 134 is the total length of the TCP packet, including the header and data.

In the next section, the data shows the systems that are communicating with each other:

7acb26dc1f44.5000 > nginx-us-east1-c.c.qwiklabs-terminal-vms-prod-00.internal.59788:

By default, tcpdump will convert IP addresses into names, as in the screenshot. The name of my Linux virtual machine also included in the command prompt, appears here as the source for one packet and the destination for the second packet. In your live data, the name will be a different set of letters and numbers.

The direction of the arrow (>) indicates the direction of the traffic flow in this packet. Each system name includes a suffix with the port number (.5000 in the code snippet) which is used by the source and the destination systems for this packet.

The remaining data filters the header data for the inner TCP packet:

Flags [P.], cksum 0x5851 (incorrect > 0x30d3), seq 1080713945:1080714027, ack 62760789, win 501, options [nop,nop,TS val 1017464119 ecr 3001513453], length 82

The flags field identifies TCP flags. In this case, the P represents the push flag and the period indicates it's an ACK flag. This means the packet is pushing out data.

The next field is the TCP checksum value, which is used for detecting errors in the data.

This section also includes the sequence and acknowledgment numbers, the window size, and the length of the inner TCP packet in bytes.

Capturing network traffic for further analysis

sudo tcpdump -i eth0 -nn -c9 port 80 -w capture.pcap &

This command will run tcpdump in the background with the following options:

-i eth0: Capture data from the eth0 interface.
-nn: Do not attempt to resolve IP addresses or ports to names. This is best practice from a security perspective, as the lookup data may not be valid. It also prevents malicious actors from being alerted to an investigation.
-c9: Capture 9 packets of data and then exit.
port 80: Filter only port 80 traffic. This is the default HTTP port.
-w capture.pcap: Save the captured data to the named file.
&: This is an instruction to the Bash shell to run the command in the background.

Now, we will use curl, to make a request and see what Tcpdump will print it out.

curl opensource.google.com

Next, we make sure that Tcpdump captures the data packets.

ls -l capture.pcap

As we can see in the screenshot above, the packet capture was successful and it was written in the capture.pcap file for further analysis.

Conclusion

In this article, we just explore a little bit about Tcpdump, how to identify interfaces, print out packets data, capturing and reading packet data file. Tcpdump is an amazing tool and very useful for anyone interested in learning about networks or cybersecutiry.

Resources

Adding A Logging Middleware To A Robyn App | Python

Carlos Armando Marcano Vargas — Wed, 24 Jan 2024 00:16:08 +0000

In this article we are going to build a logging middleware with Robyn. So, every time a request is made to the server, it will show the log in our console.

The scoop of this article is not to create a robust and complete logging system, but a simple logging middleware for learning purpose.

Requirements

Python 3 installed
Pip installed

Creating the Middleware

We create a new python project as usual:


mkdir robbyn-logging
py -m venv venv

#Windows
cd venv/Scripts
./activate
pip install robyn

First, we are going to create a middleware class with the attributes of the request information and the response information.

The request information will show the IP address, path, URL, HTTP method and request time. The response information, will show the status code, and the response type.

Let's start with the middleware that shows the request information.

app.py


from robyn import Robyn, Request, Response
from robyn.logger import Logger
from datetime import datetime
app = Robyn(__file__)

logger = Logger()

class LoggingMiddleware:

    def request_info(request: Request):
        ip_address = request.ip_addr
        request_url = request.url.host
        request_path = request.url.path
        request_method = request.method
        request_time = str(datetime.now())


        return {
            "ip_address": ip_address,
            "request_url": request_url,
            "request_path": request_path,
            "request_method": request_method,
            "request_time": request_time,

        }

In the app.py file, we import and create an instance of the Robyn class. Then, we create the LoggingMiddleware class with the request_info method, which returns the attributes of the request.

Now, let's add a route to make a request. And use before_request() method, which registers a function that runs before each request.

...
@app.before_request()
def log_request(request: Request):

    logger.info(f"Received request: %s", 
        LoggingMiddleware.request_info(request))

    return request

@app.get("/")
def h():
    return "Hello, World!"

app.start(port=8080)

Now, we restart the server and make a request to localhost:8080/ to see the response in the console.

Let's go with response data.

We add another method to the LoggingMiddleware class, but this one is with the response information.

class LoggingMiddleware:

...

    def response_info(response: Response):
        status_code = response.status_code
        response_type = response.response_type

        return {
            "status_code": status_code,
            "response_type": response_type
        }

Next, use the after_request() method to register the log_response() function. This function will show the status code of the response, and the response type.

The after_request() method register a function that runs after a request.

...
@app.after_request()
def log_response(response: Response):

    logger.info(f"Sending response: %s", 
        LoggingMiddleware.response_info(response))
    return response

@app.get("/")
def h():
    return "Hello, World!"

app.start(port=8080)

Now, we restart the server and make a request to localhost:8080/ to see the response in the console.

Complete code

from robyn import Robyn, Request, Response
from robyn.logger import Logger
from datetime import datetime
app = Robyn(__file__)

logger = Logger()

class LoggingMiddleware:

    def request_info(request: Request):
        ip_address = request.ip_addr
        request_url = request.url.host
        request_path = request.url.path
        request_method = request.method
        request_time = str(datetime.now())

        return {
            "ip_address": ip_address,
            "request_url": request_url,
            "request_path": request_path,
            "request_method": request_method,
            "request_time": request_time,

        }

    def response_info(response: Response):
        status_code = response.status_code
        response_type = response.response_type

        return {
            "status_code": status_code,
            "response_type": response_type
        }


@app.before_request()
def log_request(request: Request):

    logger.info(f"Received request: %s", 
        LoggingMiddleware.request_info(request))

    return request

@app.after_request()
def log_response(response: Response):

    logger.info(f"Sending response: %s", 
        LoggingMiddleware.response_info(response))
    return response

@app.get("/")
def h():
    return "Hello, World!"

app.start(port=8080)

Conclusion

My intention in this article was not to make a robust logging system or feature, but to show how to use the middleware feature of Robyn to create a simple logging middleware.

Thank you for taking the time to read this article.

If you have any recommendations about other packages, architectures, how to improve my code, my English, or anything; please leave a comment or contact me through Twitter, or LinkedIn.

Resources

Robyn Documentation

2023 Retrospective: A Year Of Personal Growth

Carlos Armando Marcano Vargas — Sun, 31 Dec 2023 23:19:23 +0000

Started the 2023 with the goal of publishing at one article per week. It would be 52 articles. And I accomplished this goal.

Every year we set goals and have expectations. I want a goal that would be under my control, because I didn't want to finished this year feeling like a could not accomplished anything. It was up to me to write and read every day to publish 52 by the end of the year. At first was easy because I was motivated, but it takes discipline and consistency to reach this goal. I struggled with the motivation, and lack of ideas. A lot of my articles were reused to write another articles, for example, I wrote many tutorials about Prometheus, I write an article for using it with Python, one for Go, and for Rust too.

When doing this I felt like a cheater, the only though that comfort me was that probably one of that articles was useful for someone else.

My consistency was put at a test when I found a job. I started to work at mid of November. And it was difficult to me to find time and energy to write an article, and because mostly what I write are tutorials, I have to find time to write code and test the programs for the articles.

I learned a lot by writing articles in this year. I learned about how to use Prometheus, Grafana, Docker, RabbitMQ, Apache Pulsar, Ktor. I started to learned about cybersecurity, a topic I want to write about in the 2024.

Also I continue to be more involved in contributing to Open Source projects. That was another goal that it takes discipline and consistency, but totally worth it. I'm very grateful to all the contributors and maintainers for helping me when I struggled with an issue, thanks to them I feel I did grow a lot this year as a developer. Especial thanks to the Robyn community for been so nice and welcoming.

I didn't find a job as a software developer, but that is something that is not totally under my control. What is actually under my control is to continue to apply for jobs and continue learning, to grow as a software developer and as a person.

I want to thanks to those who read my articles. Their likes, comments and feedback encourage me to continue to keep writing and help me to be a better writer each day.

This was a short article, a brief summary about how my 2023 was. Thanks for reading this article.

Happy New Year.

Carlos Armando Marcano Vargas

Adding A Rate Limiter Middleware To A Robyn Server | Python

Carlos Armando Marcano Vargas — Sun, 31 Dec 2023 01:47:51 +0000

In this article, we are going to learn how to add a rate limiter middleware to a Robyn server through robyn-rate-limits, a plugin developed by IdoKendo.

Requirements

Python 3 installed
Pip installed

Rate Limiter

According to this article, published by CloudFlare, a rate limiter is a strategy for limiting network traffic. It puts a cap on how often someone can repeat an action within a certain timeframe for instance, trying to log in to an account. Rate limiting can help stop certain kinds of malicious bot activity. It can also reduce strain on web servers.

Rate limiting is often employed to stop bad bots from negatively impacting a website or application. Bot attacks that rate limiting can help mitigate include Brute force attacks, DoS and DDoS attacks and Web scraping.

Rate limiting also protects against API overuse, which is not necessarily malicious or due to bot activity, but is important to prevent nonetheless.

Robyn Rate Limiter Plugin

According to the documentation, the rate limiter plugin enables you to implement rate limiting for your Robyn application's routes. It helps prevent abuse, and brute-force attacks and ensures fair usage of your resources.

To use this plugin, we have to install it first:

 pip install robyn robyn-rate-limits

We create a python file, app.py to create a Robyn server.

app.py

from robyn import Robyn, Robynfrom robyn_rate_limits import InMemoryStorefrom robyn_rate_limits import RateLimiterapp = Robyn( __file__ )limiter = RateLimiter(store=InMemoryStore, calls_limit=3, limit_ttl=100)@app.before_request()def middleware(request: Request): return limiter.handle_request(app, request)@app.get("/")def h(): return "Hello, World!"app.start(port=8080)

In the example above (taken from the documentation), the robyn-rate-limits plugin is used to enforce a rate limit of 3 requests per 100-seconds window for specific routes. If a client exceeds this limit, they will receive a "Too many requests" message.

We start our server as usually do, by running py app.py command in our console.

Now, we have to create a python script, to test the rate limiter.

test_limiter.py

For this script we have to import the requests library.

import requestsimport timedef test_rate_limiter(url, rate_limit, time_window): """Tests a rate limiter with the given parameters.""" start_time = time.time() allowed_requests = 0 blocked_requests = 0 while time.time() - start_time < time_window: try: response = requests.get(url) response.raise_for_status() # Raise an exception for non-200 status codes allowed_requests += 1 except requests.exceptions.RequestException as e: blocked_requests += 1 print(f"Request blocked: {e}") time.sleep(time_window / rate_limit) # Delay to respect the rate limit print(f"Allowed requests: {allowed_requests}") print(f"Blocked requests: {blocked_requests}")# Example usage:url = "http://localhost:8080/" # Replace with the actual URLrate_limit = 3 # Requests per secondtime_window = 5 # Secondstest_rate_limiter(url, rate_limit, time_window)

In the example above, we define the test_rate_limiter() function with three parameters: url, rate_limit and time_window.

This program makes requests to an URL and tracks allowed and blocked requests. response.raise_for_status() is if the status code in the response is different from a successful status code(200), it will raise an exception. And it will print the exception, in our case, it should be a "Too many request" message. time.sleep pauses execution to respect the rate limit.

We set the url to the URL address where our Robyn server is receiving requests, rate_limit to 3, and the time_window to 5.

We run this program, executing this command in another terminal: py test_limiter.py.

Now, we should receive the following response in our console:

Conclusion

This is an easy to use and an useful plugin that allow us to build more resilient services with Robyn. Also, this plugin has a feature that uses Redis to store the calls, you can take a look to the plugin's repository for an example of how to use it.

Thank you for taking the time to read this article.

If you have any recommendations about other packages, architectures, how to improve my code, my English, or anything; please leave a comment or contact me through Twitter, or LinkedIn.

Resources

robyn-rate-limits repository

robyn-rate-limits documentation

What is rate limiting

What Is a Rate Limiter?

Building a Visitor Tracker With Robyn and React| Python

Carlos Armando Marcano Vargas — Mon, 18 Dec 2023 01:23:08 +0000

In this article, we are going to build a Visitor Tracker with Robyn and React.

This is not a new project, I have built the same project using FastAPI and Rails. But, I want to build this app with Robyn to show features that I have not used in previous demos, like middleware, the request object, and the CORS. Also, to show the use of the response object.

Requirements

Python installed
Basic Python knowledge
Pip installed
Postgres installed
NodeJs installed
Basic React Knowledge

Building the Visitor Tracker

First, we create a directory for this application.

mkdir visitor_trackercd#Windows userspy -m venv venvcd venv/Scripts./activate#Linuxpython3 -m venv venvsource venv/bin/activate

We install all the dependencies we need.

pip install robyn psycopg2-binary python-dotenv

Inside the visitor_tracker directory, we create a new file, init_db.py.

init_ db.py

import osimport psycopg2from dotenv import load_dotenvload_dotenv()PASSWORD = os.getenv('PASSWORD')def get_db_connection(): conn = psycopg2.connect( dbname = "logs_db", user = "postgres", password = PASSWORD ) return connconn = get_db_connection()cur = conn.cursor()cur.execute('DROP TABLE IF EXISTS logs;')cur.execute('CREATE TABLE logs (id serial PRIMARY KEY,' 'ip_address varchar (150) NOT NULL,' 'request_url varchar (50) NOT NULL,' 'request_path varchar (50) NOT NULL,' 'request_method varchar (50) NOT NULL,' 'request_time timestamp (50) NOT NULL,' 'date_added date DEFAULT CURRENT_TIMESTAMP);' )cur.execute('INSERT INTO logs (ip_address,' 'request_url,' 'request_path,' 'request_method,' 'request_time)' 'VALUES (%s, %s,%s, %s, %s)', ('127.0.0.1', 'http://localhost:8000', '/', "GET", "2023-06-25T16:03:24.722256", ))conn.commit()cur.close()conn.close()

Here we set up a database connection to store log data. First, we load the .env file using dotenv to get the database username and password variables. Then, we define a get_db_connection() function that establishes a connection to a PostgreSQL database named logs_db. Then, the code calls that function to get a database connection and cursor.

In this file, also the code drops the logs table if it exists and recreates it with the given schema - with columns to store IP address, request URL, port, path, method, time etc. It inserts sample log data into the table with its values. It commits the changes to the database and closes the cursor and connection.

helpers.py

import collectionsdef to_dict(psycopg_tuple:tuple): tracker = collections.OrderedDict() tracker['id'] = psycopg_tuple[0] tracker["ip_address"] = psycopg_tuple[1] tracker["request_url"] = psycopg_tuple[2] tracker["request_path"] = psycopg_tuple[3] tracker["request_method"] = psycopg_tuple[4] tracker["request_time"] = psycopg_tuple[5].strftime("%d-%m-%Y, %H:%M:%S") return trackerdef list_dict(rows:list): row_list = [] for row in rows: book_dict = to_dict(row) row_list.append(book_dict) return row_list

This file has two functions: to_dict() and list_dict(). The to_dict() function converts a PostgreSQL tuple to a dictionary. The list_dict() function converts a list of PostgreSQL tuples to a list of dictionaries.

The to_dict() function takes a PostgreSQL tuple as input and returns a dictionary. The dictionary contains the values of the tuple in the same order as the tuple. The list_dict() function takes a list of PostgreSQL tuples as input and returns a list of dictionaries. The dictionaries are created using the to_dict() function.

controllers.py

from init_db import get_db_connectionfrom helpers import to_dict,list_dictimport jsondef all_logs(): conn = get_db_connection() cur = conn.cursor() cur.execute('SELECT * FROM logs;') logs = list_dict(cur.fetchall()) cur.close() conn.close() return logsdef new_log(ip_address: str, request_url: str, request_path: str, request_method: str, request_time: str,): conn = get_db_connection() cur = conn.cursor() cur.execute('INSERT INTO logs (ip_address, request_url, request_path, request_method, request_time)' 'VALUES (%s, %s, %s,%s, %s) RETURNING *;',(ip_address, request_url, request_path, request_method, request_time, )) log = cur.fetchone()[:] log_dict = to_dict(log) conn.commit() cur.close() conn.close() return json.dumps(log_dict)

The all_logs() function gets all logs from the database and returns a list of dictionaries. Each dictionary contains information about a single log.

The new_log() function inserts a new log into the database.

middleware.py

from robyn.robyn import Requestfrom datetime import datetimeclass Tracker: def visitor_tracker(request: Request): ip_address = request.ip_addr request_url = request.url.host request_path = request.url.path request_method = request.method request_time = str(datetime.now()) return { "ip_address": ip_address, "request_url": request_url, "request_path": request_path, "request_method": request_method, "request_time": request_time, }

Here we create the Tracker class and define the visitor_tracker function.

The visitor_tracker function returns the attributes: ip_address, request_url, request_path, request_method, request_time.

app.py

from robyn import Robyn, status_codesfrom robyn.robyn import Response, Headers, Requestfrom controllers import all_logs, new_logfrom robyn import loggerfrom middleware import Trackerimport jsonapp = Robyn( __file__ )@app.before_request()async def log_request(request: Request): tracker = Tracker.visitor_tracker(request=request) new_log(tracker["ip_address"], tracker["request_url"], tracker["request_path"], tracker["request_method"], tracker["request_time"]) logger.info(f"Received request: %s", tracker) return request@app.get("/")async def hello(): return Response(status_code=status_codes.HTTP_200_OK,headers=Headers({}), description="Hello, World!")@app.get("/visitors")async def hello(): logs = all_logs() return Response(status_code = status_codes.HTTP_200_OK, headers=Headers({}), description = json.dumps(logs))app.start(port=8000, host="0.0.0.0")

Next, in the app.py file we define a middleware that intercepts all the requests using the before_request() decorator. This middleware will store the request's IP address and other attributes, and it will create an instance of the Tracker class. Then it will store the data in the database.

Also, here we define two routes: "/" and "/visitors". In the hello() function we retrieve all the logs in the database or the visitors' information and show them using the Response class. The Response class requires status_code, headers and description as positional arguments.

Building a UI

We will create a React app to visualize the tracking information in a table.

Adding CORS

app.py

from robyn import Robyn, status_codes, ALLOW_CORSfrom robyn.robyn import Response, Headers, Requestfrom controllers import all_logs, new_logfrom robyn import loggerfrom middleware import Trackerimport jsonapp = Robyn( __file__ )ALLOW_CORS(app,["*"])...

In Robyn, we import ALLOW_CORS and it needs the instance of Robyn, in this case, the app variable and the origins. I use ["*"] to allow requests from any origin.

Installing Vite and React

In our command line, we install Vite with a React-Typescript template. This command line will create a folder for the project.

#npmnpm create vite@latest table -- --template react-ts#yarnyarn create vite@latest table --template react-ts#pnpmpnpm create vite@latest table --template react-ts

After all the packages are installed we run Vite with the command:

npm run dev

We navigate to localhost:5173 and we should see the Vite and React homepage.

app.ts

import React, { useState, useEffect} from "react";const url = "http://localhost:8000/visitors";interface Table { id: number, ip_address: string, request_url: string, request_path: string, request_method: string, request_time: string,}const Table: React.FC = () => { const [data, setData] = useState<Table[]>([]); useEffect(() => { fetch(url) .then(res => res.json()) .then(data => setData(data)); console.log(data); }, []); return ( <div> <h1>Logs</h1> <table> <thead> <tr> <th>Id</th> <th>IP Address</th> <th>Request URL</th> <th>Request Path</th> <th>Request Method</th> <th>Request Time</th> </tr> </thead> <tbody> {data.map((item, index) => ( <tr key={index}> <td>{item.id}</td> <td>{item.ip_address}</td> <td>{item.request_url}</td> <td>{item.request_path}</td> <td>{item.request_method}</td> <td>{item.request_time}</td> </tr> ))} </tbody> </table> </div> );};export default Table;

We start the Robyn server and the Vite server simultaneously. And then, we navigate to localhost:5173, we should see the following page:

Let's add a .css file to add style and see the data easily.

index.css

:root { font-family: Inter, system-ui, Avenir, Helvetica, Arial, sans-serif; line-height: 1.5; font-weight: 400; color-scheme: light dark; color: rgba(255, 255, 255, 0.87); background-color: #242424; font-synthesis: none; text-rendering: optimizeLegibility; -webkit-font-smoothing: antialiased; -moz-osx-font-smoothing: grayscale; -webkit-text-size-adjust: 100%;}table { border-collapse: collapse; margin-bottom: 1rem;}th,td { padding: 0.5rem; border: 1px solid #ccc;}th { text-align: left;}td { text-align: left;}.column-gap-10 { column-gap: 10px;}

Conclusion

In this article, we learned how to build a visitor tracker with Robyn and React. We use Robyn's middleware to intercept and store the information of the requests. Also, we learned how to allow CORS in Robyn to make it possible to integrate with any frontend framework.

The source code is here.

Thank you for taking the time to read this article.

If you have any recommendations about other packages, architectures, how to improve my code, my English, or anything; please leave a comment or contact me through Twitter, or LinkedIn.

Resources

Robyn documentation

4 Open Source Packet Analyzers To Explore

Carlos Armando Marcano Vargas — Sat, 09 Dec 2023 23:23:52 +0000

Recently I started a course about cybersecurity and honestly, I find the topic very interesting. One of the modules of the course talks about packet analyzers to inspect the traffic in a network.

So, I started to look for the most used Packet Analyzers in the market and found Tcpdump and Wireshark. But I wanted to find out if there are any others. I didn't find many alternatives, but two others like Sniffnet and Arkime.

Also, I want to make clear that I'm new to this topic, and I'm not an expert or a cybersecurity professional.

What is a packet analyzer?

As it says on this Wikipedia page, a packet analyzer, also known as a packet sniffer, protocol analyzer, or network analyzer, is a computer program or computer hardware such as a packet capture appliance that can analyze and log traffic that passes over a computer network or part of a network. Packet capture is the process of intercepting and logging traffic.

Tcpdump

According to this Wikipedia page, tcpdump is a free data-network packet analyzer command line interface. That allows the user to display TCP/IP and other packets being transmitted or received over a network to which the computer is attached.

Wireshark

Wireshark is one of the world's most used network protocol analyzers. It lets you see what's happening on your network at a microscopic level. Sometimes it is the de facto standard across many industries and educational institutions.

Wireshark is a project that started in 1998 and is continuously developing thanks to the contributions of networking experts across the globe.

Sniffnet

Sniffnet is a completely free and open-source desktop application written in Rust using Iced. This a relatively new project that started in 2022.

Features:

Choose a network adapter for your PC to inspect
Select a set of filters to apply to the observed traffic.
View overall statistics about your Internet traffic.
View real-time charts about traffic intensity (bytes and packets per second, incoming and outgoing).
Get details about domain names and network providers of the hosts you are exchanging traffic with.
Identify connections in your local network.
Get information about the country of the remote hosts (IP geolocation).
Save your favorite network hosts.
Set custom notifications to inform you when defined network events occur.
Choose the style that fits you the most from 4 different available themes.
Inspect each of your network connections in real time.
Save complete textual reports with detailed information for each network connection:

Arkime

Arkime is an open-source, large-scale, full packet capturing, indexing, and database system written in Javascript.

As its README says:

Arkime exposes APIs which allow for PCAP data and JSON formatted session data to be downloaded and consumed directly. Arkime stores and exports all packets in standard PCAP format, allowing you to also use your favorite PCAP ingesting tools, such as wireshark, during your analysis workflow.

Arkime is built to be deployed across many systems and can scale to handle tens of gigabits/sec of traffic.

Conclusion

This was not an extensive list, and probably there a more open source packet analyzers. Still, the ones in this article are really good options to explore if you are interested in cybersecurity and forensics.

Resources

Arkime README

TCPdump Documentation

WireShark Documentation

Sniffnet README

tcpdump Wikipedia page

Packet Analyzer

Exploring Format: The New Feature of Ruff | Python

Carlos Armando Marcano Vargas — Tue, 28 Nov 2023 02:17:10 +0000

Ruff introduces the format feature for the v0.1.2. So right now Ruff, not only is a linter but also a formatter. In this short article, we are going to learn how to use the formatter.

Requirements

Python installed

The Ruff Formatter

As the documentation says, the Ruff formatter is an extremely fast Python code formatter designed as a drop-in replacement for Black, available as part of the ruff CLI (as of Ruff v0.0.289).

If you already have a previous version of Ruff installed, run this command in your console:

pip install --upgrade ruff

For anyone who doesn't have Ruff installed, run this command:

pip install ruff

Now, to use the formatter feature in a Python file, we run the ruff format /path/to/file.py command, and it will format the file.

To format all the files in a directory, we run the ruff format . command.

Example

I have this code in a main.py file:

log = new_log(tracker["ip_address"], tracker["request_url"], tracker["request_port"],
                      tracker["request_path"], tracker["request_method"],
                      tracker["browser_type"], tracker["operating_system"],tracker["request_time"])

Then, we run the following command:

ruff format main.py

Conclusion

The formatter is an incredible addition to Ruff, it helps us to write more readable and maintainable code. Now, we have a fast linter and formatter.

Resources

Ruff Formatter

Building A Visitor Tracker with Rails

Carlos Armando Marcano Vargas — Mon, 20 Nov 2023 23:20:28 +0000

In this article, we are going to build a visitor tracker app using Rails. The app will intercept the requests and show information like the client's IP, URL, path, and request method.

Requirements

Ruby installed
Rails installed

Creating a new Rails project.

First, we need to create a new rails project.

rails new visitor-tracker

Creating the model, database, controllers, and routes.

rails g resource tracker

We go to db/migrate/<timestamp>create_trackers.rb to define the attributes of the tracker table. There will be eight attributes: IP address, request URL, port, path, method, browser, OS, time and the service name.

class CreateTrackers < ActiveRecord::Migration[7.0]
  def change
    create_table :trackers do |t|

      t.timestamps
      t.string :ip_address
      t.string :request_url
      t.integer :request_port
      t.string :request_path
      t.string :request_method
      t.string :browser_type
      t.datetime :request_time
    end
  end
end

We run the following command in our terminal, to migrate the table:

rails db:migrate

We go to app/controllers/tracker_controller.rb and create the functions to retrieve all the entries in the database, and to store the request info in the database.

Index

  def index
        @trackers = Tracker.all
        render json: @trackers
    end

Show

 def show
        @tracker = Tracker.find(params[:id])
        render json: @tracker
    end

Create

   def create
        @tracker = Tracker.create(
            ip_address: params[:ip_address],
            request_url: params[:request_url],
            request_port: params[:request_port],
            request_path: params[:request_path],
            request_method: params[:request_method],
            browser_type: params[:browser_type],
            request_time: params[:request_time],

        )
        render json: @tracker
    end

Complete trackers_controllers.rb file.

class TrackersController < ApplicationController
    def index
        @trackers = Tracker.all
        render json: @trackers
    end

    def show
        @tracker = Tracker.find(params[:id])
        render json: @tracker
    end

    def create
        @tracker = Tracker.create(
            ip_address: params[:ip_address],
            request_url: params[:request_url],
            request_port: params[:request_port],
            request_path: params[:request_path],
            request_method: params[:request_method],
            browser_type: params[:browser_type],
            request_time: params[:request_time],

        )
        render json: @tracker
    end    
end

In db/seed.rb we created a record to display when we make the GET request.

tracker_1 = Tracker.create(ip_address: "127.0.0.1", 
request_url: 'http://localhost:8000', request_port: 8000, request_path: "/",
request_method: "GET")

Then we run the following command to apply the changes.

rails db:seed

Now, let's start our server and see if it can show the data.

rails server

We navigate to localhost:port/trackers and we should see the following response in the browser or HTTP client:

Adding a middleware

We create a new folder, lib/middleware and create the lib/middleware/visitor_tracker file.

module Middleware
    class VisitorTracker
      def initialize(app)
        @app = app

      end

      def call(env)
        request = ActionDispatch::Request.new(env)

        Tracker.create(ip_address: request.ip, 
        request_url: request.original_url, request_port: request.port, request_path: request.path,
        request_method: request.request_method)

        @app.call(env)
      end
    end  
end

Here, we create our middleware, create the VisitorTracker class and create two methods, initialize and call.

In the call method, we create an instance of the ActionDispatch::Request class and then, create a new record in the database with the request information every time the middleware is executed.

Then, we go to config/application.rb file to declare our middleware.

require_relative "boot"
require_relative "../lib/middleware/visitor_tracker"

require "rails/all"

Bundler.require(*Rails.groups)

module Visitor
  class Application < Rails::Application

    config.load_defaults 7.0
    config.middleware.use Middleware::VisitorTracker

  end
end

Here we use the config.middleware.use method and pass the Middleware module and the VisitorTracker class. Rails will execute this middleware just before your application is called.

Now, to create a full-stack app, we need to go to config/routes.rb and paste the following code.

Rails.application.routes.draw do
  resources :trackers

  root "trackers#index"
  get "/trackers", to: "trackers#index"
end

Then, we need to go to the app/controllers/trackers_controllers.rb file and remove the code render json: @trackers from the index function. It should look like this:

def index
        @trackers = Tracker.all

end

Next, we go to the app/views/trackers folder and create the app/views/trackers/index.html.erb file.

<!DOCTYPE html>
<html>
    <head>
        <title>Visitor Tracker</title>
        <meta name="viewport" content="width=device-width,initial-scale=1">
        <%= csrf_meta_tags %>
        <%= csp_meta_tag %>

        <%= stylesheet_link_tag "index", "data-turbo-track": "reload" %>
        <%= javascript_importmap_tags %>
    </head>
    <h1>Visitor Tracker</h1>
    <table>
        <tr>
            <th>IP Address </th>              
            <th>Request URL </th>                     
            <th>Request Port </th>  
            <th>Request Path </th>  
            <th>Request Method </th>

        </tr>
        <% @trackers.each do |tracker| %>
        <tr>
            <td><%= tracker.ip_address %></td>         
            <td><%= tracker.request_url %></td>     
            <td><%= tracker.request_port %></td>
            <td><%= tracker.request_path %></td>
            <td><%= tracker.request_method %></td>

        </tr>
        <% end %>
    </table>

</html>

We navigate to localhost:port/ and we should see the following response in the browser or HTTP client:

Next, we create the app/assets/stylesheets/index.css file and paste the following code.


table {
    border-collapse: collapse;
    margin-bottom: 1rem;
  }

  th,
  td {
    padding: 0.5rem;
    border: 1px solid #ccc;
  }

  th {
    text-align: left;
  }

  td {
    text-align: left;
  }

  .column-gap-10 {
    column-gap: 10px;
  }

Conclusion

In this article, we learned about how to build a visitor tracker with Rails and how to create custom middleware. With this app, we can see the activity of our applications.

The source code is here.

Thank you for taking the time to read this article.

If you have any recommendations about other packages, architectures, how to improve my code, my English, or anything; please leave a comment or contact me through Twitter, or LinkedIn.

Resources

Getting Started with Rails

Using Rails and HTML to View, Sort, and Search Tables.

Rails Middleware: (Almost) Everything You Need to Know

Go RabbitMQ: Integrating RabbitMQ into Your Go Applications

Carlos Armando Marcano Vargas — Tue, 14 Nov 2023 00:02:49 +0000

In this article, we are going to learn how to integrate RabbitMQ with a Go server.

We are going to the net/http package in this article. We are going to build two servers, one that will publish messages to the RabbitMQ queue and another to consume from it.

Also, we are just using the code examples from the RabbitMQ Go tutorial and integrating them into a Go server.

I will use the same disclaimer from the RabbitMQ tutorial:

Please keep in mind that this and other tutorials are, well, tutorials. They demonstrate one new concept at a time and may intentionally oversimplify some things and leave out others. For example, topics such as connection management, error handling, connection recovery, concurrency and metric collection are largely omitted for the sake of brevity. Such simplified code should not be considered production ready.

Requirements

Go installed
RabbitMQ installed

Publisher

Here, we are going to develop a server that will publish messages to the queue. First, we have to install the RabbitMQ Go client.

go get github.com/rabbitmq/amqp091-go

Then, we create the main.go file.

package main

import (
    "context"
    "log"
    "log/slog"
    "net/http"
    "time"

    amqp "github.com/rabbitmq/amqp091-go"
)

func main() {

    mux := http.NewServeMux()
    port := ":8000"

    mux.Handle("/", http.HandlerFunc(homeHandler))

    slog.Info("Listening on ", "port", port)

    err := http.ListenAndServe(port, mux)
    if err != nil {
        slog.Warn("Problem starting the server", "error", err)
    }

}

func homeHandler(w http.ResponseWriter, r *http.Request) {

    w.Write([]byte("Hello this is a home page"))
    rabbitmqConnection()

}

func failOnError(err error, msg string) {
    if err != nil {
        log.Panicf("%s: %s", msg, err)
    }
}

func rabbitmqConnection() {
    conn, err := amqp.Dial("amqp://guest:guest@localhost:5672/")
    failOnError(err, "Failed to connect to RabbitMQ")
    defer conn.Close()
    ch, err := conn.Channel()
    failOnError(err, "Failed to open a channel")
    defer ch.Close()

    q, err := ch.QueueDeclare(
        "hello", // name
        false, // durable
        false, // delete when unused
        false, // exclusive
        false, // no-wait
        nil, // arguments
    )
    failOnError(err, "Failed to declare a queue")

    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    body := "Hello World!"
    err = ch.PublishWithContext(ctx,
        "", // exchange
        q.Name, // routing key
        false, // mandatory
        false, // immediate
        amqp.Publishing{
            ContentType: "text/plain",
            Body: []byte(body),
        })
    failOnError(err, "Failed to publish a message")
    log.Printf(" [x] Sent %s\n", body)

}

Here we create a new HTTP multiplexer (mux) and sets the listening port to 8000. It defines a route handler (homeHandler) for the root path /.

The homeHandler() function writes a simple message ("Hello this is a home page") to the response writer and then calls the rabbitmqConnection function to connect to RabbitMQ and publish a message. So, every time the route "/" is hit, it will publish the message defined in the rabbitmqConnection() function.

The failOnError() function checks for an error and logs a panic message if the error is not nil.

The rabbitmqConnection() function establishes a connection to the RabbitMQ server using the amqp.Dial function. It creates a channel (ch) and declares a queue named "hello". Then, it publishes a message with the body "Hello World!" to the "hello" queue.

The main() function creates the web server using the http.ListenAndServe function and starts listening on port 8000.

receiver.go

Now, we create a program that consumes the messages from the queue and prints them in the console.

This is the same code from the RabbitMQ Go tutorial.

package main

import (
    "log"

    amqp "github.com/rabbitmq/amqp091-go"
)

func failOnError(err error, msg string) {
    if err != nil {
        log.Panicf("%s: %s", msg, err)
    }
}

func main() {
    conn, err := amqp.Dial("amqp://guest:guest@localhost:5672/")
    failOnError(err, "Failed to connect to RabbitMQ")
    defer conn.Close()

    ch, err := conn.Channel()
    failOnError(err, "Failed to open a channel")
    defer ch.Close()

    q, err := ch.QueueDeclare(
        "hello", // name
        false, // durable
        false, // delete when unused
        false, // exclusive
        false, // no-wait
        nil, // arguments
    )
    failOnError(err, "Failed to declare a queue")
    msgs, err := ch.Consume(
        q.Name, // queue
        "", // consumer
        true, // auto-ack
        false, // exclusive
        false, // no-local
        false, // no-wait
        nil, // args
    )
    failOnError(err, "Failed to register a consumer")

    var forever chan struct{}

    go func() {
        for d := range msgs {
            log.Printf("Received a message: %s", d.Body)
        }
    }()

    log.Printf(" [*] Waiting for messages. To exit press CTRL+C")
    <-forever
}

Here we declared the name of the queue("hello") from this program will consume the messages, using the ch.QueueDeclare method. The parameters specify that the queue is not durable (won't persist across server restarts), not exclusive (can be consumed by multiple consumers), and not auto-delete (won't be deleted when the last consumer unsubscribes).

The code registers a consumer using the ch.Consume method. It specifies the queue name (q.Name), an empty consumer tag (allowing RabbitMQ to generate a unique tag), true for automatic acknowledgment (messages are automatically acknowledged after processing), and false for other parameters. The method returns a channel (msgs) that receive incoming messages.

A goroutine is started to consume messages from the msgs channel. It iterates over the messages, printing their bodies to the console.

The main function logs a message indicating that it's waiting for messages and starts a forever channel to block the main thread. It prevents the program from exiting immediately and allows the consumer loop to continue running.

The <-forever statement waits for a signal to be sent to the forever channel, which will happen when the user presses CTRL+C. This allows the program to exit gracefully when interrupted.

Consumer

Here, we create the server that consumes from the queue. We have to install the RabbitMQ client too.

package main

import (
    "log"
    "log/slog"
    "net/http"

    amqp "github.com/rabbitmq/amqp091-go"
)

func failOnError(err error, msg string) {
    if err != nil {
        log.Panicf("%s: %s", msg, err)
    }
}

func main() {
    mux := http.NewServeMux()
    port := ":8080"

    mux.Handle("/", http.HandlerFunc(homeHandler))

    slog.Info("Listening on ", "port", port)

    err := http.ListenAndServe(port, mux)
    if err != nil {
        slog.Warn("Problem starting the server", "error", err)
    }

}

func homeHandler(w http.ResponseWriter, r *http.Request) {

    body := rabbitmqConsumer()
    w.Write([]byte(body))

}

func rabbitmqConsumer() string {
    conn, err := amqp.Dial("amqp://guest:guest@localhost:5672/")
    failOnError(err, "Failed to connect to RabbitMQ")
    defer conn.Close()

    ch, err := conn.Channel()
    failOnError(err, "Failed to open a channel")
    defer ch.Close()

    q, err := ch.QueueDeclare(
        "hello", // name
        false, // durable
        false, // delete when unused
        false, // exclusive
        false, // no-wait
        nil, // arguments
    )
    failOnError(err, "Failed to declare a queue")
    msgs, err := ch.Consume(
        q.Name, // queue
        "", // consumer
        true, // auto-ack
        false, // exclusive
        false, // no-local
        false, // no-wait
        nil, // args
    )
    failOnError(err, "Failed to register a consumer")

    forever := make(chan string)
    go func() {
        for d := range msgs {
            log.Printf("Received a message: %s", d.Body)
            forever <- string(d.Body)

        }
    }()

    bodyMessage := <-forever

    return bodyMessage

}

Here, we just copy the same code from the receiver.go and wrap it in the rabbitmqConnection() function. This function will be called everytime the route "/" is called and will show the body of the message in the browser or HTTP Client.

Conclusion

In this article, we learn how to integrate RabbitMQ into a Go server. RabbitMQ is a powerful messaging broker that can be seamlessly integrated with Go applications to handle asynchronous message communication.

By utilizing RabbitMQ, Go servers can effectively decouple components, enhance scalability, and ensure reliable message delivery.

The Publisher's source code is here.

The Consumer's source code is here.

Thank you for taking the time to read this article.

If you have any recommendations about other packages, architectures, how to improve my code, my English, or anything; please leave a comment or contact me through Twitter, or LinkedIn.

Resources

RabbitMQ Go tutorial

AMQP091 README

Building a Simple Rate Limiter Middleware in Go using Gin

Carlos Armando Marcano Vargas — Mon, 06 Nov 2023 15:52:44 +0000

In this article, we are going to build a Rate Limiter middleware for a Gin server.

We will build a very simple rate limiter, one that stops accepting requests when these exceed the limit in a certain time frame. We Will not create a rate limiter that keeps track of the IP Addresses and stops them from making too many requests.

Rate Limiter

Rate limiting also protects against API overuse, which is not necessarily malicious or due to bot activity, but is important to prevent nonetheless.

Requirements

Go installed

Rate Limiter Algorithms

Fixed window algorithm: This algorithm maintains a counter of the number of requests made within a fixed time window, such as one minute. If the counter exceeds the rate limit, subsequent requests are rejected.
Sliding window algorithm: This algorithm is similar to the fixed window algorithm, but the time window slides forward as new requests are received. This means that requests that were made earlier in the window may still be counted against the rate limit, even if they are no longer within the current window.
Leaky bucket algorithm: This algorithm maintains a bucket that fills up at a constant rate. When a request is received, it is allowed to pass through the bucket if there is space available. If the bucket is full, the request is rejected.
Token bucket algorithm: This algorithm is similar to the leaky bucket algorithm, but instead of a bucket, it maintains a counter of tokens. Tokens are added to the counter at a constant rate, and requests are allowed to pass through if there is at least one token available. If the counter is empty, requests are rejected.

Simple Rate Limiter

We created a new Go project, RateLimiter. And installed Gin.

$ go get -u github.com/gin-gonic/gin

For this rate limiter we are going to use the rate package. Which implements the token bucket algorithm.

And in the main.go file, we start creating a handler that will be used to limit the number of requests made to the server.

main.go

package main

import (

    "github.com/gin-gonic/gin"

    "golang.org/x/time/rate"
)

func RateLimiter() gin.HandlerFunc {
    limiter := rate.NewLimiter(1, 4)
    return func(c *gin.Context) {

        if limiter.Allow() {
            c.Next()
        } else {
            c.JSON(http.StatusTooManyRequests, gin.H{
                "message": "Limite exceed",
            })
        }

    }

This is the RateLimiter() function, we create a NewLimiter instance that allows up to 4 requests per second. According to the documentation, the NewLimiter function returns a new Limiter, func NewLimiter(r Limit, b int) *Limiter and a Limiter controls how frequently events are allowed to happen. It implements a "token bucket" of size b, initially full and refilled at the rate of r tokens per second.

The zero value is a valid Limiter, but it will reject all events. Use NewLimiter to create non-zero Limiters.

The Allow() method of the rate limiter returns a boolean value indicating whether the request is allowed. If the request is allowed, the Next() method of the gin.Context object is called to continue processing the request. Otherwise, the JSON() method of the gin.Context object is called to send a response with the status code 429 (Too Many Requests).

func main() {
    r := gin.Default()
    r.Use(RateLimiter())
    r.GET("/", func(c *gin.Context) {
        c.JSON(http.StatusOK, gin.H{
            "message": "pong",
        })
    })
    r.Run() // listen and serve on 0.0.0.0:8080 (for windows "localhost:8080")
}

In the main() function, we create a Gin instance and use the Use() function to register the RateLimiter() middleware.

The RateLimiter() handler is called every time a request is made to the server.

In our command line, we run this file using go run main.go command.

To test this rate limiter, we created another project, TestLimiter. And use the following code in its main.go file.

package main

import (
    "fmt"
    "io/ioutil"
    "net/http"
    "time"
)

func main() {
    c := http.Client{Timeout: time.Duration(1) * time.Second}
    limit := 20

    for i := 1; i < limite; i++ {
        resp, err := c.Get("http://localhost:8080/")
        if err != nil {
            fmt.Printf("Error %s", err)
            fmt.Println("Count is: ", i)
            return
        }
        defer resp.Body.Close()
        body, err := ioutil.ReadAll(resp.Body)
        fmt.Printf("Body : %s", body)
    }
}

In this file, we define the c variable, which is a http.Client object that is used to make HTTP requests. The limit variable is an integer variable that specifies the number of times the Get() method will be called.

Then we created a loop that iterates limit times. In each iteration of the loop, the code makes a Get() request to the URL http://localhost:8080/. The Get() method returns an *http.Response object, which contains the response from the server.

The code then checks the err variable to see if there was an error in making the request. If there is an error, the code prints the error message and the current value of the i variable. Otherwise, the code reads the body of the response and prints it to the console.

We run this file, using go run main.go in another terminal.

The Rate Limiter terminal should show these messages:

And the TestLimiter terminal should show these messages:

Conclusion

In this article, we built a rate limiter for a Gin server. Building a rate limiter for your Gin server is an important step in protecting your application from abuse. By limiting the number of requests that can be made to your server, you can help improve the performance and reliability of your application for all users.

The source code is here.

Thank you for taking the time to read this article.

If you have any recommendations about other packages, architectures, how to improve my code, my English, or anything; please leave a comment or contact me through Twitter, or LinkedIn.

Resources

What is rate limiting

What Is a Rate Limiter?

Rate limiting your Go application

Rate package documentation

Seven Alternatives To FastAPI To Explore and Contribute To | Python

Carlos Armando Marcano Vargas — Mon, 30 Oct 2023 23:56:28 +0000

First of all, there is nothing wrong with FastAPI. FastAPI is a good web framework. I have used it many times in my hobby projects and works fine.

This article is about the alternatives of FastAPI for anyone who wants to try another web framework, for the sake of exploring a new software. To anyone who finds joy in trying new things or different things. Also, if you want to contribute to Open Source Software and you are interested in web frameworks.

Also, this is a biased article, because in the list below are seven Python web frameworks that we can use as FastAPI alternatives, and I'm excluding from this list Flask and Django. Not because they are bad, but because I think they are well known.

Robyn

Robyn is a fast async Python web framework coupled with a web server written in Rust.

Features:

Authentication and Authorization
Middlewares
CORS
WebSockets
Subrouters
Views
Templating with Jinja2

Example:

pip install robyn

We create the app.py file and paste the following code.

from robyn import Robyn

app = Robyn( __file__ )

@app.get("/")
async def h(request):
    return "Hello, world!"

app.start(port=8080)

Then, we run the following command in the console:

python3 app.py

Documentation

Repository

Discord

Blacksheep

BlackSheep is an asynchronous web framework for building event-based web applications with Python. It is inspired by Flask, ASP.NET Core, and the work of Yury Selivanov.

It has a similar syntax as FastAPI and also it could be used with Uvicorn. As the documentation says, it has been tested to use Hypercorn as an ASGI HTTP server too.

According to its documentation, BlackSheep offers:

A rich code API, based on dependency injection and inspired by Flask and ASP.NET Core
A typing-friendly codebase, which enables a comfortable development experience thanks to hints when coding with IDEs
Built-in generation of OpenAPI Documentation, supporting version 3, YAML, and JSON
A cross-platform framework, using the most modern versions of Python
Good performance

Example:

pip install blacksheep uvicorn

Create the server.py file and paste the following code:

from datetime import datetime
from blacksheep import Application

app = Application()

@app.route("/")
def home():
    return f"Hello, World! {datetime.utcnow().isoformat()}"

Then, we run the following command in the console:

uvicorn server:app --port 44777 --reload

Documentation

Repository

Gitter

LiteStar

Litestar is a powerful, flexible, highly performant, and opinionated ASGI framework.

The Litestar framework supports:

Plugins
Ships with dependency injection
Security primitives
OpenAPI schema generation
MessagePack, middlewares
A great CLI experience

Example:

pip install litestar[standard]

We create the app.py file and paste the following code:

from litestar import Litestar, get

@get("/")
async def index() -> str:
    return "Hello, world!"

@get("/books/{book_id:int}")
async def get_book(book_id: int) -> dict[str, int]:
    return {"book_id": book_id}

app = Litestar([index, get_book])

Then, we run the following command in the console:

litestar run
# Or you can run Uvicorn directly:
uvicorn app:app --reload

Documentation

Repository

Discord

Sanic

Sanic is a Python 3.8+ web server and web framework thats written to go fast. It allows the usage of the async/await syntax added in Python 3.5, which makes your code non-blocking and speedy.

Sanic intentionally aims for a clean and unopinionated feature list. The project does not want to require you to build your application in a certain way and tries to avoid prescribing specific development patterns. Several third-party plugins are built and maintained by the community to add additional features that do not otherwise meet the requirements of the core repository.

However, to help API developers, the Sanic organization maintains an official plugin called Sanic Extensions to provide all sorts of goodies, including:

OpenAPI documentation with Redoc and/or Swagger
CORS protection
Dependency injection into route handlers
Request query arguments and body input validation
Auto create HEAD, OPTIONS, and TRACE endpoints
Predefined, endpoint-specific response serializers

Example:

pip install sanic

We create the server.py file and paste the following content:

from sanic import Sanic
from sanic.response import text

app = Sanic("MyHelloWorldApp")

@app.get("/")
async def hello_world(request):
    return text("Hello, world.")

Then, we run the following command in the console:

sanic server

Documentation

Repository

Discord

Falcon

Falcon is a reliable, high-performance Python web framework for building large-scale app backends and microservices. It works with any WSGI server, encourages the REST architectural style, and tries to do as little as possible while remaining highly effective.

Falcon tries to do as little as possible while remaining highly effective.

ASGI, WSGI, and WebSocket support
Native asyncio support
No reliance on magic globals for routing and state management
Stable interfaces with an emphasis on backwards-compatibility
Simple API modeling through centralized RESTful routing
Highly-optimized, extensible code base
Easy access to headers and bodies through request and response objects
DRY request processing via middleware components and hooks
Strict adherence to RFCs
Idiomatic HTTP error responses
Straightforward exception handling
Snappy testing with WSGI/ASGI helpers and mocks
CPython 3.5+ and PyPy 3.5+ support

Example:

pip install falcon, uvicorn

We create a server.py file and paste the content:

import falcon
import falcon.asgi

# Falcon follows the REST architectural style, meaning (among
# other things) that you think in terms of resources and state
# transitions, which map to HTTP verbs.
class ThingsResource:
    async def on_get(self, req, resp):
        """Handles GET requests"""
        resp.status = falcon.HTTP_200 # This is the default status
        resp.content_type = falcon.MEDIA_TEXT # Default is JSON, so override
        resp.text = (
            '\nTwo things awe me most, the starry sky '
            'above me and the moral law within me.\n'
            '\n'
            ' ~ Immanuel Kant\n\n'
        )

# falcon.asgi.App instances are callable ASGI apps...
# in larger applications the app is created in a separate file
app = falcon.asgi.App()

# Resources are represented by long-lived class instances
things = ThingsResource()

# things will handle all requests to the '/things' URL path
app.add_route('/things', things)

Then, we run the following command in the console:

uvicorn server:app

Documentation

Repository

Emmett

Emmett is a full-stack Python web framework designed with simplicity in mind.

Emmett aims to be understandable, easy to learn and to be used, so you can focus completely on your product's features.

Features:

Routing system
Templating system
Websockets
Validations
Forms generations
Sesions
Authorization system
Caching
ORM

Example:

pip install emmett

We create an app.py file and paste the following code:

from emmett import App

app = App( __name__ )

@app.route("/")
async def hello():
    return "Hello world!"

Then, we run the following command in the console:

emmett develop

Documentation

Repository

Starlette

Starlette is a lightweight ASGI framework/toolkit, which is ideal for building async web services in Python.

It gives you the following features:

A lightweight, low-complexity HTTP web framework.
WebSocket support.
In-process background tasks.
Startup and shutdown events.
Test client built on httpx.
CORS, GZip, Static Files, Streaming responses.
Session and Cookie support.
100% test coverage.
100% type annotated codebase.
Few hard dependencies.
Compatible with asyncio and trio backends.

Example:

pip install starlette uvicorn

We create a server.py file and paste the following code:

from starlette.applications import Starlette
from starlette.responses import JSONResponse
from starlette.routing import Route

async def homepage(request):
    return JSONResponse({'hello': 'world'})

routes = [
    Route("/", endpoint=homepage)
]

app = Starlette(debug=True, routes=routes)

Then, we run the following command in the console:

uvicorn server:app

Documentation

Repository

Conclusion

In this article, we explore seven web frameworks for people who are willing to try something different from FastAPI. Many of them have a similar syntax and make it easy to use for people who are used to using FastAPI or Flask.

Thank you for taking the time to read this article.

If you have any recommendations about other packages, architectures, how to improve my code, my English, or anything; please leave a comment or contact me through Twitter, or LinkedIn.

Resources

Robyn Documentation

Robyn Github Page

Sanic Documentation

Blacksheep Documentation

LiteStar Documentation

Starlette Documentation

Falcon Documentation

Emmett Documentation