DEV Community: Alem Djokovic

Web Application Security & XSS Mitigation write up

Alem Djokovic — Tue, 06 Jan 2026 10:49:41 +0000

Objective

The objective of this project was to understand how Cross-Site Scripting (XSS) attacks operate in real web environments and to demonstrate the difference between secure and insecure input-handling practices. The goal was to build two versions of the same web application, one intentionally vulnerable and one protected, to see how an attacker’s input functions differently on each. We conducted a controlled experiment with both reflected and stored XSS attacks, analyzed how the browser executed or blocked injected scripts, and documented the effectiveness of detection and prevention mechanisms such as input sanitization, output encoding, and Content Security Policy (CSP). By the end of the experiment, the expected outcome was a clear comparison between the vulnerable and secure implementations, showing how specific coding decisions directly impact security. Overall, the project demonstrates that XSS vulnerabilities are created from small oversights in input handling and that layered defensive strategies significantly reduce the risk of exploitation.

Introduction

2.1 Overview of the Experiment

This experiment focuses on Cross-Site Scripting (XSS), which is a common and serious web application vulnerability that occurs when user input is not handled or displayed correctly by a browser. XSS attacks let an attacker inject malicious client-side scripts, usually JavaScript, into a trusted website, where the code then runs in the browser of an unsuspecting user. Since modern web applications rely heavily on dynamic user input, XSS continues to be one of the most widespread security problems in real-world systems.

The motivation for this experiment was to understand how XSS vulnerabilities happen at a basic level and to see how malicious input behaves in both insecure and secure environments. Instead of depending only on automated tools, this project took a hands-on approach by manually building both a vulnerable version and a protected version of the same website. This made it easier to clearly compare how different ways of handling user input can directly impact application security.

The experiment focused on two main types of XSS attacks: reflected XSS and stored XSS. Reflected XSS happens when malicious input is immediately sent back in an HTTP response, while stored XSS occurs when harmful content is saved on the server and later delivered to other users. Both attack types were demonstrated using simple form-based interactions to reflect how real web applications typically handle user input.

Previous research and industry reports consistently point to XSS as a high-risk vulnerability because it can compromise user sessions, steal sensitive information, and manipulate website content. Many existing solutions stress the importance of secure coding practices like input validation, output encoding, and browser-based protections such as Content Security Policy. This experiment builds on those approaches by showing, in a controlled environment, how these defenses work in practice and how they successfully prevent scripts from executing when they are implemented correctly.

Overall, this experiment offers a practical look at XSS attacks and their defenses, reinforcing the importance of securely handling user input and using layered security controls in modern web application development.

2.2 Problem Identification (Nature of Attack)

The primary security issue examined in this project is the improper handling of user-provided input, which can lead to Cross-Site Scripting vulnerabilities. XSS attacks happen when a web application sends untrusted input back to the browser without proper validation or encoding, allowing malicious scripts to run. Since the browser treats this content as coming from a trusted website, the injected scripts are executed with the same privileges as legitimate site code.

In this experiment, the vulnerability occurs when user input is directly placed into HTML responses or saved on the server and later displayed without being sanitized. This allows attackers to inject JavaScript code that runs within the context of the website. Once the script executes, it can carry out a variety of malicious actions that put both user security and the overall integrity of the application at risk.

Once an XSS vulnerability is successfully exploited, attackers can:

Execute arbitrary JavaScript within the victim’s browser
Steal session cookies or authentication tokens
Manipulate or deface website content
Redirect users to malicious or phishing websites
Exploit trusted user sessions without direct user interaction
Two types of XSS were examined in this project: reflected XSS and stored XSS. Reflected XSS happens when malicious input is included in a request and immediately sent back in the server’s response, which commonly occurs through URL parameters or form submissions. Stored XSS is considered more dangerous because the injected code is saved on the server, such as in a comment or message field, and then delivered to other users when the content is viewed. Since the payload remains stored, these attacks can last over time and impact a much larger number of users.

Existing research and real-world examples show that XSS vulnerabilities continue to exist mainly because secure coding practices are not applied consistently. Many web applications depend only on client-side validation or neglect proper output encoding, both of which attackers can easily bypass. This project addresses these issues by showing how XSS attacks work in a realistic setting and by highlighting the importance of handling input on the server side and applying proper output encoding.

By building both a vulnerable and a secure version of the same website, this project highlights the root causes of XSS vulnerabilities and shows how small design choices, like directly displaying user input, can lead to serious security risks. This analysis sets the groundwork for evaluating the technical approach and defense mechanisms covered in the sections that follow.

2.3 Technical Approach

This project implemented a controlled local testing environment to demonstrate and analyze Cross-Site Scripting (XSS) vulnerabilities and their mitigation. The environment was developed using PHP and Visual Studio Code, with PHP’s built-in development server used to host two separate web applications. To ensure safety and ethical testing, the setup operated entirely on localhost with no external network access. Two web environments were deployed: a deliberately vulnerable website running on localhost:8001 and a secure, hardened version running on localhost:8002. The vulnerable site was intentionally designed to echo unsanitized GET parameters directly back to the page and to store raw user input in a guestbook feature. This configuration allowed both reflected and stored XSS attacks to be demonstrated, where injected JavaScript payloads executed within the browser, highlighting the real-world risks of failing to validate or encode user input.

In contrast, the secure website implemented multiple defensive mechanisms to prevent XSS exploitation. Pattern-based input inspection was used to detect suspicious payloads, and all potentially malicious activity was logged server-side for visibility and auditing. Most importantly, user-supplied data was safely handled using HTML output encoding through PHP’s htmlspecialchars() function, ensuring that injected scripts were rendered harmless as plain text rather than executed. Additional input sanitization further reduced the attack surface by filtering dangerous characters before processing. Together, these controls effectively blocked both reflected and stored XSS attempts, demonstrating how layered defenses significantly improve web application security. This side-by-side comparison clearly illustrates the impact of secure coding practices and reinforces the importance of proper input handling, encoding, and logging in defending against common web-based attacks.

3.1 Network Topology, Configuration, and System Setup

This experiment used a single-host, isolated test environment where all components ran locally on one machine (laptop/desktop). No external network access was required because both web applications were hosted on localhost and accessed through the local browser. The topology is intentionally simple: the client (Google Chrome) sends HTTP requests to a local PHP development server, which serves two separate web applications, one intentionally vulnerable and one hardened, running on different ports. This design keeps testing safe, repeatable, and contained on one system while still representing a realistic client/server web interaction model.

Hardware Components:
1x Laptop/Desktop (single host)
Acts as the web server host, development workstation, and client browser
Minimum recommended: 8GB RAM, modern CPU (any typical student laptop is fine)
Software Components:
Operating System: Any desktop OS (Windows/macOS/Linux)
PHP runtime + local web server: PHP built-in development server (via XAMPP PHP)
Editor / IDE: Visual Studio Code
Browser (client): Google Chrome
Project folders (two applications):
vulnerable_site.html/ (intentionally insecure)
secure_site.html/ (hardened version with detection + encoding)
Directory Structure and Application Roles:
1) Vulnerable Web Application (vulnerable_site.html/)
Purpose: demonstrate how XSS occurs when input is not validated/encoded.

index.php — Reflected XSS demo (unsanitized GET input is echoed to the page)
stored.php — Stored XSS guestbook (user input written and later rendered)
comments.txt — stores unsafe/raw comments (demonstrates persistence)
2) Secure Web Application (secure_site.html/)
Purpose: demonstrate detection + prevention controls working together.

index.php — secure reflected handling (detection + output encoding)
stored.php — secure stored handling (detection + output encoding)
detection.php — pattern-based input inspection + server-side logging
comments.txt — stores comments (safe handling occurs on output/sanitization logic)
xss_log.txt — persistent log of detected XSS attempts
Port and Server Configuration (Critical Settings)
Two separate PHP servers were run simultaneously to keep environments independent:

Vulnerable site server
Command: php -S localhost:8001
URL: http://localhost:8001
Secure site server
Command: php -S localhost:8002
URL: http://localhost:8002
Key configuration choices:

Localhost binding (localhost) ensures traffic never leaves the machine.
Separate ports (8001/8002) prevent file overlap and make comparisons clean.
No external network requirement keeps testing safe and avoids accidental exposure.
3.2 Experiment Steps

Prerequisites (Software + Folder Setup)
Software used:

PHP (local runtime) — via XAMPP PHP (or any PHP install that supports php -S)
Visual Studio Code — to edit files
Google Chrome — to test reflected/stored behavior in a real browser
Project folders (two separate apps)
Create two directories (or clone your project into this structure):

Become a member
The vulnerable site was coded to directly display user input without sanitization. The reflected XSS page echoed GET input back to the browser, and the guestbook page stored and displayed raw user comments, making both reflected and stored XSS possible.

The secure site was coded with detection and prevention controls. Input was inspected for XSS patterns, suspicious attempts were logged, output was encoded using htmlspecialchars(), and a Content Security Policy was enforced to block script execution.

The vulnerable site was started by running php -S localhost:8001, and the secure site was started using php -S localhost:8002. Both sites were accessed using Google Chrome.

Reflected XSS was demonstrated on the vulnerable site by submitting a script payload, which executed immediately and displayed an alert. Stored XSS was demonstrated by submitting a malicious guestbook comment that executed on submission and page reload.

The same attacks were tested on the secure site. The payloads did not execute, warning messages were shown, and all attempts were logged, confirming that XSS attacks were successfully detected and blocked.

3.3 Results and Analysis

Testing against the vulnerable website confirmed that both reflected and stored XSS payloads managed to execute without any hindrance. Submitting alert(), JavaScript executed inline in the browser right away, causing the message to pop up, confirming that user input was being echoed back into what was probably a vulnerable website without any validation or encoding. In the stored XSS guestbook page, comments that had been written maliciously in raw form got saved and executed each time the page was refreshed, demonstrating how persistent XSS can repeatedly affect users who view the compromised page. These results reinforce how dangerous unsanitized user input becomes when it is embedded directly into HTML output.

In contrast, the secure version of the application blocked all tested payloads. Instead of executing, malicious input was encoded and displayed as plain text, and the system displayed warning messages when suspicious patterns were detected. Server-side logs consistently captured details about each attempted attack, providing insight into when and how XSS attempts were made. This demonstrated that both the detection algorithm and output encoding mechanisms were functioning properly. The CSP added a layer of defense by preventing inline scripts from running, even if an attacker attempted to bypass encoding protections.

Overall, the results showed a clear difference in the behaviors between the insecure version and the secure one. The vulnerable site allowed full script execution with minimal effort, while the secure site stopped every payload and maintained normal functionality. These findings highlight a strong correlation between proper input handling and system resilience, demonstrating that output encoding and CSP are particularly effective at reducing the risk of reflected and stored XSS attacks. The experiment also showed that using both prevention and detection made the system much stronger rather than depending on just one layer of defense.

Protocols and Algorithms for Prevention, Detection, and Countermeasures

4.1 Prevention Mechanisms

The prevention mechanisms implemented in this project were designed to stop malicious scripts from executing in the user’s browser, even when an attacker attempts to inject harmful input. Rather than relying on a single security control, the secure implementation applied multiple preventative techniques to demonstrate a defense-in-depth approach against XSS attacks.

Protocols

The primary protocol implemented to prevent XSS attacks in this project was a Content Security Policy. The CSP restricts the sources from which scripts can be executed within the browser, allowing only trusted content originating from the application itself. By enforcing these restrictions at the browser level, the CSP prevents unauthorized or inline scripts from executing, even if malicious input is successfully injected into the page. This protocol strengthens the system’s defenses by adding a browser-enforced security layer that operates independently of server-side input handling. As a result, the CSP reduces the likelihood that injected scripts can execute and mitigates the impact of potential bypasses in application-level defenses.

Algorithms

To prevent XSS vulnerabilities at the application level, this project used a server-side output encoding approach based on HTML entity encoding. This method converts potentially dangerous characters, such as <, >, “, and ‘, into safe encoded versions before user input is displayed in the browser. Output encoding was chosen as the main prevention technique because it effectively stops malicious scripts without changing or removing the user’s input. By encoding the data immediately before it is sent to the browser, the application ensures that any injected payload is treated as plain text rather than executable code. This method helps protect against both reflected and stored XSS attacks and is widely considered a best practice in secure web development.

Implementation Details

The prevention techniques and algorithms were built directly into the server-side logic of the secure version of the website. Any user input that was going to be displayed was first passed through a sanitization function before being rendered. This ensured that both new input and previously stored data could not execute scripts when shown on the page, providing consistent protection across all secure pages.

In addition, a Content Security Policy was applied at the page level using browser-supported enforcement to add another layer of defense against script execution. One challenge during implementation was keeping the website’s normal functionality while still applying strict security measures. This was handled by allowing user input to be stored and displayed as expected, but only after it had been properly encoded. By combining output encoding with browser-level protections, the application remained usable while greatly reducing the risk of XSS attacks.

4.2 Detection Mechanisms

Monitoring Tools:

To monitor potential XSS attacks, the project relied mainly on server-side logging and browser developer tools. The secure site used a logging file (xss_log.txt) to record every suspicious input submitted by the user. This log helped track repeated attempts, identify payload patterns, and confirm if an attack was successfully blocked. In addition, Chrome Developer Tools were used during testing to inspect the page’s behavior, check for blocked scripts, and verify that encoded output was displayed correctly. Overall, these tools provided visibility into how the system responded to malicious input and made it easier to confirm whether the detection logic was working properly.

Detection Algorithms:

The detection algorithm used in the secure version of the website followed a pattern-matching approach. The program checked user input for common XSS indicators such as , event handlers like onload or onclick, and other high-risk characters or sequences seen in malicious payloads. When a match was found, the system flagged the input, displayed a warning message to the user, and wrote the attempt to the server log. Although this algorithm is not as advanced as machine-learning-based detection, it was effective for this experiment because it reliably caught typical XSS payloads and prevented them from executing. It also showed how even basic pattern detection can reduce risk when combined with other security measures. Implementation Process: The detection mechanisms were built into the secure site’s PHP files and ran automatically whenever input was submitted. Incoming data was passed through the detection function before being displayed on the page. If the input was flagged as suspicious, the system encoded the output, logged the event, and displayed a warning. During testing, small adjustments were made to reduce false positives, usually by narrowing which patterns trigger alerts and ensuring normal text input was not blocked. This process helped balance security and resulted in a detection system that was both effective and practical for this project. 4.3 Countermeasures Immediate Actions: Once Cross-Site Scripting (XSS) vulnerabilities were detected, immediate response measures were implemented to limit system exposure and prevent further exploitation. All user inputs were validated and sanitized on both the client and server sides to block malicious payloads before processing. Suspicious inputs were rejected or cleaned to remove executable scripts. Output encoding was applied to ensure that user-generated content was rendered safely within the browser. By encoding dynamic data based on its context, the application prevented injected scripts from executing. Vulnerable code sections responsible for unsafe rendering were promptly patched to remove injection points. Additionally, temporary mitigation measures such as Web Application Firewall (WAF) rules were deployed to block known XSS payload patterns. These controls provided short-term protection while permanent fixes were implemented. Long-term Solutions: To provide sustained protection against XSS attacks, long-term security strategies were incorporated into the application design. Secure coding practices were enforced using modern frameworks that automatically escape dynamic content, reducing the risk of developer error.A strong Content Security Policy (CSP) was implemented to restrict script execution and limit trusted sources. By blocking inline scripts and enforcing source whitelisting, CSP significantly reduced the impact of potential XSS attacks. Regular security testing using Static and Dynamic Application Security Testing (SAST/DAST) tools was integrated into the development process. Secure input-handling libraries, safe template engines, and browser-level protections such as HttpOnly and SameSite cookies, along with HTTPS enforcement, further strengthened the application’s security posture. Implementation Strategy: The countermeasures were implemented using a layered security approach to provide defense in depth. Security controls were applied across multiple layers, including input handling, application logic, browser rendering, and network protection. Automated security testing tools were integrated into the CI/CD pipeline to enable continuous vulnerability detection. Logging and monitoring mechanisms were also implemented to detect abnormal script behavior and injection attempts in real time. Ongoing maintenance includes regular updates, code reviews, and security policy refinements, supported by continuous developer training. <ol> <li>Conclusion</li> </ol> This project demonstrated how Cross-Site Scripting attacks can compromise web applications when proper security controls are absent. By comparing a vulnerable application with a secured version, the experiment showed how reflected and stored XSS payloads can be used to inject scripts, manipulate user interactions, and expose sensitive data. The implementation of layered defenses, including input validation, output encoding, CSP enforcement, browser security settings, and continuous testing, resulted in a significant reduction in exploitable behavior. These results reinforce that XSS prevention requires multiple coordinated defenses rather than a single solution. Overall, the project highlights the importance of secure coding, proactive testing, and developer awareness in building resilient web applications. The findings contribute practical insights into effective XSS mitigation strategies. <ol> <li>Future Recommendations</li> </ol> To further enhance protection against Cross-Site Scripting (XSS) attacks, future work should focus on strengthening security throughout the development lifecycle. Adopting a Security-by-Design approach ensures that XSS prevention techniques are integrated during the earliest stages of application development rather than being addressed reactively. Automating security testing by incorporating Static and Dynamic Application Security Testing (SAST/DAST) tools into CI/CD pipelines would allow vulnerabilities to be detected before deployment. Additionally, continuously refining Content Security Policies can further restrict script execution and limit the impact of injection attempts. Regular code audits and penetration testing should be conducted to identify new or overlooked vulnerabilities as the system evolves. Ongoing developer training is also essential to keep teams informed about emerging XSS techniques and modern defensive practices. Strengthening user input controls through advanced sanitization libraries, secure template engines, and server-side validation frameworks can further reduce attack surfaces. Finally, implementing real-time monitoring and logging mechanisms would enable faster detection and response to suspicious script behavior, improving overall system resilience. References OWASP. (n.d.). Cross Site Scripting Prevention. OWASP Cheat Sheet. <a href="https://cheatsheetseries.owasp.org/cheatsheets/Cross_Site_Scripting_Prevention_Cheat_Sheet.html">https://cheatsheetseries.owasp.org/cheatsheets/Cross_Site_Scripting_Prevention_Cheat_Sheet.html</a> OWASP. (2017). A7:2017 — Cross-Site Scripting (XSS). OWASP Top Ten 2017. A7:2017 — Cross-Site Scripting (XSS) OWASP. (2023). Testing for Reflected Cross Site Scripting. OWASP Web Security Testing Guide (WSTG). <a href="https://owasp.org/www-project-web-security-testing-guide/latest/4-Web_Application_Security_Testing/07-Input_Validation_Testing/01-Testing_for_Reflected_Cross_Site_Scripting">https://owasp.org/www-project-web-security-testing-guide/latest/4-Web_Application_Security_Testing/07-Input_Validation_Testing/01-Testing_for_Reflected_Cross_Site_Scripting</a> OWASP. (2023). Testing for Stored Cross Site Scripting. OWASP Web Security Testing Guide (WSTG). <a href="https://owasp.org/www-project-web-security-testing-guide/latest/4-Web_Application_Security_Testing/07-Input_Validation_Testing/02-Testing_for_Stored_Cross_Site_Scripting">https://owasp.org/www-project-web-security-testing-guide/latest/4-Web_Application_Security_Testing/07-Input_Validation_Testing/02-Testing_for_Stored_Cross_Site_Scripting</a>

Web Application Security & XSS Mitigation Lab

Alem Djokovic — Tue, 06 Jan 2026 10:46:06 +0000

For my project, I built and tested two versions of a web application—one intentionally vulnerable and one secured—to evaluate how Cross-Site Scripting (XSS) impacts application reliability, user safety, and operational risk.

What I implemented:
• Local PHP-based web stack using VS Code + PHP built-in server
• Two environments:

Vulnerable site (no input validation or encoding)

Secure site (defense-in-depth controls)
• Demonstrated both reflected and stored XSS attacks using real JavaScript payloads

Observability & Detection:
• Pattern-based input inspection to detect injections • Server-side logging of suspicious payloads to xss_log.txt • Browser dev tools used to verify execution vs. prevention Hardening & Prevention (DevOps Lens): • Output encoding using htmlspecialchars() • Input validation and sanitization • Content Security Policy (CSP) to block inline and unauthorized scripts • Layered controls to reduce blast radius even if one defense fails Outcome: • Vulnerable site executed malicious scripts immediately • Secure site blocked execution, logged attempts, and remained stable DevOps Takeaway: This project reinforced how secure coding practices, observability, and policy-based controls are critical to application reliability. From a DevOps perspective, XSS isn’t just a security bug—it’s an operational risk that can impact user trust, availability, and compliance. <a href="https://youtu.be/yRzVNmUdgTQ">https://youtu.be/yRzVNmUdgTQ</a>

Simulated and monitored DDoS attacks Write up research

Alem Djokovic — Tue, 06 Jan 2026 10:44:15 +0000

Objective

In this project, we looked into Distributed Denial-of-Service (DDoS) attacks by simulating a SYN flood inside a virtual environment. Our goal was to examine how these attacks affect server performance and test whether basic protections could help. We used two Ubuntu servers — one with no security setup and the other secured with iptables firewall rules. The attack was launched from a Kali Linux virtual machine using hping3, targeting both servers at the same time. While the attack was running, we monitored things like CPU load, system responsiveness, and network activity using netstat, tshark, and htop. The results showed that the unprotected server couldn’t handle the attack, with hundreds of half-open connections and a noticeable spike in CPU usage. Meanwhile, the server with iptables stayed stable and kept performing well. This showed us how even simple firewall configurations can make a big difference when it comes to protecting systems from common DDoS threats [1].

Introduction

2.1 Overview of the Experiment

This project focuses on learning how Distributed Denial-of-Service (DDoS) attacks work and how they can be stopped, specifically by looking at a type of attack called a SYN flood. A SYN flood targets a weakness in the TCP three-way handshake. Instead of completing the handshake, the attacker just sends a huge number of SYN requests and never follows up with the final ACK. This tricks the server into holding open a bunch of half-finished connections. Eventually, these unfinished requests take up too much memory and CPU power, making it hard or even impossible for real users to connect.

The main goal of this experiment was to simulate a SYN flood attack in a virtual environment and see how well basic iptables rules could defend against it. To do this, we set up a virtual lab in Oracle VirtualBox with three virtual machines: one Kali Linux machine to act as the attacker, and two Ubuntu servers. One of the servers had no protection at all, while the other was set up with iptables to help block the attack and serve as a simple defense system.

We used the Kali VM to launch the SYN flood using hping3, a command-line tool that can send a large number of SYN packets. While the attack was running, we monitored both servers using tools like netstat to check connection states and tshark to watch the network traffic. These tools helped us collect real-time data and see how each server responded when facing the same attack.

This project builds on past cybersecurity research showing that SYN flood attacks, even though they’re simple to carry out, are still one of the most effective and damaging types of denial-of-service attacks. Earlier studies point out that real-time traffic filtering and rate limiting are key ways to stop these attacks, especially when using tools like iptables, which are commonly available on Linux systems [1].

In this experiment, we wanted to not only test these ideas but also show that strong defenses don’t have to rely on expensive, high-end security systems. By using open-source tools and free virtual machines, we were able to create a setup that’s easy to recreate for learning, research, or training purposes.

Fig. 1. Normal TCP handshake vs. SYN flood attack

2.2 Problem Identification (Nature of Attack)

SYN flood attacks are one of the oldest and most common types of Distributed Denial-of-Service (DDoS) attacks. They take advantage of a weakness in the TCP protocol, specifically during the handshake process that sets up a connection between a client and a server. The normal TCP handshake works like this:

The client sends a SYN request to start the connection.
The server responds with a SYN-ACK to acknowledge the request.
The client replies with an ACK to complete the handshake. In a SYN flood attack, the attacker leaves out the final ACK on purpose.
This causes the server to hold onto these half-finished connections, which can eventually overwhelm it and prevent real users from connecting.

Past research and real-world cases show that SYN flood attacks are still commonly used by attackers, either on their own or as part of bigger DDoS campaigns. One reason they’re still so popular is because they don’t need a lot of bandwidth to be effective, which makes them a cheap and easy option for causing disruption. While some large-scale attacks use botnets to send traffic from thousands of devices, our experiment showed that even one machine can create noticeable problems in a controlled setting.

One of the biggest challenges with stopping SYN flood attacks is that they look almost exactly like normal connection requests. Without the right filtering or rate-limiting in place, a server can’t easily tell the difference between a real user trying to connect and an attacker sending fake SYN requests. That’s why tools like iptables and more advanced intrusion detection systems are so important. They help spot patterns in the traffic and block the bad requests before they start draining the server’s resources.

By focusing on the SYN flood attack, this project tackles a type of vulnerability that’s easy to understand but still causes serious damage. It shows how important real-time traffic analysis is and proves that even simple countermeasures, if set up correctly, can make a big difference in protecting a system, even against steady, low-level attacks that target the core of the protocol.

Fig. 2. SYN flood overwhelming the server’s TCP backlog

2.3 Technical Approach

To explore how a SYN flood attack works and what kind of impact it has, we built a virtual setup where we could safely run the attack and see how it affects systems with and without protection. Our approach combined both the offensive side, launching the attack, and the defensive side, like monitoring the traffic and using a firewall to try and block it.

We used Oracle VirtualBox to build the environment, setting up a private internal network called DDoSNet to keep everything contained and make sure nothing outside the lab was affected. The setup included three virtual machines:

Kali Linux (Attacker): Used to launch the SYN flood using tools like hping3.
Ubuntu Server (Unprotected): Had no defenses in place, allowing us to observe the full impact of the attack.
Ubuntu Server (Protected): Set up with iptables firewall rules to help detect and block the SYN flood traffic.
Fig. 3. Virtual machines configured in VirtualBox

We used the hping3 tool to launch the SYN flood, since it allowed us to send a large number of SYN packets quickly. The attack was aimed at port 80 on both servers, but we ran the test on each server separately to keep the conditions consistent and make a fair comparison. The command we used was:

sudo: Run with root privileges
hping3: The tool used to craft and send raw TCP packets
-S: Send SYN packets
— flood: Send packets as fast as possible (flood mode)
-p 80: Target port 80 (HTTP)
-I enp0s3: Specifies the network interface to send packets from (here, enp0s3)
192.168.100.10: Target IP address ( unprotected server)
During each test, we used a few different tools to keep track of how the servers were handling the attack:

netstat: Used to check active TCP connections and look for a buildup of SYN_RECV states
tshark: Captured and analyzed packets in real time so we could see what kind of traffic was hitting the server
ping: Let us measure packet loss and latency to get a sense of how responsive the server was during the attack
Before launching the attack, we set up some basic iptables rules on the protected server to act as a simple, local defense. These rules were designed to limit the number of incoming SYN packets and slow down the flood. A breakdown of the rules and how they worked is explained in Section 4.1. This setup made it easy to run the tests in a consistent way and helped us clearly see how well the protections worked by comparing the results from both servers under the same conditions [2].

Experiment:

3.1 Network Topology

The network topology is the virtual environment in which the experiment took place using a virtual box consisting of three machines connected via an internal network so that they can all communicate locally. This set simulates a realistic local network server under attacking conditions while remaining isolated from the internet for safety concerns. All machines are connected to the same virtual network (DDosNet) to allow isolated packet exchange without risking external exposure.

TABLE I: VIRTUAL MACHINES

Component

Role

IP Address

Key Configuration

Kali Linux (Attacker)

Launches SYN flood via hping3

192.168.100.30

hping3 tool installed, connected via eth0

Unprotected Server

Target without protection

192.168.100.10

Apache2 on port 80, no firewall or limits

Protected Server

Target with iptables defense

192.168.100.20

Apache2 on port 80, iptables rate limiting

Fig. 4. Internal network settings for the virtual machine in VirtualBox

Traffic Flow and Behavior
The attacker (Kali) sends a high volume of TCP SYN packets using hping3 targeting port 80.
The unprotected server receives all traffic, quickly exhibiting high SYN_RECV counts, CPU spikes, and dropped connections
The protected server uses iptables to:
Allow a limited number of new connections (3/minute)
Drop excess SYN packets beyond that rate
This setup effectively demonstrates the difference in resilience between protected and unprotected systems.

Tools and Services
hping3: Used to simulate the SYN flood from Kali
Apache2: Runs on both servers to simulate a live web service
iptables: Configured on the protected server for DDoS mitigation
htop, netstat, tshark: Used for monitoring CPU, connections, and packet flow
Purpose and Justification
This topology was designed to replicate a realistic internal network under attack, allowing us to evaluate the effectiveness of basic firewall defenses in a repeatable, isolated setting. It reinforces the need for proactive mitigation in cybersecurity infrastructure and provides a safe platform for testing without risking actual production systems.

Configuration and System Setup

The configuration of this experiment is designed with three main iso images Kali linux this was used for the attacker and two ubuntu server images with no GUI or a graphical interface this was done to mimic actually working in a server. With all three iso images downloaded and installed we choose to host on Oracle VMware. Once all three iso images on Oracle initially want to run all three to make sure they work after that you want to go into Oracles settings and go into network for each other servers and switch them to internal network By selecting the Internal Network mode, all virtual machines (such as the attacker, protected server, and unprotected server) are placed within an isolated virtual LAN. This means that network traffic is restricted to just the virtual machines connected to this internal network and does not interact with the host system or the external internet. This isolation is especially important when simulating potentially harmful activities like DDoS attacks, as it prevents any risk of accidentally affecting other systems or networks. Naming the internal network DDoSNet serves an organizational purpose. It clearly identifies the virtual network being used for DDoS testing, which helps avoid confusion when managing multiple virtual networks in VirtualBox. This also enhances reproducibility and documentation by allowing other users to easily replicate the setup using the same network name. Using an internal network configuration ensures that the virtual machines can communicate directly with one another without relying on NAT or bridged adapters. This direct connection is essential for accurately simulating real-world attack scenarios, such as sending flood traffic from an attacker machine to a target server. Overall configuring the network as an internal adapter and naming it DDoSNet creates a safe, controlled, and well-organized environment for conducting the experiment, while also supporting good cybersecurity research practices.

Fig. 5. Network topology used in the DDoS simulation

3.2 Experiment Steps

Step 1: Download Oracle VirtualBox, then download all three Iso images: Kali Linux for the attacker, Ubuntu server image for both servers.

Step 2: Insert all ISO images into Oracle and run them. Name the Ubuntu servers, one protected and one unprotected. Step starting with the Kali Linux install tools like hping3. This was the only tool you needed for Kali. Close the Kali image after.

Step 3: On the Ubuntu unprotected server, install tshark and netstat tools, close them, and do the same thing for the protected server, but install iptables also on the protected server, and then close it.

Fig. 6. Installing hping3 on Kali Linux

Fig. 7. Installing iptables and tshark on the protected server

Step 4: Go into network settings and switch the NAT type to internal do this for all three VMs. This will allow them to talk to each other.

Fig. 8. Internal network adapter settings

Step 5: Now it’s time to assign static IPs, start up the unprotected server, and use the command sudo nano /etc/systemd/network/10-static.network, you then want to write the following into the file

Fig. 9. Static IP configuration on unprotected server

After this is complete, you want to make sure the static IP is saved, so run sudo ip a.

Fig. 10. Verifying static IP on unprotected server

After the static IP is saved, you can close the unprotected server and run the protected server and do the same thing, but make sure the IP is set to 192.168.100.20 instead of the 192.168.100.10 for the protected server.

Fig. 11. Static IP configuration on protected server

Sudo ip a to make sure the changes are applied, then do the same for Kali and assign it 192.168.100.30

Fig. 12. Launching SYN flood from Kali using hping3

Step 6: Run Kali and unprotected, on the unprotected server, run sudo netstat -ant | grep :80, then on Kali launch the flood attack to the unprotected server.

Fig. 13. ping output showing server response

Once the attack is going on the unprotected server and run sudo netstat -ant | grep :80 a couple more times to see if the queue is filling it.

Fig. 14. SYN_RECV queue on unprotected server

As you can see the server is now attacked and will become unresponsive Stop the attack crtl + c, stop the unprotected machine.

Step 7: Run the protected machine and add the IP tables rule listed below.

Fig. 15. Iptables rules applied on the protected server

Step 8: Now that these rules are enabled, run sudo tshark and sudo watch -n 1 “iptables -L INPUT -v — line-numbers”. Now, on the Kali server, run the attack on the protected server: sudo hping3 -S — flood -I eth0 -p 80 192.168.100.20. Now watch the iptables rules and watch the rule drop packets

Fig. 16. Iptables dropping SYN packets on protected server

3.3 Results and Analysis

Fig. 17. Impact of SYN flood on SYN connections and CPU usage

The unprotected server experienced a massive spike in SYN connections (from 5 to 750) and a sharp increase in CPU usage (from 10% to 95%) during the flood. This confirms its vulnerability to SYN flood attacks. The protected server, equipped with iptables rate-limiting, maintained control during the attack. SYN connections only increased slightly, and CPU usage remained low. The experiment clearly illustrates the stark contrast between the behavior of the unprotected and protected servers under SYN flood conditions the unprotected server saw an exponential increase in SYN connection attempts — from just 5 to over 750 — indicating that it was unable to control the flow of incoming traffic. In contrast, the protected server, equipped with iptables rate-limiting rules, maintained a manageable level of SYN connections (peaking at just 12) and experienced only a minor increase in CPU load. This demonstrates that basic filtering rules can significantly reduce the impact of DDoS attacks, even in resource-constrained environments.]

Become a member
These findings emphasize the importance of implementing simple yet effective security measures, especially in systems exposed to public networks. They also highlight the value of proactive defense in cybersecurity practice — waiting until an attack happens is often too late. Proper configuration, even at the firewall level, can be the difference between service continuity and system failure.

Protocols and Algorithms for Prevention, Detection, and Countermeasures

4.1 Prevention Mechanisms:

Protocols Implemented

To keep the protected server from getting overwhelmed by the SYN flood, we used iptables, which is a built-in firewall tool found in most Linux systems. It works by checking incoming packets and following a set of rules to decide if they should be allowed through, blocked, or dropped. In our case, we set up iptables to filter and limit the number of TCP SYN packets going to port 80, which is the standard port for HTTP traffic.

Firewall Rules

The following rules were applied to the protected Ubuntu server:

The first iptables rule allows a limited number of new SYN connections to port 80.
The second rule drops any extra SYN packets that go over the set threshold.
The — limit-burst option lets a short burst of traffic through before the rate limit kicks in.
The — limit option controls how many packets are allowed per second after the initial burst.
This setup helps block excessive traffic while still letting legitimate users connect to the server.
These rules used stateless filtering, which means they only checked the packet headers and didn’t track any session information. This helped keep CPU and memory usage low.

Effectiveness and Rationale

This approach was chosen because:

iptables is a lightweight tool that comes bundled with most Linux distributions.
It can be configured to drop excessive traffic before any system resources are allocated, which makes it effective against SYN flood attacks that aim to fill up the server’s connection backlog.
The rules can be easily adjusted to suit different use cases or adapt to various traffic patterns.
During the experiment, the protected server stayed fully responsive and didn’t experience any service interruption, even while under continuous attack. Monitoring tools showed that only a small number of SYN requests were accepted, while the rest were blocked by the firewall before they could use up system resources. This simple prevention method proved to be both effective and practical, showing that even basic tools like iptables can significantly improve a system’s ability to withstand denial-of-service attacks when configured properly [1].

4.2 Detection Mechanisms:

Detecting and mitigating Distributed Denial of Service (DDoS) attacks on a network is very critical for any individual and organization. DDoS attacks can inhibit the daily function of an organization by causing service disruptions since it involves overwhelming the target with high network traffic. To effectively detect and mitigate an attack, it is vitally important to implement network security tools such as the ones used in this project,namely IPtables, Tshark, Netstat, and UFW (Uncomplicated Firewall) which analyzes network traffic, capture network packets and detect anomalies signifying a DDoS attack [2]. Iptables and Tshark were installed and updated on the protected server in order to ensure that they had the latest security and threat updates, while UFW and Netstat were used on the unprotected server. A description of each tool is described below, the algorithms used for detecting the attack and the implementation process.

Monitoring Tools:

iptables:

A command-line tool used to configure the IP packet filter rules of the Linux kernel firewall.
Chains of defined rules are built into the IPtables, such a rate-limiting, and some can be created by the system administrator. These rules control traffic flow because they are defined based on ports, protocols, and IP addresses.
These rules are policies created to enhance security by logging, detecting, and blocking suspicious network traffic patterns.
Tshark:

A command-line tool like Wireshark that is used for monitoring and analyzing network traffic in real time or from PCAP files
Filters for patterns and detects anomalies in the network traffic.
This enhances security in the protected server by conducting deep packet inspection to identify malicious traffic,c and is great for forensic analysis [2].
Monitoring Tools on the Unprotected Server

Netstat:

A command-line tool that displays active connections
Identify listening ports
Monitor network traffic
Identify network anomalies
As a security enhancement tool, it tracks open ports, identifies suspicious connections, and helps to detect intrusions.
UFW (Uncomplicated Firewall):

A user-friendly command-line firewall configuration tool to set rules in IPtables to filter network traffic.
To enhance security, UFW blocks or allows traffic based on IP/port and helps mitigate DDoS attacks through rate-limiting.
On the unprotected server, UFW Status was set to inactive which allowed any network traffic to pass through the firewall.
Detection Algorithms:

To help protect the system and detect possible attacks, we used detection algorithms that monitor for anything unusual or threats. Integrating multiple detection techniques provides a layered approach to securely monitor the network. According to Koushik Chatterjee in the International Journal of Computer Science and Telecommunications [Volume 4, Issue 3, March 2013], the mechanism behind the detection algorithm is enhanced by the accessing of databases for known signatures which is vital in protecting networks from malicious actor; he further states that known attacks are easily detected and that there’s usually no false positives (Chatterjee, 2013). The detection algorithms used in this project are: traffic-based algorithms, Signature-based algorithms, and anomaly-based algorithms, which enhances the network ability to detect emerging DDoS attacks. These algorithms use different methods to detect and capture threats shown below.

Traffic Pattern Analysis: Compares real-time traffic with normal baseline traffic. A sudden spike in traffic volume can trigger an alert indicating a possible DDoS attack.
Signature-based Algorithm: Used for pattern matching, which compares real-time network traffic to known DDoS attacks stored in the Database.
Anomaly-based Detection Algorithms: Watches for deviations from normal network traffic behavior, such as excessive requests from unknown sources, which indicates a possible DDoS Attack.
Implementation Process:

In order to detect possible attacks, there is a need to configure the network security tools on the servers to detect attacks. The following step-by-step process makes sure that everything works smoothly and all alerts are accurate.

Configuring the Tools: protected server

iptables and Tshark were installed and configured on the protected server.

Iptables were configured in the command line interface (CLI) by defining rules which allow traffic only from sources that are trustworthy. This is implemented by defining two rules using the command line interface, “sudo iptables -A INPUT -p tcp –-dport 80 –syn -m -limit –limit 3/minutes –limit– burst 5 -J ACCEPT” and “sudo iptables -A INPUT -p tcp –dport 80 -syn -j DROP. These rules are set to protect the server from SYN flood attacks and to control the number of incoming TCP connection requests that are allowed through port 80 (HTTP).
The CLI input: “sudo iptables -A INPUT -p tcp –-dport 80 –syn -m -limit –limit 3/minutes –limit– burst 5 -J ACCEPT” is a rate-limiting chain rule that allows only three new incoming TCP connections per minute on port 80, and allows an initial burst of 5 connections before the limit kicks in.

SYN connections that go above this threshold will continue to the next rule in the chain which is “sudo iptables -A INPUT -p tcp –dport 80 -syn -j DROP” . This rule blocks incoming TCP connections that match the first step of the TCP handshake on port 80: it silently drops the packets by not sending an acknowledgement. Please see image below:

Protected server with iptables protection rule defined

Tshark is configured to monitor, capture and filter the network traffic by using the CLI command “sudo tshark -i eth0”.
Configuring the Tools: unprotected server

Netstat and UWF were tools installed and configured on the unprotected server. UFW was usually actively protecting the server by default, but in this project, it was disabled to make the server vulnerable to attacks. See picture below:
Fig. 18. UFW disabled on unprotected server

Netstat was configured using the following command on the CLI : “sudo netstat -an | grep SYN_RECV |wc -1”, which is a command executed by the root user to identify and count the number of incoming HTTP connections to the server. During a SYN flood attack, a large number of SYN_RECV connections are received and an unusually high number of incomplete TCP handshakes demonstrates a possible attack. Please see below :
Fig. 19. SYN_RECV count showing 182 half-open connections

Establishing a Baseline:
A baseline was first established to monitor the protected and unprotected server.

The baseline was established on the protected server by using the command “sudo watch -n 1 “iptables -L INPUT -v –line numbers” and the sudo password , which displayed in the terminal that there were no packets received, accepted or dropped as seen in the image below:
Fig. 20. Baseline of Protected Server

The baseline was established on the unprotected server by using the command “sudo netstat -ant | grep :80 ” and the sudo password, which displayed in the terminal that there was no traffic between the unprotected server and the Kali attacker server as seen in the image below:
Fig. 21. Baseline of unprotected server

Configuring Detection Algorithms

Detecting and mitigating Distributed Denial of Service (DDoS) attacks on the servers utilized various detection techniques, which included traffic pattern analysis, signature-based detection, and anomaly-based detection. Reviewing the logs and alerts enabled the knowledge of whether the rules applied were effective or defective based on the traffic behaviors.

Traffic-Pattern Analysis was used by comparing normal traffic patterns with real-time data on Tshark and Netstat. Neither tool needed adjustments because they were very effective in sending alerts of the spike in the network traffic, as seen below:
Tshark registered the packets from the attacker (kali) ip address 192.168.100.30 to the protected server ip address 192.168.100.20 during the SYN flood and we can see that they are being sent from kali to the protected server and the type of protocol was TCP.
Fig. 22. Tshark logs showing TCP SYN flood traffic to protected server

Netstat registered the hping attack that was launched by the attacker (kali) ip address 192.168.100.30 against the unprotected server at ip address 192.168.100.10. The increase in the traffic was immediately displayed in the terminal along with several SYN_RECV acknowledgements without the TCP three-way handshake being completed. Please see the image below:
Fig. 23. Netstat detects the hping attack from the attacker’s IP address.

To further analyze the traffic in Netstat, the command “sudo netstat -an | grep SYN_RECV | wc -l which displayed the number of SYN_RECV received was 182, and then a few seconds later 312 SYN_RECV was registered. Please see figures below:

The hping3 attack is requesting three way handshakes and the server is overloaded. A regular server would be a low number and a high load aka an attack would be a high number .

Netstat registered (SYN_RECV) overloading on the unprotected server.

Signature-based detection was enacted to detect known attack patterns, such as SYN flood attacks, by defining rules to detect repeated connection attempts that match malicious behavior. The following rules were implemented:
sudo iptables -A INPUT -p tcp –-dport 80 –syn -m -limit –limit 3/minutes –limit– burst 5 -J ACCEPT which limit the amount of traffic flow that is accepted.
sudo iptables -A INPUT -p tcp –dport 80 -syn -j DROP which drops all SYN packets which are suspected to be malicious.
Anomaly-based detection helped to detect abnormal network traffic behaviour by using custom rules and commands to spot strange or unexpected activity to catch common DDoS behaviors. Both a combination of the tools used in traffic pattern analysis and signature-based detection were also effective in capturing the abnormal spike in the network traffic. Below is an image of the various tools registering the anomalies:
Fig. 24. Tshark output shows server unreachable

Fig. 25. Iptables showing dropped SYN packets during flood

Fig. 26. Netstat shows abnormal connection activity.

4.3 Countermeasures:

Immediate Actions Taken on the protected server:

When the attack was detected on the protected server, the malicious IPs and traffic were immediately blocked based on the rules defined in the chain rules in the iptables seen below:
. After the first 5 connection attempts, rate limiting is activated to limit only 3 connections per minute and the number of requests that were coming from one source. There were 14 packets accepted and 8422K dropped as seen in the image below:
Fig. 27. Iptables showing 8422k packets dropped and 14 accepted

These actions mitigate the impact and prevent further damage on the protected server by \allowing a few traffic to allow legitimate users access and handle spikes in the network traffic.
Long-term Solutions:

Implement an Intrusion Detection / Prevention System like Snort to help further secure the network parameters.
Install Wireshark for real-time network traffic analysis since it is a more advanced tool.
Immediate Actions Taken on the protected server:

There were no countermeasures on the unprotected server so that the deadly effects of a DDoS attack can be clearly noted. In a real-life scenario, reenabling the UFW, installing Wireshark and an Intrusion Detection/ Prevention System (IDS/IPS) like Splunk would be very effective. Better yet, with the ever evolving landscape in the technological arena, there might be a need for a greater defensive system like an AI-powered system [1].

Conclusion:

This project was a very transformative project because it demonstrates that if someone has a computing device but cannot afford an expensive IDS/IPS, having this knowledge of enabling basic command line tools is a major step in establishing a protective home or enterprise network. This project exposes the effects of a DDoS attack on two servers, an unprotected and protected server, which demonstrates how even having the least bit of prevention mechanism, such as installed tools like Iptables, can be defensive against a DDoS attack. The outcomes were significantly great, which are:

The unprotected server was taken down by the attacker.
The protected server withstood the attack.
Spike in the network traffic was detected by tshark, and netstat was effective
Defined chain rules in the iptables helped to mitigate the attack on the protected server.
Finally, this project demonstrates that using basic command-line installed tools can also form a layered defensive approach, if high-end IDS/IPS tools are not affordable, and they are available for use by anyone who needs it.

Future Recommendations:

Further advance the DDoS attack using botnets against the protected server and monitor how it keeps up under various levels of attacks..
Enhance the attacks to use UDP Flood, HTTP flood, and ARP spoofing.
Integrate System Information Event management (SIEM) tools and a playbook to establish guidelines to identify vulnerabilities before there is a disruption in the organization’s network [2].
Design and develop a framework to mitigate DoS/DDoS attacks using IPtables Firewall (Chatterjee, 2013).
References

[1] A. S. Ahmed, M. A. Rashid, and M. A. Rahman, “Design and Development of a Framework to Mitigate DoS/DDoS Attacks Using IPtables Firewall,” ResearchGate, Jan. 2013. [Online]. Available: https://www.researchgate.net/publication/237006612_Design_and_Development_of_a_Framework_to_mitigate_DoSDDoS_Attacks_Using_IPtables_Firewall

[2] Datadome, “What Is a Network Intrusion Detection System (NIDS)?,” DataDome Learning Center. [Online]. Available: https://datadome.co/learning-center/what-is-a-network-intrusion-detection-system/

Simulated and monitored DDoS attacks in VirtualBox using Kali Linux and Ubuntu

Alem Djokovic — Tue, 06 Jan 2026 10:36:20 +0000

For my CSCI 400 final project, I designed a controlled infrastructure resilience experiment to evaluate how system reliability changes under a SYN flood (DDoS) attack — and how lightweight defensive controls can maintain service availability.

What I built:
• Isolated 3-VM lab using VirtualBox (Kali Linux traffic generator + two Ubuntu servers)
• Infrastructure configured with static IPs and internal networking to simulate production-like conditions
• Automated attack simulation using hping3 to generate sustained SYN traffic

Observability & Monitoring:
• Used netstat, tshark, ping, and htop to monitor connection states, packet flow, and CPU load
• Identified service degradation through SYN_RECV connection buildup and resource exhaustion

Reliability & Hardening:
• Implemented iptables rate-limiting and filtering on the protected server
• Verified that defensive rules dropped malicious traffic while preserving legitimate requests
• Demonstrated how simple firewall policies can function as first-line reliability safeguards

Outcome:
• Unprotected server became unreachable under load
• Protected server maintained stability and service availability

Takeaway:
This project reinforced how infrastructure hardening, observability, and proactive traffic control directly impact uptime. Even minimal, low-cost controls can significantly improve system resilience when designed intentionally.

Next steps include integrating IDS/IPS tools, log aggregation, and SIEM pipelines to automate detection and response.

https://youtu.be/5qy5xHY9ABo