DEV Community

Cover image for I Spent Hours Fighting a Silent Subnet Conflict to Build an Isolated ICS Security Lab (And What It Taught Me About the Linux Kernel)
404Saint
404Saint

Posted on

I Spent Hours Fighting a Silent Subnet Conflict to Build an Isolated ICS Security Lab (And What It Taught Me About the Linux Kernel)

#gns3 #ot #ics #cybersecurity

By RUGERO Tesla (@404Saint).

We’ve all been there. You have a free afternoon, a great idea, and a completely false sense of security about how long a deployment is going to take.

My goal for the day was simple: build a pristine, fully isolated Operational Technology (OT) and Industrial Control Systems (ICS) security sandbox on my EndeavourOS host. The blueprint in my head was beautiful: GNS3 holding a central ethernet switch, a Kali Linux VM acting as the auditor node, an OpenPLC instance simulating a programmable logic controller, and a Fuxa container hosting a custom visual HMI dashboard.

Twenty minutes, right?

Fast forward a few hours later, and I was deep in the Linux kernel virtual file system decoding hexadecimal strings over raw TCP socket structures just to figure out why my network interfaces were ghosts.

Here is the story of how a standard homelab setup turned into a masterclass in kernel routing, aggressive firewalls, and micro-container constraints—and how you can avoid the exact traps I fell into.

The Illusion of a Simple Setup

If you’ve ever worked with GNS3, you know it’s an incredible tool for virtualization. But when you mix it with Docker containers, things change. GNS3 strips away the standard Docker network daemon translation layer and binds container interface namespaces directly to its own virtual switch fabric.

I dragged my nodes onto the canvas, wired them to a central switch, and explicitly typed out what I thought was a standard static network map inside the Debian-based containers using a classic 192.168.1.X block:

auto eth0
iface eth0 inet static
    address 192.168.1.30
    netmask 255.255.255.0
Enter fullscreen mode Exit fullscreen mode

I booted the canvas, fired up my Kali VM browser, typed in http://192.168.1.30:8080 to access the OpenPLC web dashboard, and... nothing.

Connection Refused

Flat connection refused. The lab was entirely dead on arrival.

The Rabbit Hole: When the Tools Disappear

Naturally, my immediate instinct was to drop into the auxiliary terminal of the running OpenPLC node via GNS3 to check the socket statuses.

Instead of a clean bash prompt, the terminal exploded into control-character distortion. Minimalist container images don’t bundle robust interactive terminal binaries, meaning typing standard strings ended up looking like a scrambled mess: l^H^Hs^H^H.

Fine. Plan B. I dropped into my main host terminal to execute a standard docker exec command to check the listening interfaces inside the container namespace using modern replacements like ss or netstat:

sudo docker exec -it badf2aaf2595 ss -tuln
Enter fullscreen mode Exit fullscreen mode

The container immediately snapped back:
OCI runtime exec failed: exec failed: unable to start container process: exec: "ss": executable file not found in $PATH

Production-grade security containers are stripped down to the bare metal to reduce attack surfaces. High-level user-space diagnostic binaries do not exist.

Going Kernel-Level

This is where the real engineering began. If the userspace utilities are missing, you go directly to the source of truth: the Linux kernel abstractions inside the /proc filesystem. I forced the container to print out its raw, active network sockets straight from the kernel:

sudo docker exec -it badf2aaf2595 cat /proc/net/tcp
Enter fullscreen mode Exit fullscreen mode

The kernel spit back raw hex lines:

sl  local_address rem_address   st tx_queue rx_queue
 0: 00000000:1F90 00000000:0000 0A 00000000:00000000
Enter fullscreen mode Exit fullscreen mode

Let's look at that local address string: 00000000:1F90.

  • 00000000 translates to 0.0.0.0 (listening on all interfaces).
  • 1F90 converted from hexadecimal to decimal is 8080.

The kernel proved that the web panel was listening inside the container namespace! But notice what was completely absent: there was no hex line ending in 01F6 (decimal 502, the standard Modbus TCP protocol socket).

This gave me a major clue: the application container was technically alive, but the Modbus protocol engine hadn't initialized yet because it was waiting for an operator to log into the web GUI and click "Start PLC". But I couldn't reach the GUI.

The Plot Twist: The Ghost in the Network Stack

I tried bridging the GNS3 network directly to my native desktop browser using a Cloud Node to bypass VirtualBox entirely. Still, total radio silence.

I opened a host terminal and typed sudo ip addr show. The moment the output printed, the entire mystery evaporated. I saw my physical network interface:

enp0s20f0u5: inet 192.168.1.100/24 ...
Enter fullscreen mode Exit fullscreen mode

My actual, physical hardware home router was hosting my entire room on the 192.168.1.X block. By choosing that exact same subnet pool inside the virtual GNS3 switch canvas, I had created a catastrophic routing conflict in my host operating system's kernel.

Whenever my computer tried to route a packet to 192.168.1.30, the kernel routing tables panicked. It couldn't distinguish whether the target address belonged down the virtual GNS3 wire or out through my physical ethernet cable into my physical room. To add insult to injury, my host operating system (EndeavourOS) runs an aggressive default firewalld profile that was actively dropping untrusted cross-zone virtual bridge traffic.

The Resolution: Pure Isolation

The solution required an architectural shift. To build a pristine, conflict-free simulation space, you must separate your lab from reality.

I tore down the configuration files and completely re-mapped the virtual layout to a unique, non-overlapping private pool: 10.10.10.0/24.

  • Kali Auditor VM: 10.10.10.5
  • OpenPLC Engine: 10.10.10.30
  • Fuxa HMI Graphics: 10.10.10.40
  • Simulated Field Devices (VPCS): 10.10.10.101 and 10.10.10.102

GNS3 Topology

I booted the clean topology, launched the native browser inside my Kali Linux node, and navigated to http://10.10.10.30:8080.

openplc instance

The web dashboard loaded instantly. I authenticated, hit the Start PLC compilation engine to trigger the runtime daemon, and dropped back out to my Kali terminal to run a definitive
verification scan:

sudo nmap -p 502,8080 10.10.10.30
Enter fullscreen mode Exit fullscreen mode

The output printed a flawless victory signature:

PORT     STATE SERVICE
502/tcp  open  mbap
8080/tcp open  http-proxy
Enter fullscreen mode Exit fullscreen mode

Port 502 was officially wide open on the wire. The virtual industrial plant was alive, isolated, and completely transparent to my auditor node.

Lessons from the Trenches

What started as a routine lab deployment turned into a critical reminder of how low-level systems interact:

  1. Subnet isolation is non-negotiable: Never let your virtual lab environments mirror your physical host infrastructure.
  2. Know your kernel mappings: When containers are stripped of diagnostic tools, knowing how to parse /proc/net/tcp directly from the kernel space is a superpower.
  3. Beware of double-initialization: In minimal environments, rapid web UI inputs can cause underlying application binaries to spin up duplicate threads, creating internal race conditions over sockets. Slow down and verify via network scans.

Now that the networking foundation is solid, my industrial playground is ready. Next up on the roadmap is configuring custom graphic widgets in Fuxa to map live holding registers, and writing python injection scripts to interface directly with the Modbus coils.

If you want to deploy this exact sandbox for your own research without hitting the same roadblocks, I’ve documented a comprehensive, beginner-friendly UI walkthrough and a deep-dive troubleshooting ledger in the repository below:

👉 GitHub Repository: gns3-ics-security-lab

Have you ever lost an entire afternoon to a silent subnet overlap or a hidden firewall drop zone rule? Let's talk about it in the comments below!

Top comments (0)