Introduction
In the background you'll see a simple PC build, in this case
- Asus Z270 Prime-A
- 16GB Crucial RAM
- Intel 6700K
- StarTech.com Dual Port PCI Express (PCIe x4) Gigabit Ethernet Server Adapter - 2 Port Network Card - Intel i350 NIC - GbE Network Card
- EVGA Supernova 750 P2, 80+ Platinum 750W, Fully Modular PSU
In the Asus UEFI make sure to enable Intel Virtualisation option.
System Config
- Installed XCP-ng v7.6 onto single SATA SSD.
- XCP-ng is tied to a FreeNAS backend.
- Mainboard Intel NIC used for XCP-ng management access - connected to internal EdgeRouter-x (ER-x) at gateway 10.0.0.1/24
Appliance ports (yes, it's a PC though in this case...)
It's important to note that these are configured accordingly in the XCP-ng control software
- NIC0: Mainboard Intel NIC
- NIC1: I350 port 1 // pfSense WAN (xn0)
- NIC2: I350 port 2 // pfSense LAN (xn1)
Ubiquity Unifi Switch 8 Managed PoE+ Gigabit Switch
- I have Unifi controller software running in an Ubuntu VM, on my primary XCP-ng server. This handles all provisioning and control over Unifi hardware.
The switch used here is the Unifi Switch 8 Managed PoE+ Gigabit Switch with SFP shown below on the left in silver/grey.
Blue cable --> Connects to the 10.0.0.0/24 network from the ER-x router
Yellow cable --> pfSense LAN
pfSense WAN port is also tied to the 10.0.0.0/24 network from the ER-x router
Black cable --> Connect to my MacBook Pro.
Ubiquity Unifi Switch 8 Port Configuration
- Port 1: Set to default 'All' Network, which by default is VLAN1 (trunking port).
- Port 2: Set to a custom Profile tied to Native LAN VLAN2 // This is just a LAN profile with VLAN ID 2 and DHCP set to 'off'. Profile is called Profile-VLAN2
- Port 3: Black cable to the MBP - in the image below is set to Network LAN
Notice how the laptop has a 10.0.0.0/24 IP.
Now, if we change the switch port Profile to Profile-VLAN2 -- the magic happens here -- the laptop now gets a 192.168.100.0/24 IP -- YAY!! This is crucial, as this is a virtual port (xn1) within pfSense.
Great, so why would you do this?
By simply changing the port profile, any downstream traffic can now either be routed (1) straight from the main router (ER-x) or (2) through the ER-x ---> into the pFSense "Appliance" --> out the Black wire.
This setup allows you to switch the physical network tied to the Black cable 😀
I've also got IDS (Intrusion detection system) running in pFSense in the form of Suricata, which at the moment monitors the WAN interface.
I can also configure pFSense as an OpenVPN client, and all traffic through it will then be routed through a VPN -- my ultimate goal.
What's the advantage of XCP-ng (XenServer)
XCP-ng (an opensource fork of Citrix XenServer) allows us to run pFSense in a Virtual Machine (VM) which allows us to setup periodic backups of said VM via cron-jobs from within XOA (Xen Orchestra).
Since these are coupled to a FreeNAS backend, all backups are stored securely in ZFS.
Downsides to this approach
Right now, I don't have any need for VLANs through pFSense, but had I wanted to tag a VLAN entering the pFSense WAN port -- this is not possible as pfSense reports there are now VLAN capable interfaces attached.
Xen/XCP-ng has support for SR-IOV but I haven't gone down that rabbit hole.
Generating VLANs
I wanted to be able to tag traffic coming out of xn1 (pfSense-LAN port) with a custom VLAN tag, but couldn't find away to do this; (1) it seems pFSense doesn't have the ability to do this (??) but only tag/decode incoming traffic via the WAN interface (2) it wouldn't be possible since XCP-ng insulates it from VLAN capable hardware (which the I350 NICs are capable off, 802.1q).
I'd appreciate any correction to my statements above, as I'm sure there'll be someone better informed regarding tagging ports from within pFsense -- if it's possible, do let me know!
If you got this far, share your thoughts -- I'm keen to hear from you!!
Tweet to me @bsodmike :)
Top comments (1)
I use XCP-NG with separate Vlans for my router.
What you do is create a vif for each vlan you want the router to route traffic for then attach the vif to the vm. This is good for a small number of vlans and the simplest way to do it, you do have to restart the vm tho to attach new vlans.