DEV Community

Rondo
Rondo

Posted on • Originally published at dev-rondo.com

Ubuntu NetworkManager Fooled Me with Routing Metrics

Recently, I was testing a PC-based NVR that supports Dual-LAN and uses Ubuntu.

The testing environment was very simple: One NIC was connected to an isolated network that has a switch connected to a lot of cameras. And the other NIC was connected to internet.

And not surprisingly, I could clearly see the camera screens via embedded VMS viewer. Everything seemed working well.

But I soon realized that I couldn't use internet at all. That didn't make any sense for me since I used internet about just few minutes ago.

Since connecting to internet is very important for the product(It allows us debug remotely when issues happen in sites.), I had to deal with this issue.

First, I tried the simplest method : I disabled network interface that was connected to internet and enabled it. Result? I could access internet again. But only for about 4 minutes.

I repeated the same treatment several times, but the results were the same. I could access internet only for a few minutes. Every single time. I completely had no idea. I thought it was because of hardware, but my guess was completely wrong.

The key was routing metrics. I knew the concept, but I didn't know that Ubuntu's NetworkManager can change the metrics on its own. For some reason, NetworkManager kept assigning a lower metric to the isolated network than the internet interface.
(For context: a lower metric means higher priority.)

So I googled a command to control the metrics by hand. I could do this using nmcli

sudo nmcli connection modify 'connection_name' ipv4.route-metric (value)

After I fixed the metric value 999(high enough value) for the isolated net, I could finally connect to internet without disconnection.

To release this product, I needed to make sure that the product can always be connected to internet. My solution was to attach stickers to let users know which NIC is for internet and which is not and add some code to execute the command above.(It was not easy since we're using C and connection_name is necessary.)

My point is that if you're having trouble with dual-LAN hardware over Ubuntu, check your routing metrics before blaming the hardware.
Start by checking : ip route show

Top comments (0)