Docker DNS: The Odyssey

Well, well, well...How could I imagine that configuring a VPN would not be so easy? Hahahaha, ok, I'll explain.

Backstory (feel completely free to skip it right away)

Recently I changed my workstation, I used to code on a Macbook Air before, but now I moved to a desktop, and because of it, I needed to configure my VPN all again, no big deal.

I'm currently using Deepin as my distro, and I did not have much problems until today.

The problem itself

If you have skipped the last part, just to let you know, I need to configure a VPN.

"Ok, but what does this have with docker?" - you may ask.

Well, the VPN setup came up alright, I use OpenVPN and it works just fine.

But the problem came when I tried to access a private address, visible only to the ones with VPN access, and I could not, even though I was connected, but why?

Digging in, I realized that the IP of this specific host resolved on chrome to the page I wanted, so it was a DNS problem.

The .ovpn file

At first, I had a .ovpn file, that looked like this:

client
dev tun
resolv-retry infinite
nobind
persist-key
persist-tun
proto udp
cipher zzzz
comp-lzo
verb 3
remote xxxx yyyy
#ncp-disable
auth SHA512
#float
remote-cert-tls server
#ns-cert-type server
key-direction 1
<ca>
[..]
</ca>
<tls-auth>
[..]
</tls-auth>
<cert>
[..]
</cert>
<key>
[..]
</key>

And the file above always worked well, I already used this file on Windows, Ubuntu and MacOs, always without any problem.

but this time was different, and after discovering the reason why it was not working (the dns thing), I started searching all over the internet for solutions, even though I have nearly zero knowledge of how things work with docker domain name resolution, but needed it working.

The attempts

I changed my .ovpn multiple times, to different values, parameters, options, and nothing worked. At the point, i was already thinking on changing my distro.

Also, I have to mention that, a friend of mine, a Deepin user as well, told me that he could not use OpenVPN at all on his GUI on Deepin, although by the CLI it always worked fine.

We realize that OpenVPN already ships with a specific script, named "update-resolv-conf", under /etc/openvpn/, but it was not called in my OpenVPN file, and when I did it, it did not work.

Then, the closest I got to actually getting things working, was to redirect the up and down scripts of the .ovpn file to a custom script (provided by the guy on stack-overflow (trustworthy, I know)):

up /etc/openvpn/update-systemd-resolved
down /etc/openvpn/update-systemd-resolved

but, along with it, I had to add some lines above it:

script-security 2
dhcp-option DNS <DNS-IP>
dhcp-option DOMAIN <DNS-DOMAIN-NAME>
dhcp-option DOMAIN-ROUTE .

It had worked, I could access the domain I was trying without using its IP. So NICE!!! that's it?

Oh, no...

So, today I was getting the environment running, and, not to get much deeper here, we had a service which need connection with one of our hosts (and do it by connecting to its domain). But guess what? The docker container was not finding it!

I docker exec -it <CONTAINER-NAME> sh inside it, so I could try with my own hands, and it really does not work.

Searching for what could have happened, I got to the final step:

The resolv.conf file

To be brief, resolv.conf, is a file located in /etc/resolv.conf, which, on official linux man page words:

The file is designed to be human readable and contains a list of
keywords with values that provide various types of resolver
information.

and now, if you already got it, congrats, if not, remember that user-customized script that I mentioned earlier? And the one provided by openvpn?

So, they're both meant to resolve the DNS, but the one provided by openvpn, also, include the new resolved ip's into resolv.conf file, making it visible to the entire system (local, and docker).

So, at last, probably what happened was, chronologically:

1. .ovpn file did not call the script (in other versions of openvpn it does it automatically).

2. When I called it by hand, I did not have the dhcp option params on my .ovpn file (again, probably other versions and distributions of OpenVPN do it automatically, but that's not my case, unfortunately).

3. I added the scripts and the lines with dhcp option definition, and had it working, but only for my machine, not the containers.

4. When I had the correct update to /etc/resolv.conf file, the containers started seeing the domain.

Conclusion

For the record, my version of OpenVPN was:

OpenVPN 2.4.7 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Sep  1 2020
library versions: OpenSSL 1.1.1d  10 Sep 2019, LZO 2.10

But then, as a attempt, I updated my OpenVPN to version 3, it was not that easy, there were some compatibility issues between versions of Debian and Deepin, but I could workaround over it.

When I updated to OpenVPN 3, everything worked just fine, automatically, with the original file, and everything made sense again. It really does everything automatically, and that's why I had never had problems with Windows, or Ubuntu or MacOs, because all of these implementations uses v3 o OpenVPN (Ubuntu not necessarily, but I used in the time).

By the way, if you look into the downloads page, you'll see that the GUI world of OpenVPN (the one used in MacOs and Windows), are called "OpenVPN Connect v3" by the official website, maybe you now know why is it

Unfortunately, my last move was trying to integrate OpenVPN v3 with the Network Manager from Deepin, to use it in the GUI, which would be much more convenient, but I came across this issue, as the comment I highlighted indicates, there is no such integration, at least at the time this post is being made.

Hope you liked, and hope it can help you,
Thanks for reading!