Regardless of whether you work on the front-end or back-end, I think all developers should gain some proficiency in network troubleshooting. This is especially true if you find yourself gravitating towards lower level systems programming.
The ability to troubleshoot the network and systems separates good developers from great developers. Great developers understand not just code abstraction, but understand the TCP/IP model:
Source: https://www.guru99.com/tcp-ip-model.html
Some basic network troubleshooting skills
If you are just getting into networking, here are some basic tools you should add to your toolbelt:
- Perform a DNS query (e.g.
digornslookupcommand) - Send an ICMP echo request to test end to end IP connectivity (i.e.
pingcommand) - Analyze the various network hops (i.e.
traceroute X.X.X.X) - Check whether you can establish a TCP socket connection (e.g.
telnet X.X.X.X [port]) - Test application layer (i.e.
curl https://somedomain) - Perform a packet capture (e.g.
tcpdump -i any) and what bits are sent on the wire
What IP address is my browser connecting to?
% dig dev.to
; <<>> DiG 9.10.6 <<>> dev.to
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39029
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;dev.to. IN A
;; ANSWER SECTION:
dev.to. 268 IN A 151.101.2.217
dev.to. 268 IN A 151.101.66.217
dev.to. 268 IN A 151.101.130.217
dev.to. 268 IN A 151.101.194.217
Is the web server listening on the HTTP port?
% telnet 151.101.2.217 443
Trying 151.101.2.217...
Connected to 151.101.2.217.
Escape character is '^]'.
Each of the above tools helps you isolate connectivity issues. For example, if your client receives an HTTP 5XX error, you can immediately rule out any TCP level issue. That is, you don't need to use telnet to check whether there's a firewall issue or whether the server is listening in on the right socket: the server already sent an application level response.
Summary
Learning more about the network stack helps you quickly pinpoint and isolate problems:
- Is it my client-side application?
- Is it a firewall blocking certain ports?
- Is there a transient issue on the network?
- Is the server up and running?
Let's chat more about network engineering and software development
If you are curious about learning how to move from front-end to back-end development, or from back-end development to low level systems programming, hit me up on Twitter: @memattchung
Top comments (14)
Personally I prefer tcpflow over tcpdump since it provides colouring which can make things easier to view. This is especially useful on systems where you can't easily transfer files (ie, the packet capture file) from.
In my experience ICMP is usually blocked. nmap can instead be used to test connectivity on the relevant port with a syn scan.
Cool, I never heard of
tcpflow. However, it appears this tool only handlestcp, not other transport protocols likeudp, right?Also, regarding
icmp, although it is often blocked, you can modify the encapsulation so instead of ip->icmp, you can modify it such that it rides onudp.Yea, in that case you should use tcpdump or something else. I just like the colouring.
Only what is absolutely necessary is let through by the firewall in my past experience. This includes disabling inbound UDP if the application does not need it. SYN scans work because you can poke the ports being used by the application.
Thanks Jonathan, will use tcpflow next time I deal with the packets and the directions.
Completely agree, developers not having a clue about how different components of the system talk to eachother is really sad.
During college days I had inclination towards system admin but ended up on the programming side of things, but my inclination has helped me triage issues in development much faster.
Recently got into tcpdumps and port monitoring to analyse why a webservice request to one of engines was randomly erroring with connection reset.
Agree. The best developers I've worked with weren't just great at programming, but also intimately aware of the underlying system too.
Another great tool you might want to use is
strace, assuming your application is running on linux.Sure will try it out. Thanks. Yes it runs on Linux.
This is so true, I just ran into this on a project I was working on and didn’t realize my isp was auto resolving sites for me when the weren’t giving me all the wrong results. Thanks for sharing this!
Interesting ! Can you elaborate on this — was your DNS pointing at your ISP's resolver?
I searched for something and the dns responded, even though the site didn’t exist. I then called the non existent thing and it came back with a 200 response. You can probably tell I’m very new to this and so I can’t elaborate much. My mentor caught it because it just didn’t make any sense.
I think I have a better understanding. Likely the DNS record resolved returned an IP address pointed at a different address. Despite hitting that IP and receiving an HTTP 200 OK, the web site does not exist. Well, that's my guess at least.
Yeah that sounds about right.
As a developer that started his career as a sysadmin I agree with this article. Having a basic understanding of DNS and other network stuff has certainly been very useful to me.
Hey Simon,
I had a similar career trajectory, also starting off with systems administration. It definitely goes both ways: great sysadmins know some elements of programming (e.g. using a debugger, analyzing a core dump) and great programmers know some systems/network administration (e.g. checking for host-to-host connectivity, inspecting packet payloads). In this context, the 80/20 rule — 20% of activities contribute to 80% of results — rings true.
These days, what are you working on?