DEV Community: Bogdan Milojević

Networking Fundamentals for Software Developers | Part 1: OSI and TCP/IP models

Bogdan Milojević — Mon, 24 Jul 2023 14:04:28 +0000

👋 Welcome, welcome, welcome

Hi there again! As you already know (you probably don’t since you have better things to do), I want to become a better developer and I decided to learn more about networking. So naturally, I spent past few weeks wrapping my head around layers, protocols, reverse proxies and other buzzwords in networking domain.

I’ll share fundamental concepts I've learned along the way in this article series. Networking is huge topic so I won’t get into too much details for various reasons, one of them being that unfortunately, I don’t know everything about everything (yet 😅) but I’ll share a list of useful resources for further learning.

Part one focuses on a high level overview of TCP/IP and ISO OSI models with an concrete example to guide us and light the dark path ahead.

👨‍🎨 I’m particularly proud of all illustrations, which I designed myself.

📖 Bit of History: Protocol Wars

Once upon a time, few lads had a great idea. They wanted to connect two computers and share data between them. This revolutionary concept led to the birth of Arpanet. It didn’t take long for other companies and organisations to jump aboard and build their own networks.

These networks utilized proprietary technology and architecture, with their own set of protocols in place. What happened when two computers from different networks wanted to communicate? Radio silence. They couldn’t understand each other. To overcome this issue, the need for common rules and standards in the form of a networking model became evident.

After numerous meetings and intense discussions known as the 'Protocol wars', two models emerged: the one we mostly use, TCP/IP, and the OSI model. Both models are simply sets of standards which engineers adhere to when developing computer networks.

Eventually, TCP/IP protocol stack won since it proved to be more practical. Leading vendors as well as ARPA switched to TCP/IP marking the end of decades long dialogue. Although TCP/IP became the dominant protocol stack, the OSI model retained its usefulness as a reference.

If you have taken any networking classes, you may have noticed that educators primarily use the OSI model when explaining computer network concepts. This is not a big issue since layer 1 - 4 are exactly the same in both models. When network engineers refer to Application layer, they’ll always call it “Layer 7” regardless of the protocol stack.

The bottom line is that we need standardized way for establishing communication between two systems. Imagine you had to develop multiple versions of your app, one that’ll work over wi-fi, one for fibre-optic cables and every other medium there is. Fortunately, we don’t have to do this since a long time ago some smart people agreed upon standardized, globally used networking model that we take for granted.

One, Two, Three, Four… Seven?

Both TCP/IP and OSI models organize functionalities into separate layers. Each layer has a set of protocols that operate on it. Dividing the functionalities into layers provides easier development, maintenance, and interoperability of network protocols.

For example, layer 3 data unit (packet) can be transmitted across various layer 1 mediums such as radio waves, copper or fibre-optic cables. The underlying technology doesn’t matter as long as we follow common standards.

Let’s finally take a look at the TCP/IP and OSI models:

OSI models consists of seven layers and TCP/IP model has four layers (five-layer model is also commonly used like in the picture above). As already mentioned, OSI is a conceptual model, something to refer to. In practice, presentation and session layer functionalities are also handled within application layer.

When considering the flow of data within the protocol stack, each layer from 1 to 4 is associated with a specific name and has its own Protocol Data Unit (PDU) defined:

L1 - Bits
L2 - Frame
L3 - Packet
L4 - Segment
L5, L6, L7 - Data

🚀Journey Through The Layers

Lets take a journey through protocol stack with a concrete example. In this example we’ll send simple GET request with JavaScript fetch API and explore what happens at each layer:

fetch('http://api.apis.guru/v2/providers.json')
      .then(response => response.json())
      .then(json => console.log(json))

We are taking top to bottom approach meaning L7 will be the first one covered.

Once we send a GET request via HTTP, additional HTTP headers such as IP address of destination server, type of HTTP method, request URL etc. are filled in. HTTP protocol defines a number of headers. You can see a full list of standardized HTTP headers here. It’s possible to define custom headers if your application needs them.

For simplicity sake, we are going to skip layers 5 and 6, like a true network engineers.

Layer 4 - Full Message Delivery

As data passes through Layer 4, it is encapsulated within a TCP segment, which includes the source and destination ports as well as sequence number. Knowing the destination and source ports is crucial as it ensures that the data is directed to the correct process on the both systems. Additionally, the TCP segment contains other headers, as illustrated.

Layer 4 is responsible for end-to-end message transmission. It's got two major protocols under its belt, TCP and UDP – the dynamic duo of data delivery!

TCP is a stateful protocol designed to ensure error-free and complete data transmission, it will never let you down. On the other hand, UDP doesn’t bother with data integrity and transmits data without verifying if it reaches the destination, a bit like sending a message in the bottle, hoping it ends up at the final destination.

When using TCP, a three-way handshake is performed before sending actual application data in order to establish a reliable connection with the server. It’s like asking the server “Hey man, I want to send you some data, are you able to communicate with me?”

Layer 3 - All About IP

Way down we go. All the way down to the network layer, home of the famous IP address. When sending a message to your buddy, you need to know his number, right? It’s pretty much the same concept with networking.

Our TCP segment gets wrapped once more, in a IP packet this time, including source IP address, destination IP address and a bunch of other headers.

Every device with internet access has its own IP address. Go to the whatismyip.com and find out your IP address and then share it with us in the comments! Just kidding, don’t share it with anyone, not a good idea.

IP addresses are built for routing, i.e. finding a path for data to travel the vast network, across thousands of routers, switches, servers and other beautiful stuff.

Your public IP address is the address of your router, i.e. your network address, assigned to you by your ISP (Internet Service Provider). It's the address that is visible to the external world, and it is used to identify your network on the global internet.

Every device connected to your router has its’ private address which is accessible within your local network. Network address translation (NAT) is the process of mapping multiple private IPs to single public IP address. NAT is a crucial mechanism for conserving public IP addresses, as there is a limited number of IPv4 addresses available.

Layer 2 - Home of The Switch

Data link layer will take an IP packet and wrap it in a frame with source and destination MAC addresses. MAC address is also known as a burn-in address. Every device connected to a network has a unique MAC address which is assigned by manufacturer.

While IP addresses are used for end-to-end transmission, MAC addresses are used for hop-to-hop transmission. If you are sending a message to a computer outside of your network (or a subnet), all packets will be sent to your router, also known as gateway.

While router connects multiple networks, switch connects multiple hosts in a local network. Traditionally, switches used to exclusively be layer 2 devices. Nowadays there are multi-layer switches that can operate on layer 3 as well.

ARP is a layer 2 protocol used to map IP addresses to MAC addresses. Here is an awesome video that further explains MAC addresses and data link layer.

Layer 1 - I’m All About That Bits

Layer one defines physical connection between two devices and handles sending digital data through physical mediums such as coper wires, radio waves and optical cables. Layer one deals with raw bits and handles their transmission.

In other words, the main job of layer one is to take digital bits and transmit them using electrical signals, electromagnetic waves or light. This process is known as modulation.

Now back to our example. Finally, your network card will take data link frames and convert them into appropriate physical signals depending on the medium and transmit them over to your router. Your router then forwards the packets to your ISP.

Your ISP router determines the next hop router based on the destination IP and its routing table. Of course, our computer isn’t directly connected to the backend server. There is a whole jungle of switches, routers, firewalls, proxies etc. in between so this process is repeated until packet reaches its destination.

This resource is great for learning more about routing.

😱 Connection Is An Illusion

I briefly mentioned a concept of TCP connection, but the thing is, it doesn’t exist (in a sense). When I think about a connection, I imagine two sides tightly coupled to one another. In networking of course, that’s not the case, there are many nodes between the two sides, without a direct connection between.

Connection is just an abstraction which is only known to end parties, nothing in between is aware of it. At layer 3 each packet that is a part of this “connection” can take a different route from side A to B, additionally, packets will arrive out of order, some will be dropped and re-sent, depending on many different variables (internet is packet-switched network after all).

TCP takes control of this and it makes sure data arrives reliably and in the correct order. Software that is using TCP sockets can take this for granted, send and receive data smoothly without worrying about the underlying process.

So a socket is an operating-system object created on behalf of a process to represent a network connection (client/server) or listening port (server) and referred to by that process in system calls used for communicating over that connection.

→ Don’t confuse TCP socket with WebSockets layer 7 protocol

🧂 Grain of Salt

In the real word, things often aren’t cut and dry. There are protocols that may span across multiple layers or stand between some layers, without clear separation. Networking models are our best effort to make sense of everything that happens but that’s not possible all of the time, so take everything with a grain of salt.

To add to that, different networks may use different protocols, equipment and tech but the important thing is that everything is compatible and we can build software regardless of the underlying infrastructure. As a software developer, you really want to focus on layer 7 and layer 4 since that’s what you’ll most often interact with.

So thank you for reading this far, now you have a high-level overview of networking models and fundamental concepts. But we are far from done. Stay tuned.

Further learning:

Becoming the Best Backend Engineer This City Has Ever Seen

Bogdan Milojević — Sun, 25 Jun 2023 18:44:39 +0000

Yup, that was a Suits reference. It's a great show, make sure to check it out (after reading this post).

🥳 Welcome

Congratulations! You were exploring the dark depths of the web once again and found this blog. Nice to meet you. My name is Bogdan, and I've been working as a junior backend software developer for around six months now.

👉 Let's get to the point

I'd say I had a great start with the first project I worked on (let's call it Project X). Project X was quite challenging and demanding for a beginner, but it gave me an opportunity to grow as I had to adapt and learn new technologies and business logic in a short time frame.

After three months on Project X, I became more confident in my knowledge and even got promoted! Things were going great, but then... the project ended, and another firm took it over. This meant that the whole team working on it had to be transferred to another project (let's call it Project Eco). At the time, I didn't think much of the whole situation.

Colleagues assured me that the new project was going to be much better in every aspect and that we shouldn't worry. After all, Project X wasn't ideal. It had a bunch of problems regarding management, unclear requirements at times, complex business logic, and did I mention a big part of it was written in ancient Java EE with Primefaces handling the UI?

💤 Stagnation

Then, I started working on Eco, which was (and still is)... kind of chill, at least from my perspective. The workload is somewhat lower, and tickets assigned to me aren't that complex or challenging most of the time. While this may be great for someone who has been in the industry for a long time and has experienced "everything," it certainly isn't ideal environment for a junior hungry for more.

I know what you're thinking. You wouldn't trust a junior with designing and implementing the whole system architecture or some similar task that typically requires years of experience and deep domain knowledge. That's perfectly fine. I wouldn't either.

So, time passed, and I noticed I wasn't really growing as a developer as much as I did while working on Project X. I had to do something about that.

🍪 I hope I haven’t bored you with my backstory. If you are still here you are awesome! Please have a cookie.

📝 The Plan

There is one great thing about lack of the workload which is all of the extra time I get to spend on learning various topics. Over the next few months my plan is to learn a lot. While browsing LinkedIn i stumbled upon a post with some tips for becoming a better developer. One of them said ”Learn fundamentals, not frameworks” and that’s what I decided to start with.

My goal is to understand fundamental backend concepts really well and only then focus on concrete technologies and implementations. I want not only to understand these concepts but also the reasoning behind them. Why do we need them? Why are they created in the first place? What are the tradeoffs?

There is this great video (embedded below) by Hussein Nasser that inspired me to take this approach. He also has whole playlist on backend engineering as well as “Fundamentals of Backend Engineering” Udemy course.

Here is the list of some things that I’m going (or wish 😅) to study in the coming months/years:

Networking & Communication Protocols
Web Servers
Database Engineering
Caching Systems
Messaging Systems
Event Sourcing
DSA & Design Patterns
Software architecture

Such an overwhelming list! Maybe I am overambitious but shoot for the stars, land on the moon. Time-wise, I don’t want to set any deadlines for myself. I know those things take years to fully understand and master and that no one knows it all so I’ll take my time.

As Hussein said in one of his videos, there’s no such thing as an expert. An expert is someone who knows everything in a particular field and when one says they know everything they are just limiting their own growth.

🌐 Networking

At the time of writing this post I already started my journey. I decided to go with networking first as I’m already familiar with fundamentals thanks to networking class I took in college. I’m pretty sure I won’t be dealing with most of the stuff below layer 7 in my everyday work but I find networking fascinating and it won’t hurt to know how everything actually works behind the scenes.

Software developers often hide behind bunch of abstractions without really knowing how their GET requests are actually sent across the network (trivial example). I believe that comprehensive networking knowledge will make me a better engineer even if I won’t use it everyday. I briefly looked at CCNA certification but I don’t think it’s worth the time at the moment.

You can find a great free playlist that I started with below:

Networking is an excellent starting point as it lays the groundwork for understanding how information flows across systems. While you may not directly apply all networking knowledge in your day-to-day work, comprehensive understanding of the underlying mechanisms will undoubtedly make you a better engineer.

Conclusion

ChatGPT once said:

Transitioning from a challenging project to a less demanding one can lead to stagnation if not addressed effectively. The key is to make the most of the extra time available and focus on continuous learning.

In the end, becoming the best backend engineer the city has ever seen requires a combination of technical expertise, continuous learning, and effective communication. Embrace the challenges, stay dedicated to your growth, and remember that expertise is a journey, not a destination. With the right mindset and effort, you can reach new heights in your career.

Beautiful words my friend, beautiful words.