DEV Community

Evan Lin
Evan Lin

Posted on • Originally published at evanlin.com on

Strange Loop 2019: Networking and Go - An Engineer's Journey

#go #networking #tooling

Preface

Listening to podcasts is one of my ways to cultivate my listening skills and learn, and Software Engineering Daily is one of my favorite podcast channels. This time I heard a very interesting topic, which is "Go Networking with Sneha Inguva", the content is from an engineer from Digital Ocean to share why Digital Ocean wants to use Golang to build their Networking Team's network-related tools.

This article is mainly based on Sneha Inguva's recording content at the Stange Loop conference, and Software Engineering Daily's Podcast as a supplement. I hope to help everyone organize the relevant content.

Video Link: https://www.youtube.com/watch?v=XqKvgzXCoXc

Podcast Related Content: Go Networking with Sneha Inguva

“Networking and Go: An Engineer’s Journey” by Sneha Inguva

Networking and Go: An Engineer's Journey (Strangeloop 2019) from Sneha Inguva

Slides and Video: link

The introduction is mainly divided into several main themes to explain why Digital Ocean uses Golang as the programming language for Networking programming.

When to use Go

Here, the speaker summarizes some of the advantages of Golang, which will not be detailed here.

  • Concurrency
  • Server-side
  • Great for CLI tools
  • Easy-To Fuzz-test
  • Easy to write REST/RPC services

The picture above explains how Digital Ocean uses golang to build related tools.

The network stack

Here, the seven-layer architecture of OSI and the five-layer architecture of TCP/IP are introduced. The basic common knowledge will not be detailed here.

Extended Reading:

Networking primitives

Then, start to introduce how to use Golang to build some network native tools.

First, introduce Layer 7 Load Balancer:

  • Use HTTP protocol.
  • Accept client-side connections.
  • Pass client-side request to one of backends.
  • Return server-response back to client.

Introducing the package:

  • net/http

This is quite basic, that is, after reading the HTTP request connections content, go to find a backend (there is a way to choose here, or more details can be implemented) connection, that is, by sending another HTTP request to get a reply, and then return the result to the client side.

Here you can see that after receiving http.request, a backend is immediately selected through random numbers, and then the request is sent to the backend.

Next, explain how to develop a Layer 4 Load Balancer:

In the next stage, start to develop a Layer 4, which is a TCP Load Balancer: (and supports streaming socket)

  • Use TCP protocol
  • Accept client-side connection
  • Open backend connection
  • One gorutine to shuttle packet from client-side to backend.
  • Another one gorutine to shuttle from backend to client-side.

Because TCP requires three stages: SYN, SYN/ACK and ACK, it is necessary to use two gorutines to continuously serve and complete the entire process of data transmission.

Introducing the package:

  • net

The listen part needs to be changed to net.Listen() so that TCP packets can be received.

This is to open a connection to the backend.

Here, the two connections are split into two gorutines:

  • client –> backend
  • Backend –> client

This part is simply copying data from one end to the other.

Building a DHCP Server

After Layer7 to Layer4, the speaker then shared how to develop related applications on DHCP through Golang. Of course, everyone knows that DHCP (Dynamic Host Configuration Protocol) is actually built on Layer7, but the speaker shared a part that may need to use the Layer2 API, which is that they need to check whether the MAC Address of the message sender is the same as the registered information.

Introducing the package:

These packages are mainly used for:

  • The unix package is used to parse the Socket to read the IP address.
  • raw is used for ListenPacket, which can be used for IPv4 or IPv16. The speaker explained that they mainly use the IPv4 part first.
  • syscall is used as the system Error Code detection.

This paragraph is where the speaker mentioned using the dhcp4 package to compare the source MAC Address of the request with the MAC Address that needs to be bound. Confirm that the requester is consistent with his content before sending the IP to him.

Summary

The speaker mainly shared his experience of using Golang in Digital Ocean, and also gave some examples to hope that the audience can understand that in addition to making small system tools, Golang can also do more complex (or underlying) network operations, and the efficiency is also at ease. This speech is quite interesting, because in addition to understanding that Golang can also do some underlying network operations, the speaker also clearly did some basic popular science, so that the audience can understand from not having a deep understanding of the network architecture to understanding that they can develop some small tools through Golang.

Top comments (0)