DEV Community: Robb Manes

What a Video Game for Kids can Teach you about Software Engineering and Infrastructure

Robb Manes — Sun, 19 Sep 2021 17:39:25 +0000

I'm an avid gamer. I play almost everything, and most recently I've been getting in to automation games like Satisfactory or Factorio. After hundreds of hours in building factories in these games my Steam recommendations decided to show me Autonauts, and at first glance I was a bit appalled at the recommendation. It seemed very childish and the playground-toys theme turned me off of it for a while, until one day I decided to pick it up and, over the next one hundred hours of play, realized it was the absolute best simulation of all of the facets of IT Infrastructure I've ever seen.

Simulations of real-world engineering and software exist everywhere in video games these days. I was shocked at how extensive you could make faux electronical devices in Minecraft when the Redstone update was introduced, and a quick search on your favorite web browser will show people have made entire in-game CPU'S and complex computational devices in massive feats of computer engineering - all within a game about surviving and crafting.

There are quite a few pure programming games out there; I love Zachtronics games, which between Shenzen I/O, TIS-100, and Infinifactory, can satisfy my need for programming outside of work. I'm a huge fan of pure logic games as well such as Baba is You, which looks playful but carries a deep approach to puzzle solving utilizing rulesets. This being said, the target audiences for these games are clearly those who enjoy logic puzzles and already have some experience in them, perhaps an older-than-teenager audience.

Back to Autonauts. I launched the game with the images I'd seen so far in mind; everything was blocky and cute, and everything had a smile on it in a preschool-wonderland sort of way. Like any other "Survival Crafting" game, was put in an untamed area and had to scavenge for basic resources to begin the progression from humble vagrant to planetary tycoon as so many other games encourage the player to do. I made a workbench with my own hands, made an axe, and started chopping down trees and refining logs into planks.

After the initial work, the tooltips encouraged me to craft a workbench specifically to make robots, or "Bots". I did so, and then used it to craft my first Bot, and then the true genius of Autonauts began to shine.

Using my new Bot, I was able to "Teach" it to chop trees for me by simply mirroring the action. As I did so, the Bot translated this into a Scratch-like language on-the-fly with simple instructions that I could click and drag, including the addition of loops or if/else statements with drop-down qualifiers. It was very simple, and then after a few more Bots and a few more sessions of teaching and rearranging codeblocks, I had a fully automated forestry operation going, complete with re-planting.

At this point, I realized I likely never had to chop down a tree by hand in this game again.

Things began to quickly accelerate. I had quarries of stone and clay worked by machine hands, courier networks delivering to a central warehouse, and teams of assemblers/crafters distributing tools such as axes, picks, and shovels to the various resource sites, each managed autonomously and independently in teams of Bots. Bots need to be "recharged" by turning a crank, and while I did this by hand, after a hundred Bots were made I made a special team of Bots to turn the cranks on other Bots, removing all of the human intervention everywhere I could in a flurry of automation.

Eventually, after certain milestones I received upgrades; I could now craft machines that made "Colonists" and with that came a lot of new technologies to dabble in such as Agriculture.

I now had "Colonists" to take care of, babies spawned out of a machine that needed food, housing, and clothing. I set up these teams quickly, creating Bot-tended fields of berries and lettuce, and setting up orchards of Apple trees. I created a fishing team, catching and slicing up salmon, and fed everything to the colonist-babies I suddenly was responsible for. My automatic empire grew to support these colonists, and in return colonists produce a resource called "Wuv" which really are just tokens you can put in a sort of Research Machine to advance your technological level, letting you build new machines and better Bots. Eventually even the colonists themselves leveled up, growing into having more and more needs, and I had to adapt previous teams of Bots and make entire new teams to hit the ever-shifting target of needs for these growing colonists.

It wasn't until I had to do my first very-real upgrade problem of infrastructure that all of the similarities of my experience in IT, Development, and Infrastructure began to figuratively bash my head in. The problem was this: I had a large number of entry-level Bots which are slow and have multiple drawbacks, but were otherwise working just fine. I had intermixed teams of Bots of varying levels and effectiveness, and it was creating intermittent poor performance for one of my teams as sometimes there would be a surplus of materials and other times a drought as the different-level worker Bots performed tasks at different rates. I sought to consolidate skill levels by upgrading teams to the new Mk 2 level of Bot, and decommission the old ones, and went about doing so. There were a few hiccups, but I had felt I had everything under control until I scrolled to a side of my base and saw this:

That is a tower of stacked and still wriggling salmon that is reaching the stratosphere.

I couldn't find the problem immediately. I went to the fishing teams, and couldn't see which unit was responsible for the salmon; I had just replaced my older Bots. So then I realized I had switched the duty to "fetching" the salmon to my Kitchen team, which led me to the culprit; the sole "Salmon Fetcher" Bot had a "If crate of salmon is full" instead of "If crate of salmon is NOT full" clause which caused it to run once and then ponder what to do ever after. Therefore, the fisher Bot just kept fishing, and piling the salmon on top of each other, forever. I thought I had directly copied this code to fetch salmon from an older Bot but clearly I had changed something incorrectly in the course of my upgrade, and really had no way to know what was different.

This isn't the only example I could make of this being a direct allegory to positions in DevOps, Product Manager, Development, QA, Support, or any really software infrastructure position. The Colonists are really just customers who have the very-real shifting expectations of delivery as their needs are being met in various ways. Upgrading my infrastructure was always a hassle, and I had to make decisions on when and where to impact my production based on what was most important to me at the time. I had to decide which Services mattered most, and observe multiple failures to ensure that all edge-cases were caught lest suddenly there be unexpected downtime in production until the issue was caught, rectified, and prevented for the future.

One specific, important example I'd like to share is when I had to shift my "services" from a monolithic model to a microservice model. As my requirements and physical areas to access expanded, it became better to have resources not all made in my central "factory", so I undertook an effort to move certain types of goods to be made in different locations; tools would be built in the "tool" factory, food and such would be processed in a specific location, etc.

There are so many examples of how this game is the perfect, most real-world example of working in IT that I can't list them all. It is a children's learn-to-programming game that I found to be an excellent (and intensely fun) description of what working in IT Infrastructure is like, from a myriad of positions and roles which you have to take upon yourself. The game has no real consequence of failure; your colonists/customers can't leave you or "die" (yet, the game has a new "survival" mode coming out apparently) so it's very forgiving, especially when you forget to feed your customers for a week or so.

All in all, I'm overjoyed at finding this game, and I encourage everyone who's interested in this sort of thing to pick it up. It seems fairly age appropriate for all levels, and it truly is the best real simulation of my experience working in IT so far. Bravo to the developers. By the end of my 100 hours, I'd advanced decently far (I am a slow perfectionist in these games) but still have plenty to unlock, but am quite happy with my operation thusfar. Perhaps this isn't your thing, but I found this to be truly an exceptional example of "what do I do for a living" in a gamified, simplified way.

May you catch your bugs early and often, lest they pile up in a tower of wriggling salmon that pierces the heavens.

Running CoreDNS as a DNS Server in a Container

Robb Manes — Sat, 09 Feb 2019 06:17:12 +0000

If you've ever needed to or wanted to set up your own DNS server, then this is for you. I recently found myself in possession of a Raspberry Pi, and instead of relying on my home router for DHCP and DNS, I decided to serve both from containers on the Pi, so I could resolve all of my hosts with their respective names when I VPN back to my network. I intended to have all other requests forwarded to another server that weren't in my local network, but still service local systems with my customized hostnames.

Lately I find myself working more and more with containers and OpenShift, a Kubernetes distribution from Red Hat (disclaimer, I work for Red Hat), and in upstream Kubernetes one of the DNS servers provided is CoreDNS. I've been playing around with it for a while and thought I'd make this tutorial concerning how to launch it yourself, using the CoreDNS provided container image on Docker Hub.

I'd also like to note we're using a very, very small fraction of CoreDNS functionality here. One of the outstanding things about CoreDNS is its customizability with plugins, and its direct integration with Kubernetes via said plugins makes it extremely powerful indeed. Nonetheless, if DNS servers are new to you, or bind or unbound scares you a bit, maybe give CoreDNS for your personal needs.

Without further ado, here's how I got it working.

First, pull the container image down locally from Docker Hub. If you're using Docker, you can do so like this:

# docker pull coredns/coredns

Afterwards, we'll need to configure a Corefile, which serves as the CoreDNS daemon's configuration file; there are many options that can be passed in here (which can be seen in the CoreDNS manual), but I will go through the example I used.

[~/containers/coredns] # cat Corefile 
.:53 {
    forward . 8.8.8.8 9.9.9.9
    log
    errors
}

example.com:53 {
    file /root/example.db
    log
    errors
}

Let's go through the options of the Corefile one-by-one. It is important to note that each bracketed section denotes a DNS "zone", which sets the behavior of CoreDNS based on what is being resolved.

First, note the initial bracketed section. It begins with a .:53, indicating that this zone is a global (with "." indicating all traffic), and it is listening on port 53 (udp by default). The parameters we set in here will apply to all incoming DNS queries that do not specify a specific zone, like a query to resolve github.com. We see on the next line, that we forward such requests to a secondary DNS server for resolution; in this case, all requests to this zone will be simply forwarded to Google's DNS servers at 8.8.8.8 and 9.9.9.9.

Second, we have another zone which is specified for example.com, also listening on UDP port 53. Any queries for hosts belonging in this zone will refer to a file database (similar to how bind does) to do a lookup there; more on that momentarily. As an example, a query to "server.example.com" will bypass the global zone of "." and fall into the zone which is servicing "example.com", and using the file directive the database file will be referenced to find the proper record.

That's all there is to this Corefile, for a simple forwarding DNS server which also serves local clients with hostnames. Now we have to make that DNS database file we referenced, example.db, and fill it with our hosts.

Although this isn't a DNS primer, I will go over how this file works. There are two main records at play here, and I'll discuss a third; they are SOA, A, and CNAME DNS records which will make up our DNS configuration.

Initially, we must configure an SOA record, or a "Start of Authority" record. This is the initial record used by this DNS server in this zone to declare its authority to the client which is making a query, and we must begin the file with it. Here is an example SOA record which can be used in this file:

example.com.        IN  SOA dns.example.com. robbmanes.example.com. 2015082541 7200 3600 1209600 3600

To go over each section individually:

example.com. refers to the zone in which this DNS server is responsible for.
SOA refers to the type of record; in this case, a "Start of Authority"
dns.example.com refers to the name of this DNS server
robbmanes.example.com refers to the email of the administrator of this DNS server. Note that the @ sign is simply noted with a period; this is not a mistake, but how it is formatted.
2015082541 refers to the serial number. This can be whatever you like, so long as it is a serial number that is not reused in this configuration or otherwise has invalid characters. There are usually rules to follow concerning how to set this, notably by setting a valid date concerning the last modifications, like 2019020822 for February 08, 2019, at 22:00 hours.
7200 refers to the Refresh rate in seconds; after this amount of time, the client should re-retrieve an SOA.
3600 is the Retry rate in seconds; after this, any Refresh that failed should be retried.
1209600 refers to the amount of time in seconds that passes before a client should no longer consider this zone as "authoritative". The information in this SOA expires ater this time.
3600 refers to the Time-To-Live in seconds, which is the default for all records in the zone.

Once we've written our SOA to our liking, we can add additional records for each of our hosts we wish to resolve. I assign my IP addresses with static DHCP leases for certain MAC addresses, so to do so I first added the DNS server, so it can resolve to itself:

dns.example.com.    IN  A   192.168.1.2

An A record indicates a name, in this case dns.example.com, which can be canonically mapped directly to an IP address, 192.168.1.2. If I add another A record:

host.example.com.   IN  A   192.168.1.10

I can then assign a CNAME record to it, which will serve as an "alias" of sorts, directing traffic back to host.example.com:

server.example.com. IN  CNAME   host.example.com.

You can add as many entries here as you like, or look up different types of records to suit your needs. I ended up with something basic, like so:

example.com.        IN  SOA dns.example.com. robbmanes.example.com. 2015082541 7200 3600 1209600 3600
gateway.example.com.    IN  A   192.168.1.1
dns.example.com.    IN  A   192.168.1.2
host.example.com.   IN  A   192.168.1.3
server.example.com. IN  CNAME   host.example.com

Afterwards, when we're done with our DNS zone file and our Corefile, we can stick them in the same directory and prepare to export them to a newly-running coredns container. I stuck both of these files in a directory of ~/containers/coredns/:

$ pwd
/home/robb/containers/coredns

$ ls
Corefile  example.db

To run the container, the coredns binary looks in the immediate directory its in for any file named Corefile, and uses it as configuration. Unfortunately, in the coredns/coredns image we pulled from Docker Hub, it is located in the root directory of /, which can't be mounted as a volume. We'll need to manually pass our Corefile and ensure that the file directive in our zone of example.com:53 is a direct path in the container to the DNS zone database file. To do this, I mapped them to /root in the container and passed the -conf option which allows a user to specify the path to a Corefile; this is the command I used to launch my CoreDNS container:

# docker run -d --name coredns --restart=always --volume=/home/robb/containers/coredns/:/root/ -p 53:53/udp coredns/coredns -conf /root/Corefile

Afterwards, I made sure my container was running without issues by checking the logs and docker ps -a:

# docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                        NAMES
8a6c9a5c0538        coredns/coredns     "/coredns -conf /roo…"   About an hour ago   Up About an hour    53/tcp, 0.0.0.0:53->53/udp   coredns

# docker logs coredns
.:53
example.com.:53
2019-02-09T04:50:24.060Z [INFO] CoreDNS-1.3.1
2019-02-09T04:50:24.061Z [INFO] linux/arm, go1.11.4, 6b56a9c
CoreDNS-1.3.1
linux/arm, go1.11.4, 6b56a9c

We can then query our server with dig from a client in the same subnet to make sure it's working as intended. My DNS container is running on a host with an IP of 192.168.1.2:

$ dig @192.168.1.2 host.example.com

; <<>> DiG 9.11.3-1ubuntu1.3-Ubuntu <<>> @192.168.1.2 host.example.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30400
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 9faccdae88cb6576 (echoed)
;; QUESTION SECTION:
;host.example.com       IN  A

;; ANSWER SECTION:
host.example.com    0   IN  A   192.168.1.3

;; Query time: 3 msec
;; SERVER: 192.168.1.2#53(192.168.1.2)
;; WHEN: Fri Feb 08 23:01:22 MST 2019
;; MSG SIZE  rcvd: 93

And just like that, we have an easy-to-configure and maintain CoreDNS configuration running! Thusfar, I have been using docker restart to reload the container and re-read the Corefile and DNS zone file, but you should absolutely be aware of the reload plugin to CoreDNS that removes the need to restart the container.

That's a very basic introduction to CoreDNS, and I hope you get some good usage out of this great DNS daemon.