DEV Community: Gabriel Guzman

Ansible

Gabriel Guzman — Tue, 13 Apr 2021 00:00:00 +0000

I’ve been playing with ansible off and on for several years now, usually in fits and starts. Ansible is a tool for managing computers. It lets you do things like setup a new web server in exactly the same way you setup your old web server, or make sure that your new laptop has all the software you routinely use installed, or configure a cluster of computers identically regardless of their underlying operating system, package manager, or authentication system.

In short, it’s a super powerful tool for automating everything about all your computers.

Every time I manually provision a new server I tell myself, “this time I’m going to use ansible”, and every time I start I end up getting frustrated that it’s so hard to get started using ansible and I just provision the sever manually, again. Finally, I found a course on pluralsight (thanks $EMPLOYER) that walks you through setting up ansible to manage just your local machine. This was the guide I needed. It gave simple instructions that clearly told you how to get ansible working for just this one machine, from there it was pretty easy to apply the same learnings to several machines and I’m now happy to say that I’m using ansible to manage:

my work laptop
my personal laptop
my home router/firewall

This means that if I ever need to replace my router, I can just run the router playbook that I’ve created on the new machine and my router will be provisioned exactly the same way as my current router. It also means that if one of my laptops kicks the bucket, I can be up and running again on a new machine quickly. I won’t need to remember all the stuff I usually have to install, I won’t need to remember the special way I have to configure my local smtp server to relay outgoing mail to my real smtp server. I won’t have to spend time trying to get my firewall rules just right. As long as I keep only using ansible to manage these machines, I’ll have no snowflakes!

It also means that I can reuse the work I’ve done to get these machines setup to get my other machines setup as well. The ones that are left to do are:

smtpd server
httpd server
minecraft server
vpn server

Each of those has their own unique tweaks that I’ll have to port to ansible before I can fully use it to provision them, but there will also be a fair amount of overlap from the work I’ve already done.

If you have a pluralsight license, I can happily recommend this course: Getting started with ansible and if you don’t here’s a blog post that takes a similar approach (i.e. managing your personal laptop with ansible). Finally, the ansible docs are a useful if daunting resource.

I am an open source developer

Gabriel Guzman — Tue, 27 Oct 2020 00:00:00 +0000

I’m an open source developer. What does that mean to me?

Prefer open source software

I always prefer open source software. If no open source software exists to do what I’m trying to do, see the next point.

If the software doesn’t exist, create it

If there’s no software available to do what I’m trying to do, it’s my job to create it.

Contribute to open source software

If I use open source software and find a way to improve that software, or fix a bug, it’s my job to improve it.

These statements are aspirational, not absolute. There are always exceptions and I’m not going to spend my life worrying about what piece of not free software I used today.

Goodbye Google

Gabriel Guzman — Fri, 23 Oct 2020 00:00:00 +0000

I’ve been a gmail user since it first came out, and I used to be a fan of google, but now I find myself more and more concerned with their data gathering practices. I’ve decided to move all of my personal services off of google onto open source self hosted alternatives. This page will act as a checklist for each of the services I’m able to get off of. Each of the links below will talk about what I’m using now and how I set it up.

Google Services To Replace

[x] search
[x] browser
[x] email
[x] calendar
[ ] photos
[ ] contacts
[ ] maps
[ ] android
- [x] authenticator
- [x] play store

Search

This one was the easiest of the bunch. Instead of using google.com as a search engine, I now use duckduckgo.com. I’ve made duckduckgo.com my default search engine for everything. There’s even a command line tool that lets you search from the cli: https://github.com/jarun/ddgr

Browser

Instead of using chrome, I’m now exclusively using Firefox.

Email

Email was complicated since there are so many moving parts. Instead of gmail, I’m now using a combination of OpenBSD’s smtpd, the dovecot imap server, and OpenBSD’s spamd hosted on a Vultr VPS running OpenBSD. To read and send mail I’m now using aerc as well as Thunderbird as my local clients. I don’t do webmail anymore.

Calendar

For my calendar, I’m now using simple calendar synching to a self hosted instance of kcaldav. I pretty much only use my calendar on my phone so I don’t currently have a web version.

Photos

No good solution for photos yet.

Contacts

Haven’t tried to figure out contact syncing yet.

Maps

While I’ve used openstreetmap it’s not very effective where I live. I’ve been contributing to it when I can, but it can’t yet replace google maps for me.

Android

I know there are some open source forks of Android out there and I’ve tried CyanogenMod previously (which I guess is now called LineageOS), but I think ideally I’d like something like the PinePhone to fully opt out of the Android ecosystem.

Authenticator

Instead of Google Authenticator I’m using andOTP.

Play Store

Instead of the play store, I’m using F-Droid a catalog of open source software for Android.

exercism.io looking for mentors

Gabriel Guzman — Wed, 08 Aug 2018 12:59:23 +0000

Hello, dev.to'ers. exercism.io -- "an online platform designed to help you improve your coding skills through practice and mentorship" is looking for mentors! If you're looking to learn by teaching or just want to pay-it-forward by volunteering to an awesome effort to get others up and running in the language of their choice, this is a great place to start. I've been using their site since their launch to practice programming problems in different languages, and have recently started mentoring others in those same exercises. If you're interested in mentoring, check out: https://exercism.io/mentoring-faqs or if you just want to practice coding, and get feedback on your work: https://exercism.io

Currently, the PHP track needs some mentors (and I know there are a few of you out there who enjoy PHP, as I do) but there are lots of different language tracks to choose from!

Come help us out!

git bundle

Gabriel Guzman — Wed, 30 May 2018 19:53:37 +0000

I had an interesting problem at work the other day. We'd hired some external contractors to build an android app for us, and now we needed to get the code from them and into our private GitHub. The problem was, I couldn't quickly get them access to the repo since there's a fair amount of process involved in getting those approvals. They also didn't have enough seats to add me as a contributor to their GitHub account, and so we were stuck with me asking them to send a .zip file with the code. Instead, they came back to me with a git bundle file, which I'd never heard of (though I've been using git for a long time now). A git bundle file is essentially a full repository in a single file. You can have branches, history, tags, basically everything you expect in a repository, but it's all contained in a single file. This makes sharing the repository or moving the full repository pretty straightforward. Assume I have a git project called myproj, and I want to send that to someone else.

host1$ cd myproj
host1$ git bundle create repo.bundle master

That will make a bundle file (repo.bundle) that contains all the history of the master branch. You can then send this file to a friend, and they can clone it just like if it were a remote git origin:

host2$ git clone -b master ./repo.bundle myproj
host2$ cd myproj
host2$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

Another pretty neat thing about git bundles is, if you need to update one, you can send an incremental file update with only the latest changes. The recipient just puts that in place of their existing bundle, and runs git pull w/in their repo.

This isn't something you'd use every day, but in certain situations (like getting a git repo from one place to another w/out being able to share a remote origin) it works great!

More info on git bundle: https://git-scm.com/docs/git-bundle

DHCP

Gabriel Guzman — Wed, 23 May 2018 11:48:15 +0000

Once upon a time, if you wanted to connect to the internet (or even to your local network) you needed to edit a bunch of files, ask your system or network administrator for an IP address, and hope you didn't mess up your computer too badly since you didn't know what you were doing. Today, you just plug in your Ethernet cord, or login to your wifi network and you're good to go. How does that work?

Assumptions

The network cable is good
The port is active (plugged into a switch or hub at the other end)
There is a DHCP server somewhere on the network

Preliminaries

Ok, so you've got your cable plugged in, and you see some blinking lights, that's a good sign. It means that the physical interface (the network card) is receiving information over the network, and possibly sending information over the network.
You won't be able to reach hosts on the internet at this point as your computer doesn't have an IP address. Your TCP/IP network information currently looks like this:

|---------------|
|  My Computer  |
|---------------|
|IP: 0.0.0.0    |
|Router: 0.0.0.0|
|DNS: 0.0.0.0   |
|---------------|

No IP (Internet Protocol) address - The IP address is the address of your computer on the Internet. It's how other hosts on the interenet know where to reach you. When you browse to https://duckduckgo.com, you send your IP address as the "return address" so that the duckduckgo server knows where to send a reply.

No Router address - A router is just a computer that is configured to transfer packets from one network to another. All the machines on your local network can talk to each other without a router, as long as you know their IP address, you can send packets to them directly. If you want to get to another network, you need to send your packets through a router that knows about both of the networks. Usually a router is physically plugged into at least two networks via two ethernet cards, and can pass packets from one card to the other.

No DNS server address - We've talked about DNS in a previous post, but basically DNS is your address book. You ask for https://duckduckgo.com/ and the DNS server is in charge of looking that name (duckduckgo.com) up in its address book and telling you the IP address for that name is: 107.20.240.232 (promise, look it up).

No IP, now what?

What can you do w/out an IP address? We can't reach other hosts on the internet yet, but we can talk to other hosts on our own network. Not having an IP address means we can't tell others where to reach us, but we can YELL into the darkness and hope someone is listening. That's basically what your computer does when it's first plugged into a network (assuming you have a DHCP client running).

you> HELLO?
you> HELLOOOOOOOOOOOOOOO!
dhcpserver> HEY!
dhcpserver> STOP YELLING.  Use the phone, call me here: 192.168.1.2
dhcpserver> And, here's a number you can use: 192.168.1.115
dhcpserver> Also, this is the number for the router, in case you need to get on another network: 192.168.1.1
dhcpserver> make sure you let me know you're going to use this.
you> oh, hey, thanks!  I'll remember that my number is 192.168.1.115 and yours is 192.168.1.2.  
dhcpserver> Great, oops, almost forgot, here is the DNS server you can use to lookup other computers: 192.168.1.3.  See you around!

Your computer (via it's dhclient program) sends a message called DHCPDISCOVER to the network broadcast address (a special address that anyone on the network can recieve messages to) this address is known by your computer ahead of time, and on TCP/IP networks is 255.255.255.255. Sending packets to this address will send them to all hosts on the local network. Routers will not forward broadcast messages on to other networks. If there's a DHCP server listening, it will reply with a DHCPOFFER message that contains all the information you need to configure yourself as part of the local TCP/IP network. Since you don't yet have an IP address configured, the DHCP server will also use a broadcast message to send you the DHCPOFFER. Then your computer will send back a DHCPREQUEST message asking the DHCP server to reserve the number it just told you about for you. If everything goes well, the DHCP server will respond with a DHCPACK with any additional information (such as DNS server addresses) it has. This message lets you know it's ok to start using that IP address officially on the network.

Now, assuming all that went well, your network configuration should look like this:

|--------------------|
|     My Computer    |
|--------------------|
|IP: 192.168.1.115   |
|Router: 192.168.1.1 |
|DNS: 192.168.1.3    |
|--------------------|

You can confirm your current ip address using the ifconfig command on most unix like systems:

$ ifconfig iwm0
iwm0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        lladdr e8:2a:ea:ab:49:24
        index 1 priority 4 llprio 3
        groups: wlan egress
        media: IEEE802.11 autoselect (HT-MCS14 mode 11n)
        status: active
        ieee80211: nwid Odessa chan 11 bssid ac:f1:df:62:d6:9a 70% wpakey wpaprotos wpa2 wpaakms psk wpaciphers ccmp wpagroupcipher ccmp
        inet 192.168.0.115 netmask 0xffffff00 broadcast 192.168.1.255

You can confirm your router address using the route command:

$ route show
Routing tables

Internet:
Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            192.168.1.1        UGS        9      860     -    12 iwm0

And you can confirm your DNS server by looking at the contents of /etc/resolv.conf

$ cat /etc/resolv.conf
nameserver 192.168.1.3
lookup file bind

Questions

Congratulations, you're on the internet! A few questions to think about:

What would happen if you decided to run a DHCP server on your laptop and start passing out IP addresses on your network?
What more can you do before you have an IP address?
What allows you to have connections to other computers on the network before you have your TCP/IP networking configured?

Good Practices - Code Review Comments

Gabriel Guzman — Mon, 07 May 2018 21:24:23 +0000

If you do code review, you likely come across the same issues over and over again. People who are contributing to your code base for the first time may not know your team's internal standards, they may come from a different language and do things slightly differently. A new hire may need to get up to speed quickly on submitting changes to your repository. How can you make sure these cases are handled while minimizing the impact on your code reviewers? One helpful tool is the "Code Review Comments" document.

A "Code Review Comments" document is a collection of comments that come up again and again in code review sessions. For example, if you're always having to tell people "Make sure you're using prepared statements when sending input to an SQL server", you could just add that to this document, and then when you see unescaped SQL, just point the submitter to your doc.

Onboarding New Developers

When a new developer joins your team, you can point them to this document so they can get a quick overview of your standard practices. This is a good place to put things like:

All code must be linted, and conform to our internal code style standards before being submitted for review (you do have code standards right?)
Code submitted as a PR should be tested by another developer before it's merged
We prefer using "guard clauses" rather than deeply nested if / else blocks see: https://www.thechrisoshow.com/2009/02/16/using-guard-clauses-in-your-ruby-code/

Anything that's not covered by your linter/code formatter can live in this document.

Decreasing Cognitive Load

Reading code is generally considered harder than writing code so anything that places less cognitive load on the reviewer is good. If all the simple stuff is covered in your Code Review Comments document, then the reviewer can focus on the meat of the change request without having to worry about stuff that they shouldn't have to worry about. This will also help with personal style differences that may come up in a review. If it's clearly stated that your team prefers shorter variable names, then it's harder for someone to argue that in this case, they really think that loopIndexVariable is a better name than i for a loop index because it's more descriptive.

Limiting Bike Shedding

If you find yourself constantly getting into arguments over stuff that's really not that important, you can throw it in the doc (once the team comes to consensus on the best colour to paint the bike shed) and then any time someone thinks another colour would be better, just link them to the doc and be done with it. For more on Bike Shedding.

Formalizing Team Standards

Often, the longer a team works together, the more unwritten rules they will develop for how they like their code. This works great until you try to bring a new person into the group. The new person needs to discover these standards by trial and error, which can be frustrating and take time. Maintaining a Code Review Comments document can help formalize these unspoken standards and reduce friction when a new member joins the team. Instead of having to have several code reviews rejected because of internal standards that everyone else is aware of, the new developer can just read the document.

Don't Over Do It

One thing you definitely don't want to do with this type of document is to fill it with so much minutia that it itself becomes a source of cognitive load for the developer. It should be fairly succinct, and easy to read. A good example of this is the go teams Code Review Comments document. It's filled with things that have come up over and over again in code reviews, and have been added to the document to improve the development and code review experience for both the submitter and the reviewer.

Starting Your Own

It's really easy to start a document like this. Any time you're doing a code review and have to comment on the same thing more than once, add it to the document. Get your team to do the same, then you can review the doc together from time to time and decide what to keep and what to discard. When new contributors join the team, send them the doc before they submit their first PR so they can make sure their code is ready for code review.

Conclusion

Maintaining a Code Review Comments document can be a good way to onboard new developers, decrease the cognitive load on your reviewers, and formalize unwritten team standards. Instead of having to discover these things via trial and error, a new contributor can scan the document and get up to speed with the internal standards of a new team quickly. This can reduce friction for the new developer and improve the code review experience for all parties.

Do you use a document like this with your team? Feedback, thoughts? Leave a comment.

What are the constraints for uploading an image?

Gabriel Guzman — Thu, 03 May 2018 11:16:47 +0000

Tried uploading a header image and I got an error message that just said "invalid file" but didn't give me any reasons why the file was invalid. I assume it's because it's over the file-size limit, but I'm not sure what that is. Any way we could improve this error message, or barring that be more descriptive about what the requirements are for the image near the "upload image" dialog?

What's DNS?

Gabriel Guzman — Wed, 25 Apr 2018 21:53:19 +0000

DNS is basically the address book of the internet. When you type www.duckduckgo.com into your browser, the computer needs to figure out how to get your request to the right server. Routing over the internet works via IP addresses, so to get to www.duckduckgo.com, you first need to figure out what its IP address is.

This is where DNS, the Domain Name System comes in. At a high level, when you hit enter in your browsers location bar, the browser does a DNS lookup to try and find the IP address of the server name you entered. A DNS lookup consists of calling an Operating System function (a syscall) called gethostbyname(). This function takes a string such as "www.duckduckgo.com" and returns a structure that includes the IP address of the host, among other information. On my system, that struct looks like this¹:

struct  hostent { 
    char    *h_name;    /* official name of host */ 
    char    **h_aliases;    /* alias list */ 
    int h_addrtype; /* host address type */ 
    int h_length;   /* length of address */ 
    char    **h_addr_list;  /* list of returned addresses */ 
};

How does gethostbyname() know what the IP address of www.duckduckgo.com is? Unix like computers (including Macs) have a file called /etc/resolv.conf that tells the Operating System where it should look for name to address lookups, you can check yours by typing cat /etc/resolv.conf in your terminal program.

Mine looks like this:

$ cat /etc/resolv.conf
# Generated by iwm0 dhclient
nameserver 192.168.0.1
lookup file bind

That tells me a few things:

#Generated by iwm0 dhclient This tells me the file itself was generated by the dhclient program. dhclient is the program that hands out IP addresses to computers that request them via DHCP (Dynamic Host Configuration Protocol)
nameserver 192.168.0.1. This tells me the IP address of a server I can use to lookup names I don't already know about. This will be important soon.
lookup file bind This tells the computer what database it should use to 'lookup' IP addresses, 'file' tells the computer to first check the /etc/hosts file, and 'bind' tells the computer to ask DNS for the IP address.

If you're a web developer, you've likely used the /etc/hosts file to have a web address point to your computer temporarily for testing. If not, that's ok we'll talk about it now.

If you look in /etc/hosts you'll see something that looks a bit like this:

$ cat /etc/hosts
127.0.0.1       localhost
::1             localhost

This is a mapping of IP address, hostname, and aliases. In the above example, there are no aliases defined. If I wanted to prank myself, I could do something like this:

$ cat /etc/hosts
127.0.0.1       localhost
::1             localhost
127.0.0.1       www.duckduckgo.com

Now, when I try to browse to https://www.duckduckgo.com/ I get an error, because gethostbyname("www.duckduckgo.com") is returning 127.0.0.1, which is the IP address of my local computer, and not the address of www.duckduckgo.com. If I knew the IP address of www.duckduckgo.com, I could put it here and everything would work, at least until the IP of www.duckduckgo.com changed.

A long, long, time ago, there was no DNS, just a bunch of computers and a bunch of people adding entries to /etc/hosts files by hand, or maybe copy pasting from a file that was distributed by Stanford². Eventually this became too cumbersome and someone decided to write some code, thus DNS was born.

Now we come to the second option in our resolv.conf example, bind. If gethostbyname() can't find an entry in /etc/hosts it then queries the DNS server specified in the nameserver section of /etc/resolv.conf this is your DNS server.

So, the Operating System sends a query to 192.168.0.1 asking for the IP address for the "www.duckduckgo.com" server. If 192.168.0.1 knows the IP address, it will return it to you, if not... it will then ask its DNS server (also defined in the /etc/resolv.conf file but now on your DNS server) what the address is for "www.duckduckgo.com." If that server knows the IP address, it will return it, and if not, it will again ask its DNS server if that server knows the IP address to "www.duckduckgo.com" and so on... until one of two things happens:

No one knows the IP address of www.duckduckgo.com - In this case you will get an error and someone will have to go fix the DNS records for that website.
Eventually someone will say "Oh, yeah... I know that guy, here's his address" - In this case you will then open a TCP request to the IP address that you just found (there's at least one other full blog post there) and ask the server for a web page.

That's basically how DNS works, in a nutshell. But, but but, I hear you ask... how does the DNS server that finally responds know what the IP address is for "www.duckduckgo.com?" Excellent question. Basically, the administrator or the devops team, or the developer (now that things are all orchestrated and defined in code) will tell the "authoritative" name server for the domain "Hey, I just setup this server, it's going to host web pages, its name is 'www.duckduckgo.com' and its IP address is '184.72.104.138'."

You do this in what's called a Zone file.

You can play with DNS queries yourself using the dig command:

$ dig www.duckduckgo.com


; <<>> DiG 9.4.2-P2 <<>> www.duckduckgo.com
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57389
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.duckduckgo.com.            IN      A

;; ANSWER SECTION:23.21.193.169
www.duckduckgo.com.     123     IN      CNAME   duckduckgo.com.
duckduckgo.com.         59      IN      A       23.21.193.169
duckduckgo.com.         59      IN      A       107.20.240.232
duckduckgo.com.         59      IN      A       184.72.104.138

;; Query time: 51 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Wed Apr 25 17:41:15 2018
;; MSG SIZE  rcvd: 98

Most of that stuff you can ignore for now, the interesting thing is the QUESTION SECTION which basically just repeats your query back for you, and the ANSWER SECTION which in this case tells us that "www.duckduckgo.com" is a CNAME(Canonical Name)
which is kind of like an alias, to "duckduckgo.com" which has 3 IP addresses: 23.21.193.169, 107.20.240.232, and 184.72.104.138. You should be able to confirm that any of those go to the duckduckgo.com website, by copy-pasting them into your browser (this won't always work because you can have multiple websites running on the same server at the same IP address).

That's all I've got to say about that.
Questions, feedback? Leave a comment!

Scaling your engineering teams

Gabriel Guzman — Tue, 24 Apr 2018 12:23:50 +0000

I had the pleasure to work with Cathy Axais when I was a team-lead at Seedbox. She had the unenviable task of being the sole Quality Assurance resource at the company for a team of 30 developers. There was no way she would have been able to keep up with the QA load placed on her if she tried to perform the traditional Dev -> QA -> Production cycle; she needed a way to scale.

What did she do? She embedded herself with each of the development teams in turn and taught us how to test. What were her requirements? Fairly simple --

PRs need to be tested before they go to production
The developer who wrote the code can't be the developer who tests it

She spent about 3 weeks with my team, showing us what she looked for when testing a system, how to spot things that were likely to break, and how to get into the "testing mindset." This wasn't the easiest thing to do, sometimes we (developers) can be grumpy when presented with change, but from my point of view it paid off in spades. After she finished with us, she went to the next team, and so on until she'd gotten all the developers on board. I don't have the numbers, but I know the defect rate dropped substantially once we stopped testing our own code, an optimization that I take for granted now, but hadn't really considered before.

I haven't found a better way to scale a supporting function to the development process. Need your application to be secure? Don't pass it over to a security team, instead get the security team to teach the developers what to look for to spot insecure code, and show them how to fix it. Want performance? Don't call in a bunch of performance experts to fix all your performance problems, call in one, and get her to instil the "performance mindset" in your developers.

This is what full-stack means to me. You get subject matter experts who can help your teams improve the functions they might not be optimized for. Every hand off of the code before it reaches production increases the time it takes to get features out the door, especially if there is a path that returns that code to the developer where the process has to start again. How many of us have worked in environments where getting code to production requires QA sign-off, security review, architecture review, a release manager, a release date, a roll-back plan, and 4 hours of your day (if everything goes well)?

Each of those steps presents the possibility that your code might go all the way back to the starting line, possibly days after you last touched it.

If you want to move fast, it's important to keep your feedback cycles as short as possible. Often that means getting developers to do everything. Ops, security, performance, QA, deployment, site reliability. None of these things should be outside of the ability of the team.

I'm not saying you should get rid or your QA team, or your security team, or your SREs, just that you should think of them as subject-matter experts who can educate your developers on best-practices for their area of expertise. Think of them more as mentors, and tool builders. For example, Ops can build out infrastructure as code so that the development teams can define their own infrastructure needs independent of the ops team. We'll always need the ops team for when we hit the limits of our understanding, or for architecture planning assistance, but we should be able to come to them with 80% of the work already done.

Passing your code off to another group leads to bottlenecks, and there will always be defects that get through. I think it's better to find the defects quickly and push fixes just as quickly. Get the feedback cycle as short as possible and educate your developers in areas where they are weak.

How do you scale your teams?

Any plans to create a publish api for dev.to?

Gabriel Guzman — Wed, 11 Apr 2018 22:00:16 +0000

Anyone else think it would be very useful to be able to push content to dev.to via an API? Does one already exist, I couldn't find one.

What's your favorite way to version an API?

Gabriel Guzman — Mon, 12 Mar 2018 17:58:42 +0000