DEV Community: Sun-Li Beatteay

JavaScript Functions vs Methods (and other helpful tips)

Sun-Li Beatteay — Wed, 13 Jan 2021 18:43:12 +0000

If you’re coming to JavaScript after learning another backend language, you may get confused with some of the JavaScript terminology — I was. While knowing every single term in the JavaScript vocabulary won’t necessarily improve your code, not knowing them may lead to awkward situations. Because of that, I’ve created this cheat sheet to help you (and me) better understand some of the nuances when it comes to functions in JavaScript:

Functions vs. Methods

So what’s the different between the two?

Methods have a “receiver” while functions do not.

This is not unique to JavaScript but it’s still good to learn and remember. My background is in Ruby and I rarely ever heard the term “function”. It’s all about those methods (besides procs and lambdas). I never really understood the difference until I started learning JavaScript.

Also, this definition may not technically be true in JavaScript — and I’ll show you why — but it’s probably the easiest way to think of it. If you are unaware of what a “receiver” is, look at the code snippet below:

someObject.someMethod()

In this context, the someObject is the receiver, while the someMethod is the method. Compare that with a function invocation:

someFunction(arg)

Notice how there is no receiver. In JavaScript, functions are invoked without a receiver using the parens syntax (). The receiver is what separates a function from a method.

Technicalities

While it may look like a function is invoked without a receiver in JavaScript, there’s a gray area to that. Whenever a function is declared on the global scope it becomes a property of the global object. In the browser, the global object is the window:

function helloWorld() {
  console.log('hello world');
}

helloWorld();          // Logs 'hello world'
window.helloWorld();   // Also logs 'hello world'

As you can see in the last line, the window is the receiver of helloWorld which would make helloWorld a method and not a function by my own definition. However, in JavaScript, it’s still considered a function.

Another thing to note is you can easily turn a method into a function and vice-versa.

var object = {
  method: function() {
    console.log("Am I still a method?");
  }
};

object.method();          // method call
var func = object.method; 
func();                   // now it is a function invocation

Function Declaration vs Function Expression

What’s the difference?

If the line begins with the word “function” it’s a declaration. Otherwise, it’s an expression.

Here are some examples:

// This is a function declaration
function hello() {
  console.log('Hello World');
}

// This is a function expression
var hello = function() {
  console.log('Hello World');
};

// This is also a function expression.
(function() {
  console.log('Hello World');
})();

Notice in the last expression that the line started with ( and not function.

Implicit vs Explicit Function Context

What’s the difference?

Explicit context is when the context is explicitly defined and passed in. Implicit context is when the context is left off and the runtime interprets it.

And if you don’t know what I mean by “context,” I’m referring to whatever this equals when a function is executed.

This might be harder to understand, so look at the code below for reference:

function context() {
  console.log('the context is ' + this);
}

var someObject = {};

context.call(someObject);   
// Logs: 'the context is [object Object]'
// This is explicit context. The context is explicitly defined.

context();
// Logs: 'the context is [object Window]'
// This is implicit context. The context is not defined.

In the explicit context execution, we directly told the context function to use the someObject context. Therefore, this was assigned to someObject, which is why it logged out the context is [object Object].

However, in the implicit execution, no context was given. And since functions are properties of the global object, this will equal the global object. That’s why it logged out the context is [object Window].

First-Class Functions vs Higher-Order Functions

What’s the difference?

First class functions are treated like objects. Higher order functions either accept a function as an argument, return a function, or both.

When I say first class functions are treated like objects, I mean that they have states and behaviors, and they can be passed as arguments in functions and returned from functions — among other things. In this way, higher order functions essentially use first order functions and are first order functions themselves.

Examples:

// Higher Order Function
function higher(callback) {  // first order function being passed in
  callback();

  return callback;           // first order function being returned.
}

var callback = higher(function() {
  console.log('Hello World');
});
// Logs: 'Hello World'

callback();
// Also logs: 'Hello World'

While first order functions and higher level functions are not necessarily related, it’s hard to imagine a programming language supporting one and not the other.

While this is by no means the entire list of things to know about JavaScript functions, I hope you can use it as a handy reference guide. At the least it can be a nice beginner’s reference for anyone just starting out in JavaScript or anyone transferring from another backend language.

JavaScript has its quirks and flaws, but it’s here to stay. All we can do is learn to love it. Happy programming!

How Prime Numbers Keep the Internet Secure (RubyConf 2020)

Sun-Li Beatteay — Tue, 12 Jan 2021 16:38:44 +0000

Abstract

You may not know it, but you use prime numbers every day. They play a major role in internet security in the form of encryption.

In this talk, you will learn the inner workings of how the internet uses prime numbers to keep your data private as it travels over the wire. We will cover topics such as symmetric and asymmetric encryption, and why prime numbers are just so damn hard to crack. By the end, you will understand how to encrypt and decrypt data yourself with just the Ruby standard library.

So come join me as we demystify HTTPS using code, color theory, and only a pinch of math 🤓.

Presentation

I gave this presentation at RubyConf2020.

How to Build an LRU Cache in Less Than 100 Lines of Code

Sun-Li Beatteay — Mon, 11 Jan 2021 18:05:36 +0000

Note: All the code in this article can be found on GitHub.

When I was interviewing for my first software engineering job, I was asked about least recently used (LRU) caches a number of times. I was asked to both code them and to describe how they’re used in larger systems. And if you’ve done your fair share of coding interviews, it’s likely you have, too.

From an interviewer's perspective, caching makes for a versatile topic. It can be used to gauge someone’s low-level understanding of data structures and algorithms. It can also be turned around to challenge one’s high-level comprehension of distributed systems.

For job candidates, however, it can be a jarring experience — especially if you’ve never used them in a professional setting. That was the case for me. The first time I was asked about LRU caches, my mind went blank. I’d heard of them, sure. But in all my studying of binary trees and heaps, I hadn’t bothered to learn what goes into them. And live in front of an interviewer isn’t the ideal setting to try to work it out.

While I didn’t do too well in that interview, that doesn’t have to be your fate. In this article, I’m going to teach you the what and how of LRU caches. By the end, you will know how to implement your own cache in less than 100 lines of code without any third-party libraries — all within ten minutes.

So let’s get going — time is ticking.

What Is a Least Recently Used (LRU) Cache?

Caches are a type of data storage device that typically stores data in memory for fast retrieval. They are generally implemented as a key-value store, meaning you store and access the data via an identifying key of some sort.

The RAM on your computer is an example of a cache. The operating system stores data in RAM for faster access than the hard drive, and it uses the address of the memory cell as the key.

Operating system caches (Image Credit: https://www.sciencedirect.com)

LRU caches are a specific type of cache with a unique feature. When an LRU cache runs out of space and needs to remove data, it will evict the key-value pair that was least recently fetched from the cache.

Popular open source examples of LRU caches are Redis and Memcached.

Why Are LRU Caches Useful?

For tech businesses that provide an API or user interface, performance and availability are crucial. Think about the last time you visited a slow website. Did you stay and wait for it to load, or did you leave and visit another site? Most likely the latter. Slow or under-performing websites can result in millions in lost revenue.

Unfortunately, many of the same systems that rely on high uptime also have to store mountains of data. This is often the case for social media and e-commerce sites. These sites store their data in a database of some sort, be it SQL or NoSQL. While this is standard practice, the problem comes when you need to fetch that data. Querying databases, especially when they contain a lot of data, can be quite slow.

Enter the cache.

Since caches keep data in memory, they are much more performant than traditional databases. And for social media sites, where 20% of the most popular content drives 80% of the traffic, caching can dramatically decrease the load on the databases.

SQL vs. cache speed comparison (Image Credit: https://dzone.com/articles/redis-vs-mysql-benchmarks)

The next time an interviewer asks you how to optimize an API endpoint or workflow that requires fetching the same data over and over, caching is a good place to start.

However, knowing how caches work and how to use them is easy. Knowing how to build one yourself is the hard part. And that’s exactly what we’ll be focusing on.

How to Build an LRU Cache

For the purposes of this article, I will be using Python to implement the LRU cache. It’s succinct, easy to read, and a lot of people know it. However, if Python isn’t your thing or you’re curious about how to implement it in other languages, you can check out my examples repository.

Requirements

Before we start building our cache, we have to understand what the requirements are. The first will be the API. Which methods do we need to implement?

While production quality caches are feature rich, we’ll keep it simple. You’ll only need to create two methods:

get(key)
set(key, value)

But that isn’t all. The LRU cache itself has a number of requirements:

When the max size of the cache is reached, remove the least recently used key.
Whenever a key is fetched or updated, it becomes the most recently used.
Both get and set operations must complete in O(1) time complexity (meaning that no matter how large the cache is, it takes the same amount of time to complete).
When fetching a key that doesn’t exist, return a null value.

With these requirements in mind, we can get to work.

Photo by ConvertKit on Unsplash

Data structure

The first question that we need to answer is, which data structure should back our LRU cache? After all, a cache is not a data structure itself.

Since the cache uses get and set and needs to operate in O(1) time, you might’ve thought of a hash map or dictionary. That would indeed fulfill some requirements. But what about removing the LRU key? A dictionary doesn’t allow us to know which is the oldest key.

We could use a time stamp as part of the dictionary value and update it whenever the key is fetched. This would tell us which key-value pair is the oldest. The problem is, we would need to loop through all the dictionary entries to check which is the oldest — breaking our O(1) requirement.

So what’s the answer?

This is where I should let you in on a secret. We’re actually going to need two data structures: one for fetching the values (dictionary/hash map) and one for keeping the items sorted by frequency of use.

The second data structure

So what should the second data structure be? If you thought of an array, you’re getting close.

We can use an array to keep the items sorted. And each key in the dictionary can reference the index of a value in the array. Whenever a key is fetched, we can move that value to the front of the array, pushing the rest of the items back, and update the index in the dictionary.

However, there’s still a small issue. Under the hood, arrays are a continuous row of data cells. If we need to move values to the front of the array and push the rest down, that operation will get slower as the array increases in size. In other words, inserting items at the beginning of an array takes O(N) time.

Image Credit: Author

But we are close. We just need a data structure with the same sorting benefits of an array that can also get, set, and delete data in O(1) time. And the data structure that fits all of those requirements is a doubly linked list (DLL).

A doubly linked list is the same as a linked list, except each node in the list has an additional reference to the previous node as well as the next node.

Image Credit: https://medium.com/flawless-app-stories/doubly-linked-lists-swift-4-ae3cf8a5b975

Each key in the dictionary will reference a node in our linked list. This way, we’ll be able to easily fetch data, as well as update and delete it.

Image Credit: https://corvostore.github.io/#LRU

Building the LRU Cache

Now that we know the data structure makeup of our LRU cache, we can start building it!

class LRUCache(object):
    def __init__(self, max_size=10):
        if max_size <= 0:
            raise Exception('Max size must be larger than zero')
        self.max_size = max_size
        self.list = DoublyLinkedList()
        self.nodes = {}

    def set(self, key, value):
        node = self.nodes.get(key, None)

        #if the key already exists, update the value
        if node != None:
            node.data.value = value
            self.list.move_front(node)
            return

        # if the cache has reached its max size, remove the least recently used key
        if self.list.capacity == self.max_size:
            expired = self.list.remove_tail()
            del self.nodes[expired.data.key]

        self.nodes[key] = self.list.unshift(KVPair(key, value))

    def get(self, key):
        node = self.nodes.get(key, None)

        # if the key doesn't exist, return None
        if node is None:
            return None

        # make the key to the front of linked list (most recently used)
        self.list.move_front(node)
        return node.data.value

Looking at this code, you may assume that we’re done. But there’s a catch. Did you notice the DoublyLinkedList on Line 6? Python doesn’t come with a DoublyLinkedList in the standard library.

In fact, the majority of programming languages don’t come equipped with a DLL. And since live coding challenges rarely let you use third-party libraries, we will have to implement it ourselves.

Building the Doubly Linked List

When it comes to crafting a DLL, the most important aspect is the design. There are many different types of linked lists with different tradeoffs, but for our use cases, we’ll be making a circular DLL. This will give us quick access to the head and tail, which is important for an LRU cache.

Circular doubly linked list w/ root node (Image Credit: Author)

class Node(object):
    def __init__(self, data):
        self.data = data
        self.next = None
        self.prev = None

class DoublyLinkedList(object):
    def __init__(self):
        self.root = Node(None)
        self.capacity = 0

        # the root node should point to itself when the list is empty
        self.root.next = self.root
        self.root.prev = self.root

While linked lists usually come with many convenience methods, we only need to implement a small subset of them:

move_front(node)
unshift(value)
remove_tail

MoveFront

Whenever a key is fetched from our cache, we will need to make that key the most recently used. This involves moving that key to the front of the DLL.

Implementing move_front can be a bit confusing as it involves both removing a node from the list and inserting it in a different location. We do this by changing and swapping the next and prev on a number of surrounding nodes.

Move Front animation (Image Credit: Author)

def move_front(self, node):
    # guard against empty nodes
    if node is None:
        return None

    # Step 1: remove the node from its current position
    node.next.prev = node.prev
    node.prev.next = node.next

    # Step 2: change the node so it points to the root and the old head node
    node.prev = self.root
    node.next = self.root.next # old head node

    # Step 3: Update the root node so it points to the new head
    self.root.next.prev = node # update old head prev pointer
    self.root.next = node
    return node

Unshift

The unshift method for a DLL works very similarly to an array where we insert a value at the front or head of the list. Our LRU cache will use this method whenever set(key, value) is called.

Luckily for us, unshift shares a lot of logic with move_front. If we update move_front slightly, we can use it to implement unshift.

Unshift animation (Image Credit: https://visualgo.net)

def unshift(self, data):
    node = Node(data)
    self.move_front(node)
    self.capacity += 1
    return node

def move_front(self, node):
    if node is None:
        return None
    elif node.prev is not None and node.next is not None:
        # if node is already in the list, remove it from its current position
        node.next.prev = node.prev
        node.prev.next = node.next

    node.prev = self.root
    node.next = self.root.next

    self.root.next.prev = node
    self.root.next = node
    return node

RemoveTail

If our cache reaches its max size, we will need to remove the least recently used key. This involves removing the tail of our DLL. Similar to unshift, remove_tail shares some common logic with move_front. While we can’t reuse move_front for remove_tail, we can abstract some of the common logic.

Remove Tail animation (Image Credit: https://visualgo.net)

def remove_tail(self):
    # if the list is already empty, bail out
    if self.capacity == 0:
        return None

    removed = self.isolate(self.root.prev)
    self.capacity -= 1
    return removed

@staticmethod
# isolate removes a node from its current position
# it also sets its next and prev pointers to None to prevent memory leaks
def isolate(node):
    node.next.prev = node.prev
    node.prev.next = node.next
    node.next = None
    node.prev = None
    return node

Putting It All Together

When we combine our LRU cache and DLL, this is what we get:

class KVPair(object):
    def __init__(self, k, v):
        self.key = k
        self.value = v

class Node(object):
    def __init__(self, data):
        self.data = data
        self.next = None
        self.prev = None

class DoublyLinkedList(object):
    def __init__(self):
        self.root = Node(None)
        self.capacity = 0

        self.root.next = self.root
        self.root.prev = self.root

    def unshift(self, data):
        node = Node(data)
        self.move_front(node)
        self.capacity += 1
        return node

    def move_front(self, node):
        if node is None:
            return None
        elif node.prev is not None and node.next is not None:
            self.isolate(node)

        node.prev = self.root
        node.next = self.root.next
        self.root.next.prev = node
        self.root.next = node
        return node

    def remove_tail(self):
        if self.capacity == 0:
            return None

        removed = self.isolate(self.root.prev)
        self.capacity -= 1
        return removed

    @staticmethod
    def isolate(node):
        node.next.prev = node.prev
        node.prev.next = node.next
        node.next = None
        node.prev = None
        return node

class LRUCache(object):
    def __init__(self, max_size=10):
        if max_size <= 0:
            raise Exception('Max size must be larger than zero')
        self.max_size = max_size
        self.list = DoublyLinkedList()
        self.nodes = {}

    def set(self, key, value):
        node = self.nodes.get(key, None)
        if node != None:
            node.data.value = value
            self.list.move_front(node)
            return

        if self.list.capacity == self.max_size:
            expired = self.list.remove_tail()
            del self.nodes[expired.data.key]

        self.nodes[key] = self.list.unshift(KVPair(key, value))

    def get(self, key):
        node = self.nodes.get(key, None)
        if node is None:
            return None

        self.list.move_front(node)
        return node.data.value

And just like that, 10 minutes later, we’ve created a basic LRU cache in less than 100 lines of code. For extra practice, you can try adding the ability to delete a key, creating simple unit tests to prove it works, or creating it in a different language.

As I mentioned before, if Python isn’t your thing, don’t worry. I have more examples for JavaScript, Ruby, and Go. I’m also open to pull requests for other languages.

I hope you found this deep dive into LRU caches useful and informative. It can be a confusing topic to grasp, but with some practice, you’ll be able to build one in your sleep. So the next time an interviewer tries to challenge you on caches, show them what you’ve learned.

Public vs. Private vs. Protected

Sun-Li Beatteay — Fri, 08 Jan 2021 17:49:07 +0000

For anyone learning Object Oriented Programming or Ruby or both, you know that there is a lot to learn. I’m still learning new things everyday. One of the topics that bothered me, and still bothers me, is the difference between public, private and protected instance methods.

As an attempt to better understand them, I’ve done some research on the topic and am writing down what I’ve learned. Hopefully this article will not only clear up the matter in my own head but help anyone else who is confused as well.

A basic understanding of Object Oriented Programming will be needed to understand this article. If you are unfamiliar with OOP, I suggest reading up on it first.

First Things First

What are public, private, and protected instance methods? Why do we use them? When do we use them?

To make it simple: public, private and protected methods are just that. Methods. You use them to perform certain functions on your code. The difference between the three comes from who and/or what has access to them.

Why do we use them? To provide a easy and simple interface for our classes. They allow us to write clean code and provide a enjoyable experience for the consumers of that code while protecting any sensitive data at the same time.

As for when do we use each one, that will require us to go more in-depth.

Public

Public methods are the generally considered the easiest to understand out of the three.

What are they?

Public methods are methods that are accessible both inside and outside the scope of your class. Any instance of that class will have access to public methods and can invoke them.

How do you use them?

You use them as you would any other method in ruby. Using dot notation, you can invoke a public method on a caller.

Let’s see it in action:



class Person
  def initialize(name, age)
    @name = name
    @age = age
  end

  public

  def name
    @name
  end

  def age
    @age
  end
end

joe = Person.new('Joe', 30)
joe.name                    # => 'Joe'
joe.age                     # => 30

You can see above that joe is an instance of class Person. Since name and age are public instance methods, the joe object can access them even outside the scope of class Person.

As a side-note: you don’t need to preface public instance methods with public like I did in the code snippet above. All instance methods are public by default. I just did that to make it more clear.

When do you use them?

You certainly want to use public methods when displaying information for everyone to see.

A public method in human form.

You should also keep a method public if it is a part of that class’s interface. Meaning that other classes interact with that class via that method. Since that last sentence made no sense, here’s an example:



class Dog
  attr_reader :name

  def initialize(name)
    @name = name
  end

  def speak
    'Arf!'
  end
end

class Owner
  attr_reader :name, :pet

  def initialize(name, pet_name)
    @name = name
    @pet = Dog.new(pet_name)
  end

  def call_pet
    puts "#{name}: Hey #{pet.name}! Come here boy!"
    puts "#{pet.name}: #{pet.speak}"
  end
end

tom = Owner.new('Tom', 'Rufus')
tom.call_pet  

# => "Tom: Hey Rufus! Come here boy!"
# => "Rufus: Arf!"

The above example is a bit long, but you can see that the speak method in the Dog class is part of its interface. The Owner class uses that method in it’s call_pet method to allow the pet to speak. Also, we could use *speak *method by itself if we want the dog object to speak on it’s own. Therefore we should keep it a public method

It’s also important to keep your method public when you want the program/client to be able to change the state of your object. For instance, say you have a car object and you want to spray paint it a different color:



class Car
  def initialize(color)
    @color = color
  end

  def spray_paint(new_color)
    @color = new_color
  end

  def color
    @color
  end
end

my_car = Car.new('pink')
my_car.color               # => 'pink'

my_car.spray_paint('black')
my_car.color               # => 'black'

Private

Private methods are probably the next easiest to understand as protected are a bit more nuanced.

What are they?

Private methods are not accessible outside the scope of the class whatsoever. If you try to invoke a private method with a class object, an error will occur.



class TopSecret
  private


  def data
    "Top secret information"
  end
end 

secret_file = TopSecret.new
secret_file.data           
# => NoMethodError: private method `data' called for #<TopSecret:0x007f8051041ab8>
```

### How do you use them?

You may have also noticed that in order to make a method private, you have to preface it with the keyword *private. *Any methods that you create underneath *private* will be considered private methods and off limits.

(*private* is actually a ruby method itself, but it may be easier to think of it as a keyword/section divider)

Furthermore, private methods do not use dot notation and can only be called implicitly. This means that you will never reference **self**, or any other class instance, on a private method.

If you want to use a private method within the proper scope, just call it by its method name. Simple as that.

```ruby
class Mathy
  def output_result(num1, num2)
    puts crazy_algorithm(num1, num2)
  end

  private

  def crazy_algorithm(num1, num2)
    value = num1 * num2
    value *= (num1 + num2)
    value /= num1
    value -= num2 / num1
    value**2
  end
end

equation = Mathy.new
equation.output_result(3, 5)    # => 1521
```

### When do you use them?

Private methods are very useful when it comes to managing the interface of both your class and your program. They should be used when dealing with:

* Sensitive information, such as passwords, personal information, etc.

* Methods that serve as the inner workings of your class and don’t output or display information.

* Methods that don’t add to the interface and do not collaborate with other classes.

If you’re a developer building a video game app, you wouldn’t want the player to have access to the game settings, computer AI, or any other of the inner functions.

![What happens when you don’t use private methods](https://cdn-images-1.medium.com/max/2000/1*HmYW8e8Z1YoKHT7AGsk8wA.gif)*What happens when you don’t use private methods*

What about when you want certain people to access the information, but not everyone? That’s when you can use a public method to act as a go-between. This allows you to create a filter that only lets some people through. A password is a good example.

```ruby
class TopSecret
  def access_data    
    puts "Please enter password:"
    password_attempt = gets.chomp

    if password_attempt == '12345'
      puts data
    else
      puts 'Nice try, bozo.'
    end
  end

  private

  def data
    "Top secret information"
  end
end

secret_file = TopSecret.new
secret_file.access_data
```

In the above example, the *data* method is private so neither the client nor the program has access to it. But a secret is only fun if some people have access to it and they can shove it everyone else’s face, so I created a accessor method called *access_data*.

The *access_data* method requires a password from the user before it gives up its secrets. A wrong password attempt leaves them being called a bozo.

![](https://cdn-images-1.medium.com/max/2000/1*zzp3G8tT6cDL9HHZnjdW7w.gif)

## Protected

We now come to the protected instance methods, the odd one of the group.

### What are they?

Protected methods are a balance between public and private methods. They are similar to private methods in that they cannot be accessed in the public scope. Neither the client nor the program can invoke them.

```ruby
class Person
  def initialize
    @age = rand(50)
  end

  protected

  def age
    @age
  end
end

me = Person.new
me.age          # => NoMethodError: protected method `age' called

However, objects of the same class can access each other’s protected methods.



class Person
  def initialize
    @age = rand(50)
  end

  def >(other_person)
    age > other_person.age
  end

protected

def age
    @age
  end
end

me = Person.new
you = Person.new

me > you         # => true

The ages are generated at random so the result of the above code won’t always be “true.” However, it will never produce an error, which is the most important take away.

Since you *and *me are objects of the same class — Person — they can access each other’s protected methods without issue.

How do you use them?

If you want to make a method protected, you must declare the protected *method first, similar to private methods. However, with protected methods, you *can *use dot notation and call **self* or another receiver.



class Person
  def younger_than?(other_person)
    self.age < other_person.age
  end

  protected

  def age
    @age
  end
end

When do you use them?

The best time to create a method under the protected heading is when you want to compare and/or transfer data between objects of the same class, but you don’t want that information to be made public.

Say I’m building a banking game and I want to keep the account balance information private. However, I want to include features that let me know whether I have more money than a friend and to steal from them.



class BankAccount
  def initialize
    @balance = rand(100)
  end

  def richer_than(other_account)
    if balance > other_account.balance
      puts "You're rich!"
    else
      puts "Better luck next time."
    end
  end

  def steal_from(other_account, amount)
    if amount > other_account.balance
      puts "Hold it there, they aren't that rich."
      return
    else
      @balance = @balance + amount
      other_account.balance = other_account.balance - amount
      return
    end
  end

  protected

  def balance
    @balance
  end

  def balance=(num)
    @balance = num
  end
end

my_account = BankAccount.new
friend_account = BankAccount.new

my_account.richer_than(friend_account)
# => "Better luck next time."

my_account.steal_from(friend_account, 25)

my_account.richer_than(friend_account)
# => "You're rich!"

my_account.balance    # => NoMethodError

Let’s break down the above code.

I made the balance getter method protected, so it is not available to the public scope and there’s no way of actually knowing how much money is in the account.

The richer_than method takes another BankAccount object as an argument and lets you know if your own account has more money than theirs.

The steal_from method takes two arguments, a BankAccount object and how much money to take from it. However, since you don’t know how much money the other bank account has, you have to make a guess of how much to take. That is why it lets you know if you’ve taken too much.

Finally, I invoke the getter balance method at the end just to reiterate that calling a protected method will result in an error.

Summary

Public instance methods:

Accessible from global scope.
Part of the class interface.
All instance methods are public by default.
Use if displaying information or interacting with other classes and/or the client.

Private instance methods:

Accessible only from within class scope.
Part of the class inner-workings.
Must preface with private method.
Use when hiding sensitive information or crucial class methods.

Protected instance methods:

Accessible from within class scope and by objects of the same class.
Must preface with protected method.
Use when wanting to compare/interact with other class objects, but don’t want the program or client to have access to them.

There you have it. An in-depth look at public, private and protected methods. I hope this has been as helpful to you as it has been to me.

Now go and make some awesome object oriented programs!

The Bit Wars: Why Nintendo was obsessed with the number 64

Sun-Li Beatteay — Thu, 07 Jan 2021 17:44:20 +0000

The Christmas of 1995 was a memorable one for me. I had just turned 5 years old and was reveling in my transition from a small child to a slightly larger child. However, my birthday wasn’t what made it special. No sir. On that day, I was in for a big surprise.

I remember sitting in the living room, right beside our tree. I was starring down one of the largest presents I’d ever received, still in its pristine wrapping. With bated breath, I waited for my parents to give me the signal. And when they did, I tore into it like only a kid hopped up on Christmas cookies could.

I instantly recognized the man with the red hat and cape riding atop a cartoonish green dinosaur.

Mario and Yoshi! I exclaimed. I recognized from TV. They were always in my favorite Nintendo ads. I loved watching game footage and would always shout the tag line:

Now You’re Playing With Power, Super Power!

What I was holding was the Super NES. I could barely contain my excitement. As soon as all the presents had been opened, I remember dashing to my room to set it up. Stomping on Goombas had never been so satisfying.

However, unbeknownst to me, as I battled my way through Super Mario Worlds' first castle, were the other wars brewing in the real world.

The Bit Wars.

What were the Bit Wars?

If you’re a fan of the Angry Video Game Nerd like I am, you might’ve already heard about the Bit Wars before.

The short answer is that the Bit Wars were a period of time between the mid-1980s to mid-to-late 1990s where video game titans battled it out for market share and dominance. Since this is more of a historical observation rather than an actual war that took place, the timeline is subjective.

For a better answer, however, we must go back in time to the Video Game Crash of 1983. Let me set the scene.

The video game industry was in shambles. Most game companies were beaten down with their tails between their legs, trying their best to prevent financial ruin. Atari, one of the largest names in video games at the time, slashed its workforce from 10,000 employees to just 400.

During this turmoil, one company would do the unthinkable. Only two short years after the collapse, Nintendo would release a home video game console in America. The Nintendo Entertainment System.

While it certainly wasn’t the first home console, it was the first one to perfect it. The simple construction, ease of use, and high-quality games. It was everything its predecessors hadn’t been, and that was by design. Nintendo dedicated itself to avoiding the mistakes that had lead to the video game crash.

Further aiding the NES’s success was an aggressive marketing campaign and licensing agreements for third-party developers. All of this would culminate in Nintendo taking a 75% share of the video game market and revitalizing the entire industry.

But as it often does, success begets competition.

Witnessing the dominance of Nintendo, arcade industry veteran Sega threw its hat in the ring. As did other smaller players, such as Atari. But when a market is already dominated by a single entity, how does one make a name for itself? Game library and brand image help, certainly. But in the late 1980s and early 1990s, there was another characteristic that caught on like wildfire, bits.

And thus, the Bit Wars began.

Looking back now, it’s bewildering how “bit” campaigns were. Few people really knew what bits were and how they worked. Not the kids playing the games, nor the parents buying them, nor even the marketers creating the ads. But that didn’t stop us from parroting the phrases “16 bit”, “32 bit”, and “64 bit” anytime we debated which were the better consoles.

It also didn’t stop Nintendo from naming an entire console, and a portion of its game library, after its bit architecture.

For those of you, like me, who grew up during the 90s and never understood what bits were, now’s the time to learn.

What are bits?

Bit stands for Binary Digit and it is a singular piece of data (or datum, to be precise). A bit can either be 0 or 1. And they can be strung together to make a long series of 1s and 0s. But why this is important?

Because machine code — the thing that tells computers what to do — is made up of bits. All the games we know and love, with their complex graphics and storylines, can be boiled down to a large string of 1s and 0s.

But how can this be? You might ask. How can a 1 or a 0 tell a computer what to do?

It all depends on the context. For example, take the machine code 0111 0011. In a certain context, those bits can represent the number 115. But in another context, it can represent the lowercase letter s.

The same goes for this code here:

0110 0110 1011 1000 0000 0100 0000 0000 0000 0000 0000 0000

In one context, this gibberish represents the number 112940527124480. But in another, it stands for the assembly language command:

MOV eax, 4

As you can see, bits can mean many different things. It’s up to the Central Processing Unit (CPU) and Graphics Processing Unit (GPU) on each gaming console to process all those bits and turn them into the games we enjoy playing.

The Left and Right Brains

While the “left brain right brain” concept is inaccurate and outdated, it makes for a somewhat fitting analogy for the CPU and GPU.

You can think of the CPU as the left brain of the computer. It’s the logical piece in charge of processing the controller inputs and updating the game’s state based on those inputs. The GPU, on the other hand, is in charge of rendering the graphics on the screen. In other words, the artsy stuff. Thus, the right brain.

When you hear phrases like 8-bit or 16-bit, they refer to the number of bits that a CPU or GPU can process in a single clock cycle. The amount of bits that can be processed at once can play a crucial role in the graphics and game play of a video game.

For example, the original NES had an 8-bit CPU. To put this in perspective, the largest number an 8-bit processor can handle at any given time is 256. This is why the original Nintendo games had simplistic graphics and controls. When your CPU can only handle 8-bit operations, simpler is better.

However, as chip sets and processors became more powerful, they were able to handle more and more bits. When Sega launched their Genesis console with 16-bit capabilities, it was immediately evident it was a technically superior system than the NES. Sonic the Hedgehog’s colorful scenery and blazing fast speed left Mario in the dust.

This may lead some people to believe that larger processors always result in faster and more complex games, but that isn’t the case. There are several factors that play a role in the performance of a gaming console:

Number of processors (the more processors there are, the more that can be done in parallel).
Clock speed (how fast the CPU and GPU are).
Memory size (the more memory, the larger the games can be).
Type of memory (certain types of memory are faster than others, leading the faster load times).

Even when a system has high marks in all of these areas, it fall flat if it’s confusing and had to work with. This was the case for the Atari Jaguar. The Jaguar had very impressive hardware specs compared to it’s main competitors at the time, the Genesis and Super NES. It boasted itself as the first 64-bit console in a now infamous “Do The Math” marketing campaign.

While many debate whether the Jaguar was a true 64-bit console, it’s hardware was still the most advanced on the market. The problem was that it used customized hardware chips and did not provide a coherent development kit. This meant that few game studios actually built games for the Jaguar. And of the few studios that did, even fewer created games that took advantage of its specialized chips.

Despite the Jaguar’s superior hardware, its games look and play little better than its 16-bit counterparts. It also didn’t help that it’s controller was notoriously clunky and awkward to use.

All in all, the Atari Jaguar was the prime example of more bits not equaling better games.

So if the bit architecture alone doesn’t make better games, why were game companies so obsessed with it? Why was Nintendo obsessed with the number 64? Two main reasons.

The first is marketing. As I mentioned before, the video game industry was dominated by one main player in the early days. New and smaller companies had to make themselves stand out in some way. This was the especially true for the Atari Jaguar and 3DO Interactive Multiplayer as they competed against Sega’s Genesis and Nintendo’s SNES.

The second is potential. Sophisticated hardware won’t make a gaming console great in itself, nor will it produce video game marvels either. However, it does mean that games have the potential for better graphics and complex game play. When you combine this with games that have engrossing stories and fun mechanics, masterpieces are born.

How did the Bit Wars end? Who won?

Seeing as how the Bit Wars were not a recorded event in history, there’s no official end date. But there are plenty of conjectures.

Many believe the release of the Atari Jaguar was the catalyst. Up until that point, consumers had made the association that more bits meant better games and better graphics. The Jaguar was a letdown for many consumers and shattered their bit-beliefs.

Some others believe that the final nail in the coffin came with the releases of the PlayStation and Nintendo 64. Not only was Sony a new player in the gaming industry, the PlayStation was only a 32-bit system. It was competing against Nintendo, an industry titan, with its impressive 64-bit console. By all measures, Nintendo should’ve wiped the floor with PlayStation.

However, PlayStation would end up on top. To be more precise, it destroyed the Nintendo 64, selling 3 times as many consoles.

The true end of the Bit Wars is probably somewhere in the middle. The success of the PlayStation, when juxtaposed to the failure of the Atari Jaguar, was proof to the rest of the industry that buyers no longer cared about bits. They just wanted entertaining games. The Nintendo 64 would be the last console to puts bits at the center of it’s marketing.

Additionally, there is no official winner. Though it’s probably not too controversial to say that the PlayStation crashed onto the video game scene and made the rest of the industry look like squabbling children. However, I propose that the real winners of the Bit Wars were us, the gamers.

Speaking from personal experience, the Bit Wars was a surreal time to be a gamer. We witnessed the resurrection of an industry. Rapid technical innovation took us from this:

to this

in just over a decade. Not to mention the number of software engineers that must’ve gotten their start during this period. With all the marketing about bits going on, an entire generation was subtly introduced to the world of computer science.

If the Bit Wars has taught me anything, it’s how persuasive marketing and advertising can be. In a time when ads are becoming the go to monetization strategy for many companies, the old adage rings true:

Don’t believe everything you hear, especially if you hear it from a marketer.

How Prime Numbers Keep the Internet Secure

Sun-Li Beatteay — Wed, 06 Jan 2021 17:32:09 +0000

Note: I spoke about this topic at RubyConf 2020. If you would like to checkout the video, you can see it here:

Whether you know it or not, you use prime numbers every day. Do you see that lock symbol in the address bar of your web browser? The one that looks like this:

That lock means you’re using prime numbers at this very moment. That’s because the internet uses prime numbers. In fact, prime numbers are so ingrained into the fabric of our daily lives that the world would be a drastically different place without them. We’d still be doing all our banking in person and buying everything in cash. And forget about texting, because we’d still all be pen pals.

So what is it about prime numbers that make them so special?

Firstly, they’re unique. There aren’t any combination of numbers that can be multiplied together to create a prime number.

Secondly, every number can be broken into it’s prime components. For example, 10 can be broken down into:

10 = 2 * 5

Lastly, while the average human might not be able to look at this number and immediately detect if it’s prime …

9307398526401816703683197763617206082269079617576835286211259044095385462270542532346398139788788003092515521098292832872130802035097419307557532476688659

… it’s relatively simple for computers. You might’ve written your own primality checker in the past, similar to this …

// shout out to Reddit for the correction
function isPrime(a) {
    const n = BigInt(a);
    const zero = BigInt(0);

    if (n <= BigInt(3)) {
        return n > BigInt(1);
    } else if (n % BigInt(2) === zero || n % BigInt(3) === zero) {
        return false;
    }

    let i = BigInt(5);
    while ((i*i) <= n) {
        if (n % i === zero || n % (i + BigInt(2)) === zero) {
            return false;
        }
        i += BigInt(6);
    }

    return true
}

… and know that many conventional methods for checking prime numbers is slow. But there are more advanced methods, such as the Miller-Rabin primality test, that make it very fast.

It’s for all of these reasons that prime numbers are the perfect tools for encryption!

Encryption

For those who don’t know, encryption is the act of turning information into an unreadable format called a cipher. Decryption is the opposite process of turning a cipher back into the original information.

In other words, encryption allows us to keep information private and out of the hands of people who might use it for malicious purposes. That’s why it has become a cornerstone of the modern internet.

Without encryption, I wouldn’t be able to do most of the things I do online, such as buy groceries, pay off debts, or message my friends — at least not securely. Encryption prevents hackers from stealing my banking information and spying on my private conversations.

It’s not just the internet that uses encryption but many modern devices, such as computers, smartphones, or even smart fridges. They all use encryption. Suffice it to say, encryption is important and everywhere.

But how does encryption work?

Encryption algorithms use keys to encrypt and decrypt data. How those keys are used depends on the type of encryption, of which there are two: symmetric and asymmetric. Both of which have different use cases.

Symmetric encryption

Symmetric encryption gets its name because it uses the same key for both encryption and decryption. Since it uses a single key for both encryption and decryption, symmetric encryption is very fast — but fragile. The key must always be kept private and only shared between trusted parties.

Because of this, one of the main uses for symmetric encryption is securing data at rest. This means encrypting devices like computers, databases, or IoT devices. If you remember the drama that occurred between Apple and the FBI — that was a battle over iPhone encryption.

While symmetric encryption works well, it has an inherent flaw. In order for multiple parties to have encoded communication via symmetric encryption, they must all agree on a key ahead of time. And in the context of the internet, where you’re communicating with hundreds of servers a day half-way across the world, that’s not possible.

That’s where asymmetric encryption comes in.

Asymmetric encryption

Asymmetric encryption uses two keys, one for encryption and one for decryption. This works because the keys are complements of one another. When they’re used together, they cancel each other out — similar to how complement colors cancel one another out into white.

Correction on the above image: Cipher should be orange

The key used for encryption is known as the public key. As you might guess, it’s safe to share this key with anyone.

The decryption key, on the other hand, is called the private key because it must be kept private. Only the holder of the private key can decrypt ciphers that were encrypted with the public key. Even if a malicious user were to intercept a ciphertext, they’d just see gibberish.

This makes asymmetric encryption an ideal tool for sharing sensitive data. Not only that, but since a private key should only be owned by a single entity, it works well for authentication as well. That’s exactly how it’s used in the TLS handshake.

The trapdoor

One of the reasons that asymmetric encryption is as important as it is is because it works as a trapdoor function.

This means it’s very simple to execute in one direction but very difficult to reverse — unless you have special information, otherwise known as the trapdoor or secret.

In the context of asymmetric encryption, it’s very simple to encrypt data but very difficult to decrypt it using only the public key. It becomes simple again with the private key.

But not all asymmetric-encryption algorithms are built the same. How laborious it is to reverse the trapdoor function determines an algorithm’s security. To see just how secure asymmetric encryption can be, let's explore one of the most popular algorithms in use today: RSA.

RSA encryption

RSA was invented in 1977 by three cryptographers: Ron Rivest, Adi Shamir, and Leonard Adleman — hence the name. Since its inception, it has spread to nearly every corner of the earth.

If you’ve ever used Secure Shell (SSH) …

… or GNU Privacy Guard (GPG) …

… you have RSA to thank for it. However, it’s most known for its use in TLS and HTTPS to prevent man-in-the-middle attacks.

While RSA is nearly half a century old, it’s one of the most commonly used asymmetric-encryption algorithms in the world. Its ubiquity is a testament to its security.

But why is it so secure? Short answer: prime numbers. Long answer? That’ll involve some math. But the best answer would be to try and break it ourselves.

Breaking RSA

Here’s the scenario: We’re hackers trying to impersonate Medium’s server. We want to intercept all traffic going to Medium’s website in order to steal user credentials and ransom their data.

Using Wireshark, we’re able to get a copy of Medium’s RSA public key and website certificate.

But in order to impersonate Medium and fool users into connecting to our phishing server, we need the private key. Luckily, all is not lost.

One thing I haven’t mentioned is that RSA keys are just numbers. An RSA private key is just a single number, which we’ll call d. The public key is made up of two numbers, e and N. And N is the product of two more numbers, p and q.

I know, that’s a lot of numbers to track of. But it’s just those last two numbers, p and q, that we need to focus on. Because according to RSA’s key-generation algorithm, if we know e, p, and q, we can recreate the private key.

“Well, perfect,” one might say. “Since we have the public key, we know e and N. And since we know N, we just need to split it apart to get p and q. How hard could that be?”

Not so fast, person I just made up to ask loaded questions — p and q are prime numbers. Gasp!

I mentioned before that detecting that generating prime numbers and checking if they’re prime are relatively simple for computers. However, what isn’t simple is prime factorization.

How hard, you might ask?

RSA typically uses numbers 1024, 2048, or 4096 bits long. As you can see in the graph above, it only takes seconds to minutes to create N, but it’d take millions to billions of years to factor it apart.

The reason for this is — for average, nonquantum computers — there isn’t a fast method for factoring a number into its prime components. One of the best methods we know is the Number Field Sieve, but even then, for a number like this, it will take a while:

12647218591793774062037539860814590913847656969568852342569985866826731647633698490555162899129013020883082990527279827064849704038819915244363097120031062841681483530795022535252488366169730386558454292994968234214045666016756933262308367238453012386845278265898125397947728757013541963782671274800429212175737617916738370351721854897974375037404102868790995317383226110430324268401945063200233204784127599950729869495397377610047121343931821194220803396259107891220452870079636709770538139479748696178546655932056530040495898965404702415803790560056325250086900175615221136804225865647753477561884491932551643726743

While it’s not impossible, the level of effort is astronomical and not worth it. We’d all be long dead by the time we could generate Medium’s private key.

So long story short, prime numbers are pretty darn hard to break. And that’s how they keep the internet secure.

Parting Thoughts

As a software developer, I’m often intimidated by all the different moving parts on the internet. It can feel like a magical and bewildering place. And as a result, I usually feel like I have no idea how any of it works or what I’m doing.

But any time I learn something new about the systems I use on a daily basis, the world becomes just a little less chaotic and magical. I hope this article has helped demystify some of the mysteries of the internet for you as well.

How I Became a Senior Software Engineer

Sun-Li Beatteay — Tue, 05 Jan 2021 18:47:43 +0000

Ever since I started learning to code, I have dreamed of becoming a senior engineer. Some may think this a bit odd. Shouldn’t my main goal be to get a job first and worry about a promotion later?

Generally, yes. But, as someone who has changed careers four times in as many years, I didn’t want to simply change my career. I wanted to thrive in it. And there was no better example of thriving in the tech field than being a senior engineer.

Senior engineers are the masters of their craft. They are rockstar developers who churn their code at breakneck speeds. They are the pinnacle. Right?

Well, after years of working and learning, I happy to proclaim that I’ve made it. I am now among those with the coveted senior title. But you know what the craziest thing is? I’m none of the things I described above.

Rockstar code-smith producing code at 10X speed? Not quite. I’m your average 1x engineer who has Stack Overflow bookmarked. How does someone like me end up a senior?

That’s what this article is going to focus on.

Over the years, I’ve come to realize how propped-up my image of a senior was. Many articles define the characteristics of a senior developer. Fewer discuss how to become one. I want to address that gap.

This post will show the steps I took to arrive at this point in my career and dispel some myths that I’ve seen newer engineers, including myself, fall prey to. Keep in mind, this was just my path and is in no way prescriptive, although I do hope it provides some insight.

So, how did I become a senior software engineer?

I Asked

Photo by Scott Graham on Unsplash

Seriously, I just asked.

After my one-year anniversary at the company, I approached my manager. I recapped my accomplishments and expressed my interest in getting to the next level in my career.

I followed up with a simple question: “How can we make that happen?”

After that meeting, my manager and I came up with a development plan and within that year, I earned my promotion.

A trap I see many new developers fall into is believing that our industry is a meritocracy. They believe that they will get what they deserve if they work hard enough. Once they’ve proven their worth, their employer will reward them with a raise or promotion.

While the tech industry may be more of a meritocracy than other industries, there are countless undervalued developers. I know several myself.

There are many reasons why engineers may be underpaid and under-leveled. I don’t pretend to understand anyone else’s situation. However, the reason I became a senior is that I pushed for it.

Photo by Tim Mossholder on Unsplash

But how do you know when to ask?

This is a great question and is also highly subjective. The problem I have with defining “senior engineer” is that each company has its own definition. Taking a blog post to your manager and saying, “according to this, I should be a senior engineer” isn’t going to work.

The criteria for becoming a senior engineer at a ten-person startup will be different from a FAANG company.

Luckily, many companies provide detailed criteria for the senior role. This was the case for me. Whenever I made an achievement of note — shipping a new feature, refactoring old code, or handling an incident — I kept a record of it. After a while, my accomplishments began to look like those of a senior engineer. That’s when I knew to ask.

If your company doesn’t have a clear benchmark, I would suggest talking to your coworkers. Ask other senior engineers what they’re working on, review their code contributions, and/or ask them, “how did you become a senior engineer?” I’m sure you’ll find at least one person willing to talk your ear off about their accomplishments. I’m doing it right now.

Speaking up for myself and pushing for my own goals was crucial. But, it would’ve been for naught if I didn’t have the proof to back up my value. Which brings me to my second point.

I Utilized My (Non-Coding) Strengths

Photo by Clark Tibbs on Unsplash

Another misconception I see about senior engineers is that they are all masterful programmers. I’ll be the first to admit that I’m a pretty average coder. I know a thing or two about a thing or two, but I won’t be winning any awards any time soon.

But being a senior isn’t about being a code-smith. As Patrick McKenzine wrote in Don't Call Yourself A Programmer, And Other Career Advice:

“Engineers are hired to create business value, not to program things.”

It’s all about providing value to the business. As an engineer, a lot of the value you bring will be a direct result of the code you write, but there are other ways, too. Each of us has different strengths other than coding. Utilizing those other skills can help you stand out.

For me, it came in the form of writing, speaking, and teaching.

During my first year at the company, I had difficulty understanding how the system worked. Hundreds of moving parts, combined with halfway-refactored services, made for an opaque structure. While our code base wasn’t old by any means, it had its fair share of tech debt with which we were still grappling.

Many of my coworkers felt this same struggle, though not all. There was a mental fissure in the engineering organization. Several engineers dated back to the company’s founding and knew it’s historical baggage. However, there were even more engineers that didn’t. The knowledge gap was a divide that was widening with every new hire.

So, unable to piece things together myself, I began talking with my tenured coworkers. I asked them how the technology worked and how it had evolved. What trade-offs were made, and what role did each team play in the system? Some of these conversations lasted for hours.

Through these discussions, my mental model formed and solidified. I had obtained the coveted knowledge that so many of my coworkers sought. The trouble was that it was all trapped in my head, which didn’t benefit anyone except me.

That’s when I realized what value I could provide! I was in a unique position to fill the knowledge gap. By putting pen to paper, I recorded the company’s technical history. From it’s founding to the present day, covering the technical decisions, trade-offs made, and tech debt accumulated.

While I’ve never won an award for my coding, my writing is a different story.

Photo by Pereanu Sebastian on Unsplash

I interviewed senior and staff engineers past and present, and one of the founders. What started as an internal document blossomed into a public article and a tech talk. It has aided in onboarding newer engineers, as well as giving back to the developer community.

I received accolades from many of those who viewed it, including the VP of Engineering, the Head of Product, and CTO. For a mid-level engineer, that level of attention was foreign but exhilarating. It also played a key factor in my future promotion.

This is not to say that everyone has to write a blog post, that’s just what I did. There are always ways to stand out and add value. If you’re a coding savant, then let your code speak. But if you’re a mortal like the rest of us, then play to your other strengths.

I Became Essential to My Team

Photo by Marvin Meyer on Unsplash

I had always thought of senior engineers as code machines. They were the ones finishing the most tickets, shipping the most features, and writing the most code. Imagine my disappointment when I realized that, along with being an average coder, I’m also a remarkably average producer.

However, I’ve come to learn an important lesson. While speed is nice, consistency is just as important. And what I lack in speed, I make up for in consistency. The tickets to which I commit myself, I finish. I don’t shy away from giving estimates. And if I get blocked, I update my manager and stakeholders and let them know how long I’ll be delayed.

In other words, I’m a reliable team member. I’m honest about what I can accomplish, and even more important, with what I can’t. I do my best to support my team and make their lives easier. This extends to emergencies as well.

Last year, my tech lead messaged me out of the blue to let me know that our metering service wasn’t working properly. A bug was merged into production that was preventing usage from being recorded. When I asked when the bug had been introduced, my heart sank when my lead responded with “over a week ago.”

This was a major problem. The system that we relied on for accurate billing had been down for over a week. No one knew why and how we hadn’t detected it earlier. To make things worse, the engineer who had built the service, and knew all its ins and outs, had left the company.

We needed to get the service back up and running as soon as possible. Not only that, but we also needed to backfill multiple weeks worth of usage data. With my tech lead in emergency meetings and the only senior engineer on my team unavailable, the task fell to me.

Photo by Tonik on Unsplash

Even though I didn’t cause the bug nor did I write the impacted service, I took ownership of the situation. I identified the bug, made the patch, and, crafted a solution for fetching the missing data. I also kept in constant communication with stakeholders, letting them know of updates, and assisting in the postmortem. When my team was in trouble, I stepped up.

But, I didn’t stop there. The experience had been so stressful that I vowed it wouldn’t happen again. I improved the tests, health checks, and added more substantial alerting. If something were to go wrong again, we would know about it immediately.

These added measures are what caught the attention of my director. He told me that mistakes happen — even mistakes that result in lost revenue. Ensuring those mistakes never happen again improves the team as a whole. And sure enough, my team has not had an issue with that service since.

Becoming a senior engineer, for me, was realizing that reliability is essential. The more dependable I am, the more my manager and the team will trust me. The added authority and responsibility that comes with a senior title can only be earned through trust. And trust can only be built over time through consistency.

A meaningless sign — Photo by Micaela Parente on Unsplash

A parting thought: Titles are meaningless.

However, they do signify growth and progress, which is what many of us seek in our careers. The money doesn’t hurt either.

Don’t seek the senior role just for the sake of a new title. Trust me, it won’t change anything. You might have some extra responsibilities and a slight uptick in recruiter emails, but you’ll be the same engineer you were before.

Growth and self-improvement should be your real goal. If you focus on mastery, the titles and accolades will come along the way. But don’t forget to trust your worth and advocate for yourself — don’t wait for someone else to do it for you.

Why I Didn't go to College for Software Engineering

Sun-Li Beatteay — Mon, 04 Jan 2021 18:55:44 +0000

The sounds of the keys clicked and clacked as I rushed my fingers across the laptop keyboard. I only had 30 seconds left.

“Gotta finish this fast,” I thought.

After a couple more finishing touches, I hit “Enter” and watched as a popup window appeared on-screen.

ALERT: MALWARE DETECTED! PLEASE INSTALL NEW ANTI-VIRUS UPDATE!

Beneath the popup were two buttons: Update and Cancel. Just as I had practiced.

A few moments later, I heard a door open and footsteps coming down the hallway. I shut the laptop, tip-toed to the filing cabinet across the room and began rifling through the papers, as if searching for something important.

Penny, my co-worker, strolled around the corner and made a beeline for her laptop on the counter. My pulse quickened in anticipation. Out of the corner of my eye, I saw the alert materialize as she opened her laptop.

Penny said nothing as she read the alert and quickly clicked Cancel. I bit my tongue — the last thing I wanted was to give myself away with an unintentional titter. The silence was broken when a new alert popped-up on Penny’s screen.

WARNING: New Virus Detected! Shut Down Now!

A red danger symbol accompanied the text, emphasizing the apparent danger:

Photo by Michael Geiger on Unsplash

“What the hell?” Penny whispered to herself.

She immediately powered down her laptop and slammed the screen shut as soon as the screen turned black. I bit my tongue harder but to no avail. I let out a muffled snort that devolved into a fit of laughter.

“Was this you?” Penny said, turning to me. I nodded. She sighed in frustration and relief.

“You really shouldn’t do that. We work with medical patients. It wouldn’t be good for them to think that our computers are infected.”

“Yeah, you’re right. Sorry. I won’t do it again.” I sighed, though my persistent smile revealed my true feelings. It worked! I thought to myself with glee.

A couple minutes later, my other coworker walked into the room holding her computer. Her eyes were wide, seeking assurance.

“Can one of you help? I think there’s something wrong with my computer.”

I looked to Penny, who was glaring at me. “That was the last one, I promise.”

This happened back in April 2016 but I’ve recently found myself reflecting back on that day quite a bit. For two main reasons.

The first is that it’s a memory that fills me with a sense of pride. Admittedly, the script I wrote wasn’t anything special and could have landed me in trouble with HIPPA. But it was the first time anyone had used code I had written, besides me.

The second reason is because it happened four years ago. Why is that length of time special? Because it was at this job that I had to make a difficult decision that would impact my life for that exact amount of time.

Two Roads Diverged

At the time, I was working as a technician in a physical therapy clinic. The pay wasn’t anything to brag about and there wasn’t any upward mobility. But it was a fine job for what it was: a means to an end.

I knew I wasn’t going to stay there for long. But I also wasn’t going to quit until I found a calling. Something that motivated me to continuously improve myself. Something that would help me achieve the life I wanted to live.

I had found it in coding.

Photo by ThisisEngineering RAEng on Unsplash

Programming made sense to me. It was logical yet creative. It was a craft that was welcoming to novices but difficult to master. I wanted to make this my career. The question was how?

Up until the day my coworker opened her computer to find a suspicious popup, I was self-taught. I would often practice and study on lunch breaks and after work. When I decided to make this my profession, I saw my road ahead diverge into two distinct paths.

Do I continue to teach myself to code? Or should I go to college?

Believe me, it was a tough choice. While I had been learning on my own for free, the idea of a degree was enticing.

I lived in Seattle, only a stones throw from the University of Washington. If you aren’t aware, the University of Washington has one of the best computer science programs in the country. Seattle itself is also home to some of the largest tech giants in the world.

Seattle | Photo by Felipe Galvan on Unsplash

In fact, a large portion of the patients I worked with were either software engineers or computer science students. I often talked to them about their jobs and studies. Who knew that sitting in front of a computer for over eight hours a day with bad posture can lead to physical ailments?

When I was at my crossroads, I asked some of the patients I worked with how they had landed their first jobs and whether they thought getting a degree was the best way to get a job.

To no surprise, most of the engineers had computer science pedigrees. And the students all thought that a degree, or at a certificate at the minimum, was the best way to get that coveted first job.

I agreed with them. A top-notch Computer Science education combined with internships at high-profile tech companies would open any employment door. It was a straightforward plan. However, before I made a decision, I needed to consider how life would look over the next four years if I followed this path.

If I chose to pursue a degree, it’s no doubt I would get a top-tier education and a significant boost to my career, but it would come at a price. Judging by the tuition rates, four years would costed me nearly $45,000 in student loans.

There was no question that I would have to take out loans — I had hardly a penny to my name. After all, I was still trying to pay off the debt from my first degree. A second trip to college would leave me with a total bill of around $70,000. While still in the five figures, the thought of accruing that much debt made my heart race.

But the even more intimidating factor was the time commitment — four years.

Photo by NeONBRAND on Unsplash

For a single eighteen year old with no life obligations, it’s not an issue. In fact, it’s expected. However, for someone nearly in their thirties, four years is an eternity to put life on hold.

I had loved my time at college the first time. I made great memories and friends. But I wasn’t 18 or single any more. I had just moved in with my girlfriend and we were enjoying a new chapter in our lives. I didn’t want to force her to live the starving college lifestyle all over again.

Carving My Own Path

So with all of those thoughts swirling in my head, I made my decision. I rejected the sage advice of the patients I worked with and dived headlong into the self-taught route.

I was gripped with fear at the idea of striking off into the unknown. I didn’t know anyone who had successfully taught themselves to code, much less land a job as a self-taught engineer.

But I also had hints of excitement and hope. In the months that I’d been studying, I had made leaps and bounds in progress. If there was even a chance that I could start my career in less than the four years it would have taken to earn a diploma, it would be worth it. Any time saved would be extra time spent with my family — enjoying the fruits of my labor.

And that’s exactly what happened.

After deciding that college wasn’t in the cards for me, I threw myself into self-study.

Photo by Clark Tibbs on Unsplash

I picked the brains of my techie patients until I eventually quit to study full time. My girlfriend and I moved to New York City and I signed up for every several online courses and tutorials. That is, until I settled on LaunchSchool.

From there, life is a blur. Programming, note-taking, and the (not-so) rare video-game break. All while sitting in my tiny New York apartment.

My original hope was to get my first job and start my career within four years. Through the power of dedicated study, practice, and Zelda, I was able to do it in two.

Ever since getting my first job, life has been anything but a blur.

As the alternate version of myself would have been sitting in classroom adding to his student loan debt, I paid off all of mine. And while college version of myself would have graduated into the worst job market in a decade, I got promoted and became a project lead.

While I’m proud of my career milestones, I’m even more proud of the time I was able to spend with my family and the adventures I’ve taken. In the past four years I’ve I road-tripped across the entire United States.

Photos from my road-trip

I’ve visited several countries.

Three of the countries (Thailand, Italy, and Iceland)

And proposed to the love of my life at Yosemite National Park.

Proposing at Yosemite National Park

All while the other version of myself was heads down in the books.

So Did I Make the Right Choice?

If you’ve read up to this point, you can probably guess that my answer is yes.

Four years ago, I was working in a physical therapy clinic and pranking my coworkers with fake viruses. I was treating software engineers on a daily basis while daydreaming of having their lives. Now, I’m one of their peers.

Not only that, teaching myself to code has resulted in a faster start to my career and more time spent with my family. I wouldn’t trade that for anything.

Photo by Daniel Cheung on Unsplash

But let me be clear, this was the right choice for me. I took a risk and it paid off. Not everyone who has made the same choice has been as fortunate. This article is the definition of survivor-ship bias and shouldn’t be seen as prescriptive.

For those of you learning to program, you may have to decide to either teach yourself, get a diploma, or something in between. If you’re in this position and are unsure what direction to take, I have some advice.

Remind yourself of your goals and which aspects of your life mean the most. — use that to shape your decision. The most important lesson I’ve learned in this process is to not sacrifice what means the most to you.

Don’t worry if other people are getting a job faster or a better starting salary. Breaking into tech isn’t the important part. It’s all about what you do when you get here.

Is a Computer Science Degree Worth It?

Sun-Li Beatteay — Thu, 07 May 2020 17:39:01 +0000

TL;DR: Is a computer science degree worth it? For me, partially. For you? You tell me.

In August 2018, I enrolled in college to get my computer science degree. However, I wasn’t your typical CS student. First off, I was almost 30 and already had a college degree from nine years earlier. Secondly, I was already a professional software developer with a full-time job.

The vast majority of the other students in my classes were looking either to break into the technology field or to leverage the degree for a better job. But I wasn’t trying to do either of those. So why did I even go back to school if I was already a software engineer? That’s a question I heard a lot. And rightfully so. It seems like a waste of time and money. Or at least it did to the people I told.

There were a number of reasons I chose to get a CS degree, such as improving my computer science and algorithm fundamentals — those areas aren’t my strongest. But there was one primary reason that motivated me to go back to school: to see what all the hype was about.

You see, I’ve written about my journey into tech as a self-taught developer a couple of times. I’ve heard people claim that a CS degree is the only way to become a real software engineer. While those people are in the minority, they were loud enough to make me scratch my head and wonder if they knew something that I didn’t.

“If you don’t have a software engineering degree you’re not a real software engineer. … You only know how to do some code.”

Despite the rise of free or cheap online resources, was it still worth forking over an arm and a leg for a college degree? Was it still the best way to become a software engineer?

Fortunately, I was in a position to answer that question myself. I had an employer that offered reimbursement for college courses, and I found a university with an online CS program that I could do part-time outside of work hours. So I decided to conduct an experiment. I would earn my CS degree and finally see for myself if it is really worth it.

But what does being “worth it” mean? Everyone has their own definition. For my experiment, I decided a CS degree would be worth it if it both offered more job opportunities and would have prepared me for my job better than my self-taught route did.

Fast forward one and a half years to April 2020, and my experiment has concluded. I am now a computer science graduate with the diploma to prove it.

Yep, that’s my degree

As for the results of my experiment? That’s what the rest of this article will be focused on.

As a side note: Take all of this with a grain of salt. This experiment was meant for entertainment rather than as peer-reviewed analysis, and my findings pertain only to myself. I hope my experience offers some enlightenment, though.

A Degree Still Matters (to Some People)

Photo by Clem Onojeghuo on Unsplash

One of the parameters of my experiment was to determine whether having a degree would result in job opportunities. It didn’t take long to find the answer.

When I first enrolled in the CS program, one of the first things I did was I updated my website, resume, and LinkedIn. Hey, I was excited and wanted everyone to know about it. I’m allowed to be a bit vain from time to time.

Within a couple of weeks of updating my LinkedIn, there was a noticeable uptick in InMails from recruiters. Not only that, they outright referenced my degree in their messages.

An email I got from a recruiter

But that itself wasn’t very convincing. So to test whether recruiters would notice the CS degree, I re-uploaded my resume to Hired. I had created a Hired profile six months earlier, though the results had been pretty dismal. I was curious to see if I would fare better this time around.

I did. By a lot.

Six months prior, I had only gotten a couple of hits, all of which tried to negotiate me down from the salary I was seeking. With the updated resume, I got eight hits within a couple of days, all of which accepted my suggested salary (which I had increased).

Now, I doubt the new section on my resume stating “computer science degree (2020)” was to thank for all of this newfound interest. I did have six months more work experience, after all. But I do think it played some role, as many job listings these days have a line saying “computer science degree (or related experience).” My degree made for a nice tick in that checkbox.

Photo by Glenn Carstens-Peters on Unsplash

I can’t say I was surprised by the new attention. A CS degree can signify to recruiters and hiring managers that you have been vetted by an accredited institution and have a basic understanding of software engineering that they have come to expect for entry-level engineers.

A self-taught engineer, on the other hand, is a wildcard. Since they don’t have prior experience, it’s difficult to know if they are ready for a job without diving into their portfolio. But most recruiters don’t know how to do that or don’t have the time.

It’s worth pointing out that not every recruiter or hiring manager will care about a computer science degree. I got a job without one and many companies are eliminating the degree requirement from their job listings. However, from what I’ve seen, having one is definitely a boon to your resume, especially early in your career.

A Degree Is Not Job Training

Photo by JESHOOTS.COM on Unsplash

The second portion of my experiment was to assess whether a CS degree would’ve prepared me for my job better than my self-education did. I thought this would be a close call.

I grew up in Seattle, home to Microsoft, Amazon, and one of the most prestigious computer science programs at the University of Washington. Every software engineer I knew had a CS degree and most companies wouldn’t hire anyone without one. I grew up thinking that a CS degree, and college in general, was the best training to get a job.

However, going through college twice now has made me realize that this couldn’t be further from the truth. A computer science degree is not job training. The curriculum is broad, shallow, and covers a wide array of subjects.

Speaking from experience, I took courses covering calculus, discrete math, operating systems, computer architecture, data structures and algorithms, network security, project management, artificial intelligence, machine learning, and much more. Not only that, but I learned about three different programming languages over the span of a year and a half.

While all of those subjects are important to learn about over the course of a software engineer career, it’s a lot upfront. It’s tough to master a specific skill or language when the course material jumps from subject to subject every month or so.

And mastery is important because employers want expertise. They want people who can perform well and provide value in a given role as soon as possible, be it back-end, front-end, data science, etc. If you’re a candidate who is kind of knowledgeable in several different skills and languages, you’re more likely to be passed over for someone with a more focused proficiency.

Let me be clear. I’m not saying college is bad. College was designed as a gateway to higher learning and is fantastic at propelling prospective students into careers in research and academia.

However, too many people confuse a degree for direct job training. For someone who wants a job in web development and isn’t interested in learning how Linux avoids POSIX thread deadlocking, a computer science degree isn’t what they’re looking for.

In the past, a CS degree was enough to get a job, but times are changing. The job market gets more and more competitive every year, and employers’ standards for entry-level engineers keep growing higher.

From what I’ve seen, the computer science curriculum itself is not enough to prepare students for jobs. Oftentimes, they have to supplement their learning with side-projects, open source, and, if they’re lucky, internships. And the students who don’t, fall behind.

This explains why more and more CS students are attending coding boot camps and why more and more new graduates are emailing me for help finding a job.

So Was It Worth It?

Photo by Brendan Church on Unsplash

Based on the criteria of my experiment, the answer to whether my CS degree was worth it is: kind of.

I have more recruiters popping into my inbox than I did before I got my shiny diploma. However, in regards to whether it would’ve trained me for my job better than my self-teaching did, the answer is resounding no.

There were certainly courses that were relevant to my day-to-day work, but even more that weren’t. And when it comes to moving into a new career, whether it’s your first career or your fifth, any time spent learning information you won’t use is an opportunity cost.

However, that’s not to say that my CS degree was worthless. In terms of supplemental education on top of my self-taught background, it was perfect. It expanded my knowledge of the computer science field and solidified my fundamentals. For example, while I don’t directly use cgroups and namespaces in my job, I do use container systems like Docker. Learning how an operating system uses cgroups and namespaces taught me a lot about how containers operate under the hood.

Despite all the articles titled “Self-Taught vs. CS Degree” and “How I Got a Job Without a CS Degree,” it doesn’t have to be one or the other. After all, there are gaps in both approaches. It’s all about finding balance.

Photo by Christophe Hautier on Unsplash

For those of you studying for your computer science degree, make sure to hone your programming skills with side projects. And for you self-taught developers, take time to learn the computer science fundamentals. You don’t need to earn a degree as I did; you can simply teach yourself CS.

But this article isn’t to offer career advice. In fact, take everyone’s opinion on education with a hefty level of skepticism. While I have my own preferences, that doesn’t mean they’ll work for you. It’s all about trade-offs and finding the ones that work for you.

There is no right or wrong way to learn, just your way.

How to Convert ES6+ into ES5 using Babel

Sun-Li Beatteay — Wed, 11 Mar 2020 16:51:39 +0000

So you want to start using ES6 but are worried about browser support and backwards compatibility for your application? Not to worry, Babel can help us there.

Babel is a transpiler (translates code in one language to another computer language at the same abstraction level) that can turn our ES6 code into ES5. I’m going to assume you know how to use both node and npm and have them both installed on your machine. If you don’t, start here.

1. Install Babel as Dev Dependency

First things first is to download the Babel command line interface, which will allow us to use Babel in our terminal. Navigate to your project folder in your command line and type the following command:

$ npm install -D babel-cli

2. Write ES6+ Code

If you already have ES6+ code written then you’re ahead of the game. If you don’t, do that now. My suggestion would be to house your JavaScript files in a folder, such as src or JavaScript. I’m going to make an example file:

    // src/example.js

    class Hello {
      static world() {
        console.log('Hello, World!');
      }
    }

    Hello.world();

3. Install env preset and create .babelrc file

Now that we have our JavaScript code, we need to set up Babel to transpile it for us. Unfortunately, Babel can’t do much out of the box. It needs the appropriate plugins and presets — curated groups of plugins — to do the work.

In our case, we’ll be using env preset and installing it as a dev dependency in your project.

$ npm install -D babel-preset-env

Now we need our .babelrc file. Much like how npm relies on package.json, Babel uses .babelrc to keep track of the plugins and presets we’ll be using.

Create your .babelrc file:

$ touch .babelrc

And write the following:

    // .babelrc

    {
      "presets": ["env"]
    }

4. Create a npm build command

You don’t need to do this, but it’ll make it faster and simpler if you need to transpile your code often.

Go to your package.json file where we’ll create a npm build script using the babel command. The babel command takes two arguments: the path to your ES6+ code and a path to where you want your ES5 code to go.

If you have all your JavaScript code housed in a directory, then you can add the -d flag to the command to tell Babel for look for directories instead of files. For my example, I have my JavaScript code in my src directory but want my ES5 code to be put in a build directory.

    // package.json

    ...
    "scripts": {
      "build": "babel src -d build",
    },
    ...

5. Run the Babel command

With your .babelrc file created and your build command ready, just run it in your command line.

$ npm run build

You should see your converted JavaScript files in a build directory. Now all you need to do is include/require them in your application and you should be ready to go!

Am I a “real” Software Engineer yet?

Sun-Li Beatteay — Wed, 04 Mar 2020 18:31:46 +0000

Am I a “real” Software Engineer yet?

This question has haunted me for years. And it seems I’m not alone. Many people experience this same insecurity. They desire to know if they’ve made it. *Are they finally *good enough?

While “Software Engineer” is the standard title handed out by employers, many in the software community have different standards. And to new programmers joining the field, especially those without CS degrees, it can feel like the title is safe-guarded. Only bestowed on the select that have proven themselves.

Many people refer to this sense of deficiency as Impostor Syndrome, though it goes by many names. Developers experience it in different ways and have differing reactions. To some, it can be crippling, but others might not notice it whatsoever.

In this article, I’m going to recount my own ordeals with Impostor Syndrome. While I don’t pretend to have a cure, I hope to shed some more light on the topic and help others who are dealing with it.

Note: All the comments I’ve shared are real comments I’ve found on the internet. Not all of them were directed towards me, but I’ve heard similar remarks. I’m sure you have too.

When it all began

Photo by Danielle MacInnes on Unsplash

For me, it started in July of 2016. I had started studying web development as a New Years Resolution. I wasn’t satisfied with my career at the time and was looking for a change. A common origin story that I’m sure many people can relate to.

I focused on front-end development as I had heard that HTML, CSS and JavaScript were easy to pick up. I spent my evenings after work parsing through Treehouse, Lynda, and Codecademy tutorials. The first three months of 2016 passed by like this. By April, my notebook was full of thorough notes and my GitHub contained a few static sites. But I wasn’t satisfied. I wanted to sink my teeth into a larger project.

That was when I decided to create a portfolio site for my wife, who was a product designer. For my skill level at the time, it wasn’t an easy undertaking. I struggled a lot and it took the better part of four months to complete.

It’s important to note that while working on my wife’s website, I did my best to surround myself with tech culture. I watched YouTube videos, listened to podcasts, and read blog posts from experienced engineers to keep myself motivated. I daydreamed what it would be like to stand in their shoes. To have a job at a tech company, work on the latest technology, and write “Software Engineer” as my Occupation on tax forms. Silicon Valley made it all look so glorious.

That’s why I couldn’t have been happier when my wife’s website went live in July of that year. I had done it. I had made my first real website that was going to receive real traffic. Could I finally start calling myself something of a Software Engineer?

“Web development isn’t real programming”

Not according to the Internet:

“being a fully competent and complete software is much more … ”

“Web development is not real programming” — Read more here

“Web development isn’t real programming”, “JavaScript isn’t a real programming language”, and “a front-end developer isn’t a Software Engineer” were all phrases I heard often. While I was determined to change careers, I was also self-conscious. I couldn’t shake the thought that the naysayers were right.

If I’d had a Plan B or a stable career to fall back on, I might’ve given up right then and there. Fortunately, I didn’t. For better or worse, I had no other career or skills to fall back on and I had already invested too much time. So I trudged on, with the sunk cost keeping me afloat.

I spent the next 18 months studying software development full time. I quit my job and moved in with my in-laws — which was a journey in-and-of itself. I put everything I had into my career move. I studied Ruby, Node, and some Go and built some small web apps.

By January of 2018, it had been two years since the start of my career switch and I had learned a great deal. I knew the fundamentals, I could program in multiple languages, and I had started contributing to open-source projects. I also had a decent portfolio to show for it. I was excited to finally call myself a Software Engineer.

“Not so fast,” said the Internet.

“software engineer” after 1–2 years? yeah no.”

“If you don’t have a software engineering degree you’re not a real software engineer. ... You only know how to do some code.”

At this point, I had dealt with many doubters. While it still didn’t feel great to hear criticism, I had learned to ignore it for the most part. Furthermore, I had an Ace up my sleeve. There was one final step I could accomplish to put all the doubts to rest: Getting a job.

Yes, a software engineering job. The holy grail. I would have the title, the paycheck, and the acclaim. No more self-doubt. Only deserved validation from an employer.

And that’s what I did. I started interviewing in early 2018 and signed my first offer letter by March of 2018. My official role was “Software Engineer”. That was that, the hunt for acceptance was finally over.

From bad to worse

“You’re still a newbie.. and will be for a few more years at least. Software engineer by title, that’s all.

“I’ve been in software engineering for 30 years. …You haven’t even begun to understand what you don’t know yet. Good luck to you!”

At least that’s what I thought. It seemed the gate-keeping extended even into the realm of employment. But it was over. I was a Software Engineer — I had the W4 papers to prove it. The moment I stepped into the office for my first day on the job, all my worries would disappear.

But that wasn’t the truth. Who knew that adding obligations and deadlines would make one’s insecurities worse, not better?

My anxiety hit a fever pitch the moment I arrived for my first day of onboarding. Slack conversations I didn’t understand. GitHub repositories with thousands of lines of code I didn’t comprehend. Development tools I had never heard of or used. My first week on the job was a whirlwind of stress.

Don’t get me wrong, I was thrilled to be surrounded by such experienced engineers. My team was full of some of the smartest engineers I’d ever met and they were incredibly understanding. They mentored me and ramped me up to speed. But it was still intimidating as hell.

Many of my coworkers had advanced degrees, some had been programming since they were teens, and others were core maintainers of large open source projects. And who was I? Some guy who learned to code because it looked cool on TV. I felt out of place and the feeling didn’t disappear.

My thoughts began to resemble the negative comments that I had worked so hard to prove wrong:

How the hell did I get here? Was it luck? Did someone make a mistake?

I realized that I didn’t need Reddit to point out my inadequacies — I was quite capable of that myself. It’s true that, with time, I became more familiar with my environment. I began to understand the Slack conversations, the code became more familiar, and I learned how to use the various tools. And yet, I couldn’t shake the feeling that I was a stranger in a foreign land. Every mistake I made only served to prove that point further.

I anticipated the day that I made one mistake too many and someone finally asked, “How on Earth did you get hired?” That fear consumed me. I began reading whitepapers, frequenting Hacker News, and taking classes to get a Computer Science degree. All in the hopes that I would finally feel like I fit in. That I was a “real” software engineer. But none of it worked.

Luckily, I found something that did.

Finding my solution

The solution that I found for myself was simple yet terrifying.

Talking.

I had to talk to someone. I couldn’t keep it bottled anymore. However, I was too nervous to ask any of coworkers a question like, “Have you ever felt like a fraud?” Instead, I started with my close friends. But I soon found myself discussing the topic to anyone I could, including coworkers and teammates. Because the more I talked and listened, the more I realized I wasn’t alone.

As it turns out, impostor syndrome is so common that it has become a cliché!

“Another impostor syndrome article …. Walking in to work everyday with a complete mastery over everything you do and write is a problem of its own.”

Many professionals at the top of their careers experience it. Tina Fey, John Steinbeck, and even Albert Einstein. It was comforting to find out that I was in such good company. Furthermore, when talking with a close friend, he said something that struck a chord with me:

“The only time impostor syndrome is bad is when you don’t have it. Feeling like a fraud is a sign that you’re learning. Having anxiety in a new and uncomfortable environment is perfectly normal. The trouble comes when you feel like you know exactly what to do and how everything works. If you find yourself in that situation, you aren’t learning anymore.”

It was in that moment that I realized impostor syndrome wasn’t something to fight or ignore. With the right mindset, it can be a tool. That may sound like some feel-good BS, but public speakers employ a similar trick to convert anxiety into excitement.

This isn’t to say that my insecurities have magically disappeared. Far from it. Whenever I’m given a new project to work on, my panic makes an unwelcome return. My heart rate spikes and my mind conjures up fantastical feats of failure. However, I’ve learned to interpret this nervous energy as a sign that I’m learning and pushing myself to new heights.

When my manager gives me a complicated task, it means that they trust that I can get it done. I may need to ask for help, research the topic, or simply go for a walk to cool my nerves, but I know I’ll get through it. I still make mistakes all the time and that will never change. It’s all a part of the process.

After all, no one knows everything, not even within their own specialty. Dan Abramov doesn’t know flexbox and David Heinemeier Hansson still looks up ActiveRecord methods he wrote himself. If perfection was a feasible goal, Stack Overflow wouldn’t be as popular as it is.

Am I a “real” software engineer?

So after all this time, am I a “real” software engineer yet? LinkedIn seems to think so.

The better question is, “Does it matter?” I haven’t always been passionate about tech and I lack the enticing origin story of a Rockstar Engineer. But I’m here now and I’m here to stay. So to those in the tech community who have strong opinions about my title, call me whatever you want. A software engineer, programmer, script kiddie...

My title no longer matters to me. What matters is what I do. And what I do is use code to solve problems and architect solutions to make peoples’ lives better.

Best of all, I’m damn good at it.

Building Conclave: a decentralized, real time, collaborative text editor

Sun-Li Beatteay — Fri, 28 Feb 2020 17:06:21 +0000

Try Conclave here

Read our in-depth Case Study here

Note: Conclave is no longer maintained and not production ready. However, the code is open source and available for learning purposes

In 2018, GitHub released TeleType. Teletype is a plugin that was built by a remote three person team and allows for collaborative editing in the Atom code editor.

Teletype accomplishes this by utilizing WebRTC and Conflict-free Replicated Data Types. It is a really cool piece of software and you should definitely check it out.

However, this is not the story of how Teletype was built.

This is the account of a different remote three person team who built a different open-source collaborative editor using the exact same technology in the same time period.

This article is about Conclave.

Conclave is a decentralized, real time, collaborative editor for the browser. This article is going to discuss the many challenges we faced as well as the solutions we found. By the end of this piece, we hope that many of you will want to create a collaborative editor yourself. It is a very fun project and we highly recommend it to any developers interested.

Even if you are not interested in creating your own editor, you will still learn a lot from this article. Creating and scaling a real time application has many lessons that apply to other realms of software engineering.

If you are interested in either:

distributed systems
advanced data structures
maintaining consistency without a central data store

Keep reading.

Meet The Team

Before we get started, I want to give a big shout out to the ragtag team of developers who created Conclave.

What a handsome bunch.

There is Elise Olivares from San Francisco, CA, Nitin Savant in Minneapolis, MN, and Sun-Li Beatteay based in Brooklyn, NY. It was a pleasure getting to build this project together.

Now, onto the tech.

Why a collaborative editor?

A sensible question to ask is: Why even build a collaborative editor in the first place?

Having used Google Docs quite a bit as a team, we had always been intrigued by how it worked under-the-hood. Ironically, Googling how Google Docs works is fruitless. They are quite secretive about their proprietary software.

In the end we decided that, if Google would not tell us, the next best way to learn was to build it ourselves.

Interesting Software Engineering Challenges

Besides being a creative way to learn how Google Docs works, creating a real time, collaborative text editor poses some interesting software engineering challenges:

Maintaining consistency across all users while merging conflicting operations.
Reducing latency as much as possible to achieve a real time effect.
Scaling a real time application.

Let us go over each of these challenges in detail.

Challenge 1: Maintaining Consistency / Merging Conflicting Operations

The most important part of a collaborative editor is guaranteeing consistency across all users. After all, if the document did not stay consistent, it would be useless.

But how is this achieved?

To answer that, it is important to understand exactly what a text editor is and how it works.

What is a text editor?

For our project, we defined a text editor as a space where you can insert or delete text characters. Each character has a value and a numerical index that determines its position in the document.

For example, with the text “HAT”, the first character has a value “H” and a position of 0, “A” has position 1, and “T” has position 2.

A character can be inserted or deleted based on its positional index. To insert a “C” at the beginning of the text, the operation is insert("C", 0). This insertion causes all the other letters to shift their position to the right by 1.

To delete the “H” would require the operation delete(1).

One user editing a document is simple, but what if we want multiple users simultaneously editing the same document?

Multiple concurrent users

First, we will need to provide each user with a local copy of the document and allow them to make edits. Remember, we are aiming for “real-time”. We want users to be able to apply their edits immediately as if they were using a simple text editor.

Next, we need a way for users to inform other users of edits they made. We will introduce a Central Relay Server to facilitate this communication.

Two users connected through a central relay server.

The problem with this situation comes when users attempt to make concurrent edits.

Commutativity

As an example, say there are two users who both start with the word “HAT”. One user inserts a “C” while the other deletes the “H” and both of their edits are sent to the other person to be incorporated.

Oh no! One user has a “HAT” and the other user has a “CAT”. Their documents did not converge to the same state.

The reason for this divergence is because the insert and delete operations did not commute.

Commutativity occurs when different operations produce the same result no matter the order in which they are applied. Addition and multiplication are all commutative operations.

Idempotency

Let us try another example where the users simultaneously decide they want to delete the “H” from “HAT” to get “AT”.

The documents did converge but we have another problem! Both users ended up with “T” instead of “AT”. Neither of them wanted this result. This occurred because the delete operations are not idempotent.

Idempotency is when repeated operations produce the same result. For example, multiplying by 1 is an idempotent operation. No matter how many times you multiply a number by 1, the result is the same.

Consistency Requirements

Looking at the previous two example, we can see that a collaborative text editor must have the follow properties to remain consistent across all users:

Commutativity: Concurrent insert and delete operations converge to the same result regardless of the order in which they are applied.
Idempotency: Repeated delete operations produce the same result.

With the problem identified, how do we go about solving it? For our team, this required plenty of researching before we came upon two possible solutions.

Operational Transformation (OT)

If you are familiar with distributed systems, you may have thought “Operational Transformation can solve this problem.” That was the first solution we found as well.

To keep this article as brief as possible, we will not go into what OT is. To give you an idea of why we decided against OT, read this quote from a Google engineer:

“Unfortunately, implementing OT sucks. There’s a million algorithms with different tradeoffs, mostly trapped in academic papers. The algorithms are really hard and time consuming to implement correctly. […] Wave took 2 years to write and if we rewrote it today, it would take almost as long to write a second time.”
Joseph Gentle (Google Wave/ShareJS Engineer)

If you are interested to learn more about OT, you can read the Operational Transformation section of our case study.

Conflict-Free Replicated Data Type (CRDT)

The other solution we found was the Conflict-Free Replicated Data Type (CRDT). CRDTs were first created by researchers who were trying to simplify OT. While OT relies on a complex algorithm to maintain consistency, CRDTs utilize a more advanced data structure.

CRDTs operate by converting each character in the document into a unique object with specific properties.

siteId: an id to identify which user created it.
value: which letter the object represents.
position: a list of integers that represent the position of the character in the document. This position is relative to the characters around it.

Conversion of letter into character object

Since each character is unique and can be identified by these properties, we can prevent any character from being inserted or deleted more than once. This enables commutativity and idempotency.

The downside to this approach is the large amount of metadata. This increases the memory consumption of our app. However, since the overall complexity of CRDTs is significantly less than OT, we were content with this tradeoff.

How To Create A Relative Position

Unless you are already experienced with CRDTs, you are probably wondering “what is this ‘relative position’ property he mentioned and how is it created?” Allow us to elaborate.

Relative positions are the key concept that differentiates a CRDT from OT. The positions of characters in a CRDT never change even if the characters around them are removed. Furthermore, the relative position can always be used to determine the location of the character in the document.

The problem now is: How do we create these relative positions?

We can do this by imagining the characters and their positions like nodes in a tree. When we type a letter, it is given a position that is greater than the character before it but less than the character after.

If we write the word “CAT”, each letter might get a position as in the diagram below.

Example positions

However, what if we want to insert a character between two adjacent positions? If we want to turn “CAT” into “CHAT”, there is no integer between 2 and 3. For this, we need to move down to the next level of the tree and pick a position on the that level.

Inserting characters between adjacent positions.

This creates a fractional index. “C” has a position of 1, “A” has a position of 2 and “H” has a position of 1.5. In code, we represent this fraction as an array of integers.

Fractional positions as arrays of integers.

CRDT Commutativity and Idempotency

If we go back to our previous example, we can see how CRDTs maintain commutativity and idempotency. The fractional indexes of the characters are included for reference.

CRDT Commutativity

Using relative positions allows us to be more specific about which letter we are deleting and where it is located. Due to this specificity, commutativity is not an issue.

CRDT Idempotency

Additionally, since each character is unique, we cannot delete it more than once from the CRDT.

To read more about how to implement a CRDT in code, check out the Coding the CRDT section of our case study.

Challenges 2 and 3: Reducing Latency and Scaling a Real-Time Application

Now that we have gone over how to merge conflicts and keep a consistent document, it is time to tackle the two remaining problems: latency and scaling.

Our current system architecture relies on the client-server model of communication. Each user is connected to a central server via a WebSocket connection. The central server acts as a relay by forwarding operations from each user to all the other users in the network.

Multiple users connected through a central relay server.

Are there improvements that can be made to this model? In order to find improvements, we must first define the limitations.

Limitations of a Central Relay Server

The first limitation is an unnecessarily high latency between users. All operations are routed through the server. Even if users are sitting right next to each other, they must still communicate through the server.

It takes around 200–300ms for two users in California to communicate with each other through a server in New York. This latency directly impacts how “real-time” our application feels. If they could send messages directly to each other, it would only take a few milliseconds.

Latency across the United States.

The second limitation is that a central server can be costly to scale. As the number of users increases, the amount of work the server must do increases accordingly. To support this, the server would require additional resources, which costs money.

For a well-funded startup, this would not be an issue. As a team creating an open source project, we wanted to minimize the financial cost as much as possible.

Finally, reliance on a central server creates a single point-of-failure. If the server were to go down, all users will immediately lose their ability to collaborate with each other.

Peer-to-Peer Architecture

We can remove these limitations by switching to a peer-to-peer architecture. Rather than having one server and many clients, each user can act as both the client and server.

Whenever a user makes a change, or receives an operation from another user, they can relay that operation to all the the users they are connected to. This would allow messages to be passed directly between users from one end of the network to the other.

In distributed systems, this is known as the Gossip Protocol.

P2P architecture

How to create a P2P system?

To allow users to send and receive messages directly between each other, we used a technology called WebRTC. WebRTC stands for Web Real Time Communication and is a protocol that was designed for communication over peer-to-peer connections.

While WebRTC enables our users to connect directly, a small server is required to initiate those peer-to-peer connections in a process called “signaling”.

It is important to mention that while WebRTC relies on this signaling server, no document content is ever sent through it. It is simply used to initiate the connection. Once a connection is established, the signaling server is no longer necessary.

Establishing WebRTC connections between users.

For the sake of brevity, we will not delve too deep into how WebRTC works. For Conclave, we use a library called PeerJS to handle most of the grunt work for us.

To read more about creating a P2P system, WebRTC, and how secure WebRTC is, check out the P2P section of our case study.

Causality

One concept we have not covered yet is how to maintain causality. Causality is the relationship between cause and effect. To maintain causality is to guarantee an effect when there is a cause.

In the context of a collaborative text editor, to guarantee causality means that all operations will be received in the order they were made.

While maintaining causality in a server-client model is hard enough, it becomes even more difficult with a P2P system using WebRTC. The reason for that is because WebRTC uses the UDP transport protocol.

UDP helps reduce latency because it allows users to send messages quickly without a response from the receiver. The tradeoff is that it does not guarantee in-order packet delivery.

This presents a potential issue. What if a user receives a message to delete a particular character before it has inserted that character?

In the diagram below, there are three peers collaborating on a document. Two of the peers are next to each other while the third is far away. Peer1 types an “A” and sends the operation out to both peers. Since Peer2 is nearby, it quickly receives the operation but decides it does not like it and promptly deletes it.

Peer1 inserts a character and Peer2 immediately deletes it.

Now both the insert and delete operations are on their way to Peer 3. Due to the unpredictability of the Internet, the delete operation races past the insert operation.

The delete operation arrives at Peer3 before the insert operation.

What happens if the delete operation arrives at Peer3 before the insert operation? We would not want to apply the delete first because there would be nothing to delete and the operation would be lost. Later, when the insert is applied, Peer3’s document would look different from the others. Causality would be lost.

We need to find a way to delay the delete operation until after we have applied the insert.

Version Vector

To solve this problem, we implemented what is called a Version Vector. It sounds fancy but it is simply a strategy that tracks which operations we have received from each user.

Whenever an operation is sent out, in addition to the character object and the type of operation (insert/delete), we include the character’s Site ID and Site Counter value. The Site ID indicates who originally sent the operation and the Counter indicates which operation number it is from that particular user.

When a peer receives a delete operation, it is immediately placed in a Deletion Buffer. If it were an insert, we could apply it immediately. However, with deletes, we have to make sure the character has been inserted first.

After every operation from another user is received, the deletion buffer is “processed” to check if the corresponding character has been inserted. If it has, the deletion operation can be executed.

In this example, the character that is supposed to be deleted has a Site ID of 1 and Counter of 24. To check if the character has been inserted, Peer3 consults its version vector. Since Peer3 has only seen 23 operations from Peer1, the delete operation will remain in the buffer.

The first time the buffer is processed, the delete operation is not ready to be applied by Peer3.

After some more time, the insert operation finally arrives at Peer3, and its version vector is updated to reflect that it is seen 24 operations from Peer1.

Since we have received a new operation, we again process the deletion buffer. This time, when the deletion operation’s character is compared to the version vector, we see that the complement insert has been made. The delete operation can be removed from the buffer and applied.

This time the delete operation can be applied by Peer3.

Final System Architecture

With a Version Vector in place, the collaborative text editor is fully functional. The final system architecture for the application we made is shown below.

Final System Architecture

An example user flow may look something like this:

A user inserts a letter to their text editor
That change is added to their CRDT and converted into a character object.
That local insertion is broadcasted out to the rest of the users using a Messenger class — which is essentially a wrapper for WebRTC.
That same Messenger class is also responsible for receiving operations from other users. Those received operations are verified against the Version Vector and CRDT before being incorporated into the editor.

A Controller class is used to communicate between all the different components and make sure everything runs smoothly.

Conclusion

We hope you enjoyed reading about our journey as much as we enjoyed the journey itself! If you would like to read more about Conclave and learn how to implement a collaborative editor yourself, please check out our full case-study here.

Thanks for reading and happy coding!