DEV Community: Stanislav Kozlovski

Kotlin - Unit Testing Classes Without Leaking Public API!

Stanislav Kozlovski — Sat, 05 Dec 2020 08:34:23 +0000

Introduction

Kotlin is an amazing new up-and-coming language. It's very actively developed and has a ton of features that make it very appealing.
It's been steadily gaining market share ever since Google added Android Development support for it (back in 2017) and made it the preferred language for such development exactly one year ago (May 2019)

For any Java developer coming into the language, spoiler alert, there is one big surprise that awaits you --- the package-private visibility modifier is missing. It doesn't exist.

Access Modifiers

An access modifier is the way to set the accessibility of a class/method/variable in object-oriented languages. It's instrumental in facilitating proper encapsulation of components.

Java

In Java, we have four access modifiers for any methods inside a class:

void doSomething()- the default modifier, we call this package-private. Any such declarations are only visible within the same package (hence the name, private for the package)
private void doSomething() --- the private modifier, declarations of which are visible only within the same class
protected void doSomething() --- declarations only visible within the package or all subclasses.
public void doSomething() --- declarations are visible everywhere.

This wide choice of options allows us to tightly-control what our class exposes. It also gives us greater control over what constitutes a unit of testable code and what isn't. See the following crude example:

package doer.something;

/**
 * Does something.
 */
public class JSomethingDoer {

  private int motivationLevel;

  /**
   * @param motivationLevel a 0-100 integer, denoting how motivated the #{@link JSomethingDoer} is.
   */
  public JSomethingDoer(int motivationLevel) {
    this.motivationLevel = motivationLevel;
  }

  public void doSomething() {
    doOneLittleThing();
    maybeDoSecondLittleThing();
  }

  void doOneLittleThing() {
    System.out.println("Doing one thing.");
  }

  // package-private for testing
  void maybeDoSecondLittleThing() {
    if (feelingAmbitious()) {
      doSecondLittleThing();
    }
  }

  private void doSecondLittleThing() {
    System.out.println("We're overachievers, doing a second thing!!!");
  }

  private boolean feelingAmbitious() {
    return motivationLevel > 90;
  }
}

We can only directly test three of the five methods above. Because doSecondLittleThing() and feelingAmbitious() are private, our unit tests literally cannot call those methods, hence we cannot test them directly.

The only way to test them is through the package-private method maybeDoSecondLittleThing() , which can be called in our unit tests because they're in the same package as the class they're testing.

package doer.something;

import org.junit.Before;
import org.junit.Test;

public class JSomethingDoerTest {

  private JSomethingDoer doer;

  @Before
  public void setUp() {
    doer = new JSomethingDoer(11);
  }

  @Test
  public void Test_doSomething() {
    doer.doSomething();
    // assert
  }

  @Test
  public void Test_doOneLittleThing() {
    doer.doOneLittleThing();
    // assert
  }

  @Test
  public void Test_maybeDoSecondLittleThing() {
    doer.maybeDoSecondLittleThing();
    // assert
  }
}

The package-private modifier comes very useful in this example, because it allows us to test the maybeDoSecondLittleThing() method directly!
Testing would be more cumbersome if we had to test all code paths of the maybeDoSecondLittleThing() method by calling the public doSomething() method.

The package-private modifier also ensures that we don't leak any such internal methods to packages outside of this one.

Notice the different package doer in the Main class.

Kotlin

Kotlin is a bit weirder in this regard. It also has four modifiers:

private fun doSomething() --- vanilla private modifier
protected fun doSomething() --- the same as Java's protected --- only visible within this class and subclasses.
public fun doSomething() --- visible everywhere

These three are pretty standard. I want to focus on the fourth one:

internal fun doSomething() --- Internal declarations are visible anywhere inside the same module.

This is as good as public inside the same module, in that sense. Many small projects have no need for multiple modules, but they're likely to have multiple packages.

There is basically no way to limit access to a Kotlin method to only be within the same package.

What does this mean? It means that we either need to make our methods private, thus making our testing life harder, or make them public, thus exposing unnecessary API to components in the same module.

Our same example from above, again in the doer.something package:

package doer.something

/**
 * Does something.
 * @param motivationLevel a 0-100 integer, denoting how motivated the #[SomethingDoer] is.
 */
open class SomethingDoer(private val motivationLevel: Int) {
  fun doSomething() {
    doOneLittleThing()
    maybeDoSecondLittleThing()
  }

  protected fun doOneLittleThing() {
    println("Doing one thing.")
  }

  private fun maybeDoSecondLittleThing() {
    if (feelingAmbitious()) {
      doSecondLittleThing()
    }
  }

  private fun doSecondLittleThing() {
    println("We're overachievers, doing a second thing!!!")
  }

  private fun feelingAmbitious(): Boolean {
    return motivationLevel > 90
  }
}

But this time, from the doer package, we can call every method:

How do we solve it?

Solution 1 --- Refactor!

It seems to be commonly believed that we shouldn't test methods that are/should be private. The idea being that:

all private methods are reachable by the public methods
the public methods expose the interface/contract of a class
private methods are implementation details, and we want to test the class' functionality, not implementation

While that makes sense in theory, reality is not as black-and-white.

Regardless, you should treat any need to test non-public methods as a code smell.
Re-evaluate your design and rethink whether it makes sense to refactor the logic around such that you get the same test coverage by testing public interface contracts, rather than implementation details.

In some cases, though, it is cleaner and more maintainable to get full test coverage by testing each tiny private method code thoroughly, rather than testing each conditional branch via public methods or creating unnecessary boilerplate (e.g additional classes).

Solution 2 --- Reflection

An alternative solution is to keep the methods private and use the language's reflection features to call into said private methods.

Add a dependency on the kotlin-reflect library and code away!

  fun callPrivate(objectInstance: Any, methodName: String, vararg args: Any?): Any? {
    val privateMethod: KFunction<*>? =
        objectInstance::class.members.find { t -> return@find t.name == methodName } as KFunction<*>?

    val argList = args.toMutableList()
    (argList as ArrayList).add(0, objectInstance)
    val argArr = argList.toArray()

    if (privateMethod?.javaMethod?.trySetAccessible()!!) {
      privateMethod?.apply {
        return call(*argArr)
      } ?: throw NoSuchMethodException("Method $methodName does not exist in ${objectInstance::class.qualifiedName}")
    } else throw IllegalAccessException("Method $methodName could not be made accessible")

    return null
  }

Woah.

Here, our callPrivate() test helper is the star of the show! It allows us to call into private methods, albeit in a funky way.
We need to specify the method name by a string, hence we lose all type checking guarantees. This results in fragile code that's also hard to read.

We can leverage a compiler plugin, like Manifold, to get type-safe meta-programming. But you can argue that takes it a bit too far, as it introduces a large dependency that involves a lot more meta-programming behind the scenes.

Solution 3 --- Open the class, protect the methods and test a wrapper!

A third, perhaps preferable way to solve this is with a bit of subclassing work.

First, we need to make our class open for extension via the open keyword. Then, we need to make our private methods protected.

open class SomethingDoer(private val motivationLevel: Int) {
  fun doSomething() {
    doOneLittleThing()
    maybeDoSecondLittleThing()
  }

  protected fun doOneLittleThing() {
    println("Doing one thing.")
  }
}

Finally, we can create a subclass inside the test class whose sole goal is to expose the protected method directly.

class SomethingDoerTest {

  class TestSomethingDoer(motivationLevel: Int) : SomethingDoer(motivationLevel) {
    fun testDoOneLittleThing() {
      super.doOneLittleThing()
    }
  }
  private lateinit var doer: SomethingDoer
  private lateinit var doerWrapper: TestSomethingDoer

  @Before
  fun setUp() {
    doer = SomethingDoer(99)
    doerWrapper = TestSomethingDoer(99)
  }

  @Test
  fun doSomething() {
    doer.doSomething()
    // assert
  }

  @Test
  fun Test_doOneLittleThing() {
    doerWrapper.testDoOneLittleThing()
    // assert
  }
}

This is yet another imperfect solution to the problem which results in more boilerplate and a non-totally-private method.

It is better than the reflection approach in that it's type-safe, therefore less prone to breakage and significantly easier to maintain.

Summary

In this short article we went over the access modifiers of both Java and Kotlin and saw where Kotlin comes short. We proposed some solutions to Kotlin's lack of the package-private modifier with relation to testing, all with sufficient code examples!

Speaking of which, the code for this article live on GitHub --- here.

References

This problem has been hit by many, unsurprisingly. It makes sense given that most people come from the same background --- we miss our good old Java features 😭

Kotlin Discussion Forum --- https://discuss.kotlinlang.org/t/kotlin-to-support-package-protected-visibility/1544; https://discuss.kotlinlang.org/t/another-call-for-package-private-visibility/9577
KT Issue on the matter --- https://youtrack.jetbrains.com/issue/KT-29227
Another take on Medium --- https://medium.com/virtuslab/on-the-missing-package-private-or-why-java-is-better-than-kotlin-in-this-regard-4a1c9ecbe40c

Working With Multithreaded Ruby Part II

Stanislav Kozlovski — Mon, 16 Oct 2017 07:19:09 +0000

We're back at it with another edition of Multithreaded Ruby where we'll continue to dive into concurrency using our beloved language!
Today, I'm going to introduce you to a famous multi-process synchronization problem called the Producer-Consumer problem and we're going to look at Ruby's ConditionVariable class.

Back to Deadlock

A paragraph into the new article and we're at deadlocks again? Well yes, they're pretty prevalent and we did not actually touch on a solution to the problem last time.
Let's bring back the deadlock example we used in Part I, modified just a tiny bit.

require 'thwait'

item_accessories = {}
item = {}
item_acc_lock = Mutex.new
item_lock = Mutex.new

a = Thread.new {
  item_acc_lock.synchronize {
    sleep 1  # pretend to work on item_accessories
    item_lock.synchronize {
      # pretend to work on item
      sleep 1
      puts 'Worked on accessories, then on item'
    }
  }
}

b = Thread.new {
  item_lock.synchronize {
    sleep 1  # pretend to work on item
    item_acc_lock.synchronize {
      # pretend to work on item_accessories
      sleep 1
      puts 'Worked on item, then on accessories'
    }
  }
}

ThWait.all_waits(a, b)

> enether$ ruby item_worker.rb
/Users/enether/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/thwait.rb:112:in `pop': No live threads left. Deadlock? (fatal)

No surprise here, thread a obviously takes a hold of item_acc_lock, thread b takes a hold of item_lock and each of them waits for the opposite lock in an endless loop. So how could we avoid this?
What if we had a way to temporarily release one of the locks at a specific point in our program where we could afford doing so? That way, the other thread could take the lock, do its thing and return it back for the original one to finish its work.

Enter ConditionVariable

ConditionVariable is a Ruby class which lets you block a thread until another thread signals it OK to continue. It is a way to say - "I'm waiting for a lock and I can give up mine at this exact time". It is an ideal way to synchronize our a and b threads here:

require 'thwait'

item_accessories = {}
item = {}
item_acc_lock = Mutex.new
item_lock = Mutex.new
cv = ConditionVariable.new

a = Thread.new {
  item_acc_lock.synchronize {
    sleep 1  # pretend to work on item_accessories
    # At this point, we've just finished work on item_accessories and we're at a window where we
    #   might not care if item_accessories changes. So: let somebody else take it and give it back
    cv.wait(item_acc_lock)  # Temporarily sleeps the thread and releases the lock
    puts 'Gained back access to item_acc_lock'  # on this line, item_acc_lock is re-acquired 
    item_lock.synchronize {
      # pretend to work on item
      sleep 1
      puts 'Worked on accessories, then on item'
    }
  }
}

b = Thread.new {
  item_lock.synchronize {
    sleep 1  # pretend to work on item
    item_acc_lock.synchronize {
      # pretend to work on item_accessories
      sleep 1
      puts 'Worked on item, then on accessories'
    }
    cv.signal
    puts "I'm still working, but I'm finished with item_acc_lock"
  }
}

ThWait.all_waits(a, b)

> enether$ ruby synchronized_item_worker.rb
Worked on item, then on accessories
I'm still working, but I'm finished with item_acc_lock
Gained back access to item_acc_lock
Worked on accessories, then on item

What we achieved here is a sort of synchronization: we can now be sure that one b thread will always reach its cv.signal line before an a thread starts working on item_accessories.
Here is a picture visualizing the process:

You might want to open this in another tab - High-Resolution

It is worth noting that the ConditionVariable#signal method will only wake up one thread which is waiting for the variable. This means that if we have two threads waiting on a ConditionVariable and its signal method is called only once, the thread that does not get called will end up waiting forever for the ConditionVariable, resulting in a deadlock

require 'thwait'

lock = Mutex.new
cv = ConditionVariable.new
threads = []
2.times do
  threads << Thread.new {
    lock.synchronize { cv.wait(lock) }
  }
end

threads << Thread.new {
  lock.synchronize { sleep 1 }
  cv.signal
}

ThWait.all_waits(*threads)

> enether$ ruby cv_pitfall.rb
/Users/enether/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/thwait.rb:112:in `pop': No live threads left. Deadlock? (fatal)

In such a scenario, you need to either call signal as many times as there are waits or use another method - ConditionVariable#broadcast, which will wake up every thread that is waiting on the condition variable.

Producer-Consumer

The producer-consumer problem consists of at minimum two threads, one representing a producer and one representing a consumer.

Producer - Sole job is to create an item and put it into the buffer
Consumer - Sole job is to take the item from the buffer and process it

Here is a sample implementation in Ruby, where the tasks array acts as the buffer with a fictional limitation of having at most 2 items in it at once:

require 'thwait'

threads = []
tasks = []
mutex = Mutex.new

# producer
2.times do
  threads << Thread.new do
    loop do
      mutex.synchronize do
        if tasks.length < 2
          tasks << "Task :)"
        end
      end
    end
  end
end


# consumer
5.times do
  threads << Thread.new do
    loop do
      task = nil
      mutex.synchronize do
        if tasks.length != 0
          task = tasks.shift
        end
      end
      unless task.nil?
        100000.times do
          # Simulating task execution's CPU work
          # also doing it outside the mutex so we don't block the tasks array (other producer might want to take a task as well)
        end
      end
    end
  end
end

ThWait.all_waits(threads)

Producer-Consumer problem

As the name implies, something is not quite right with the code above. Finding problems in concurrent code is hard, so I'm going to give you a couple of minutes to figure out what is wrong.

...
...
...

Okay, if you managed to figure it out - great, if not - don't fret, multithreaded programming is unintuitive.
The problem with the code above is that it wastes time. You see, as we can't control when and to which thread the OS switches to, there is the possibility that we leave our consumer thread and enter the producer's when there is no reason to.
Imagine all our consumer threads are currently executing a task and our tasks array is full (has two elements). If the OS decides to do a context switch and gives control to a producer thread it would only waste time. Since the tasks array would be full, the producer would loop only to check that the array is full and not do anything else, only managing to take precious CPU time from our consumer threads.

Let's track exactly how much useless iterations this thing does:

require 'thwait'

threads = []
tasks = []
mutex = Mutex.new

to_exit = false
times_tasks_added = 0
times_time_wasted = 0
executed_tasks = 0

# consumer
2.times do
  threads << Thread.new do
    loop do
      Thread.kill(Thread.current) if to_exit  # a way to stop execution

      mutex.synchronize do
        if tasks.length < 2
          tasks << "Task :)"
          times_tasks_added += 1
        else
          # time here is absolutely wasted
          times_time_wasted += 1
        end
      end
    end
  end
end

# producer
5.times do
  threads << Thread.new do
    loop do
      Thread.kill(Thread.current) if to_exit  # a way to stop execution
      task = nil
      mutex.synchronize do
        if tasks.length != 0
          task = tasks.shift
        end
      end
      unless task.nil?
        100000.times do
          # Simulating CPU work
          # also doing it outside the mutex so we don't block the tasks array (other producer might want to take a task as well)
        end
        executed_tasks += 1
        if executed_tasks >= 100  # don't loop forever
          to_exit = true
        end
      end
    end
  end
end

ThWait.all_waits(threads)

puts "Total tasks added: #{times_tasks_added}"
puts "Total times we branched out into the useless else statement: #{times_time_wasted}"

And here are the results:

> enether$ ruby squander_of_time.rb
Total tasks added: 102
Total times we branched out into the useless else statement: 1633
> enether$ ruby squander_of_time.rb
Total tasks added: 102
Total times we branched out into the useless else statement: 848282
> enether$ ruby squander_of_time.rb
Total tasks added: 102
Total times we branched out into the useless else statement: 356418

We know that thread context-switching is non-deterministic and these results further prove it. Sometimes we do as little as 1633 useless executions of the else branch and some - as much as 848k, 8316 times more than we need!
This is bad because it will cause your program to run slower at some times and behave seemingly normal in others.
It's good to note that I personally found it hard to figure out the problem in this code even though it's specifically made to illustrate it. Imagine how hard it would be to spot such a thing in an established codebase!

This slow-down is not acceptable, so let's fix it. Thinking the problem through, our problem seems to boil down to having a producer thread wake up when it doesn't make sense.
What would be perfect is if we had the ability to somehow control when we resume the producer thread - specifically when a task is removed from the tasks buffer so we're sure there's room to add another one.

ConditionVariable to the rescue!

The fix is simple: We're just going to put a condition variable which is going to give up the mutex lock whenever we detect that we do not need to continue looping in the producer. We're also going to need to tell the producer he can resume when we have an empty spot in the tasks buffer.

require 'thwait'

threads = []
tasks = []
mutex = Mutex.new

to_exit = false
times_tasks_added = 0
times_time_wasted = 0
executed_tasks = 0
cv = ConditionVariable.new

# consumer
2.times do
  threads << Thread.new do
    loop do
      Thread.kill(Thread.current) if to_exit  # a way to stop execution

      mutex.synchronize do
        if tasks.length < 2
          tasks << "Task :)"
          times_tasks_added += 1
        else
          times_time_wasted += 1
          cv.wait(mutex)  # no need to continue looping in such a case, only continue after it makes sense
        end
      end
    end
  end
end

# producer
5.times do
  threads << Thread.new do
    loop do
      Thread.kill(Thread.current) if to_exit  # a way to stop execution
      task = nil
      mutex.synchronize do
        if tasks.length != 0
          task = tasks.shift
          cv.signal  # one new task can now be added
        end
      end
      unless task.nil?
        100000.times do
          # Simulating CPU work
          # also doing it outside the mutex so we don't block te tasks array (other producer might want to take a task as well)
        end
        executed_tasks += 1
        if executed_tasks >= 100  # don't loop forever
          to_exit = true
        end
      end
    end
  end
end

ThWait.all_waits(threads)

puts "Total tasks added: #{times_tasks_added}"
puts "Total times we branched out into the useless else statement: #{times_time_wasted}"

> enether$ ruby saver_of_time.rb
Total tasks added: 101
Total times we branched out into the useless else statement: 50
> enether$ ruby saver_of_time.rb
Total tasks added: 100
Total times we branched out into the useless else statement: 45
> enether$ ruby saver_of_time.rb
Total tasks added: 100
Total times we branched out into the useless else statement: 42

Woo, performance!

In reality, it's worth noting that the previous example and this one actually run in about the same time. 400k useless iterations sound like a lot but our computers are fast enough to not let us notice this inefficiency. Regardless, I hope this example managed to clearly illustrate the problem.

Further optimization

Do we even need to enter the else branch at all? Could we not put cv.wait inside the block which adds the tasks and have it call wait when the buffer is full? We can and that way it should never enter the else block, as it would only be resumed to add a task and sleep if the buffer is full.

# ...
loop do
  Thread.kill(Thread.current) if to_exit  # a way to stop execution

  mutex.synchronize do
    if tasks.length < 2
      tasks << "Task :)"
      times_tasks_added += 1
      if tasks.length >= 2
        cv.wait(mutex)  # no need to continue looping in such a case, only continue after it makes sense
      end
    else
      times_time_wasted += 1
    end
  end
end
# ...

> enether$ ruby no_time_wasted.rb
Total tasks added: 100.
Total times we branched out into the useless else statement: 16
> enether$ ruby no_time_wasted.rb
Total tasks added: 102
Total times we branched out into the useless else statement: 13
> enether$ ruby no_time_wasted.rb
Total tasks added: 101
Total times we branched out into the useless else statement: 372021

What the hell? We still got entered the else block and we even got back to the previous levels of needless execution!

Concurrent programming is hard. We're using two producer threads here and when one fills up the tasks array it frees the mutex. The other producer thread seems to get resumed (remember we free the mutex only when tasks is full) and enters the else branch as the tasks buffer is full.

You might want to open this in another tab - High-Resolution

Okay, well the simplest thing to do is put a wait back where we had one. This should limit the useless calls as much as possible:

# ...
loop do
  Thread.kill(Thread.current) if to_exit  # a way to stop execution

  mutex.synchronize do
    if tasks.length < 2
      tasks << "Task :)"
      times_tasks_added += 1
      if tasks.length >= 2
        cv.wait(mutex)
      end
    else
      times_time_wasted += 1
      cv.wait(mutex)
    end
  end
end
# ...

> enether$ ruby no_time_wasted_fixed.rb
Total tasks added: 101
Total times we branched out into the useless else statement: 5
> enether$ ruby no_time_wasted_fixed.rb
Total tasks added: 101
Total times we branched out into the useless else statement: 8
> enether$ ruby no_time_wasted_fixed.rb
Total tasks added: 101
Total times we branched out into the useless else statement: 3

Well I'm afraid this is as much as we can do with the ConditionVariable. The reason we still enter the else block a couple of times is most likely the one depicted in the image above.
Although there is one other possibility:

Spurious Wakeups

A spurious wakeup is when a ConditionVariable gets woken up without getting signaled to. This might sound stupid but it makes sense, since it seems to boost performance in some cases. According to David R. Butenhof's Programming with POSIX Threads (ISBN 0-201-63392-2): "Spurious wakeups may sound strange, but on some multiprocessor systems, making condition wakeup completely predictable might substantially slow all condition variable operations.".
They also enforce robust multithreaded code, essentially enforcing you to take care of such cases. This is why it is strongly recommended that you always put your ConditionVariables inside a loop which always checks the appropriate condition (as we do with if tasks.length < 2).
Here is an interesting discussion on the topic: comp.programming.threads

I personally could not identify when a producer thread was woken up spuriously or simply got scheduled when a previous producer went to sleep. I did dig through the MRI code to verify that cv.wait is vulnerable to spurious wakeups.
Here is the way it gets called - rb_condvar_wait -> do_sleep -> mutex_sleep -> rb_mutex_sleep -> rb_mutex_sleep_forever -> rb_thread_sleep_deadly_allow_spurious_wakeup -> sleep_forever
It seems to boil down to calling the sleep_forever function which calls native_sleep in a loop. After the code exits from the sleep, Ruby checks if the thread was woken up on purpose (in RUBY_VM_CHECK_INTS_BLOCKING(th)) and schedules it to be interrupted if so. Since ours isn't, it likely enters the if (!spurious_check) block and breaks the loop, effectively stopping the sleep.

Summary

We touched on a couple of important topics in multithreaded programming.
We learned about the precious ConditionVariable class and more specifically how it allows you to pause threads at will and schedule a resume when you decide to.

ConditionVariable#wait(mutex) - puts the current thread to sleep, releases the given mutex for the time being and gets resumed strictly after a signal
ConditionVariable#signal - allows one thread that holds the given condition variable that to resume
ConditionVariable#broadcast - allows all threads that hold the given condition variable to resume

We dabbled into the producer-consumer problem, trying to optimize it on our own and further explored concurrency problems and in the end learned about spurious wakeups.

Working with Multithreaded Ruby Part I

Stanislav Kozlovski — Tue, 26 Sep 2017 06:47:26 +0000

Introduction

Multithreaded Ruby is a niche topic in our community and to no surprise. Most Ruby applications are web servers built on Rails or Sinatra, those are single-threaded frameworks and developers on such projects rarely even need to know about threads, as the framework usually has got your back.

Even if you do not use it, some basic knowledge of multithreading (and its basic concepts) in an interpreted language like Ruby will surely come in handy throughout your career.

I assume you know about the GIL (Global Interpreter Lock). In case you don't know what it is, you can read my article Ruby's GIL in a nutshell

GIL != the end of the world

Even though it limits parallelism, Ruby's GIL does not completely stop it. As we know, it exists to guard the interpreter's internal state. As such, it only applies to Ruby operations. In our normal day-to-day code there are a lot of operations that are not the job of Ruby's interpreter to handle.

A good example is I/O operations. While waiting for an external service to load something, there is no need to hold the GIL, as this external service cannot harm our internal state.
Ruby's PostgreSQL library is written in C and its method call for a DB query releases the GIL. The following example shows that:

require 'thwait'
require 'pg'

start = Time.now

first_sleep = Thread.new do
  puts 'Starting sleep 1'
  conn = PG::Connection.open(dbname: 'test')
  conn.exec('SELECT pg_sleep(1);')
  puts 'Finished sleep 1'
end

second_sleep = Thread.new do
  puts 'Starting sleep 2'
  conn = PG::Connection.open(dbname: 'test2')
  conn.exec('SELECT pg_sleep(1);')
  puts 'Finished sleep 2'
end

random = Thread.new do
  puts 'In a random thread'
end

ThWait.all_waits(first_sleep, second_sleep, random)

puts "Time it took: #{Time.now - start}"

Here we spin up two threads, create a connection to different databases and run a sleep query for a second. Without parallelism, this should take at minimum 2 seconds.

> enether$ ruby async_pg.rb
> Starting sleep 2
> Starting sleep 1
> In a random thread
> Finished sleep 2
> Finished sleep 1
> Time it took: 1.074824

But it runs in 1 second!
This proves that the PostgreSQL query does not hold the GIL and lets the other thread take control. Not only does it not lock the interpreter but it actually runs the query in parallel with the other query, that's the only way in which we could achieve a 1 second execution time to run two sleep queries!

Reminder: The GIL does not protect you

A problem can occur when two or more threads access shared data and try to change it. This is called a race condition.
Because Ruby's thread scheduling algorithm can swap between threads at any time, you don't know the order in which the threads will attempt to access the shared data. Therefore, the result of the change in data is dependent on the algorithm and seemingly out of your control.
It is therefore possible for two threads to modify data in such a sequence where you get an unexpected outcome.

Here is an example of the so called check-and-act race condition, where you check for a variable's value and then act in regards to it.

require 'thwait'

def send_money(amount)
  puts "Sending $#{amount}"
  sleep 1  # Simulate network call sending of money PS: This is I/O, so you know Ruby releases GIL here
end


threads = []
money_is_sent = false

2.times do
  th = Thread.new do
    unless money_is_sent
      send_money 10
      money_is_sent = true
    end
  end
  threads << th
end


ThWait.all_waits(*threads)

We obviously want to send the money only once but running the code shows that this is not the case

> enether$ ruby balling.rb
> Sending $10
> Sending $10

This is what happens here

As you saw, what looks like straightforward code can end up producing a huge problem (losing us money!) when executed concurrently. It is up to you to make your code thread-safe.

How to protect yourself

So how could we avoid such race conditions?
Simple, you can take the same approach as the Ruby Core team and introduce your own lock (kind of like the GIL), which would be a local lock on a block of code.
This is called a Mutex (Mutual Exclusion) and it helps you synchronize access to blocks of code, acting like a gatekeeper.

require 'thwait'

def send_money(amount)
  puts "Sending $#{amount}"
  sleep 1  # Simulate network call sending of money
end

lock = Mutex.new
threads = []
money_is_sent = false

2.times do
  th = Thread.new do
    lock.synchronize {
      unless money_is_sent
        send_money 10
        money_is_sent = true
      end  
    }
  end
  threads << th
end


ThWait.all_waits(*threads)

We define a Mutex and call the synchronize method. When we enter the block in the synchronize method, our mutex gets locked. If another thread tries to access code through lock.synchronize it will see that the lock is locked and pause until it is unlocked.

> enether$ ruby balling_on_a_budget.rb
> Sending $10

Be sure to note that lock.synchronize only prevents a thread from being interrupted by others wanting to execute code wrapped inside the same lock variable!
Creating two different locks will obviously not work.

2.times do
  Thread.new do
    Mutex.new.synchronize {
      unless money_is_sent
        send_money 10
        money_is_sent = true
      end
    }
  end
end

> enether$ ruby lock_city.rb
> Sending $10
> Sending $10

yeah, no way

Mutexes are not perfect

Now that we know about these locks, we need to pay attention to how we use them. They offer protection but there is also a possibility where that can backfire on you if not used correctly.
It is possible to end up in a so-called deadlock (sounds scary, doesn't it?). A deadlock is a situation where one thread that holds mutex A waits for a mutex B to be released but the thread that holds mutex B is waiting for mutex A.

require 'thread'
require 'thwait'

first_lock = Mutex.new
second_lock = Mutex.new

a = Thread.new {
  first_lock.synchronize {
    sleep 1  # essentially forces a context switch
    second_lock.synchronize {
      puts 'Locked #1 then #2'
    }
  }
}

b = Thread.new {
  second_lock.synchronize {
    sleep 1  # essentially forces a context switch
    first_lock.synchronize {
      puts 'Locked #2 then #1'
    }
  }
}

ThWait.all_waits(a, b)

> enether$ ruby dead_lock.rb
> /Users/enether/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/thwait.rb:112:in `pop': No live threads left. Deadlock? (fatal)

They are both holding what the other thread wants and waiting for what the other thread has.
Of course, this is a pretty specific example and there are not many cases in which you might use two mutexes in such a way, but it is essential to know about this pitfall.

Summary

We saw that regardless of the GIL you can still do tasks asynchronously (I/O and native libraries) and confirmed that it won't save you from your thread-unsafe code.
You learned about the most common pitfall - the check-then-act race condition, we introduced a way of handling the problem through our own little GIL-esque lock (Mutex) and we saw that even that can backfire.

I hope I've managed to showcase how tricky multithreaded programming can turn out to be and how it can introduce problems you would not consider programming synchronously.

Crystal - the Ruby you've never heard of

Stanislav Kozlovski — Mon, 18 Sep 2017 12:19:29 +0000

What?

Crystal is a new, elegant, multi-paradigm programming language that is productive and fast. It has ~~Ruby's~~ a Ruby-inspired syntax and compiles to native code. It is actually unreal how similar to Ruby this language looks like.
This language combines efficient code with developer productivity, adds full OOP, a great concurrency model and a compiler that holds your hand.

This article is meant to give you a short overview, a direct performance comparison to Ruby and show some things that set it apart. It is advised you know at least some Ruby before continuing on reading.

Starting with the fun stuff - a performant example

Let's actually get a feel as to how performant Crystal is.
I wrote an AA Tree in both Crystal and Ruby.
Note: Code quality might not be top-notch. Some lines of Crystal code are intentionally written more explicitly

We are going to be running this code to benchmark each implementation:

elements_count = ARGV[0].to_i  # first command line argument
root = AANode.new(value: elements_count, level: 1)
tree = AATree.new(root)

start = Time.now

elements_count.times do |num|
  raise Exception.new("Tree should not contain #{num}") if tree.contains?(num)
  tree.add(num)
  raise Exception.new("Tree should contain #{num}") unless tree.contains?(num)
end

elements_count.times do |num|
  raise Exception.new("Tree should contain #{num}") unless tree.contains?(num)
  tree.remove(num)
  raise Exception.new("Tree should not contain #{num}") if tree.contains?(num)
end

puts "Time it took: #{Time.now - start} seconds."

What this essentially does is it adds numbers to our tree (which sorts them internally) and then removes each one, one by one. We also check if the tree contains the given number twice per addition/deletion.

The code snippet above is actually Crystal code.
Like I said, these languages are identical at first glance. Rewriting the code from Crystal to Ruby took me a total of 50 line changes for a 360 line file. 27 if you were greedy
It is worth noting that those changes are simply removing .as() method calls and type annotations.

Okay, they look identical but how much faster is Crystal?

Let's build the executable and start testing

> enether$ crystal build AA_Tree.cr -o crystal_tree --release

# 100 elements
> enether$ ./crystal_tree 100
Time it took: 0.0006560 seconds.
> enether$ ruby AA_Tree.rb 100
Time it took: 0.00172 seconds.

# 10K elements
> enether$ ./crystal_tree 10000
Time it took: 0.0044000 seconds.
> enether$ ruby AA_Tree.rb 10000
Time it took: 0.288619 seconds.

# 100K elements
> enether$ ./crystal_tree 100000
Time it took: 0.0498230 seconds.
> enether$ ruby AA_Tree.rb 100000
Time it took: 3.414404 seconds.

# 1 million elements
> enether$ ./crystal_tree 1000000
Time it took: 0.5007820 seconds.
> enether$ ruby AA_Tree.rb 1000000
Time it took: 39.370083 seconds.

# 10 million elements
> enether$ ./crystal_tree 100000000
Time it took: 5.6283920 seconds.
> enether$ ruby AA_Tree.rb 100000000
# Still running

As you can see, it runs laps around Ruby and proves to be ~80 times faster if we were to judge by our 1 million elements example.

Quirks and differences to Ruby

Despite the similarities, there are substantial differences to Ruby, here we will highlight the most obvious and interesting ones.

Types, type checking and type unions

The most apparent difference is that Crystal uses and mostly enforces types for variables. It has great type inference - if you do not explicitly define the type of a variable the compiler figures it out itself.
The way this language does typing is a sort of mix between static and dynamic typing. It allows you to change a variable's type

a = "Hello"
puts typeof(a) # => String
a = 42
puts typeof(a) # => Int32

but it also allows you to enforce a variable's type

a : String = "Hello"  # a should be a string and only a string!
a = 42 # error:  type must be String, not (Int32 | String)

Type Unions

Were you wondering what the (Int32 | String) type was in the error message above?
This is a so-called type union, which is a set of multiple types.
If we were to enforce a to be a union of Int32 and String, the compiler would allow us to assign either type to that variable as it knows to expect both.

a : (Int32 | String) = 42
a = "Hello"
# Completely okay

# But if we were to try to assign another type to it
a = true # => type must be (Int32 | String), not (Bool | Int32 | String)

Type Inference and Type Checking

The compiler can figure out the type of a variable himself in most cases. The type inference algorithm is specifically built to work when the type of the variable is obvious to a human reader and does not dig too deep into figuring out the specific type.

In the cases where multiple conditions are plausible, the compiler puts a union type on the variable. Crystal code won't compile if the possible types do not support a given method invoked on them.

if rand() > 0.5
  a = "String"
  puts typeof(a) # => String
else
  a = 42
  puts typeof(a) # => Int32
end
puts typeof(a) # => (String | Int32)
puts a.camelcase  # => undefined method 'camelcase' for Int32 (compile-time type is (Int32 | String))

This is the way the compiler protects you from silly mistakes with mismatched types, something that is really common in dynamic languages. Its like having your very own programming assistant!

The compiler is smart enough to figure out when a variable is obviously from a given type

if rand() > 0.5
  a = "String"
elsif rand() > 0.75
  a = 42
else
  a = nil
end
puts typeof(a) # => (String | Int32 | Nil)
unless a.nil?
  # a is not nil for sure
  puts typeof(a)  # => (String | Int32)
end
if a.is_a?(String)
  puts typeof(a)  # => String
end

There are ways to ensure the compiler that the appropriate type is set.

puts a.as(String).camelcase

This checks that the a variable is a string and if it is not, it throws an error.

Enforcing types

As we said, we have the option to enforce a variable's type or let it be whatever.
This holds true for a method's parameters as well. Let's define two methods:

def generic_receiver(item)
  puts "Received #{item}!"
end

def string_receiver(item : String)
  puts "Received string #{item}!"
end

I assume you can already imagine what'll happen with the following code:

generic_receiver(1)
generic_receiver("Hello")
generic_receiver(1.5)
generic_receiver(true)
string_receiver("Hello")
string_receiver(1)  # error!

It is usually good practice to not enforce a variable, as it leads to more generic code.

Concurrency

Its concurrent model is inspired by that of Go, namely CSP (Communication Sequential Processing).
It uses lightweight threads (called fibers) whose execution is managed by the runtime scheduler, not the operating system. Communication between said threads is done through channels, which can either be unbuffered or buffered.

Pictured: A lot of fibers who communicate between each other through channels

Crystal currently runs in a single thread but their roadmap intends to implement multithreading. This means that it currently has no support for parallelism (except for process forking), but that is subject to change.
Because at this moment there's only a single thread executing your code, accessing and modifying a variable in different fibers will work just fine. However, once multiple threads is introduced in the language, it might break. That's why the recommended mechanism to communicate data is through channels.

Metaprogramming

Crystal has good support for metaprogramming through macros. A macro is something that pastes code into the file during compilation.

Let's define our own version of Ruby's attr_writer

macro attr_writer(name, type)
    def {{name}}=({{name.id}} : {{type}})
        @{{name}} = {{name.id}}
    end
end

Calling attr_writer foo, Int32 will evaluate to

def foo(foo : Int32)
 @foo = foo
end

class Greeter
    attr_writer hello_msg, String

    def hello_msg
        @hello_msg
    end
end


gr = Greeter.new
gr.hello_msg = "Hello World"
puts gr.hello_msg # => Hello World
gr.hello_msg = 11 # => no overload matches 'Greeter#hello_msg=' with type Int32

Crystal macros support iteration and conditionals and can access constants.

MAX_LENGTH = 3
macro define_short_methods(names)
  {% for name, index in names %}
    {% if name.id.size <= MAX_LENGTH %}
        def {{name.id}}
          {{index}}
        end
    {% end %}
  {% end %}
end

define_short_methods [foo, bar, hello]
puts foo # => 0
puts bar # => 1
# puts hello => undefined local variable or method 'hello'

Miscellaneous

Crystal has taken a lot of cool features from other languages and provides various syntax sugar that is oh so sweet!

Initializing class instance variables directly in a method

def initialize(@name, @age, @gender, @nationality)

is equal to

def initialize(name, age, gender, nationality)
  @name = name
  @age = age
  @gender = gender
  @nationality = nationality
end

Implicit object notation

Switch statements support invoking methods on the giving object without repeatedly specifying its name.

case string_of_the_gods
when .size > 10
  puts "Long string"
when .size == 5
  puts "Normal String"
when .size < 5
  puts "Short String"
end

case {1, 1}
when {.even?, .odd?}
  # Matches if value1.even? && value2.odd?
end

External keyword arguments

My personal favorite - Crystal allows you to name a function's parameters one way for the outside world and one way for the method's body

def increment(number, by value)
  number + value
end
increment(10, by: 10)

Compiler

As you saw earlier, this language is compiled to an executable. Regardless, it still has something like a REPL which proves to be similar to our beloved irb - https://github.com/crystal-community/icr
You can also directly run a file without having to compile it and then run it, via the crystal command.

> enether$ crystal AA_Tree.cr 200000
Time it took: 0.536102 seconds.

This runs a little bit slower because we do not make use of the optimizations that the --release build flag brings with itself.

C Bindings

There is a way to write a performant library in Crystal which you can run in your Ruby code. The way you do this is to bind Crystal to C, which allows you to use it from Ruby.
I did not delve too deep into this but apparently it is easy and you can do it without writing a single line of C. That is awesome!

Conclusion

If you write Ruby, picking up Crystal is natural and can quickly find yourself writing performance-critical software in it. I believe it has a lot of potential and can yield a lot of benefits to our community but also to non-ruby programmers, as the syntax is just too easy to pass up. It is a joy to write and it runs blazingly fast, that is an unique combination which very few languages can boast with.
I hope I've sparked your interest by these short examples! I strongly encourage you to take a look at the language for yourself and notify me if I've missed something.

Here are some resources to read further up on:
Google Group
Gitter Chat
IRC
Subreddit
Newsletter

Ruby's GIL in a nutshell

Stanislav Kozlovski — Mon, 11 Sep 2017 07:12:20 +0000

Liquid syntax error: Unknown tag 'endcomment'

Managing RESTful URLs in Django Rest Framework

Stanislav Kozlovski — Mon, 14 Aug 2017 05:37:47 +0000

We've all been taught about RESTful API design. It does not take much to realize that these endpoints

POST /products/1/delete
POST /products/1/update
GET /products/1

are inferior to

DELETE /products/1
PUT /products/1
GET /products/1

You could also imagine that these multiple URLs per object stack up quickly and if we were to introduce more objects like /merchants/, /shops/ we'd quickly be managing a lot of URLs which can get confusing. Nobody wants to read a 100-line urls.py file.

But Django Rest Framework does not support mapping the same URL to different class-views based on the request method. How could we map this one URL with different methods in our urls.py file?

Let's first create the initial project with our bad URLs

Initial Project

We'll have a dead simple project which allows us to interact with product objects. We want to be able to update a product, get information about it and delete it.

models.py

from django.db import models
from rest_framework import serializers


class Product(models.Model):
    name = models.CharField(max_length=500)
    price = models.DecimalField(decimal_places=2, max_digits=5)
    stock = models.IntegerField()


class ProductSerializer(serializers.ModelSerializer):
    class Meta:
        model = Product
        fields = '__all__'

views.py

from rest_framework.generics import DestroyAPIView, UpdateAPIView, RetrieveAPIView

from restful_example.models import Product, ProductSerializer


class ProductDestroyView(DestroyAPIView):
    queryset = Product.objects.all()
    serializer_class = ProductSerializer


class ProductUpdateView(UpdateAPIView):
    queryset = Product.objects.all()
    serializer_class = ProductSerializer


class ProductDetailsView(RetrieveAPIView):
    queryset = Product.objects.all()
    serializer_class = ProductSerializer

and our ugly urls.py

from django.conf.urls import url
from django.contrib import admin
from restful_example.views import ProductDestroyView, ProductUpdateView, ProductDetailsView

urlpatterns = [
    url(r'^admin/', admin.site.urls),
    url(r'^products/(?P<pk>\d+)/delete$', ProductDestroyView.as_view()),
    url(r'^products/(?P<pk>\d+)/update$', ProductUpdateView.as_view()),
    url(r'^products/(?P<pk>\d+)$', ProductDetailsView.as_view()),
]

Wait, don't forget tests!

tests.py

from rest_framework.test import APITestCase

from restful_example.models import Product, ProductSerializer


class ProductTests(APITestCase):
    def test_can_get_product_details(self):
        product = Product.objects.create(name='Apple Watch', price=500, stock=3)
        response = self.client.get(f'/products/{product.id}')
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.data, ProductSerializer(instance=product).data)

    def test_can_delete_product(self):
        product = Product.objects.create(name='Apple Watch', price=500, stock=3)
        response = self.client.delete(f'/products/{product.id}/delete')
        self.assertEqual(response.status_code, 204)
        self.assertEqual(Product.objects.count(), 0)

    def test_can_update_product(self):
        product = Product.objects.create(name='Apple Watch', price=500, stock=3)
        response = self.client.patch(f'/products/{product.id}/update', data={'name': 'Samsung Watch'})
        product.refresh_from_db()
        self.assertEqual(response.status_code, 200)
        self.assertEqual(product.name, 'Samsung Watch')

Run our tests and you'd see that this works.

Becoming more RESTful

Okay, now it's time to fix our URLs into more sensible ones. As we said, we want all of these views to point to one exact URL and differ only by the method they allow.

The idea here is to have some sort of base view to which the request on the url gets sent to. This view will have the job to figure out which view should handle the request given its method and send it there.

class BaseManageView(APIView):
    """
    The base class for ManageViews
        A ManageView is a view which is used to dispatch the requests to the appropriate views
        This is done so that we can use one URL with different methods (GET, PUT, etc)
    """
    def dispatch(self, request, *args, **kwargs):
        if not hasattr(self, 'VIEWS_BY_METHOD'):
            raise Exception('VIEWS_BY_METHOD static dictionary variable must be defined on a ManageView class!')
        if request.method in self.VIEWS_BY_METHOD:
            return self.VIEWS_BY_METHOD[request.method]()(request, *args, **kwargs)

        return Response(status=405)

This simple class requires us to inherit it and define a class variable named VIEWS_BY_METHOD. A dictionary which will hold our method names and their appropriate handlers.
Using this base class, creating the ManageView class for our Product model is trivial:

class ProductManageView(BaseManageView):
    VIEWS_BY_METHOD = {
        'DELETE': ProductDestroyView.as_view,
        'GET': ProductDetailsView.as_view,
        'PUT': ProductUpdateView.as_view,
        'PATCH': ProductUpdateView.as_view
    }

It is worth mentioning here that any permission classes must be defined in the separate views and will not work if put in the ManageView

We need to quickly edit our urls.py's urlpatterns

urlpatterns = [
    url(r'^admin/', admin.site.urls),
    url(r'^products/(?P<pk>\d+)$', ProductManageView.as_view()),  # this now points to the manage view
]

Let's test this new view as well
tests.py

class ProductManageViewTests(APITestCase):
    def test_method_pairing(self):
        self.assertEqual(len(ProductManageView.VIEWS_BY_METHOD.keys()), 4)  # we only support 4 methods
        self.assertEqual(ProductManageView.VIEWS_BY_METHOD['DELETE'], ProductDestroyView.as_view)
        self.assertEqual(ProductManageView.VIEWS_BY_METHOD['GET'], ProductDetailsView.as_view)
        self.assertEqual(ProductManageView.VIEWS_BY_METHOD['PUT'], ProductUpdateView.as_view)
        self.assertEqual(ProductManageView.VIEWS_BY_METHOD['PATCH'], ProductUpdateView.as_view)

    def test_non_supported_method_returns_405(self):
        product = Product.objects.create(name='Apple Watch', price=500, stock=3)
        response = self.client.post(f'/products/{product.id}')
        self.assertEqual(response.status_code, 405)

We change the previous tests' urls to use the new one and we can see that they pass

Creating test database for alias 'default'...
...
----------------------------------------------------------------------
Ran 3 tests in 0.037s

OK
Destroying test database for alias 'default'...

And voila, we have the same functionality but in one url!

Summary

What we did was create a main view which dispatches requests to the appropriate views given the request method. I believe that this is the right way to handle multiple methods per URL in DRF when you want to have different class-based views handling each method.

This, however, is not the optimal case with our dead simple example. For views which are somewhat to extremely generic (like ours, no custom logic inside) or function views, there are Routers and ViewSets

Bonus - Routers and ViewSets

Since DRF is awesome, they provide us with the ability to define all the basic operations on a model in just a few lines.
With a ViewSet class, we can define our create, read, update and destroy logic without writing any code.

views.py

from rest_framework import viewsets
class ProductViewSet(viewsets.ModelViewSet):
    queryset = Product.objects.all()
    serializer_class = ProductSerializer

and we need to define the URL in our urls.py using a Router class

from rest_framework.routers import DefaultRouter
router = DefaultRouter(trailing_slash=False)
router.register(r'products', restful_views.ProductViewSet)

urlpatterns = [
    url(r'^admin/', admin.site.urls),
]

urlpatterns += router.urls

This viewset, believe it or not, has all the functionality we implemented above plus some additional like creating a Product(POST /products) or getting a list of all Products(GET /products)

ViewSets also allow you to override the views and create custom ones. This allows us to create function views for one URL but not class-based views.
They are absolutely amazing for our example and other simple projects, but sub-par for managing multiple class-based non-generic views with custom logic inside them.

From Zero to Hero (How I became a professional developer in a year)

Stanislav Kozlovski — Tue, 25 Jul 2017 05:39:50 +0000

I had always thought that I would have a hard time finding a job as a programmer. When starting out, I was easily stifled by various problems. Thoughts that this was not the right thing for me were frequent. I kept going though, as I did not have a better alternative.

Graduating from high school, I decided to ditch university in favor of coding bootcamps. This was just at a time when they were gaining popularity here in Bulgaria and they gave me hope I would be able to land a job quicker than going through the 4 years of university.
I had a couple of key reasons for not enrolling in an university

I did not want to waste 4 years for attaining something that could have taken less. Yes, of course those 4 years are not 'wasted' and everything learnt there sooner or later pays off in a direct or indirect(as the case with math) way.
I did not want to be a financial burden on my parents for 4 more years. I knew that I could not sufficiently learn if I had to work simultaneously, so getting a job was out of the question.
I was unsure if this was the right thing for me and I was scared of studying for years just to figure out I didn’t like it once I got a taste of the professional world

The Beginning

So I ventured out into the world of coding bootcamps, at first online, learning the very basics of programming. For this, I want to give a shout out to Software University (www.softuni.bg). Their introductory courses proved invaluable for a beginner like me. Two hours for a lecture on Arrays, live coding and numerous challenges on the material might seem like overkill now, but they proved to be immensely useful at the time. SoftUni has world-class introductory programming lessons and it’s a real shame that there are no english lectures, as the world would greatly benefit from them. Take it from somebody who tried to learn programming through other courses/tutorials, I have not been able to find an equally good beginner programming course online. Out of the others that I have seen, most go through the material too quickly and do not give the needed problems for the student to solve in order to fully grasp the material. Some do not even have homework!

Regardless, a couple of introductory courses are simply not enough. I quickly realized that one bootcamp was not going to cut it and at first decided that building up a portfolio to go alongside those certifications was the thing to do.
I started simple, building a RPG console game in Python. Of course, this was a dead-simple application but I did grow it to around 5k lines of code, which, at the time, were a lot for me.
This still was not enough and I ventured out to other courses (some online, some in-person). My workload grew and I found myself consistently spending 10+ hours to stay on top of everything. It felt good though, as I could actually see the progress.

We were now close to the new year and I had a team project to do on a course. Working with a team was not easy, it was especially hard to divide the work. Knowing me, I did most of it just because I wanted to(you learn by doing, after all) and nobody else had anything against that. This is where we created a really silly Facebook clone for pets (https://github.com/codeBusters-softuni/petbook)

This was cute

At this point, I felt reasonably confident in my skills and thought that landing an internship sometime in the new year was the next best step. It was time to surround myself with professionals if I wanted to continue growing.No offense, but most people in coding bootcamps are not that good. You can't blame the bootcamp though, it's just that this format attracts people who want the 'easy' way to get into an industry with high salaries, but I digress.
I decided that in order to get in a good company, I'd need to know a fair amount of Data Structures and Algorithms. No problem, there were plenty of resources on the topic.
Well, this material turned out to be the toughest yet. Suddenly, I was back to the feeling that maybe I’m not cut out for this. This all culminated when it took me two-three days to implement a Red-Black tree (the classical kind, none of these left-leaning garbage variants). This was around Christmas too and as such I feel that an obligatory xkcd is in place here:

On the upside, finally grasping a structure/algorithm was rewarding enough to keep me going.

Very quickly, me and a friend got addicted to the direct problem solving aspect of algorithms and started competing in online programming competitions. Obviously, we did not score that well, but it was fun and it incentivized further learning. I felt I made significant progress in my algorithmic thinking here and I encourage everybody to give competitive programming a go.
For the following months (December-March) I spent a fair amount of time solving problems, writing out data structures and learning algorithms.
In retrospect, I'm very glad I spent some time on those topics as they built resiliency, logical thinking and confidence in my skills. I actually continue to take online courses on algorithms (I recommend this one) and I will get back into competitive programming some day.

My 'trophies'

Inspired by the platform for algorithmic problems and competitions Hackerrank, I wanted to make something like that but with features I felt were missing. In February I started my third project - Deadline. This was initially planned as a hackathon project and while it won us tickets to the 2017 WebIt festival both me and my partner knew we would continue to develop this (and we still do actually, track its progress here: deadline)

Time to work

We were now around April and it was time to find a job. I felt confident I could land any internship, as I thought the main thing companies look for in an intern is their motivation and potential. I had plenty of motivation and around 8 months of non-stop Github activity to prove it.
From December I had wanted to take a shot at Uber even though it seemed infeasible (a year of bootcamps/courses and getting to work at a top technological company where some of the world’s most elite engineers are? yeah, right).
Well, against all odds, I did get an invite to an interview. I was ecstatic and had fun throughout the whole process. After it, I thought I had a fair chance of getting in, as I did my best to show my motivation, personality and ambitions. The technical questions were not hard at all and I felt I did well on them.
I did not get chosen. Not a big surprise, albeit disappointing nonetheless, I'm glad I strived for it, as it helped me become a better engineer.

As the things with Uber were over, it was now time to continue my search. I wanted to get in a company where the problems were interesting and I would find maximum growth as an engineer. My next choice was VMware, but their recruiting process was insanely slow and I decided I had enough of waiting. I started to apply to other companies and landed a position as a Junior Ruby Developer at SumUp. I had no previous experience with Ruby, but I made sure to quickly learn the basics and at this point I feel more than comfortable writing it.

Here we develop our own product and that is something I value in a company, I did not want to work in an outsourcing company.
I love my current job, the colleagues are great and even though I did not get to work at the place I initially wanted, I'm glad things turned out this way.
I look forward to a long career with them, trying to make the biggest impact I can. They trusted me and I will make sure to pay that off.

Conclusion

In the end, I feel I am better off this way due to my decision to skip university for the time being. I can now afford to enroll in uni due to desire and curiosity in the field, not out of seeming necessity. And if I do enroll, which I very well might, I will have a clearer vision on the material taught there and I will appreciate it more, as I would be able to more easily see its applications.

As a moral of the story, I want to emphasize that you do not have to follow the beaten path to become successful in any endeavor and sometimes that can even be the better choice.
Know that everything is possible with sufficient determination and work.

Hi, I'm Stanislav Kozlovski

Stanislav Kozlovski — Sun, 23 Jul 2017 13:07:39 +0000

I have been coding for 1.5 years.

You can find me on GitHub as Enether

I live in Sofia, Bulgaria.

I work for SumUp

I mostly program in these languages: Ruby, Python, Javascript.

I am currently learning more about everything.

Nice to meet you.