DEV Community: Marko Kruljac

The cost of abstraction

Marko Kruljac — Tue, 15 Dec 2020 10:14:12 +0000

Credits to the image: Dave Wolanski

Yesterday I was driving down the highway and it was very foggy. All along the highway, there were digital (variable) speed limit signs, which were showing a speed limit of 60km/h (instead of the default 130km/h) due to the bad road conditions.

And I thought to myself, how cool. The constructor of the highway decoupled the actual speed limit from the sign, just for cases like this. Now they have the flexibility to dynamically adjust the limit, based on the road condition. That was a very smart design choice!

The traditional solution would be to just place a metal sign, but this hardcodes the speed limit and cannot be easily changed (someone has to go to the sign, and place a new one in it's place, revert the change when conditions reset, etc.).

Does this mean all signs everywhere should be digital? Probably not. A digital sign can easily cost two orders of magnitude more than just a metal plate sign. And not to speak of the operating and maintenance cost.

Every abstraction comes with its cost.

The point of my post is to show that the same applies to software architecture. It would be great to have "digital signs" everywhere, decouple all of the things and have infinite flexibility. As much as it would be great to have a new Tesla with full equipment and all features unlocked. But do you have the budget to pay for the cost?

As much as we balance our budget when buying a new car, we as engineers should balance our budget for abstractions.

Knowing when and where in your product to invest in abstraction, like the digital road signs, and where to just place a metal hardcoded one can mean the difference between success and failure.

Where things need to change, invest in abstraction and layers. Where things are simple and static, go with the low cost solution. And if the simple thing starts getting complicated and changes frequently, THEN invest your budget in refactoring and abstraction. Your abstraction must provide more value than its cost, or else you are in trouble.

I would really like to hear your thoughts on this. How do you approach complexity and abstraction? Do you consider the cost of these abstractions?

Reading/listening material:

PHP session quirks

Marko Kruljac — Mon, 16 Nov 2020 07:21:00 +0000

Hello there, fellow developer!

Did you know that PHP Sessions are blocking on a single server instance, but vulnerable to race conditions bugs on multi-server architecture?

Here are the important things you should know about how sessions work in PHP.

First thing you should know is how sessions are stored.
The default session save handler is called “files”, which just saves all the session data in a file. The file is conveniently named exactly like the value of the PHPSESSID, which is how the server knows where is your session data and if your session even exists on the server and how to retrieve the session data.

Second thing you should know is that “files” session handler is blocking by design, and there is no way to disable this constraint on this session handler. What this means is that every time the server tries to open your session file, it locks the file (using flock) which prevents any other processes from opening the file – until the lock has been lifted, which happens automatically after the PHP script/request has finished. This is actually a great technique to prevent race-conditions. You can imagine the following snippet of code.

<?php

if ($_SESSION['received_payment'] === false) {
  $_SESSION['received_payment] = true;
  sendMoney();
}

Running this code in parallel, and without locks could result in the sendMoney() being called multiple times! This is a race condition which is solved by locks. Remember, while PHP is single-threaded, you can achieve concurrency by running multiple processes in parallel, Apache or Nginx does this for you. The same trick is used by pm2 to parallelise node processes.

So there is no problem, right? Wrooong 🙂

The problem is that this pattern scales poorly with regards to the total time required to completely process all requests it received in parallel. The requests themselves are received in parallel, but due to the locking they are executed in sequence. This means that if you have 10 parallel ajax calls to process, and let say that each call takes 500ms to process, you will have to wait a total of 5 seconds until all the ajax requests have been resolved. Even worse is if the first call needs 4 seconds to complete, and the rest 9 call need 100ms each. You will still end up waiting 5 seconds, but you will wait a full 4 seconds before seeing any results!
There is a great demo with which you can fiddle with.

I also made my own experiment, here is with "slow" sessions https://github.com/krukru/php-session-quirks/blob/master/example_1/screenshots/with-sharding/sharding.gif
And here is what happens when sessions get closed as soon as they are opened. https://github.com/krukru/php-session-quirks/blob/master/example_0/screenshots/with-sharding-max-workers.png
There are also some other things to take into consideration, like the browser connection limit, and your web server concurrency settings - but these are beyond the scope of this post.

So how to mitigate this issue?

There are two viable solutions.

The first solution is to close the session as soon as you are finished with reading session data. Sessions are most often used just to determine if the user is logged or a guest. After that point the session is no longer needed (in most cases) and if you close the session early, you are allowing the next request to be processed concurrently.

The second solution is to use the read-only session flag, when you will only be doing “read” operations from the session. Again a good example is checking if the user is a guest or logged in user. Here you are only reading from the session, not writing anything – this has the nice property that there is no possibility for race conditions (since data is not being changed) and there is no need for locks!
This approach has its caveats. Read-only sessions are only supported from php 7, which is an issue for frameworks who wish to support php 5 (looking at you Yii2). Another issue is that major frameworks like Zend, Symfony are slow to support this, see https://github.com/zendframework/zend-session/issues/39 and https://github.com/symfony/symfony/issues/24875

So your best bet is to just close-early and avoid sessions as much as possible 🙂

Remember, this only applies to ajax calls from the same user (the same PHPSESSID) and only if the session is being used (session_start() called anywhere in the script lifecycle)!

Ok, but what about multi-server architecture? Well, now you can no longer use “files” as your session handler, since a session could exist on one server instance, but not on another.

How you approach this issue is by using some shared memory space to manage your sessions, redis and memcached being the strongest candidates for the job.

Redis session handler does not support locks at all, and memcached has started supporting it with various degrees of success (there are bugs https://github.com/php-memcached-dev/php-memcached/issues/310).

This means that you cannot get that sweet sweet race-condition safety you get with “files” session handler. The trivial snippet with “received_payment” session gets very difficult to implement correctly.

The solution for this case, is unfortunately to change your code logic and use either a database for locking or specifically some locking mechanism (like https://symfony.com/doc/current/components/lock.html), and again avoid sessions as much as possible.

How do you approach session management? How do sessions work in a node backend environment? Please share your thought and experiences in the comments below! :)

And happy developing!

Links and resources:

The original post that got me thinking about the issue https://pasztor.at/blog/stop-using-php-sessions/
My attempt at recreating the various experiments https://github.com/krukru/php-session-quirks

Is remote work bad for innovation?

Marko Kruljac — Fri, 04 Sep 2020 08:58:41 +0000

The world’s largest work from home experiment has entered its midgame. So far, it was all about adapting quickly and making the best of it. Individual productivity has increased, there is no more commuting, which saves both time and money, work hours are flexible, and there are other advantages as well.
After all, this is what agile taught us to do, embrace change and adapt quickly to it.
We have won the earlygame, but now I ask question: “What did we lose in the process and are we prepared for the midgame?”

There are many disadvantages to remote work, but I would like to focus on a big one, innovation, or creativity if you will.

I have observed that thoughts and ideas, along with chitchat and casual smalltalk travel very difficulty across a remote medium, in contrast to the office environment where this happens naturally all the time.
In a remote environment, you are usually not going to video-call your colleague just to tell him how you solved X or how Y happened and made you wonder about Z and so on. And texting just doesn’t cut it, we communicate so much non-verbally and explaining thoughts and ideas via text is just cumbersome and time-consuming.

However, it is from this collision of ideas that great innovation and progress is created. It is from these small, seemingly meaningless social interactions that creativity and innovation come from. It is from talking face-to-face to a colleague that you build a relationship and make a friend. Without these things, creativity and team’s cohesion is slowly being sapped away.

As a great counter-example, let's look at the legendary Bell Labs, which fostered 9 Nobel Prizes and 4 Turing Award winners. Bell Labs is home to the transistor, UNIX and C among many many other innovations.
When these people were asked what was so special about this building, the common answer was that it was an open environment packed with smart people who collaborated together and openly exchanged thoughts and ideas. (and had unlimited AT&T funding, which helps, but would not produce the innovations by itself).

I would really like to hear about your experiences, are you noticing the same things at your workplace? How are you addressing these issues? Let's discuss!

Thanks!

A post about delegation, responsibility and accountability

Marko Kruljac — Mon, 31 Aug 2020 13:00:02 +0000

Cover Photo by rob walsh on Unsplash

My sister recently started taking her driving lessons, and I thought that this is an excellent metaphor to explain the topics from the title of this post.

How do driving lessons usually look like?
In the first session, the driving instructor will give you a theoretic overview of the car, its parts and all the other important things you need to know before driving. After that, he will demonstrate this theory in practice. So far in the process, you only absorb and observe.

But very soon after that, the student becomes the one driving the car on the streets. The instructor delegates the driving responsibility to the student. The student is now responsible for following all rules and regulations and to safely drive the car.

The instructor still shares the responsibility with you, as he has his own set of driving pedals in case anything goes really badly wrong, he can reach the wheel, he is watching the road with you, has special rearview-mirrors, etc.
It is your responsibility to drive well, follow instructions, yet the instructor is accountable for your driving actions, in case of an accident.
If you have an accident while driving, it is most likely because the instructor delegated too much responsibility too quickly. He is accountable for your failure.

This is exactly how a healthy team is structured. You have a team leader or manager, who delegates and shares responsibility with his team members. The manager is, like the driving instructor, accountable if anything goes wrong. If a team member fails in their responsibilities, it is most likely because the manager either was not clear with his instructions, delegated too much, too difficult or it could be many many other reasons. If you fail, the manager is accountable for this, and must find a way for you to succeed.

Responsibility is something to be shared, and delegated away. "Delegating is a gift with two recipients". One person receives extra time to do other work, while the other person learns something new and interesting, and grows in the process.

For example, you would never learn to drive a car, if you just watched your instructor do it flawlessly, if he never delegated the responsibility to you. Consequently, your team members cannot grow, if you have your hands on the wheel all the time.

It is only when responsibility is delegated, that a person can grow. This is really important to remember.

Delegating responsibility is the hardest thing in the world to learn and do well, but it is vital, both to the manager’s success and to the growth of members of the team. Be prepared to practice this a lot.

Accountability, on the other hand, is not transferable. It is a burden the leader carries for the team.
Giving praise for everything that goes well, and taking responsibility for everything that goes wrong – is the essence of leadership (as said by Dwight Eisenhower)

If you want to learn more about management topics, there is an excellent series of podcasts which you can find here. They have been producing brilliant content for the last 15 years, and many thanks to them for that!

As you become more and more senior, you will need to pick up on these skills. If you’re going for a engineering manager position, these skills will be your bread and butter, to the point that you will need to forget lots of coding skills. Be prepared to accept that.
If you’re going down the technical senior/lead/architect track, don’t think that you are off the hook. Mastering these skills is still a very important tool in your road to success.

Happy delegating, and keep on growing, both yourself and others.

Problem-solving and the dirty dishes principle of software development

Marko Kruljac — Mon, 06 Jul 2020 08:22:40 +0000

In this story, I would like to explore with you the cost of solving various tasks and problems.

More specifically, let’s explore how the cost of solving a problem grows with respect to the size of the problem.

Some types of problems have a linear cost function. This means that if a problem is doubled, the cost to solve it is also doubled.

An example would be reading a book, or travelling — if it takes you one hour to read fifty pages of a book, then you can conclude that it would take you twice as long to read twice as many pages, or if it takes you one hour to travel a fixed distance, then it will take you twice as long to travel twice the distance (providing that you maintain the same average speed).

However, there are many problems where the cost function is nonlinear. In practice, this means that as the problem increases in size, the solution cost becomes disproportionately higher.

Dirty dishes principle?

Take for example washing dishes. If all you need to wash is one dirty plate right after lunch, it’s no big deal. But as soon as you have a sink full of dirty plates laying around for weeks, the problem becomes much more difficult than simply cleaning every single plate individually (unless you are using a dishwasher, but that is cheating).

All of a sudden, you have less free space in your sink, you need to be extra careful not to damage the rest of the dishes, you will probably need to do much more scrubbing and soaking as the dirt on the plates has hardened, and that in turn will require more soap, bacteria can form, etc, etc.

The goal of cleaning dishes suddenly looks unachievable and the task is not fun for anyone.

A bunch of new problems (which need to be solved, and now have a cost) emerged just by deferring and accumulating the problem and letting it increase in size. These problems did not exist when there was only one dirty plate. These problems would never have showed up had you washed every dish as soon as it became dirty.

I call this the dirty dishes principle and it maps very well to software engineering.

How does it transfer to software development?

Take for example the exponential cost of fixing bugs. If you detect a bug early on, as early as in the inception phase, the cost of fixing it is very low. The bug was not around for enough time to cause any real damage and you already know which part of the code needs fixing — the context is well known and understood, which all contributes to a quick and easy solution.

But if you fail to find the bug, and time passes — the problem becomes harder and more costly to solve. Now all of a sudden, you need to invest time just to figure out what and why is wrong. The bug probably caused some issues with customers, which damaged the product image and will negatively impact future revenue. The original author of the bug is months gone in a different company, documentation is scarce, no one knows how the code works, and you need a hero to save the day.

A bunch of new problems (which need to be solved, and now have a cost) emerged just by deferring and accumulating the problem and letting it increase in size. These problems did not exist when there was only one dirty ~~plate~~ bug. These problems would never showed up had you ~~washed every dish~~ fixed every bug as soon as possible.

This is exactly what happened to the pile of dirty dishes, and the best part is that it could have easily been avoided by investing in QA and in software development best practices. For example, by using a static analysis checker.

Another infamous example that clearly explains this principle is the Broken Windows Theory, which states that leaving small problems unattended will contribute to the overall situation becoming worse and thus more costly to solve. And this applies well to software engineering and highlights the importance of refactoring technical dept and maintaining a high quality bar.

The point is — handle problems on time…

Software engineering is absolutely filled with these costly nonlinear problems. Be it code maintenance, system design, bug fixing, algorithm analysis and many many others. That’s why it’s important to know which problems have a nonlinear solving cost, detect them early and address them as soon as possible.

Do not let your problems accumulate, as the price of solving them could end up costing you more than the sum of its parts.

Happy dishwashing! 🙂