DEV Community: Dmitry Kandalov

Limited WIP — test && commit || revert

Dmitry Kandalov — Sat, 08 May 2021 11:34:13 +0000

Limited work-in-progress doesn't seem to be a very well-known idea in the software development world but there are few practices which we use without realising that they're limited work-in-progress (WIP) techniques. One example is pomodoro which suggests to focus on a task for 25 minutes and then take a break (without checking email or social networks), i.e. pomodoro limits the amount tasks that can be done in parallel. A less obvious example is TDD which makes us focus on the problem taking one step at a time, i.e. TDD limits what we can do at each stage of red-green-refactor cycle (for example, we can't add production code when tests are green). Another not-so-obvious example is continuous integration (CI) which is about regularly combining all changes into a working piece of software, i.e. CI is about limiting merge debt.

Unfortunately, there is no common practice to quantify merge debt and doing CI can a bit more subtle than most CI tools want us to think. In particular, it might seem ok to stay on a branch as long as it can be merged into trunk (aka main/master branch). The problem is that there is no guarantee that pending changes on the branch can be integrated with the pending changes on all other active branches. You can optionally make the problem more interesting by adding code reviews via pull-requests and multiple "main" branches (e.g. dev/uat/prod).

Without suggesting that it's the best, easy or even practical solution, one extreme way of doing CI might be developing on trunk using "test && commit || revert" (TCR). TCR is a workflow inspired by TDD and intended to encourage small changes which are safe to integrate.

The idea

TCR can be summarised with the following three rules:

You can't commit without running tests.
If the tests pass, all changes are automatically committed.
If the tests fail, all changes are automatically reverted.

There is an analogy with some arcade computer games where if you reach a safe point in the game, then the next time character dies, it respawns at the last safe point. Similarly, with TCR if the code is broken (i.e. has any failing tests), you're taken to the last safe point when all the tests were green. Unlike arcade games it's up to you to decide where the safe points are and there are no lives or score in TCR (it might be an interesting idea to explore though).

TCR might seem hard because of the constant pressure to get the code and the tests right on the first attempt. If anything goes wrong (even a typo in assertion), all uncommitted work is deleted. The trick is to find ways of working in tiny increments so that at each step it's nearly impossible to fail and if you do fail, there isn't much to lose anyway (this is quite similar to auto-revert constraint).

The most continuous integration

Because TCR makes us work in small increments, merging code becomes much easier. Small frequent changes are less likely to have conflicts and even when a conflict occurs, it's easier to resolve it or to just drop local changes and do them again. Since TCR only allows to commit on passing tests, all commits should be safe to share with other people. Obviously, frequent pushes will also ensure that you receive updates from everyone else, so it might somewhat like using a collaborative code editor except that it's done via version control. From this point of view, "test && push || revert" (TPR) could be a better name to promote continuous integration at the smallest possible level.

Given that the commits are done automatically, there is a question of when/how to specify commit messages. One straightforward solution might be to edit commit message just before it's done automatically. Or you can try specifying commit message in advance before making changes, e.g. in some environment variable. Although these options are likely to get in the way of TCR flow and with lots of small changes you might discover that writing a commit message after each modification is too tedious. So another approach is to use a "default" commit message for a series of commits and squash them later. Or use a placeholder message and later squash commits into more meaningful chunks with better commit messages.

It's also interesting to challenge the whole idea of writing a commit message for each code modification. Having an explanation of why code has changed is good, but should it really be part of the commit? Maybe there is more value in decoupling incremental changes from the task description and decisions made on a project which often span many commits anyway. And as a nice bonus we'll get a detailed history of code modifications.

TCR with TDD?

Following the basic TCR rules, there is no way to have a failing test without code being reverted. This means that you can't really use TCR with TDD but there are few workarounds. One approach is to bend TCR rules a bit and only revert production code. Another approach is to write negated assertion(s) before implementation and fix it once the implementation is done. Or you can try negating/modifying assertions after a successful commit to confirm that they fail (somewhat like a manual mutation testing).

TCR is perfect for refactoring though. It encourages the refactoring strategy to run tests after each change and go back if anything went wrong without wasting time trying to figure out why refactoring didn't work. After a failed attempt you can do a simpler, smaller or just different refactoring. Trying to analyse failure from a stacktrace or debugging code with TCR is problematic anyway, because the stacktrace will be pointing to the code which doesn't exist anymore, and you can't rerun a failed test with a debugger (although I suppose you could run tests in debugger in the first place).

There is a question of which tests should be run when using TCR. The ideal answer is all tests. In practice, running all tests might be too slow, so a subset of tests which are the most relevant for modification is a more pragmatic choice. Unfortunately, on many code bases even running a single test might be too slow because of build/compilation time. It's not a great experience to spend 2 minutes writing a bit of code and then wait for a minute to see it all reverted. There are two possible conclusions. One is that TCR is not usable on some projects and with certain toolsets. Another is that some projects and toolsets are not very usable themselves. The second conclusion is particularly interesting because it's hard to say that many mainstream toolsets are really focusing on performance. This is especially sad because as software developers we created all these tools ourselves (editors, compilers and testing frameworks). What stops us from focusing on a really good tool performance (in the range of milliseconds so that it's instant from human perception point of view)? Maybe a hanging code editor or one minute compilation time should be socially unacceptable similar to a major security breach or personal data leak. Maybe testing frameworks should be more opinionated and fail unit tests which run for longer than a second.

Try it yourself!

The easiest way to try TCR is by setting up a toy project or a code kata and manually following the rules. Writing a small script (or finding one on the Internet) might be a bit more effort but is worth it to make TCR more interactive and a bit like playing a mini-arcade game. I ended up adding TCR-mode to the Limited WIP plugin for IntelliJ IDEs which already had similar functionality. Either way don't take it too seriously, experiment with rules and make sure to have fun.

Limited WIP — Auto-revert

Dmitry Kandalov — Wed, 05 May 2021 17:02:14 +0000

As software developers we seem to be mostly interested in technology and tools rather than in the human side of the process such as software development workflows. For example, consider TDD — one of the few workflows that made it into the mainstream. These days pretty much every project has some kind of unit testing library imported and everyone has TDD on their CV. However, in practice even when people do write tests, they often do it after writing production code (which contradicts the definition of TDD) or are following red-code-code-debug-green approach instead of small red-green-refactor steps.

The interesting part is failing at red-green-refactor steps by making too many changes. It's easy to focus on the solution, start exploring how it can be expressed in a programming language and end up writing/changing way more code than is necessary to pass existing tests. When we finally add more tests for the implemented functionality, the new code might not work as expected, so we need to debug it, therefore red-code-code-debug-green pattern (notice that the "refactor" step is missing). It's quite a common problem so Adrian Bolboaca came up with baby steps constraint for code retreats which essentially adds a time limit to red-green-refactor steps.

The idea of limiting work-in-progress (WIP) based on time is not specific to TDD and can be used in other contexts as well. Similar to other limited WIP techniques, the benefits are about doing more focused work with fewer mistakes while constantly keeping project in a releasable state. In this blog post I'll describe a generic version of the baby steps constraint which I call "auto-revert".

The idea

As the name suggests, auto-revert is about automatically reverting uncommitted changes after a time interval:

Start a countdown timer, e.g. 5 minutes.
On each commit, restart the timer.
On a timeout, revert all changes.

There is also an implicit rule that you can only commit when the code compiles and doesn't have any failing tests but, hopefully, it's not necessary to mention.

It might seem ridiculous to give up control of when the code is reverted and use a timer instead. It can also be difficult emotionally because of the perceived time pressure to finish the task before timeout and the harsh penalty of code being reverted if it's not done on time. Plus with a low timeout duration (e.g. 5 minutes) it might seem impossible to get any work done. Given the above, it's reasonable to assume that using auto-revert is not going to work.

The trick with is to change your ways of working to make small incremental code modifications (baby steps) so that the code is never too far away from being in a working state which can be committed. Once you get into this flow, auto-revert fades into the background and reverts become very rare. Even when some changes are reverted, it's not a big deal anymore because it's only few minutes of work. You can also think about redoing reverted changes as a micro-codekata. Interestingly, the code might not end up in the same place where it was before the revert, because the process of making the same changes again can give us insights into a better way of doing it.

Another positive effect of auto-revert is better time awareness. It seems that we often underestimate even the most basic tasks (perhaps, because we imagine the size of the change in terms of text editing and downplay the probability of things not going according to the plan). Using a timeout makes duration of tasks more visible. It also helps to avoid distractions because you know that you only need to concentrate for a certain period of time. This is similar to pomodoro technique which suggests to work in 25 minute blocks with regular breaks.

Thought experiments

To explore the idea, let's perform a couple of thought experiments:

Auto-revert with 10 year timeout will be impossible to notice because the changes that were not committed for that long will be forgotten about and will be out-of-date by that time anyway. Considering the probability of hardware failures over 10 years, we are already working with this constraint.
Similarly, with a timeout of months, weeks or several days there will be no effect on the workflow for most people. From this point of view, there is nothing radical about auto-revert.
It becomes more interesting with a timeout measured in hours and minutes. This is the range where the learning opportunities are and auto-revert will be uncomfortable until you change your ways of working.
Finally, if the auto-revert timeout is just couple seconds, making any change becomes impossible. One interesting solution could be to use some kind of code transformation tool which can do safe code modifications and immediately commit them. There is a parallel here with using the smallest maximum change size threshold.

What about real-world projects?

Auto-revert is a great technique for learning at code retreats but there is a question of how useful it is on real-world projects. One of the problems is complex code which takes a long time to understand let alone to make the required change within few minutes. The solution is to work in read-only mode during the exploration phase or to make focused improvements which can be done and committed within timeout. If it looks like it's impossible to implement the feature in a series of small steps, then it might be worth asking yourself "why", "how did we get here" and if the code can be refactored to make it possible.

Another problem is slow build and tests. In order to be usable, auto-revert timeout has to be longer than the duration of an average feedback loop on the project. For example, if it takes 2 minutes to compile and run all relevant tests, then it's not practical to revert changes every 5 minutes. Because of this, long feedback loops will push the lowest usable auto-revert timeout to the point when the change size is quite big and expensive to revert. What's worse is that if the feedback loop is too long, we naturally get distracted (start checking email or chatting to someone) and forget about auto-revert timeout. Surely, this can be interpreted this as "auto-revert doesn't work on real projects". Or you can use this an opportunity to ask yourself "why". In particular, if running tests take too long, was it a conscious decision based on some trade-offs or it just happened to slow down over time. Slow compilers, build tools and editors/IDEs also often get in the way of fast feedback. It's less common to continuously measure the speed of a compiler or a build tool on the project — it feels that it's outside our circle of influence anyway. But maybe we should, because looking at modern programming environments, many of them don't provide particularly fast feedback, so unless we collectively make a big deal out of toolchain performance, it's likely to stay just around "ok".

Finally, when using auto-revert, you might start committing really often. This brings up the question about the meaning of commit messages. If there are changes every few minutes, commits can become too fine-grained to write a separate message for each of them. One option might be to squash tiny commits into more chunky ones which can have more meaningful messages. Another idea is to decouple the process of integrating changes with other people from documenting project history (so commit on the trunk / main branch will be essentially like a "Save" button for distributed work).

Try it yourself!

The simplest way to try auto-revert is to use a timer on the command line or on your phone. Although if you're not pair-programming, it's easy to negotiate with yourself and skip the revert "just this one time". A bit more advanced approach might be to write a script which will perform git reset every few minutes and restart timer if there are no local changes. I ended up going down the path of creating Limited WIP plugin for IntelliJ IDEs.

Auto-revert is an absolutely essential constraint to try on a codekata to challenge your current ways of working and observe reaction to code being reverted. It can be more tricky to use on a large-scale project but it's worth trying just to find the bottlenecks in your working environment. Similar to other workflows, nothing is set in stone with auto-revert, so feel free to experiment with it and have fun.

Limited WIP — Maximum change size

Dmitry Kandalov — Sat, 24 Apr 2021 08:36:10 +0000

There is a lot of focus on the technical side of software development: programming languages, libraries/frameworks, architecture, etc. But not that much focus on what humans are supposed to do while using the tools. There seems to be an implicit assumption that developers will do things in the most optimal way. Unfortunately, this assumption is often false. In the same way that people don't always follow the lifestyle which is best for them, we don't always take the best path when creating software.

To give a specific example, let's say while working on a feature, you spot some code which can be improved. So you go ahead with the improvement and go back to the feature. This happens few more times and before you know there are twenty modified files with changes including both code improvements and the feature itself. The feature is not complete yet but the change feels big enough that it needs to be committed. The bad news is that it's now hard to disentangle code improvements from the feature, and you have to choose between (1) figuring out what to include into a partial commit, (2) committing mixed changes with an incomplete feature or (3) continuing with already oversize change. Not ideal. It would be good to not end up in situations like this in the first place.

This is where techniques to limit work-in-progress come to rescue by stopping us from doing too many things in parallel. The benefits of limiting work-in-progress are not just about better commits (which is often a symptom), they are about more focused work, fewer mistakes and improved workflow. In this blog I'll describe the technique of setting the maximum change size.

The idea

Using maximum change size to limit work-in-progress is quite simple — if the size of changes is bigger than some threshold, you're not allowed to commit. Instead, you have to find a way to split the work into smaller committable chunks. It also means that as soon as you reach the threshold, there is no point making more changes because you won't be able to commit them anyway. Reaching the threshold should be a rare event though. You should keep yourself aware of the current change size, for example, by monitoring diff size in version control or IDE, and dealing with it before it gets too big.

Many developers already have some kind of implicit change size limit. It manifests itself as the feeling that it's time to commit during a long coding session. So you might be already using maximum change size constraint without being fully aware of it. The advantage of an explicit change size limit is that you can quantify it and keep yourself honest by comparing change size to some threshold. You can also use a tool or script which will watch uncommitted changes and notify you as soon as the change size gets close to or exceeds the threshold instead of relying on the gut feeling to know when there are too many changes. It's also usually harder to "negotiate" with a script (e.g. a Git hook) compared to convincing yourself that the commit is still has a reasonable size "just this one time".

After a while this technique should naturally nudge you into working in a more focused way (unless you're doing it already) along the lines of making one change at a time until it's finished and committed. And if something else comes up during the change, you can add the new task to TODO list for later or shelve the current modifications, finish the new task and unshelve. Sounds simple but it's surprising how often we divert into more complex workflows.

Calculating change size

Calculating change size is a bit more tricky than it seems. The most obvious question is what to count: lines, words, characters or even the amount of actions/keystrokes in the editor since last commit? Should empty lines and whitespace characters be excluded? Regardless of the choice, there is a problem that modifications of the same size might have different real-world weight. For example, in Java adding an import statement is usually less work compared to writing a line of code that will be executed. Another problem is that conceptually simple changes can produce large diffs. For example, renaming a function which is used across the project will create a large diff even though it's a single rename. There are other problems of this kind and none of them have simple solutions just because it's extremely hard to measure complexity and impact of code. In practice though, I found that using a simple metric like non-empty lines of code works quite well.

Another question is how to calculate modifications on diffs. If we are measuring change size in lines, then adding a brand-new file with 10 lines of code should count as 10. But what if the change is replacing 10 lines of code with completely different 5 lines? Should the change size include only the new code, both old and new 10 + 5 = 15 or maybe average (10 + 5) / 2 = 7.5?

Finally, there is a question about how to quantify file system changes such as moving/renaming files or creating directories and how well the diff algorithm can determine if a file was moved or it's a new file with similar content.

Given the above, the maximum change size is obviously a heuristic which you will need to experiment with to make it useful. A good strategy might be to start with a threshold which is quite low and gradually increase it. Note that you can also track your progress in version control by analysing how commit sizes changed over time.

Thought experiments

To explore the idea, let's perform couple thought experiments:

With the maximum change of 1_000_000 lines of code you will never notice that there is a change size limit because pretty much nobody changes a million lines of code on daily basis.
If we go down to maximum change size of 10_000, once again it's unlikely to be noticed, except for large-scale code migration and restructuring.
It becomes more interesting with the change size limit around 100 and below. This is where the learning opportunities are.
Finally, with the limit of 1 or 2 lines, it becomes impossible to make any significant changes. Unless we cheat and redefine the constraint to be 1 or 2 actions/keystrokes in the editor. Then automated code transformations are allowed (for example, inline refactoring or create new class). The interesting aspect of it is that if transformations don't break existing functionality, then they are perfectly fine to commit and share the with other people after every single change. There is probably not enough functionality in your editor/IDE to actually do this but it was just a thought experiment.

Try it yourself!

Maximum change size is an easy constraint to try. You can start by watching change size manually on command line (e.g. with git diff). Or you can roll out a proper script to regularly check the change size and prohibit commits above threshold. I went down a different rabbit hole of writing a Limited WIP plugin for IntelliJ IDEs. Whatever path you choose, please remember that just like with other techniques nothing is set in stone, so feel free to experiment with it and have fun.