Tool knowledge pays back in the first week, but each new version resets part of it, so little of it builds up. Judgment pays back more slowly because it builds on itself: every problem you understand makes the next one easier. It is also about the problem and not about the tool, so it stays useful when the tool changes. Learn the tool well enough to do the work, and also study the problem it solves.
In embedded Linux work, a team uses a build system, which is a program that builds the software image for a product. A software image is the file that contains the whole system software. Yocto and Buildroot are two common build systems, and each comes with its own commands, file formats and rules.
Suppose two engineers join different teams. Both learn the commands of their team's build system and can produce an image within a week. (This example is illustrative, not measured.)
The second engineer also studies the questions that a product team has to answer, whatever tool it uses:
- How long must this product be supported, and how long will the kernel and the other components it depends on be maintained?
- When a vulnerability is published, how do we learn whether we ship the affected component, and how fast must a fix reach customers?
- How will a fix reach devices already in the field safely, with signed updates, and what does the customer's domain require?
To answer these you must know exactly what shipped and be able to rebuild it. The build system is how you act on these decisions, not the decision itself. Small teams often have no separate security group, and choices such as key storage are hard to change later.
A year later both teams change their build system. The first engineer must learn the new commands and also, for the first time, those questions. The second engineer learns only the new commands, and brings a year of answers that have already been tested against real releases.
Now imagine that a vulnerability is published in a library. The first engineer can rebuild an image, but does not know whether the product ships that library, which version, or who must approve a fix.
The second engineer has a record of every component in each released image and checks quickly. That engineer also asks whether an update can fail safely on a device that loses power halfway, and who holds the signing key.
Tool knowledge pays back in the first week. Judgment pays back more slowly, but it builds on itself and survives the tool.
What compounding means here
Tool knowledge is knowing how to operate a tool: its commands, settings and file formats. Judgment is knowing which question matters, what is likely wrong, and what a change will do, in situations the manual does not cover.
To compound means that each gain builds on the gains before it, as interest does. I use the word as a comparison, not as a measurement. I am not claiming that judgment grows at a number you can calculate, only that it has this shape: slow at the start, and larger each year because it rests on what came before.
Why tool knowledge pays back quickly but does not compound
People make tools, and people change them. Commands are renamed, options are removed, and a project can replace a tool completely. Knowledge of those details is tied to one version, so some of it becomes less useful each time the tool changes. The next version starts partly from zero, which is why years of tool knowledge add up to less than they seem to.
Tool knowledge is also written down in manuals, and an AI assistant can now find it in seconds.
A large field study of customer-support agents found a similar effect in another job. Agents with an AI assistant and two months in the job performed as well as colleagues with more than six months in the job and no assistant, measured by issues resolved per hour. The authors report only suggestive evidence that the agents also learned from the assistant, and customer support is not engineering, so treat it as weak evidence.
You still need the tools, because you cannot work without them, but knowing the tool is no longer enough by itself. What is still uncommon is described in Your Code Got Cheap. Your Judgement Did Not.
Why judgment compounds more slowly
My current explanation is that judgment develops from useful experiences. An experience is useful when three things happen: you find the real cause of the problem, you see the result of your decision, and you understand why the result happened.
That last step is what makes it build up. When you understand why a cause produced a result, you recognise the same pattern in the next problem and reach the cause faster. Each useful experience becomes the starting point of the next one, but you need many of them before the effect is noticeable, so the start is slow.
Many hours of work contain none of these. For example, you may fix a failure by trying changes until it stops, and never learn which change mattered. Nothing is added, so nothing compounds. Why the person who made a decision often never sees its result is explained in Ten Years of Work Is Not Ten Years of Experience.
Research on professional intuition, which means fast expert judgment, names two conditions: an environment regular enough to predict, so that the same causes give the same results, and long practice in it with rapid, clear feedback. Feeling confident is not a reliable sign of being right.
Judgment compounds from useful experiences, not from hours of work alone.
Why judgment compounds more durably
My view is that judgment is about the problem, and problems usually change more slowly than the tools used on them. Because the problem stays, what you learned about it carries over to the next tool instead of being reset. The product questions above are one example. Another is shared data.
When two things can run at the same time, the data they share needs protection, or one of them can read it partly updated. Different systems protect it with different mechanisms, such as locks, atomic operations or disabling interrupts, and the mechanisms change over the years. The reason for the protection does not change.
An engineer who knows the reason can usually read a new system's version of it much faster. This applies to related problems, not to every field. Reasoning about shared data is useful on the next system you work on, not in an unrelated job.
A new tool is a different method for the same job.
Where this argument has limits
Some tool knowledge lasts. A tool that keeps its core behaviour stable for decades, such as the core of the POSIX shell, is worth the years spent learning it. Use the replacement test below on what you learn, not on the age of the tool.
Tool-specific knowledge can also be a main part of the job. Knowing the quirks of the exact platform your product ships on can remain useful for as long as that product is sold. Learn it deeply in that case, and also learn why it is that way.
Judgment can also be wrong, and it can go out of date when the problem itself changes. It compounds only while you keep checking it against results.
A tool lets you do the work. Judgment lets you decide whether the work is correct.
What to do this week
- Run a replacement test. List what you learned in the last six months, and for each item ask: if the tool were replaced next year, could I still use this? Mark each item yes or no, and learn the no items quickly.
- Write down the problem behind your main tool. Choose the tool you use most and write down what problem it solves, what you would do without it, and what you would check to know it worked. Then ask whether you could do the same job with a different tool.
Here are examples for the replacement test. The command that signs an update is no, but why a device must verify the signature, and who guards the signing key, is yes. How your tool lists component versions is no, but how you find out whether a published vulnerability affects your product is yes.
If you hire or train engineers, ask a candidate to explain a task they completed, then change the tool and ask what would remain the same.
This is why TECH VEDA sessions are built around problem scenarios and not around the menus and windows of one tool. Each scenario is based on one problem, and you learn the tool while you solve it. The sessions are live and mentor-led. If you want to learn the build system and the product questions together, see the Embedded Linux BSP Development training.
The tool you learn this year will probably be replaced or changed. The problem it solved is likely to exist when you learn the next tool, and what you understood about it will carry forward.
— Raghu Bharadwaj
Top comments (0)