DEV Community: Mansur Fattakhov

How to Handle Incidents in a Small Team

Mansur Fattakhov — Sat, 14 Nov 2026 18:49:27 +0000

As long as the service is running, it's easy for a small team to assume it doesn't need an incident process. Everyone knows everyone else, you can get hold of anyone, and if something breaks, you'll figure it out in chat. That agreement gets its first real test when one person is rolling back a release, another is changing the configuration, and someone in chat is asking when things will work again. Everyone is doing something useful. Nobody has the whole picture.

My article on alerts was about noticing a problem. This is what comes next: the alert arrives, somebody opens a laptop, and then what? In a small team, one or two people should be able to put a process in place without a dedicated department or a form that takes half an hour to fill in. You need a few agreements that help you restore service and leave you with something to show for the outage besides exhaustion.

A shared channel: who's taking what?

My starting point is a shared incident channel. When a report comes in, that's where we say who's picking up which piece. Everyone needs to be able to see it: one person is checking application errors, another is looking at infrastructure, a third is finding out which users are affected. A read receipt doesn't tell you who's actually working on the problem.

For a major incident, we open a dedicated channel, for example inc-2026-10-06-http-outage in Slack. The date and short description tell you which outage it is and make the discussion easy to find later. We leave a link in the shared channel and move the conversation to the incident channel. That's where we collect checks, results, and decisions.

Get on a quick call to divide the work and assess the risks

When an incident is serious, it helps to get on a call straight away. In those first few minutes, we need a common picture of what's broken, how many users are affected, and what could happen if the problem continues. Current impact and potential risk can be very different. Errors might still be rare, but the incident is hardly minor if data could be lost or the failure could spread.

On the same call, we agree on who does what and who keeps track of the overall situation. People need to know what the others are doing. Otherwise, someone may change a setting just as someone else rolls back a release, leaving us to guess which action helped and what else changed in the meantime. You don't need an elaborate set of roles. Divide the work explicitly and talk through the next few steps.

Everyone records their own actions and results

It's worth posting a short account of the call in the incident channel: what we know about the impact, what each person is working on, what we've decided to try, and when we'll post an update. Anyone joining later can catch up without asking for a recap. Once the service is back, there's already material for the postmortem.

As the work progresses, it helps if each person notes the time and what they did: when they took the task, what they started investigating or changing, and what happened. Short personal notes are fine; they can go into a shared timeline after the incident. This matters especially when several people work in parallel. Even after a brief outage, it's surprisingly hard to reconstruct the order of events from memory.

If people are already talking in the incident channel, they can post progress there. Each message already has a name and a timestamp. An engineer might write: “I'll take restoring database access. First I'll check whether the database is reachable from the internal network.” Then they come back with the result: “I can reach the database from the internal network, but connections from the application nodes aren't getting through. I'm checking network access from those nodes.” This is an illustrative entry. What matters is the link between what the engineer set out to check and what they found, whatever the eventual cause.

These messages serve two purposes. The team can see where things stand, what's been checked, and who's working on what. The postmortem gets a sequence of events with names and times. Even if someone keeps private notes, the key findings should still go back into the channel. After recovery, we can bring the notes and messages together in a single timeline. Keep the checks that led nowhere, too: they explain what we knew at the time and why the next step made sense.

In the working record, it helps to distinguish observations from assumptions: when we noticed the problem, what users couldn't do, what we tried, and what happened. “The errors stopped after the rollback” is an observation. “The latest release definitely caused the errors” is still a hypothesis. Write both with equal certainty and the postmortem will start with people defending the first guess that paid off.

Restore service first

During an outage it's easy to get absorbed in the explanation: why did this happen, which line is responsible, who changed the setting? What users need right now is a working service. Sometimes rolling back, disabling the failing path, or temporarily limiting a feature is enough to give them that. We'll still need to find the cause. Recovery and investigation solve different problems and can move at different speeds.

Metrics tell an engineer how the system responded to a fix: whether errors dropped, latency returned to normal, and requests are being processed again. That's valuable evidence. It's even better if we can repeat the user journey that actually broke. If the team has testers, they can run through it and check the outcome. If the error only affects a particular user and we can't reproduce it ourselves, we can get in touch and ask them to try again.

This is where the project manager and the rest of the team can help. They can run checks, contact the user, and bring the answer back to the channel. Communication and verification can happen while the engineer works on recovery. The shared record then shows both what the metrics say and what the user experienced. If the graphs are back to normal and the user hasn't replied, say exactly that: we see technical recovery and we're still waiting for confirmation on this particular case.

When choosing the next step, I'd look first at how easy it would be to undo. Disabling a new processing path may be safer than reworking shared configuration on the fly. A rollback isn't harmless by default, either: if the release has already changed the data, the old version may no longer be able to handle it. Before acting, take a moment to say what you expect to see and what you'll do if things get worse. That gives everyone the same way to judge the result of the recovery step you're about to try.

Time permitting, save the evidence that the change will erase: a section of the logs, when the errors started, the running version, settings that changed, or the state of a queue. Don't delay recovery for the sake of a perfect archive. Someone who isn't applying the fix can gather what's available and leave links in the channel. Sensitive data stays in the storage intended for it rather than spreading through screenshots in the shared chat. After a restart, a process may look perfectly healthy and tell you nothing about what was wrong ten minutes ago.

A blameless postmortem with clear ownership of the fixes

Choosing not to look for someone to blame doesn't let everyone off the hook once the meeting is over. It's more useful to ask why an action seemed reasonable with the information available, which safeguard was missing, and what will now change. You can't verify “be more careful.” You can verify a task with an owner, a deadline, and a clear definition of done.

By the time we write the postmortem, our action items are already links to issues in the tracker, each with an assignee and a due date. “Add a database connectivity check” on its own doesn't tell you who will do it or when. The issue should make the intended change and its acceptance criteria clear. From the postmortem, anyone can jump straight to the discussion and current status.

Several actions may be necessary: fix the cause, add a missing check, clarify the recovery procedure. Each gets its own issue, owner, and due date. That's how a postmortem becomes part of the regular rhythm I described in the processes track. The team's commitments make it into its work plan, where they stay visible long after the incident stops feeling urgent.

Keeping the outcome alive a week later

The next day, the team is back to its usual work. The urgent messages scroll out of view, the incident channel goes quiet, and the promised fix has to compete with the next release. Even a good postmortem can come to nothing here: everyone understood what happened, but a month later nobody remembers what became of “we should do something about that.”

I'd keep a short entry for each incident in a shared register: the date, the impact users saw, links to the channel, timeline, and postmortem, then the follow-up issues and their current status. A spreadsheet or a page of links is enough. There's no need to copy the whole postmortem; the point is to find a particular incident and spot unfinished work.

Revisit those issues during normal team planning. If a deadline has passed, it's more useful to understand why and agree on what to do next than to move the date. Sometimes the original fix turns out to be too big; sometimes another change makes it unnecessary. Leave that decision on the issue, too: why it was replaced or closed and what risk remains. A green incident status means the service is back. It doesn't mean every promised improvement is in place.

Before the next postmortem, look for similar incidents in the register. You might find that you've seen this outage before, the follow-up issue is still open, and the team has just spent the first twenty minutes running the same check again. At that point the useful question practically asks itself: what's stopping us from finishing the fix?

This simple setup has limits. If several incidents run at once, multiple teams get involved, or ownership regularly gets lost in handoffs, agreements in a single chat won't be enough. You'll need explicit arrangements for coordination, on-call coverage, and handoffs. Add that complexity when you've seen the problem it will solve. To start with, it's enough that everyone can tell who's restoring service, who's talking to users, and where to find the shared record.

What should be left after an incident?

Before everyone goes back to their day, I'd check five things:

It's clear who picked up the initial report and how the work was divided.
Actions and results are in one place, with names and timestamps.
Metrics and a check of the affected user journey confirm recovery; any remaining uncertainty is recorded.
The postmortem links to issues with assignees, deadlines, and acceptance criteria.
The issues have a place in the team's work plan, where their status will come up again.

You can tell an incident process is working from a small moment: somebody joins the conversation and quickly understands what's going on and how they can help. They don't have to piece the story together from three private chats or interrupt the engineer who's restoring database access. And a week later, the team can open the record and see what changed because of the outage. That's worth agreeing on in advance.

Automating Support with AI: From a User's Question to a Response

Mansur Fattakhov — Mon, 09 Nov 2026 08:38:54 +0000

A user asks a question that support has answered before. An agent reads the message, clarifies the details, finds the right instructions, and explains what to do. A little later, another person asks something similar. For them, the problem is new. For the team, it is another trip down a familiar path.

This is an illustrative scene, but it captures exactly the routine we wanted to automate in our support operation. The goal was practical: answer standard user requests as automatically as possible and reduce support costs in the process. When a request already has a clear procedure, you want a person to step in where their knowledge or authority is actually needed.

That led to a service connecting language models to support work. Messages arrive from Telegram and Discord, while agents manage conversations in Chatwoot. But the most interesting part turned out to be what happens between the incoming question and the outgoing reply: what knowledge the model receives, what decision it returns, and what the service does with that decision.

At first, this path did not end with a message to the user at all. The assistant left suggestions in private notes visible to agents. We could see how it handled a request before giving it permission to respond on its own.

This is the first article in a series on automating support with AI. We will follow a single request from context assembly through an LLM call, result validation, and the choice of a next action. Later parts will look more closely at the knowledge base and conversation history, model tiers and processing costs, human handoff, and protocol failures. Chatwoot and the messaging platforms provide a concrete example; the core decisions also apply to other support systems.

The conversation already lives somewhere

Users write wherever it suits them. In our case, that means Telegram or Discord. Agents, meanwhile, need to see the request, its history, and its current state. Chatwoot handles that part.

The integrations deliver messages and connect them to the right conversation. Processing begins after that: is an automated response allowed right now, what information should go to the model, and what should happen to its result? In autonomous response mode, a Chatwoot agent-bot event triggers this processing. The trial mode starts from the channel integration and saves the result as a private note. These are alternative ways to trigger processing.

Looking inside an autonomous response gives us this flow:

Telegram / Discord message → Chatwoot conversation
                                     ↓
                         processing eligibility checks
                                     ↓
                  history + knowledge base + response rules
                                     ↓
                              LLM call at T1
                                     ↓
                       parse result and select action
                           ↙         ↓          ↘
                        reply     next tier    human agent

Each subsequent tier also returns a result that needs parsing. Sending a message is just one possible outcome. Chatwoot remains the agent's workspace, with the history, request state, and means to continue the conversation already in place.

A human agent saw the first result

In shadow mode, the assistant receives the message, the knowledge base, and the available conversation history. It then proposes a response or a decision to hand the request to a person. The result is saved in Chatwoot as a private note. The user cannot see it, and sending a reply remains the agent's responsibility.

That is how we started. Before allowing the assistant to speak on behalf of support, we gave it a place beside the person already handling the conversation.

For an engineer, this offers a useful observation point. You can open a request and see what the assistant would have suggested in that context: did it find the right information, understand the question, or start answering where a person was needed? Alongside the result, you can see the processing tier, token usage, and cost, if the provider returned it.

The presence of a note does not, by itself, mean the answer was good. The agent may not have read it, may have used just one sentence, or may have resolved the issue in a completely different way. To measure suggestion quality, we need to agree separately on how to evaluate it. A count of generated notes is no substitute.

Still, this starting point has an important practical benefit: we can examine the assistant's work on support requests without turning every generation mistake into a message to the user. Data access restrictions and the terms under which context is sent to a model remain part of the system's design. A private note changes who can see the result; it does not eliminate the external service call.

Assemble the task before calling the model

A message saying “it isn't working again” rarely tells you exactly what happened. Earlier in the conversation, the user may already have described the problem and the agent may have asked them to run a check. If the model receives only the latest sentence, it has to guess at part of the conversation.

The service fetches history from Chatwoot. It includes user messages and public replies, while excluding private notes. Among other things, this prevents a trial response from the assistant from automatically returning in the next request as though it had already been part of the conversation with the user.

The second component is a knowledge base from Confluence. It contains information needed for answers, alongside separate instructions about when the bot should respond and when it should hand the request to an agent. These materials serve different purposes. Knowing a procedure does not mean the assistant has permission to make every decision associated with it.

In the path examined here, the knowledge base text is loaded and cached inside the service process. Connecting Confluence does not, on its own, turn the system into vector search: what matters is which data actually enters the request. History length, knowledge base completeness, and update frequency affect both answer quality and the amount of context you pay for.

That is why I would also begin work on such an assistant by looking at our own documentation: does the team have a sufficiently clear answer to a typical request? If the instructions contradict themselves, the model may faithfully restate that contradiction. A more expensive tier cannot create a missing product rule.

An LLM call must end in an actionable decision

For autonomous operation, a string containing well-written text is not enough. The service needs to know what to do next. The model's result therefore contains an action and the data needed to carry it out.

The code provides four options:

Decision	Service action
Reply to the user	Send the prepared message
Hand off to an agent	Initiate a human handoff
Reply and hand off	Send the message and continue the handoff
Request the next tier	Repeat processing with the next model configuration

This contract makes behavior inspectable: we can see which command arrived and which handler executed it. A restricted command list does not guarantee the right choice, however. The model can still propose an inappropriate response. Code checks, prompt rules, and quality evaluation address different parts of that problem.

There is another step between the API response and command execution: the service must parse the JSON and interpret its fields. This is where a tidy diagram meets everyday integration work. A response may be invalid JSON, or valid JSON with an unexpected structure. Those are different situations, and they are handled differently.

For broken JSON, there is a separate request to repair the format. It receives the previous call's output, without the knowledge base or conversation history: its job is to restore the structure. If the JSON is readable but the command is expressed differently, the handler tries to recover the action from known fields. This tolerance has a cost: a recovered command can result in a message being sent to the user. Recovery rules therefore deserve as much attention as the normal response path.

When to call the next model

There are three processing tiers. Each can be configured with a separate model. It is also possible to use one model with different reasoning depths, which is what the defaults do. It is therefore more useful to think of T1, T2, and T3 as processing configurations than as a mandatory list of three different providers.

The reasoning behind a cheap-to-expensive strategy is straightforward: a routine question may need only the first call, while additional resources should be spent where they can help. In the current loop, however, a specific action triggers the transition: call_next_model. A numerical score is recorded, but it does not itself act as a switching threshold. If the first tier decides to answer or call a human agent, the loop ends there. If the third tier requests another tier, the request goes to a person.

This is an important boundary: there is a limit to how long computation can continue. And needing a human agent does not always mean the model was insufficiently capable. Sometimes the answer requires access or a decision the assistant simply does not have.

After handoff, the bot must yield the conversation

When an agent takes over, writing a log entry saying processing has finished is not enough. The user needs to understand what is happening to their request, and the agent needs to know why it was handed over and where to pick up the work.

In this service, handoff includes notifying the user, adding a private note for the agent, and changing the conversation state. Separate checks take the handoff label and participation of a human agent into account. Their purpose is to stop the assistant from continuing to answer over the person.

This is now a distributed-system state problem. Events arrive through a webhook, a message can be repeated, and an agent can join between two processing steps. Message deduplication exists at the entry point, but that mechanism alone does not prove that all possible overlaps have been eliminated. Handoff needs to be treated as a scenario of its own, with its own checks.

This is exactly where the simple picture of “receive text, call a model, send a response” stops explaining the service. We need to account for who is currently handling the conversation and whether a decision made a few seconds ago is still valid.

Savings begin with the cost of the whole request

We came to this problem to automate routine work and reduce costs. Looking only at the price of the first request would therefore be insufficient.

One support request may end after T1. Another may pass through several tiers. A third may need an additional call to repair the response format and still end up with a human agent. The code collects token usage, duration, and available call costs, but human work falls outside the model's meter.

To understand the outcome, we need a wider view: was the issue resolved, did the user have to contact support again, how long did they wait, and how much work remained for the agent? Cheap generation is useful until the cost of fixing its consequences outweighs it.

I will not claim a percentage saving in this article; that conclusion requires comparable data. What matters here is showing a design that lets us gradually automate standard requests and examine what happens at each step.

For us, that path began with a fairly modest result: a note the user never saw. Autonomous replies followed, with a limited set of actions and a way to hand the conversation to a person. Around the model call, the main engineering work gradually revealed itself: context, conversation state, routing, and result validation.

In the next part, “Context for AI in Support: Knowledge Base, History, and Response Rules,” we will begin with a short user message that is not enough on its own to resolve the issue. We will explore how to assemble the necessary information around it, what to leave out of the request, and why more context does not automatically make support better.

I also wrote about setting context and task boundaries in “How to Give AI a Task in an Existing Project”.

How to Learn to Work with a Coding Assistant When Time Is Tight

Mansur Fattakhov — Sat, 07 Nov 2026 12:49:14 +0000

At last, there is a spare half hour in the evening. You could open that walkthrough of a new tool you saved: the tab has been waiting for a week. It will probably contain three more links, unfamiliar terms, and a suggestion to install a few things first. Then morning brings the usual work again: a service bug, a question from a colleague, and a task you promised to finish yesterday.

On an evening like that, it is easy to decide that getting properly acquainted with a coding assistant requires a holiday. Or at least a quiet week when nobody asks you for anything.

I would not wait for that week. It has an unfortunate habit of never arriving.

For a developer, some of the learning material is already sitting in the issue tracker. There are familiar tasks, clear constraints, and people who can tell you whether you delivered what was needed. All that remains is to choose a small piece of work where you can try a new technique and make sense of the result. You do not always need a separate practice project for that.

A familiar task gives you something to hold on to

In the previous two articles, we worked through a hypothetical example: an email sent after an order is completed. First, we found our way around the existing project, then checked the change before release. Imagine that work is finished. Another small task appears in the tracker: adjust the data in a similar notification.

Previously, I would have opened familiar files and started looking for the right place. Now I can hand part of that search to the assistant. Before making the first request, though, it helps to decide exactly what I want to try. Learning to get an explanation I can verify against the code, for instance.

That is a small enough goal for an ordinary working day. There is no need to learn a new editor, switch models, assemble a toolkit, and invent the perfect system of rules all at once. In a familiar part of the project, it is easier to notice where the assistant has found something useful and where it has confidently filled a gap in its knowledge.

For a first attempt, I would choose a task where I can judge the answer myself. If the tool, the language, and the domain are all unfamiliar, it is much harder to distinguish help from a convincing mistake. Too much has to be taken on trust.

I also need enough time to allow for an unsuccessful attempt. Restoring a service in a hurry is a poor moment to get acquainted with a new way of working. A small change I know how to make myself leaves me the option of stopping and finishing it the familiar way.

One question per attempt

Back to the email. Suppose I need to understand where the recipient's name comes from before it reaches the template. I already have a rough idea of the route, so I can compare the explanation with what I know.

The request can be very simple:

Find how the recipient's name reaches the email sent after an order is completed. Show the path through the files and functions. State separately what the code shows and what you are assuming. Do not change anything yet. I need to be able to follow this path myself using your answer.

That final sentence sets a fairly specific expectation. I need an answer that lets me open a few places in the project and understand how they connect.

The assistant might find the template quickly but miss the data transformation that happens before it. That is already a useful learning outcome: I can see where its search stopped and where I might have been too quick to agree. I can point out the omission and ask it to continue tracing the path. Or I might notice that the request was too broad and start with a particular handler instead.

Another time, the answer may be accurate but so long that reading it takes more time than finding the function myself. I would narrow the next request: only the key steps, with links to the code, without an explanation of every line.

That is how a personal understanding of the tool develops. It is made up of small observations: where to narrow a question, when to ask for evidence, which part of the context to supply at the outset. After a few attempts, these things begin to save attention.

Do not change everything at the first inconvenience

An unhelpful answer often makes you want to try another tool. Sometimes that really is sensible. But if you change the whole set of conditions every time, it becomes difficult to tell what you have actually learned.

Suppose the assistant proposes far too large a change. On the first attempt, I asked it to “tidy up the notifications.” On the second, I specified a particular field, ruled out changes to the shared sending mechanism, and described the expected result. If the second answer was more useful, I learned something about defining a task. That lesson will stay with me even if I switch tools.

So, for a few comparable tasks, I would keep my usual environment and change one technique. Today, I ask for the evidence behind a conclusion. Next time, I agree on a plan first. Later, I ask it to compare the completed change with the requirements. You can choose your own order; what matters is giving yourself time to notice what has changed in the work.

These attempts need a clear limit. If half an hour has gone into persuading the assistant to notice an obvious file, it is perfectly reasonable to stop. I would note what did not work and return to the task the usual way. A learning experience does not become more valuable because it derails the entire day.

Leave yourself a few lines

Once the email works, you want to close the task. That is precisely when it is easiest to lose the small discovery that made the whole exercise worthwhile.

I would leave a few lines alongside the task: what I tried, where the answer helped, what needed correcting, and what I will repeat next time. For example:

The assistant found the template but did not trace the data preparation. Asking for the full path from the event to the email exposed the gap. Next time, I will start with that path. I am keeping responsibility for checking the result.

A week later, a note like that is more useful than a collection of polished prompts without context. It shows which problem the technique addressed and why it helped. If a similar situation comes up, I will have somewhere to start.

When the same request comes up again and again, it can become a project rule or a small repeatable procedure. I arrived at this through my own repetitions: I did not want to explain the same constraints from scratch every time. But first, it helps to see how the rule works across several tasks. Otherwise, it is easy to spend an evening writing a lengthy instruction that nobody can confidently explain afterward.

A rule needs a reason that makes sense without the chat history. “Show me a plan before making changes” helps us agree on the scope. “Do not commit without a separate instruction” keeps the decision to record the changes under my control. Agreements like these are easy to carry between tasks because their purpose is clear.

How to notice an improvement

The initial impression of speed can be misleading: the code appeared in a minute, but then it took an hour to work out what it did. I would therefore look at the whole journey to a result I am willing to accept.

How much time went into explaining the task? Did it become easier to find the right place? How often did I have to bring the assistant back to the original constraints? Can I explain the change to a colleague afterward? These questions do not require a separate tracking system. Sometimes a short note beside the previous attempt is enough.

Comparisons also deserve care. One email may be assembled in a single file; another may pass through several services. If the second task took longer, that does not by itself mean the new way of working became worse. It is more useful to look for recurring observations across similar tasks: fewer unnecessary changes, faster discovery of context, less need to explain the boundaries again.

Sometimes the assistant does not make the work faster but does help me understand an unfamiliar part of the project. That counts as a result too, if that understanding was what I lacked. It is important to name it honestly, so I do not later promise myself time savings where I actually spent time learning.

What can wait

At almost any moment, there will be a new tool that looks interesting to try. But getting acquainted with it has to take time from somewhere. I would return to my setup when I can name a specific inconvenience in my work.

If the assistant keeps forgetting project constraints, there is a reason to look into rules. If it spends too long finding context, I can examine what it can access and how the material is organized. If I want to continue a task from another device, I can explore a remote workspace. Each next step gains a purpose that makes its result easier to assess.

Until then, some news can remain unread. Not every announcement needs to trigger a rearrangement of your working day.

If I were starting now, I would choose one small, familiar task for the coming week, one technique, and a short note after the attempt. Then I would try the same technique again. After a few cycles, I would have material for the next decision: what to keep, what to change, and what I do not need yet.

That spare half hour in the evening can still go toward an interesting walkthrough. Only now, it will be easier to recognize my own question in it. Why does the assistant lose the connection between the handler and the template, for example? Or how can I give it the constraints once instead of repeating them in every chat? Reading with a question like that is usually more useful than trying to learn everything just in case.

And in the morning, there will be another ordinary task in the tracker. That is where I can pick up again.

The Assistant Wrote the Code. How Do You Know It Is Ready to Ship?

Mansur Fattakhov — Sat, 07 Nov 2026 03:34:25 +0000

The email arrived. It has the right name, the right order number, and even the button leads where it should. The assistant says the task is done, the checks passed, and the changes are ready. At this point, closing the tab and moving on sounds very appealing. Especially if this little task has already taken up half the day.

I would leave the tab open a little longer.

In the previous article, we worked through an illustrative example: adding an email after an order is completed in an existing service. We first worked out what “completed” meant, found the existing notification mechanism, and agreed on the constraints. Now imagine the assistant has implemented that plan. We still have to decide whether we are ready to release the change to users.

An email arriving at a test address is a satisfying moment. For the first time, you can see the result you were working toward. But alongside that email, I would reopen the original task. We promised the customer a notification at a particular point, and the team that email problems would not disrupt the order. I want to know how much of that promise the result actually confirms.

A short request is enough to begin:

Show me the path from the relevant order event to the email being sent. What changed, and why? Compare the result with what we agreed: what has been checked, what evidence supports it, and what is still unknown. Do not fix anything yet.

That last sentence helps. Otherwise, a discussion of the result can easily turn into another development cycle: the assistant spots an improvement and immediately starts making it. First, I want to look at what we already have.

Suppose the reply points to the order handler, the code that adds a job to the queue, and the email template. I open the changes and follow that path myself. Where does the address come from? Why this particular status? What data goes into the message? Has the new code affected neighboring notifications?

You do not need to learn the entire project for this review. But I do want to understand the parts responsible for the behavior we promised. If the explanation ends with “this uses the standard mechanism,” I would ask to see that mechanism. If the code is still unclear, I would bring in a colleague who knows it. A polished description does not make unfamiliar code familiar.

Sometimes this is where an unexpected expansion of the task becomes visible. We were adding one email, yet the changes include a new way of processing every notification. There may have been a reason. It is worth recovering that reason before release, while the necessary change is still easy to separate from incidental cleanup. The more we changed, the more behavior we need to check.

After reading the code, I would return to that email. How did it reach the inbox? Did we take a test order through the relevant state, or did we call the sending function directly? In the second case, we have learned something useful about the template and email delivery, but we have not yet traveled the path from order to notification. That needs a separate check.

Imagine that works too. The order reaches the agreed state, and the email goes to the test address. Now we can introduce a little trouble into this happy story.

For example, give the handler the same event again.

In the last article, we agreed that a repeated event should not create an extra notification. Now we can make that agreement observable. How many jobs were created? How many times did the system try to send the email? What lets it recognize an event it has already processed? In a test, a replacement for the sending service can record calls so we can inspect them.

Suppose the test list contains the reassuring name “does not send the email twice.” I would still look inside. It might only check that a particular field exists in the job. That is a useful check of the field. The promise about duplicate sending remains a separate question.

It is easy to end up in a closed loop here: the assistant writes the code, writes tests to match that code, and reports that everything agrees. That is why I return to the behavior we discussed before implementation. What should the customer see? What should remain unchanged? What outcome would we consider wrong? These questions give the checks a reference point outside the implementation itself.

The next problem is an unavailable email service. We reproduce it in a test environment or with a test double and see what happens to the order. Is its status preserved? Can the person using the service carry on? Can the team see that the notification was not sent, and is it clear what happens next?

For our example, that matters more than checking once again that the email button looks nice. We agreed upfront that an email failure must not interfere with the order. Now we can verify that condition while the consequences stay inside a test.

There is a more awkward case too: the email service accepted the message, but its acknowledgment never reached us. Sending may look unsuccessful to the application even though the email has gone out. If the assistant promises that duplicates are impossible under any circumstances, I would ask it to examine this particular moment. How does the system find out the result of the first attempt? What does it do if it cannot find out? We need to understand the solution’s actual limits and agree on acceptable behavior before promising the customer more than the system can deliver.

These scenarios do not have to become a large, separate testing program. I would choose checks based on what this change could break. For an order email, that means timing, recipient, repeated events, and keeping order processing independent of email delivery. If the review shows that a shared notification mechanism has changed, we also need to look at its other callers.

After that, “the checks passed” carries much more meaning. Ideally, the actual results are there too: which checks ran, in which environment, and what they confirmed. Separately, there should be a record of what could not be run. Perhaps the local tests passed, but access to the test email service was unavailable. That means part of the path still needs checking. Someone with access can do it, and we can return to the release decision once we have the result.

I would also ask the assistant to compare the final changes with the original plan. The code may have changed while we investigated retries and failures. Checks performed before the last edit do not necessarily describe what we are about to ship. It helps to finish with one clearly identified version of the changes and check results for that version.

Even then, one question remains before release, a question that rarely comes to mind while looking at an email in your own inbox: what will we do if a problem appears once users have the change?

In our example, I would find out in advance whether we can stop the new emails while keeping order processing running. Who will notice a failure? Where can we see unsuccessful attempts? Will jobs already created continue sending after the feature is disabled? If that control is needed for a safe release and does not exist, we have another concrete task to complete beforehand.

An ordinary code rollback cannot recall an email that has already been sent. Being able to stop has a practical purpose: it helps contain the consequences within the next few minutes, instead of continuing to send while the team works out what happened.

By this point, the release decision usually feels calmer. I can explain what changed, show checks of the important scenarios, and name the remaining limits. I know how to observe the new sending behavior and how to stop it. If a significant “probably” remains in the story, it has an address: a specific piece of code, an unchecked scenario, or a question about product rules. We can work with that.

I like how this approach gradually changes the conversation with the assistant too. Instead of a vague “check again that everything is definitely fine,” we ask questions with verifiable answers. We stop circling through reassurances and start examining specific behavior.

The email in the test inbox still feels good. Only now I can see the whole path behind it: the right order, the right moment, the right recipient, and understood behavior when something fails. I can explain why I am ready to let this code meet users.

Over time, these questions become a habit. That does not always require a separate practice project or a free month to learn a new tool. In the next article, we will look at learning to work with a coding assistant through everyday tasks, while the regular work is still there to be done.

How to Give a Coding Assistant a Task in an Existing Project

Mansur Fattakhov — Tue, 20 Oct 2026 17:32:31 +0000

“We just need to send an email after an order.” My favorite word in requests like this is “just.” While you're saying it, the work sounds almost finished: there's an order, there's the customer's address, and all that remains is connecting the two.

Then you open the project.

In one place the order is paid, in another it's packed, and in a third it's handed over for delivery. Emails are already being sent somewhere, but it's not yet clear whether you can reuse that mechanism. Before writing a few lines, you have to answer a question the original request never raised: what exactly do we mean by completing an order?

Let's use this as a worked example. We have a running service and need to add an email to the customer when an order is completed. I want to use it to show how to approach a change with a coding assistant when a project already has plenty of established behavior and you can't hold all of it in your head.

In the previous article, I suggested starting with a small change whose result you can see and check. This task is a little harder: before changing anything, we need to work out where its boundaries are.

The most tempting first request is “find the order handler and add email sending.” It sounds specific. But several decisions are already hidden inside it: that a handler is the right place, that the email should be sent from there, and that we agree on the right moment.

I'd start with what we know for certain: the customer should receive a notification when the order is completed, and email problems must not disrupt the order itself. Then I'd ask the assistant to first investigate the existing process.

We need to email the customer when an order is completed. A sending failure must not interfere with order processing. Don't change any code yet. Look at how an order moves through the system, where emails are already sent, and what checks exist nearby. Show the relevant places with references to the code. If “completed” is ambiguous, raise that as a question.

This is an example request, not a required formula. What matters to me is having a goal, a constraint, and a next step at the start of the conversation. After that request, I expect an explanation I can compare with the project.

To begin, the assistant needs access to the relevant repository and its working instructions: how the main parts are organized, how to run checks, and which conventions to follow. If I know where order processing starts, I'll provide that lead. If there's an issue describing the expected behavior, I'll share that too. Secrets and real customer data aren't needed for this: code and anonymized examples are enough.

At the same time, I don't want to explain the entire project upfront. The investigation has a route: follow the order to the relevant state and find the existing notification mechanism. Other details can wait until that route leads to them.

Suppose the assistant returns with some findings: the status changes in one module, emails go through a queue, and a similar notification already exists for another event. Now we have something we can verify. I can open those locations, read the surrounding code, and ask why this particular point was chosen.

I'd pay special attention to “completed.” The project may have several similar statuses, each meaning something different. A field name helps you search, but it doesn't define the promise you make to the customer. A “Your order is complete” email after a successful payment could be quite a surprise if the order still needs to be packed.

This is a useful point for the assistant to stop. It can show the options and explain what happens in each. Deciding which event to tell the customer about must follow the product's rules. If the answer is unknown, we need to establish it first. Otherwise, a few tidy lines will turn a guess into system behavior.

Let's say we've settled when to send the email. Now we can examine what happens around that moment. Can the handler receive the same event twice? How does the project retry failed sends? Where can we see that an email never went out?

It's easy to mistake these questions for overcomplicating a small task. But each brings us back to the same user request. The customer needs one clear notification. Running the handler again shouldn't turn into another email, and an unavailable mail service shouldn't break an order.

Here I'd ask the assistant to distinguish what it found in the code from what it's assuming. There's a huge difference between “the neighboring handler already guards against duplicates” and “systems like this usually guard against duplicates.” We can inspect and reuse the first. The second still needs checking.

Once those questions are resolved, the plan usually becomes much more concrete. We know which event matters, which sending mechanism already exists, how to add the email to it, and which scenarios to check. If something remains unknown, we can say so explicitly instead of hiding it inside the future implementation.

Before editing, I'd ask for a short account of the plan: what changes, why here, and how we'll know we got the intended behavior. That conversation helps catch unnecessary scope. If one notification suddenly calls for rebuilding the entire email module, it's worth asking whether that's really unavoidable.

Once we've agreed, we can move to code. The change needs its own branch with a clear starting state. If other tasks are running in parallel, a separate working tree—a worktree—can help. I use them to keep changes from different chats from getting mixed together. That separates the work, though the results still need to be checked for compatibility when merged.

At this point the request can be brief: implement the agreed plan, use the existing sending mechanism, and show the changes and check results. By now, most of the meaning has been assembled in the conversation and grounded in the code.

The checks should cover the behavior we agreed on. The email is produced after the right event. A repeated event is handled without creating an extra notification. A mail service failure doesn't interfere with the order, and the failure is visible through the project's established mechanism. Test addresses and a sending stub are suitable for these checks; real customers shouldn't be part of the experiment.

I'd also ask which checks actually ran and what remains unverified. A persuasive explanation doesn't mean the implementation has been exercised through the relevant scenarios. If something couldn't be checked, that's useful information for the next step.

This task leaves more than a new email behind. We've learned where the relevant order stage ends, how notifications are supposed to be sent, and which constraints matter. That knowledge belongs alongside the code or in the project documentation. Next time, we won't have to hunt for it again through a long conversation.

I like that you can start with perfectly ordinary words. You don't need to know the solution's architecture immediately or compose an ideal prompt. Explain the outcome, name the important constraints, and assign a first step you can verify. The conversation then becomes more precise: specific code locations, behavioral questions, and a small plan emerge.

“Add an email” gradually becomes a task both of you understand the same way. That shared understanding is what you can rely on when the assistant brings back the changes.

The next question comes just before release: is reasonable-looking code and a set of passing checks enough? In the next article, we'll look at how to judge whether the result is ready to ship.

How I Built a Workspace for My Coding Assistants

Mansur Fattakhov — Sat, 10 Oct 2026 18:53:01 +0000

I'm starting a series about how I gradually put together a workspace for my coding assistants. It will cover shared rules across projects, personal plans, working from my phone, and my attempts to stop depending on a single laptop. The story is still unfolding: I'm trying tools, changing my mind, and will be writing about how it goes. But first, I want to explain why I felt the need to build any of this.

This story has a fairly modest beginning: I got tired of repeating the same requests.

Don't commit until I've reviewed the changes. Don't add your own sign-off to the commit message. Small things, hardly worth interrupting your work for. Yet they came back in the next task, another project, a new conversation. And each time I had to briefly set aside whatever I'd opened my laptop to do and explain, once again, how I wanted us to work together.

By then, coding assistants were already part of my daily development work. With their help, I'd built quite a few products: things I used myself, things other people used, things real businesses relied on. Over time, I grew comfortable with that way of working. I learned where I was happy to trust an assistant and where I wanted to stop and inspect the result myself. Along with that experience came working agreements that, for some reason, I still had to carry from one conversation to the next.

I generally enjoy learning new tools. Lately, though, barely had I settled into one before I could see another way to do part of the work. A few months later, I'd revisit my process and find more to reconsider. There was plenty to be curious about, but I had little appetite for explaining my own habits all over again with every change.

I have a low tolerance for repetitive work. I can do it manually for a while, but somewhere in the back of my mind I'm already looking for a way out. So after making the same requests several times, I started writing them down as project rules.

At first, it was a small collection of agreements in one repository. Each had a clear reason behind it: the assistant had rushed ahead here, I'd had to explain the workflow again there, and this was something I'd already asked for in the previous task. Rules appeared as they became necessary, and I remembered exactly why each one was there.

Then I noticed that whole sequences of actions were repeating too. Creating a task, describing the intended change, preparing the result for review: I had preferences about those as well. That was how skills appeared alongside the rules, describing the way I usually worked. Things I'd once had to spell out every time gradually found a permanent place in the project.

Naturally, I wanted to take the useful discoveries with me. If one project had become easier to work on, why not give the others the same treatment? I copied the agreements, refined them, and soon ran into the next nuisance: now I had to keep all the copies consistent. Change something in one place, then try to remember everywhere else it needed to go.

When it came to sharing rules across projects and teams, rulesync proved useful. I collected the source rules in one main repository and set up a process that generated files for different agent tools and distributed them to the projects. The whole arrangement had a simple purpose: give me one place to make a change and a way for that change to reach the others.

Little by little, the system acquired more detail. Habits, small conveniences, solutions to recurring frustrations all found their way into it. I liked being able to carry experience from one project into the next without spending so much time explaining it.

Around the same time, I started planning a big camping trip for the following summer. I needed to put together a camp, work out what to bring, and think everything through. I was also paying more attention to my health and keeping track of what I ate. I kept notes for these things, and at first I didn't connect them with what I was gradually building around my development work.

Then I noticed I was returning to them with some very familiar questions. What had I already decided? What still needed figuring out? Where had I left off? Before taking the next step, I had to reconstruct the previous ones. In my work projects, I was already learning to spare myself that effort by keeping context somewhere the assistant could read it too.

I wanted to try the same approach with personal matters. Separate areas for development, health, camping, and other projects appeared in the shared repository. Each had its own notes and plans, while familiar ways of working could be used throughout.

And so the camping plans moved in next to the code.

It was surprisingly easy to get used to that arrangement. I could talk through plans with the assistant, return to recorded decisions, and work through what I still didn't understand. Starting a project no longer meant spending ages deciding where its first notes should go and how to avoid losing them. There was a place for it in an arrangement I already knew.

I felt the change most clearly when picking things up again. When everything is scattered across separate notes, even a short break creates extra work: you have to remember what you meant, find the right entry, and gather your thoughts. Here, continuing became easier. The project retained what I'd already figured out about it, and I could move on.

It made me noticeably more organized. More than that, I found myself wanting to return to my plans more often. They were easier to pick up, talk through, and see a next step in. The system I'd once started building out of irritation was slowly becoming a place I liked coming back to.

That place, however, lived on my laptop.

The files were there, Git was there, and that was where the agent ran, reading the material and making changes. As long as it was all about development, I barely noticed the tether: I sat down at a computer to work on code anyway. But thoughts about a trip or a personal plan rarely wait for you to open your laptop. They turn up in a taxi, in a coffee queue, in the middle of something else entirely.

At those moments, I wanted to take out my phone and continue the conversation. Add something I'd just remembered. Clarify a question. Check where I'd left off. Everything I needed was already gathered together; I just needed to reach it.

When tools for working with agents remotely became available, I was able to try that too. The laptop still did the work, but I could now direct it from my phone. My projects became easier to reach throughout the day: a quick check-in no longer required sitting down at the computer.

You get used to that convenience quickly. Especially when the conversation follows you between devices and all the rules and material you've collected remain within reach. I'd spent so much time putting it together, and now I could finally use it where the need arose.

All I had to do was remember to leave the laptop on.

Sometimes I forgot. Or its battery ran out while I was traveling. Then the whole arrangement, with its agreements, projects, and remote access, had to wait until I got back to the computer.

I think that was when I realized how far I'd come from those first requests about commits. I now had a workspace I wanted to reach at any moment. What remained was to make sure it didn't switch off along with my laptop.

I'll come back to taking that life beyond the laptop. First, though, I want to look more closely at what we'd actually be moving. In the next article, I'll return to those early rules and skills: how they grew out of everyday requests, how I gathered them in one place, and how I used rulesync to distribute them across projects. Those small agreements were where everything else began.

How to Start Vibe Coding When You Already Know How to Code

Mansur Fattakhov — Thu, 08 Oct 2026 16:45:32 +0000

When you already know how to code, your first attempt at working with a coding assistant can be frustrating. You could have opened the file and changed a few lines by now. Instead, you are explaining the task, clarifying details, and reading a long response. Code appears on the screen, along with a perfectly reasonable thought: “I could have done this myself already.”

Perhaps you really would have finished that particular task faster. But one attempt still leaves you wondering how to fit an assistant into your usual work. Handing over an entire application feels risky; asking it to finish an obvious line feels pointless.

I would start with a small change in a familiar project. Something where you can picture the result in advance and spot a mistake. That leaves you free to pay attention to the process itself: how to explain the task, what happens on the first attempt, and where your involvement is needed.

A small annoyance on a large screen

Imagine you have a personal blog. You usually open it on your laptop, and it looks fine. Then one day you open the page on a large monitor and notice that the article list has drifted apart. Titles sit far to the left, images stretch toward the right, and there is so much space between them that they no longer feel like parts of the same card.

Meanwhile, the header and author description look normal. Nothing is particularly bothersome on your phone, either. You do not want to redesign the whole site. You want to bring the wide layout together while preserving what already works.

This makes a useful first task. It has a visible defect and clear boundaries. You can decide what “done” means before you start: the list aligns with the rest of the content, images keep their proportions, titles remain readable, and the phone layout has no horizontal scrollbar.

That clarity will come in handy when the assistant announces that everything is fixed. You will have something to compare its answer against.

Before experimenting, save your current work in the usual way and create a separate branch. Being able to undo a change calmly makes it easier to try things without worrying about every unsuccessful attempt.

The conversation starts with what you see

The assistant needs the same material a colleague would find useful: the page, a screenshot of the problem, and a few words about what bothers you. If the tool can read your project, you can ask it to find the template and styles. In a regular chat, you will need to provide the relevant snippets yourself. Page code and a screenshot are enough to investigate the layout; credentials and user data have no role here.

An ordinary description is enough for the first request:

On my blog's author page, the article list spreads wider than the header and author description on a large screen. The titles and images end up too far apart. I am happy with the layout on smaller screens.

Find what controls the list's width, explain the cause, and suggest a small change. Do not edit any files yet. I have attached a screenshot and the page address.

You do not have to guess the cause or dictate which class to add in your first message. Describing what you observe and asking the assistant to investigate is enough. It also gives you a chance to check whether you understand the task the same way before changes appear in the project.

A useful explanation can be checked against the code. For example, the assistant might show that the header sits inside a container with a width limit, while the article list sits outside it. You open the template and see whether that is true. If the explanation does not match the file, it is better to discuss the discrepancy immediately.

At this point, it is worth asking which other pages use the same template or style. A small change can have neighbours: fixing the author page might also alter a category page without you noticing.

When the code arrives

Once the explanation makes sense, ask the assistant to apply the proposed change and show the diff. The work now becomes familiar: you have a set of changes to read.

It is tempting to glance at the browser and move on. But a few minutes with the changed lines help you understand what happened. Why was the width constraint added here? Was an existing container reused? Where did that change to the neighbouring block come from?

If the assistant also renamed classes, reformatted the whole file, and “improved” a couple of other things, ask why those changes are necessary. A compact diff is easier to connect to the original task. Anything unrelated can wait for a separate conversation.

There is no need to pretend you understand an unfamiliar construct. You can work through it, ask why it is needed, or suggest an approach already used in the project. After all, the code will still be in your repository after you close the chat.

A regular chat adds one manual step: copying the suggested snippet into the file. After that, the process is the same: inspect the diff and open the page.

The page needs to survive resizing

The blog may now look excellent on a large monitor. That is satisfying, but the task had another condition: preserving smaller screens.

I would open the page in a wide window, at a typical laptop width, and at a narrow phone width. Then I would simply drag the edge of the window. Sometimes the problem appears between your chosen sizes: a title suddenly wraps, an image becomes too narrow, or an unexpected gap appears.

If the change affects a shared template, check another page that uses it. The build and project checks are useful too, but they will not tell you whether a title is comfortable to read next to its image. It helps to ask what the assistant actually checked and what still needs your attention.

Suppose the wide layout looks good, but an extra gap has appeared on the left on mobile. That is a concrete observation you can bring back to the conversation:

The article list now aligns correctly on a wide screen. At a width of 390 pixels, there is an extra gap on the left. Here is a screenshot. Find which part of the last change introduced it and suggest a fix that preserves the wide-screen result.

A working rhythm gradually emerges. You point out a discrepancy, the assistant investigates, and you check the next version. If changes keep accumulating and the explanation stops matching the result, you can stop and return to the saved state. Sometimes restating a narrow task is easier than continuing a tangled chain of fixes.

After the first attempt

Once the page looks right, pause briefly before moving to the next task. Where did the assistant save time? What was difficult to explain? Which mistake did you catch yourself, and how did you resolve it?

Those answers are more useful than an overall feeling of liking or disliking the experience. Perhaps finding unfamiliar templates was quick, but discussing the design took too long. Or perhaps the code worked immediately, but you had to describe your acceptance criteria much more precisely than usual.

A small blog fix is enough for a first attempt. You can approach input validation or a report change in the same way: choose a familiar task, picture the right outcome, and work through to checking it.

One such change is enough to get started. Afterwards, you will have your own experience: useful wording, unnecessary edits, and places where work stalled without your understanding of the project. That gives you something to build on when choosing the next task.

I wrote about my attitude toward this way of working in “Yes, I Vibecode”.

Alerts Worth Waking Up For

Mansur Fattakhov — Tue, 06 Oct 2026 04:32:28 +0000

The most honest way to test your monitoring is to wait until everything goes down and then see who tells you about it first: the system or a real person. I once had a project where that examination arrived without warning and ended badly. The machines sat with a single provider, Kubernetes with Prometheus and Grafana lived right there among them, and nobody checked anything from the outside — why bother, it seemed, when everything is plainly visible from within. On the day the provider took every machine down at once, not a single message arrived. The graphs did not turn red, because they had stopped updating; the rules did not fire, because there was nothing left to fire; the notification channel held its ordinary workday silence, the very silence that for all the preceding months had meant that things were going well.

We found out about the outage not from monitoring and not straight away. And this was not a story about poorly chosen thresholds or unfortunate rules — it is a story about how monitoring that runs on the same servers as the services falls silent together with them, and monitoring that has fallen silent looks exactly like monitoring for which everything is fine.

Ever since, I have designed alerts not from the question "what else could we measure" but from the question "and if this does not work, who will notice, and when". What follows is what comes of that: which messages are worth sending to a human being, where thresholds come from, and why a good share of the alerts in an average project ought not to exist at all.

An alert is a promise that someone will get up

It all starts with a single distinction that spares more nerves than any technique. An alert is neither an observation nor a curiosity, but a promise: when it arrives, a particular person will take a particular action. If no action is implied, what you have in front of you is not an alert but a graph, and its place is on a dashboard people visit of their own free will, not in a channel that wakes them.

The easiest way to tell the two apart is a thought experiment: imagine the message arriving at three in the morning. If the only thing you will do is glance at it and lie back down, then it has no business arriving at night; and if you would do nothing about it in the daytime either, then it has no business arriving at all. I have written about this before in the piece on KPIs, where the share of alerts that people actually respond to turned out to be a far more honest metric than their number: an alert everyone ignores does more harm than a missing one, because it does not merely keep quiet about a problem — it trains people not to look at the rest of them either.

This rule has a reverse side that gets remembered less often. If action is implied and there is no alert, then you are seriously counting on someone noticing the breakage by accident. Usually it is the users who notice, and usually later than you would have liked.

Silence that looks like order

The breakage hardest to see does not look like a red graph but like the absence of a graph. A process died, an agent dropped off, the network cut away half the environment — and the panel does not flush crimson, it simply freezes. The last green dot is indistinguishable from a normal one, and an eye trained to hunt for red slides calmly past.

The rule "the service is not answering the scraper" is no salvation here, because a dead host fails to answer not out of pain but because, as far as the scraper is concerned, it no longer exists. What is needed is a separate class of rules — ones that watch not for bad values but for the very fact that data is arriving, and that fire when a metric stays quiet longer than is reasonable. I keep such a rule for every host, and it is the only thing that catches "everything has vanished" rather than "everything has broken".

Out of the same story with the provider grows a second conclusion: part of your checks must live on the outside. Internal monitoring knows everything about the services but dies along with them; an external check knows shamefully little — whether the domain answers, whether the certificate is valid — yet it outlives the collapse of everything else and asks precisely what a user would ask. Two loops cost pennies and cover different classes of outage: from the inside you see that things have gone bad, from the outside that you are no longer there.

And the last layer of the same onion is a watchman over the watchman. Rules may fire faithfully while the messages never arrive: the bot token went stale, the notification service went down, someone renamed the channel. The silence that results is indistinguishable from the silence of a calm day, and here the circle closes exactly where it began. So I keep alerts for notifications failing to be delivered and for the external monitor being unreachable — and yes, I understand that this watchman could in theory fall silent too; it is simply that with every layer the probability of a simultaneous failure grows less and less interesting.

Outages known about in advance

There is a separate breed of breakage that it is plainly shameful to learn about after the fact, because the date is known in advance and written into the object itself: a certificate expires, a domain runs out, an integration token goes stale. All of it happens on schedule and, at the appointed hour, brings the service down without fail.

On a busy website such an outage does not live long: the certificate expired, the page stopped opening, a quarter of an hour later somebody wrote in the chat, another quarter and it was mended. But on a website that gets only a few visits a day there is nobody to be first — and that same expired certificate quietly holds the page shut for hours, while you learn of it when you finally drop by yourself. The most galling part is that it is prevented by a single line: a warning two weeks out, then a week, then a day. I keep such checks in the external loop, the very one that will outlive the fall of the internal one.

Beside them stand the background jobs, and the logic is the same, only turned inside out. A backup, a nightly export, a cleanup of old data do not break loudly: they do not fall over with an error, they simply stop happening. So what has to be watched is not the failure but the absence of a successful run: my blog's nightly backup knocks on the monitor itself after every successful attempt, and if no knock has come by morning, a message arrives — regardless of whether the process died, never started on schedule, or quietly hung halfway through.

Where thresholds come from

A threshold taken from thin air is the chief producer of pointless nighttime wake-ups. "Let us say eighty percent" sounds judicious right up to the day it turns out that the service lives perfectly well at eighty-five and wakes you every night until somebody thinks to look at what those numbers have been over the past six months. The order ought to be the reverse: first the actual distribution, then a threshold slightly above the usual maximum, and after the very first false firing — higher still, because reality always proves noisier than it looks on a week's worth of graph.

The second thing without which a threshold is useless is the "for" duration. The rule must require the condition to hold for some time, otherwise any one-second spike will raise the alarm and you will find yourself in front of a laptop at the moment when everything has long since been fine. A couple of nights like that are enough for a person to start opening the messages every other time, and from there see above: an alert that gets ignored is worse than no alert at all.

There is a third consideration as well, far less obvious: a threshold lives not in a vacuum but in concert with the rest of the automation, and that automation has to be known. On my server there is a cleanup after the build runner which begins pruning images when about three quarters of the disk is taken. The free-space alert is set not where the number looks prettier but where it falls between the soft and the hard level of that cleanup: let it fire earlier and it will shout every time the system is already tidying after itself; let it fire later and it arrives when it is too late to mend anything and images must be torn out by hand on a live server. A good threshold is always an answer to the question "what is already happening without me", and not a round number out of one's head.

Severity as a promise, not as ornament

Severity levels make sense only if behind each of them stands an obligation that can be checked. I have three, and they read literally: critical wakes me at night, warning I look at today, info does not reach the main channel at all. The moment a fourth level appears, of the "important, but not that important" sort, the system begins to come apart, because the boundary between neighbouring levels ceases to be checkable and therefore ceases to be observed.

Hence the most reliable symptom of ailing monitoring: if everything of yours is critical, then nothing is critical. Marking by severity says nothing about how dear a metric is to you; it says what a person is obliged to do on seeing the message — and if the answer is the same for every level, the levels can safely be thrown out.

One cause, one message

When a node goes down, a conscientiously configured system sends two dozen messages: about the node itself and about every container that lived on it. Formally it is all true; in essence it is noise, and the single line that matters drowns in it. So messages have to be grouped by cause, and repeats sent no more often than once every few hours while the problem is alive: a person needs one signal and one place to look, not a chronicle of disintegration.

And without fail — a separate message about recovery. Without it there is no telling whether the outage has ended or you have merely turned away from the channel; and the habit of assuming that things will somehow sort themselves out will one day coincide with the case where they did not.

Alerts on meaning, not only on mechanics

Technical rules answer honestly the question of whether the system is working, but say nothing about whether it is doing the thing it exists for. A service can return an impeccable HTTP 200 to every request, all the graphs will be green and even, and inside it is already empty: the data supplier broke, the integration dropped off, an update changed the format, and the page now honestly and quickly shows nothing at all.

So on top of the technical ones I keep a few rules that watch for business-level signals: click-throughs have fallen to zero, the data sources have stopped answering, a regular job has not refreshed the data mart by its appointed hour. Such alerts catch precisely what the mechanics miss, and it is usually they that save your reputation, because a user does not see your uptime — they see an empty page and draw a conclusion about you as a whole.

The rules live in the repository

Last in order but not in importance: alerts are code, and they are worth handling as code. The rules lie next to the configuration, pass a check before rollout, and change by the ordinary working process rather than by mouse in an interface on the fly. The reason is not fastidiousness: monitoring is almost always patched in a hurry and in a foul mood, right after an outage, and it is exactly those changes that matter most to keep in the history — so that six months later one can see not only what the threshold is, but what kind of night gave birth to it.

Checklist

⬜ For every alert it is clear who does what when it arrives

⬜ There are rules for the absence of data, not only for bad data

⬜ There is an external loop that will outlive the fall of the internal one

⬜ There is an alert for the failure of notification delivery itself

⬜ Certificate, domain and token expiry dates are checked in advance

⬜ Background jobs alert on the absence of a successful run, not only on failure

⬜ Thresholds are taken from actual values and have a "for" duration

⬜ Severity levels mean a concrete promise, and critical is not on everything

⬜ One cause yields one message, and recovery arrives separately

⬜ There is at least one alert on business meaning, not only on mechanics

⬜ The rules lie in the repository and pass a check before rollout

Yes, I Vibecode

Mansur Fattakhov — Sun, 27 Sep 2026 15:40:42 +0000

Only a year ago, putting together a working application from a single description in plain language was a party trick people bragged about on Twitter. Today that is how internal tools get built in teams that used to wait months for a slot in a developer's queue, and nobody finds it remarkable any more. Vibecoding has stopped being a sideshow and become something anyone can simply do — much the way building a website without knowing HTML once stopped being a miracle.

Along with that, a stock reply has appeared in developers' conversations, and I hear it constantly now: "vibecoding is not coding." It is delivered in every possible tone, from the condescending to the openly angry, but the meaning is always the same: what those people are doing does not count. I thought that way myself for quite a while, and that is exactly why I want to work out why it is a bad reaction — not because it is factually wrong, but because it misses the target and spoils the person who says it.

Where the irritation comes from

It is worth starting with the fact that the irritation is entirely understandable. Behind the person feeling it lie years spent on things nobody sees from the outside: algorithms that had to be learned twice, nights spent debugging someone else's code without a single line of documentation, postmortems in which it turned out that you were the cause yourself, and the slowly collected bruises out of which professional instinct is eventually made. And right next to that, someone describes a task in the words they would use with their grandmother, and gets the result in one evening.

It feels as though the part that was hard is precisely the part being devalued. As though years of experience had collapsed into an ability to phrase things neatly in plain language. This is not greed and not snobbery — it is an ordinary human jealousy over one's own labour, and it would be strange not to feel it. The problem is not the feeling but the conclusion drawn from it.

Honestly: I thought the same

At first I caught myself on exactly that thought — nonsense, not serious, toys for people who do not want to learn properly. I watched a couple of demos, looked inside the resulting code, saw things there I would not let into prod even at the point of a deadline, and calmed down: not a competitor.

The mistake was not in my assessment of the code — the code really was mediocre. The mistake was that I was comparing the wrong things with each other: someone else's result after one evening against my own result after months of work. The comparison that would have made sense sounds different — what I myself can do with this tool, and how that will differ from what I do without it. The moment I put the question in that form, there was no one left to argue with.

The tool does not level, it amplifies

It turned out that the better a person understands how systems are built, the more they get out of code generation. That contradicts the first impression — you would think a tool that writes code for you ought to level out the strong and the weak. It does not level anything out; it multiplies what is already there.

The reason is simple: an engineer's work never did come down to typing text. It begins long before the first line — with framing the problem, where you have to formulate not only what the result should be, but also what happens when everything goes wrong: what we do on a timeout, what we do when two requests race each other, where the load limit lies beyond which the solution stops working. Half the problems of the future task are visible right here, and they are visible to the person who has already stepped on them.

Next you need context the model cannot know in principle: how this particular service is put together, why there is a queue here rather than a direct call, which decisions were already taken and written down a year ago and why they should not be reopened along the way. Then the resulting code has to be read with the eyes of the person who will answer for it in prod — which means looking not for typos but for the quiet places: an unhandled timeout, idempotency lost somewhere along the road, a query that works beautifully on a hundred test rows and falls over on a real million. And finally all of it has to be rolled out, the metrics watched, rolled back if it went wrong, and answered for — that part is delegated to absolutely no one and is covered by no tool at all.

Generation covers the middle: the actual writing of the code, the very part that was already the fastest for an experienced person. The edges stay where they were, and it is in them that everything an engineer is paid for lives.

Two different occupations under one name

This, it seems to me, is where the whole argument grows from: one word is used for two different things, and then people are surprised that they cannot come to an agreement.

The first is vibecoding as entertainment and as a way into the profession. Someone puts together something of their own, takes pleasure in the fact that what they had in mind works, and sees a result they would never in their life have seen without this tool. For most of them it will stay an evening pastime, and there is absolutely nothing wrong with that. More than that, it is through such trifles that new people come into the industry — and they come several years earlier than they would have come through a textbook on algorithms, if they ever arrived at all.

The second is an engineer who uses generation professionally. The tool is exactly the same, but the tasks are of a different scale and carry a different cost of error: data migrations that cannot be replayed, integrations with someone else's payment systems, rewriting a service that is holding load right now, infrastructure you answer for with night shifts. This is no longer vibecoding in the sense in which the word is usually used — it is engineering work in which generation stands in the same row as the debugger and the profiler, and just as surely decides nothing on its own.

The difference, in short, is not in the tool but in what happens before it and after it.

About the musician at his friends' gig

When an engineer, in reply to "but you're vibecoding," starts explaining that he is not vibecoding at all but doing real, difficult work, he is arguing about a word — and from the outside that looks not like an argument but like a defence of status, whatever he may actually have meant.

I like the comparison with music here. Picture a musician from a good orchestra who has come to a concert by his friends' band: they got together six months ago, they play in a garage, they miss half the notes, the drummer rushes. And there he is all evening, wincing and telling everyone around him how bad this is and how much he dislikes it. Formally he is absolutely right — they really do play worse than he does. In human terms he has simply ruined the evening for people who took nothing away from him.

The people putting together their first application from a description today are the friends in the garage. They are not laying claim to a seat in the orchestra and they are not taking it away from anyone. Some of them will come a year from now to learn to play seriously, and it will be precisely because they had a good evening, not because someone explained to them how bad they were.

How to answer it

So when you are told that you are vibecoding, the best thing you can do is not to take offence. Smile and agree: yes, and I do it brilliantly and professionally. After that the conversation either moves into a normal professional channel or ends — and both options are better than an argument about terms.

Along the way I have forbidden myself three things. Not to prove that my work is difficult: if its difficulty is not visible in the result, no words will explain it, and if it is visible, there is nothing left to explain. Not to correct other people's terminology: the phrase "this isn't vibecoding, it's engineering with code generation" has never yet made anyone smarter, but it reliably ends the conversation. And most importantly — not to devalue someone else's interest: a person showing you a thing they put together over the weekend is not waiting for a review, what they need is for someone to notice that they made it at all, and that is worth saying out loud even if inside there is terrifying code and three hardcoded passwords.

The last of these matters more to me than the first two. Interest is the rarest and most fragile resource in our profession: it is put out with a single condescending comment, and lighting it again afterwards almost never works.

What this means for an engineer

The skill does not disappear, it shifts. Value moves out of the speed of typing code — where, frankly, there was never much of it to begin with — and over to where there was always the most of it: understanding the domain, framing a task so that it can be solved at all, seeing the boundaries, and answering for what runs in prod at three in the morning.

That, incidentally, is exactly the same set the team lead is responsible for — there was a separate post about it. And it is exactly what cannot be generated: no model knows what downtime counts as acceptable at your company, why the last migration was rolled back at the final moment, and whom you need to talk to before touching this service.

And how a working everyday tool gets assembled out of generation — projects, goals, routine — I will tell separately, closer to December.

In short

⬜ Vibecoding has gone mainstream; arguing with that is too late and pointless

⬜ The tool does not level, it amplifies: the better you understand systems, the greater the return

⬜ Generation covers the middle of the work, but not the framing, the context, the review or the responsibility

⬜ Entertainment and engineering work share one word — they are different occupations, and both are fine

⬜ If you are told "you're vibecoding," agree and add that you do it professionally

⬜ Do not review someone else's interest: notice it and praise them for having made the thing

KPIs That Don't Turn Into a Stick

Mansur Fattakhov — Sun, 27 Sep 2026 14:42:03 +0000

You now have several teams, each with a lead, and you want the system to run predictably — without diving into every team every day. The first reflex is to "hand out KPIs." And that is exactly the place where it's easy to ruin everything.

It all breaks against one regularity the economist Charles Goodhart formulated half a century ago: once a metric becomes a target, it stops being a good metric. Not because people are dishonest, but because you can almost always work on the number directly — bypassing the thing it was introduced for. Give every lead a share of uptime and you'll get cautious releases and arguments about whose incident it was, not a more reliable product.

What follows is how I built KPIs for team leads on a gamedev project with six teams while working around that trap. The main question there turned out to be not "which metrics do we take" but "what is a lead responsible for in the first place."

This is a spin-off of the "How to bring order to a project" series — this time about management.

How it started

Six teams: the mobile client, the launcher, the game server and the rest. At the C-level we wanted transparency — to see how things were going without climbing into every team daily.

But there was also a concrete trigger, not just manageability in general. In several teams, churn was eating the focus on the things that never look urgent: ANRs and crashes. Nobody worked on them deliberately — not because they couldn't, but because it was nobody's zone. A feature has a deadline and an owner; a stability number has neither. So beyond transparency, the KPIs had to place responsibility: not "who's to blame for the crash," but "whose zone of attention is this at all."

The starting point was the CTO's own KPI. It already existed — put together quickly, but it existed, and that turned out to matter more than the precision of the wording: there was a single outcome everyone worked toward. The mistake would have been to slice it into shares and hand them down — more on that trap below. What was valuable was something else: it set a direction, and from there you had to answer your own question — what is this particular lead responsible for within that direction.

Why the C-level wants this — honestly

The motivation from the top usually sounds like "transparency and manageability," but it's worth unpacking all the way:

Seeing the state of the teams without micromanagement. Not "what is everyone busy with" but "is the system healthy" — and noticing degradation before an incident.
Predictability. Promises to the business rest on something measurable, not on a feeling that "it seems fine."
Scaling yourself. A CTO with six teams physically cannot be the entry point to every decision — lead KPIs are a way to delegate responsibility, not tasks.

And right here is the honest trap you have to see in advance: the easiest way to "decompose the CTO's KPI" is to cut it into pieces and hand them down. The CTO's goal is "uptime and product quality" — so give each lead a share of uptime. It looks logical and works badly: the lead gets a metric they only partly influence, starts optimizing the number instead of the system, and defends themselves with reports instead of doing the work.

What the leads think about it

Leads have their own set of motivations — and fears, and ignoring them is the most expensive option of all:

What a lead gets from honest KPIs:

clarity: "what is expected of me" stops being telepathy — these are literally the expectations spoken out loud from the people track, only at the level of roles;
protection: written criteria are insurance against arbitrary evaluation "by mood" and against rules changed after the fact;
arguments: "we need one more person" sounds weak; "we won't hold the stated recovery time without a second on-call" sounds strong.

What a lead is afraid of:

that the metrics will become a stick: any red number is grounds for an inquest;
that they'll have to work "for the number": producing reporting instead of value;
that the goals will be set above their head — from a world where their teams don't exist.

Everything below is, in essence, an answer to those fears.

The key turn: a lead owns the system, not the outcome

The central decision everything rests on: a lead is responsible not for the final business metric, but for the system and the processes that make the outcome inevitable.

CTO KPI → processes → team → outcome

The outcome is a consequence of a working system, not the subject of a monthly evaluation. The QA lead doesn't own "zero bugs in production" — they own the regression suite being current, a verdict existing before a release, and defects that escaped to production being reviewed so they close the hole in the system. The infra lead doesn't own "zero incidents" — they own alerts being meaningful, runbooks existing, and MTTR going down.

And this is the answer to the question "how do I make everyone work on product quality rather than on the CTO's KPI": don't translate the outcome downward — ask about the health of the system that produces that outcome. You can "work on" an outcome number in a report; you can't do that to a working system — it either exists or it doesn't.

The evaluation is then phrased through the system as well, in three levels:

Good — the system works and is developing; the outcome is a stable consequence of it.
Satisfactory — the system exists but has gaps: coverage is incomplete, it runs irregularly.
Unsatisfactory — there is no system; the work is reactive, held together by manual labor and heroics.

Notice: "unsatisfactory" is not "bad numbers," it's "no system." The difference is fundamental — and it takes the fear of the stick away.

The construction: standing KPIs plus KPIs of the month

The second finding that made life much easier was to split KPIs into two kinds:

Standing (S) — the hygiene of the role: the same bar every month. Monitoring covers the services, the regression suite is current, RCAs get written for incidents, releases go through a verdict. They don't change, they aren't renegotiated — they simply have to be met.
KPIs of the month (M) — three or four goals set for the month: close a specific piece of tech debt, raise feature coverage, lower MTTR, run a Game Day. They live for one cycle and are set anew at the monthly review.

The standing ones give stability ("what normal means"), the monthly ones give movement ("what we're improving right now"). Without S, monthly goals turn into firefighting; without M, standing ones turn into stagnation.

Metrics: ground them in facts, not in wishes

The selection rules I arrived at:

A metric has to be readable from a live system — from monitoring, the tracker, the app store. A metric the lead counts by hand at the end of the month will be dead within a quarter.
Thresholds are set from facts, not from dreams. Before writing down a response-time threshold, I pulled the real percentiles out of monitoring. It turned out that almost all the services were already inside the corridor and one stood out — and the threshold became honest: "pull up the outlier," not "everyone improve everything." A threshold taken from thin air demotivates worse than no threshold at all.
Few metrics. Three or four standing plus three or four monthly per team is the ceiling. Past that, the evaluation system costs more than it's worth.
Examples of "system → its metrics" pairs by domain: QA — the currency of the regression suite and the trend of defects escaping to production; infrastructure — recovery time and the share of alerts that actually get a response (an alert everyone ignores does more harm than no alert at all); backend — SLOs and error-budget burn; the mobile client — crash-free and ANR, whose thresholds the app store sets for you.

The stabilization pattern: when the goal is far away

The most common reason KPIs "don't land": the goal is objectively unreachable this month. Between the current availability level and the target one there isn't a step but several quarters of work. Making the target level the monthly bar means condemning the lead to an "unsatisfactory" — and to cynicism.

The pattern that works is burn-down plus a ladder toward the horizon:

the absolute goal is a reference point, not a monthly bar: it's honestly written down as the horizon;
the monthly bar is a step: noticeably reduce the number of errors, close their main source, climb one rung of the ladder you laid out from the current level to the horizon;
what gets evaluated is the step: taken — "good," even if the horizon is still far away.

This is also the answer to "how do we agree": the conversation with the lead stops being a haggle about the unreachable and becomes planning for the next rung.

How to present it and how to agree

The procedural part is short, but everything depends on it:

Warn in advance. The leads knew from ordinary conversations, long before the first card, that KPIs were being prepared and that we'd sit down to discuss them soon. It's a cheap step that removes half the tension: the system doesn't fall on them as a surprise, it arrives at an expected moment.
Don't hand down a finished thing. The frame (S/M, the levels, the pattern) is brought by the CTO; filling the card in is the subject of a conversation with the lead. Thresholds are discussed from the actual numbers, out loud, one on one. This is exactly the spoken expectations, in both directions — the lead also says what they need in order for the system to work.
The CTO sets the goals of the month — the final word is theirs, this isn't a democracy; but goals set without a conversation don't work.
A monthly review — by the card, half an hour: how the standing ones are doing, what happened to the goals of the month, what we set for the next one. A red metric is grounds for the question "what's in the way," not for a reprimand. One or two cycles in that mode and the fear of the stick goes away.
Be careful with money. Tying this to a bonus from the first month breaks the whole construction: a metric that has become money stops being a measurement instantly — that's the same Goodhart from the opening, only in his fastest form. First a quarter or two of simply keeping the rhythm and building trust in the numbers.

What came out of it

Not once did I hear a flat "no." That doesn't mean the cards were accepted in silence: every item was worked through one on one — and not once at launch, but continuously, at every review. That is the central point of the process itself: not to hand it down and forget it, but to talk and to agree until the person has their own understanding of why the metric is there. The absence of loud resistance is the result of those conversations, not a sign that nobody cared.

There were corrections along the way, and always of one kind. Either a wording that could be read two ways, or a number that couldn't be reached this month. The first ones we rewrote. The second we replaced with tracking a trend: not "hold the threshold" but "move toward it" — with a check at the next review. Not once did we have to throw a metric out entirely: the argument was always about the threshold, never about the meaning.

What changed noticeably:

ANRs and crashes became the leads' concern, not the CTO's. Exactly what the whole thing was started for: the tracking settled on the teams' side — the lead looks at the numbers and arrives with a plan, instead of waiting for a question from above;
QA stopped needing to be pulled in by hand. Before, the lead had to be prodded both about the release process and about taking part in postmortems. Once the cards existed, the lead and the team started joining in on their own;
responsibility stopped being a matter of guesswork. It became visible who owns what — and that understanding became shared, rather than living "in the CTO's head".

I don't have a case where the scheme wouldn't fit. There is a condition without which it doesn't work: agreeing in a way that leaves everyone understanding the goal. Without that, the cards turn into work for the sake of work — and it won't be the scheme's fault.

Alternatives — and when they're better

An honest section: lead KPIs are not the only instrument.

OKRs — if the task is not "keep the system healthy" but "move toward ambitious goals": inspiring objectives, measurable key results, 70% attainment is normal. Weaker for hygiene, stronger for pushes. Nothing stops you from combining them: the S part as KPIs, the M part in the spirit of OKRs.
Team health checks (a regular self-assessment against a checklist: code, deployment, tests, mood) — softer, without thresholds, good as a quarterly reflection. Not enough if the C-level needs manageability.
Just agreeing on expectations without numbers — works in a small team with strong trust; stops working roughly when there are more than three leads and "keeping it in your head" no longer works.

The choice is simple: the more teams you have and the more expensive failures are, the closer you move to a formal system. But it's always worth starting with the conversation about expectations; KPIs are its written form, not a replacement for it.

Checklist

⬜ The lead owns the system, not the outcome; the outcome is a consequence

⬜ The frame: standing KPIs (hygiene) plus three or four goals of the month; a monthly review

⬜ Three levels of evaluation, through the maturity of the system, not through "red numbers"

⬜ Metrics are read from live systems, thresholds are grounded in facts

⬜ A distant goal is the horizon; the monthly bar is a step (burn-down / one rung)

⬜ The frame is brought by the CTO, the content is a conversation with the lead from actual numbers

⬜ A red metric is the question "what's in the way," not a reprimand

⬜ A number that's unreachable this month is replaced by a trend, not by an "unsatisfactory"

⬜ Don't tie money to the metrics until the system has lived through a couple of quarters

People: Bus Factor, Expectations and Day One

Mansur Fattakhov — Fri, 18 Sep 2026 11:20:01 +0000

Part of the "How to bring order to a project" series — the people track. It comes after infrastructure and architecture not because it matters less, but the opposite: it's the most important and the slowest track. A config can be moved into git in a day; trust, habits and the understanding of "who owns what" take months to rebuild.

Let me repeat the track's thesis from the hub, because it's the main one: code with a bus factor of 1 is not an asset, it's a hostage. And the other way around: a team where knowledge flows freely can fix any code.

Step 1. Bus factor: find the hostages

The source material is the "who knows what" map from the state snapshot. If the "who knows it" column shows the same name for five services, you have five hostages and one irreplaceable person — who, by the way, isn't enjoying it either: they can't take a vacation or move to new work.

The treatment is three moves, in order of rising cost:

Documentation on touch — the cheapest: figured out a piece — leave a passport or a note (this is already built into the architecture track)
Pairing on tasks — a second person on tasks in a "solo" zone; slower, but the knowledge transfers live
Ownership rotation — the most expensive, for critical zones: a service officially gets a second owner

Don't try to push the bus factor to two everywhere — life is too short. Prioritize by the risk map: first what is both critical and lives in one head.

Step 2. Ownership: assemble from facts, don't draw it

The classic mistake is to sit down and draw a beautiful responsibility matrix the way it "should be." A matrix like that dies within a week, because it doesn't match reality.

The working recipe is to assemble it from facts: pull half a year of closed tracker tasks and the git history of the repositories, and see who actually does the work and who actually decides. That's how our RACI matrix across all services was born: for each one — who executes (R), who's accountable for the result (A), who's consulted (C), who's informed (I). Not "as designed" but "as is" — with all the uncomfortable discoveries, like a service whose executor sits in one team while accountability sits nowhere in particular.

From there the matrix works both ways:

a mismatch of "one does it, another answers for it" becomes a conversation and a decision instead of a silent conflict;
every service gets a name next to it — and that name comes from reality.

Updates happen on touch, like everything in this series: the owner changed — the row changes in the same merge request as the passport.

Step 3. Delegate — and spell out expectations

The most personal part of the track. When you're bringing order to a project, it's easy to fall into a trap: you know how to do it right, so you do everything important yourself. The outcome is predictable — you become the very bus factor of 1 you're fighting.

You have to delegate. But delegation has an entry price that often goes unpaid: spelled-out expectations. A familiar scenario: you hand over a task, the person works for a month, brings the result — and you expected something else. Nobody's guilty: you didn't say, they didn't ask, both filled the gaps with assumptions. A month burned, trust dented on both sides.

What to spell out at handover — four things, literally as a list:

The result: what "done" looks like — not the process, the end state. Better in writing, two sentences.
The deadline and checkpoints: not "show me in a month," but "in a week we look at the direction." An early checkpoint is cheap insurance against "I expected something else": turning the work around after a week costs days; after a month it costs a month.
The boundaries of freedom: what the person decides on their own, and what they bring to you. Without this they either ping you about everything, or silently make decisions at your level.
The why: the task's context. A person who knows "why" makes the right micro-decisions where the instructions are silent.

And the symmetric rule: expectations are spelled out in both directions. Ask what the person needs for the task to move — access, context, uninterrupted time. Half of the "didn't get it done" cases turn out, on review, to be "waited three weeks for access and was too shy to nag."

Step 4. Onboarding and one-on-ones

Onboarding is not "handed over a laptop and added to the chats." Two metrics worth measuring it by:

environment in a day — by the end of day one the person runs the project locally;
a production task in a week — within the first week they ship something real, however tiny.

For that to work, the onboarding documents must exist before the hire, and there are exactly three levels — no more needed:

company — what the product is, which teams exist, who leads them, where to look;
engineering — how we work: a task's lifecycle from discussion to deploy, the standards, the decision process;
glossary — the domain dictionary: internal terms and abbreviations "everybody just knows."

And separately — one-on-ones for the early period. Regular short private meetings: weekly for the first month or two, less often later. Not a status update (the tracker exists for that), but "how is it going for you": what's confusing, what's in the way, what surprised you. In their first weeks a fresh person sees your oddities better than any veteran — then they get used to it and that vision is gone. One-on-ones are how you collect it in time. They're also the natural place to re-sync the expectations from step 3 — while a mismatch costs days, not months.

Step 5. Hiring: an honest footnote

Hiring gets one step in this plan, and a short one — not because the topic is small, but because it's a separate big one. The market is full of people; picking the one who will strengthen your particular team is a discipline of its own that doesn't fit into a section.

What does fit is one rule that costs little and gives a lot: write the job post from the project's real tasks, not from a list of technologies. Not "Python, Docker, K8s, 5 years of experience," but "here are three tasks you'll work on in your first quarter." A text like that filters by itself: the people who respond are the ones interested in the tasks, not the ones who matched the keywords. And the interview gets a ready-made script — you talk about those three tasks.

The rest — the funnel, evaluation, probation — deserves its own article; it's in the backlog.

What came out of it

The irreplaceable stopped being irreplaceable — and breathed easier themselves
Every service has a name from reality, not from the org chart
Delegated tasks arrive at the intended result — because the intended result was spelled out up front
A new person is useful in their first week, and their fresh perspective is collected, not lost
Job posts answer "what will I actually do here," and that's who applies

The track's checklist

⬜ Bus factor from the snapshot map; treatment by cost: docs → pairing → rotation

⬜ Responsibility matrix assembled from facts (tracker + git), not drawn

⬜ Every service has a name; "does vs. answers" mismatches resolved

⬜ Delegation = result + checkpoints + boundaries of freedom + the "why"

⬜ Expectations spelled out both ways, in writing

⬜ Onboarding: environment in a day, a production task in a week

⬜ Three documents: company / engineering / glossary — existing before the hire

⬜ One-on-ones early on: weekly for the first months, not status but "how is it going"

⬜ Job posts — from real tasks, not from a list of technologies

The other tracks of the series are on the map.

Originally published at mind.mansur.expert.

Architecture: Write It Down Before Rewriting

Mansur Fattakhov — Wed, 16 Sep 2026 07:13:17 +0000

Part of the "How to bring order to a project" series — the architecture track. Same context: a growing gamedev project, about twenty services on different stacks, an architecture that "grew historically." Nobody designed it badly — nobody designed it at all: services appeared as needs arose, boundaries were drawn by circumstance, decisions lived in chats and heads.

That's a normal phase — up to a point. The signals that the point has arrived:

incidents increasingly happen at the seams between services, not inside them;
every change starts with an argument about "whose area is this anyway";
estimates keep growing, because "we need to be careful there, we don't know what we might break";
the only answer to "why is it built this way?" is "historically."

The track's main thesis: order in architecture starts not with rewriting, but with writing down. Half of the "architecture problems" dissolve once the system is honestly described as it is. The other half turns from "scary to touch" into tasks with a price and a plan.

Step 1. Service passports

The first tool is boring to the point of indecency: every service gets a passport. One page:

# <service>
Why it exists: one sentence.
Area of responsibility: what it does.
Boundaries: what it does NOT do, even when "it would be convenient here."
Stack, port, status (production / deprecated / experiment).
Owner: a name, not "the team in general."
Operations: how it deploys, where the configs are, where it logs.

Three rules that keep passports alive instead of becoming a wiki graveyard:

The owner writes it, not the architect. A passport is "I'm responsible for this and here's how it works," not "management told me to document it."
Half an hour, no more. A passport is not design documentation. If it takes a day, you're writing something else.
Updated on touch. Changed the boundaries or the deploy method — update the passport in the same merge request. Dedicated "documentation days" don't exist; they don't work.

Filling in the passports is diagnostics in itself: you keep catching moments of "wait, why is this even here?" — and those are the right questions, there was just nobody to ask them before. A couple of our services honestly moved to deprecated status during passportization — it turned out habit was the only thing keeping them alive.

Step 2. The system map

The passports add up to a map — two artifacts:

The registry — a table of all services: type, stack, status, owner. Boring, but this is where you see that there aren't "about fifteen" services, but exactly nineteen, three of which are alive only in legends.

The call graph — in text, per service: whom it calls, over which protocol, and why. For example:

shop → accounts service    HTTP REST   ownership check at purchase
shop → game API            HTTP REST   progress levels (callback)
shop ← payment webhooks    HTTPS       events from payment providers

No pretty arrows at the start — a table is enough. And it has a property a slide-deck diagram never will: it lives in git next to the code, so it can be reviewed and diffed. Changed an integration — the map's row changes in the same merge request; a mismatch is caught in review, not a year later.

The rule is the same as in the state snapshot: trust reality, not documentation. A map that shows the desired state is worse than no map — people make wrong decisions with full confidence.

Step 3. Boundaries: where a service ends

The most expensive architecture bugs live not inside services but between them — where two services consider the same thing "theirs." The signs of broken boundaries are recognizable:

two services write to the same database table;
the same business logic is duplicated in two places "for speed";
a service reaches into a neighbor's internals past its API, "because it's simpler";
the first half hour of every incident goes into figuring out whose area it is.

Our classic was the payment flow: historically payments started in the accounts service, then the shop grew its own purchases and webhooks — and the logic sprawled across two homes. Every payment incident opened with archaeology: which of the two services was supposed to handle this event? Duplicated handlers masked the problem right until the day they both handled the same event.

The cure is a boundaries document. For every service, explicitly:

inside: what the service is solely responsible for
outside: what's adjacent but belongs to someone else (and to whom exactly)
interface: how others interact with it — and that's the only door

The order of the fix matters: document first, code second. First the team agrees on paper where payment events live (in our case — webhooks in the shop, accounts only read status via API), and only then the code is brought to the agreement. The reverse doesn't work: code without an agreement sprawls back within a quarter.

The goal isn't beauty but two practical properties: no overlaps (every entity has one owner) and growth control (a new feature first gets an address — which service it goes to and why — instead of settling wherever).

Step 4. Decisions on paper

Architecture "in heads" dies when people leave. Two document genres stuck with us, and the difference between them matters:

RFC — before a change. Half a page, by template:

Problem: what hurts, in numbers or incidents.
Options: 2–3, including "do nothing."
Decision: which one and why.
Cost: work, risks, what about rollback.

The lifecycle is simple: Draft → Review → Approved/Rejected → Implemented. A new RFC is automatically announced in the team channel — the review is open, and objections at the Draft stage cost one comment instead of a production rework. A live example: the proposal to replace direct calls between game services with a message bus went through three review iterations — and the final design (with a journal and delivery guarantees) had almost nothing in common with the first draft. All those iterations happened before the first line of code.

ADRs and research — around a decision. A short record: what was decided, why, what was considered and rejected. Plus research for specific risks — "how to back up the analytics database," "the plan for moving data off the game host." The difference from an RFC: nothing is being proposed here; knowledge is being recorded — so the same research doesn't get redone a year later.

And one rule on top of both genres: before any architecture work — check the index first. Don't reinvent what's already decided; don't start a migration if there's an open document about it.

Step 5. Tech debt with a price tag

Everyone has a tech debt list; it only works with two extra columns:

Debt	What it blocks	Cost to fix
Shop and accounts share payment logic	every payment incident ×2 in time; new providers go into two places	~3 weeks: boundaries + moving the webhooks
No backups for the analytics DB	losing all analytics on a disk failure	research done, ~1 week to implement

"Blocks" is the main column. Debt that blocks nothing is not debt — it's a trait of the system; cross it out and stop carrying it. Debt that makes every feature in a service cost ×2 is the first candidate for work — and now that's visible not only to the tech lead, but to the product manager who sets priorities.

The list's sources are the state snapshot and incident reviews. Debt is paid down two ways: small — by the touch rule (touching a service — leave it cleaner), large — through an RFC, like any architecture change. Once a quarter the list is revised: some items closed by touch, some stopped blocking, some grew into an RFC.

How not to slide into bureaucracy

The honest risk of this track: getting carried away and demanding an RFC for renaming a variable. The antidote is explicit thresholds. For us, an RFC is mandatory when a change:

moves a boundary between services (or creates a new service);
touches data: a shared DB schema, a migration, a storage change;
touches money: payment flows, billing;
is irreversible or expensive to roll back.

Everything else is a regular merge request with a regular review. And the second antidote: all documents live next to the code or in one wiki with a single index, get updated on touch, and fit on a page. A document nobody can find or finish reading — that's what bureaucracy is.

What it buys you: evolution without revolution

With the map, the boundaries and the decision process in place, big rebuilds stop being leaps of faith. From the same project's practice — three rebuilds, all through RFCs and along the boundaries:

The most loaded service was rewritten in another language — entirely behind its own boundary. The API contract was pinned in the passport and the boundaries document, so "rewrite the internals" became a local task: the other seventeen services noticed nothing. Without a pinned contract this would have been "rewriting the system."
The shop was extracted from the "main API" that did everything — first the boundaries document got the line "purchases, cases, storefront — a separate service," then the code moved along that boundary. Extraction by document rather than intuition is also a ready checklist: what moves, what stays, where the interface is.
Data was separated from compute onto different hosts — following a research document with a migration plan, in stages, with rollback points. Not "overnight on Saturday," but a boring sequence of steps, each of which could be stopped.

None of these rebuilds was a "big rewrite." Each was a local operation with a clear price and a rollback. That's the point of the track: architectural order is not a pretty diagram on the wall — it's cheap changes. The diagram is a by-product.

The track's checklist

⬜ A passport for every service: why, area, boundaries, owner, operations

⬜ Owners write the passports; updates on touch, in the same merge request

⬜ Service registry + call graph in text, in git; verified against reality

⬜ Boundaries document: inside / outside / interface; one owner per entity

⬜ Fixing boundaries: agreement on paper first, code second

⬜ RFC before a change (problem → options → decision → cost); open review in the channel

⬜ Explicit RFC thresholds: boundaries, data, money, irreversibility — the rest is a regular MR

⬜ ADRs and research: record knowledge, "index first"

⬜ Tech debt with "what it blocks" and "cost" columns; quarterly revision

⬜ Small debt — by touch; large — through an RFC

The other tracks of the series are on the map.

Originally published at mind.mansur.expert.