DEV Community: Mansur Fattakhov

Yes, I Vibecode

Mansur Fattakhov — Sun, 27 Sep 2026 15:40:42 +0000

Only a year ago, putting together a working application from a single description in plain language was a party trick people bragged about on Twitter. Today that is how internal tools get built in teams that used to wait months for a slot in a developer's queue, and nobody finds it remarkable any more. Vibecoding has stopped being a sideshow and become something anyone can simply do — much the way building a website without knowing HTML once stopped being a miracle.

Along with that, a stock reply has appeared in developers' conversations, and I hear it constantly now: "vibecoding is not coding." It is delivered in every possible tone, from the condescending to the openly angry, but the meaning is always the same: what those people are doing does not count. I thought that way myself for quite a while, and that is exactly why I want to work out why it is a bad reaction — not because it is factually wrong, but because it misses the target and spoils the person who says it.

Where the irritation comes from

It is worth starting with the fact that the irritation is entirely understandable. Behind the person feeling it lie years spent on things nobody sees from the outside: algorithms that had to be learned twice, nights spent debugging someone else's code without a single line of documentation, postmortems in which it turned out that you were the cause yourself, and the slowly collected bruises out of which professional instinct is eventually made. And right next to that, someone describes a task in the words they would use with their grandmother, and gets the result in one evening.

It feels as though the part that was hard is precisely the part being devalued. As though years of experience had collapsed into an ability to phrase things neatly in plain language. This is not greed and not snobbery — it is an ordinary human jealousy over one's own labour, and it would be strange not to feel it. The problem is not the feeling but the conclusion drawn from it.

Honestly: I thought the same

At first I caught myself on exactly that thought — nonsense, not serious, toys for people who do not want to learn properly. I watched a couple of demos, looked inside the resulting code, saw things there I would not let into prod even at the point of a deadline, and calmed down: not a competitor.

The mistake was not in my assessment of the code — the code really was mediocre. The mistake was that I was comparing the wrong things with each other: someone else's result after one evening against my own result after months of work. The comparison that would have made sense sounds different — what I myself can do with this tool, and how that will differ from what I do without it. The moment I put the question in that form, there was no one left to argue with.

The tool does not level, it amplifies

It turned out that the better a person understands how systems are built, the more they get out of code generation. That contradicts the first impression — you would think a tool that writes code for you ought to level out the strong and the weak. It does not level anything out; it multiplies what is already there.

The reason is simple: an engineer's work never did come down to typing text. It begins long before the first line — with framing the problem, where you have to formulate not only what the result should be, but also what happens when everything goes wrong: what we do on a timeout, what we do when two requests race each other, where the load limit lies beyond which the solution stops working. Half the problems of the future task are visible right here, and they are visible to the person who has already stepped on them.

Next you need context the model cannot know in principle: how this particular service is put together, why there is a queue here rather than a direct call, which decisions were already taken and written down a year ago and why they should not be reopened along the way. Then the resulting code has to be read with the eyes of the person who will answer for it in prod — which means looking not for typos but for the quiet places: an unhandled timeout, idempotency lost somewhere along the road, a query that works beautifully on a hundred test rows and falls over on a real million. And finally all of it has to be rolled out, the metrics watched, rolled back if it went wrong, and answered for — that part is delegated to absolutely no one and is covered by no tool at all.

Generation covers the middle: the actual writing of the code, the very part that was already the fastest for an experienced person. The edges stay where they were, and it is in them that everything an engineer is paid for lives.

Two different occupations under one name

This, it seems to me, is where the whole argument grows from: one word is used for two different things, and then people are surprised that they cannot come to an agreement.

The first is vibecoding as entertainment and as a way into the profession. Someone puts together something of their own, takes pleasure in the fact that what they had in mind works, and sees a result they would never in their life have seen without this tool. For most of them it will stay an evening pastime, and there is absolutely nothing wrong with that. More than that, it is through such trifles that new people come into the industry — and they come several years earlier than they would have come through a textbook on algorithms, if they ever arrived at all.

The second is an engineer who uses generation professionally. The tool is exactly the same, but the tasks are of a different scale and carry a different cost of error: data migrations that cannot be replayed, integrations with someone else's payment systems, rewriting a service that is holding load right now, infrastructure you answer for with night shifts. This is no longer vibecoding in the sense in which the word is usually used — it is engineering work in which generation stands in the same row as the debugger and the profiler, and just as surely decides nothing on its own.

The difference, in short, is not in the tool but in what happens before it and after it.

About the musician at his friends' gig

When an engineer, in reply to "but you're vibecoding," starts explaining that he is not vibecoding at all but doing real, difficult work, he is arguing about a word — and from the outside that looks not like an argument but like a defence of status, whatever he may actually have meant.

I like the comparison with music here. Picture a musician from a good orchestra who has come to a concert by his friends' band: they got together six months ago, they play in a garage, they miss half the notes, the drummer rushes. And there he is all evening, wincing and telling everyone around him how bad this is and how much he dislikes it. Formally he is absolutely right — they really do play worse than he does. In human terms he has simply ruined the evening for people who took nothing away from him.

The people putting together their first application from a description today are the friends in the garage. They are not laying claim to a seat in the orchestra and they are not taking it away from anyone. Some of them will come a year from now to learn to play seriously, and it will be precisely because they had a good evening, not because someone explained to them how bad they were.

How to answer it

So when you are told that you are vibecoding, the best thing you can do is not to take offence. Smile and agree: yes, and I do it brilliantly and professionally. After that the conversation either moves into a normal professional channel or ends — and both options are better than an argument about terms.

Along the way I have forbidden myself three things. Not to prove that my work is difficult: if its difficulty is not visible in the result, no words will explain it, and if it is visible, there is nothing left to explain. Not to correct other people's terminology: the phrase "this isn't vibecoding, it's engineering with code generation" has never yet made anyone smarter, but it reliably ends the conversation. And most importantly — not to devalue someone else's interest: a person showing you a thing they put together over the weekend is not waiting for a review, what they need is for someone to notice that they made it at all, and that is worth saying out loud even if inside there is terrifying code and three hardcoded passwords.

The last of these matters more to me than the first two. Interest is the rarest and most fragile resource in our profession: it is put out with a single condescending comment, and lighting it again afterwards almost never works.

What this means for an engineer

The skill does not disappear, it shifts. Value moves out of the speed of typing code — where, frankly, there was never much of it to begin with — and over to where there was always the most of it: understanding the domain, framing a task so that it can be solved at all, seeing the boundaries, and answering for what runs in prod at three in the morning.

That, incidentally, is exactly the same set the team lead is responsible for — there was a separate post about it. And it is exactly what cannot be generated: no model knows what downtime counts as acceptable at your company, why the last migration was rolled back at the final moment, and whom you need to talk to before touching this service.

And how a working everyday tool gets assembled out of generation — projects, goals, routine — I will tell separately, closer to December.

In short

⬜ Vibecoding has gone mainstream; arguing with that is too late and pointless

⬜ The tool does not level, it amplifies: the better you understand systems, the greater the return

⬜ Generation covers the middle of the work, but not the framing, the context, the review or the responsibility

⬜ Entertainment and engineering work share one word — they are different occupations, and both are fine

⬜ If you are told "you're vibecoding," agree and add that you do it professionally

⬜ Do not review someone else's interest: notice it and praise them for having made the thing

KPIs That Don't Turn Into a Stick

Mansur Fattakhov — Sun, 27 Sep 2026 14:42:03 +0000

You now have several teams, each with a lead, and you want the system to run predictably — without diving into every team every day. The first reflex is to "hand out KPIs." And that is exactly the place where it's easy to ruin everything.

It all breaks against one regularity the economist Charles Goodhart formulated half a century ago: once a metric becomes a target, it stops being a good metric. Not because people are dishonest, but because you can almost always work on the number directly — bypassing the thing it was introduced for. Give every lead a share of uptime and you'll get cautious releases and arguments about whose incident it was, not a more reliable product.

What follows is how I built KPIs for team leads on a gamedev project with six teams while working around that trap. The main question there turned out to be not "which metrics do we take" but "what is a lead responsible for in the first place."

This is a spin-off of the "How to bring order to a project" series — this time about management.

How it started

Six teams: the mobile client, the launcher, the game server and the rest. At the C-level we wanted transparency — to see how things were going without climbing into every team daily.

But there was also a concrete trigger, not just manageability in general. In several teams, churn was eating the focus on the things that never look urgent: ANRs and crashes. Nobody worked on them deliberately — not because they couldn't, but because it was nobody's zone. A feature has a deadline and an owner; a stability number has neither. So beyond transparency, the KPIs had to place responsibility: not "who's to blame for the crash," but "whose zone of attention is this at all."

The starting point was the CTO's own KPI. It already existed — put together quickly, but it existed, and that turned out to matter more than the precision of the wording: there was a single outcome everyone worked toward. The mistake would have been to slice it into shares and hand them down — more on that trap below. What was valuable was something else: it set a direction, and from there you had to answer your own question — what is this particular lead responsible for within that direction.

Why the C-level wants this — honestly

The motivation from the top usually sounds like "transparency and manageability," but it's worth unpacking all the way:

Seeing the state of the teams without micromanagement. Not "what is everyone busy with" but "is the system healthy" — and noticing degradation before an incident.
Predictability. Promises to the business rest on something measurable, not on a feeling that "it seems fine."
Scaling yourself. A CTO with six teams physically cannot be the entry point to every decision — lead KPIs are a way to delegate responsibility, not tasks.

And right here is the honest trap you have to see in advance: the easiest way to "decompose the CTO's KPI" is to cut it into pieces and hand them down. The CTO's goal is "uptime and product quality" — so give each lead a share of uptime. It looks logical and works badly: the lead gets a metric they only partly influence, starts optimizing the number instead of the system, and defends themselves with reports instead of doing the work.

What the leads think about it

Leads have their own set of motivations — and fears, and ignoring them is the most expensive option of all:

What a lead gets from honest KPIs:

clarity: "what is expected of me" stops being telepathy — these are literally the expectations spoken out loud from the people track, only at the level of roles;
protection: written criteria are insurance against arbitrary evaluation "by mood" and against rules changed after the fact;
arguments: "we need one more person" sounds weak; "we won't hold the stated recovery time without a second on-call" sounds strong.

What a lead is afraid of:

that the metrics will become a stick: any red number is grounds for an inquest;
that they'll have to work "for the number": producing reporting instead of value;
that the goals will be set above their head — from a world where their teams don't exist.

Everything below is, in essence, an answer to those fears.

The key turn: a lead owns the system, not the outcome

The central decision everything rests on: a lead is responsible not for the final business metric, but for the system and the processes that make the outcome inevitable.

CTO KPI → processes → team → outcome

The outcome is a consequence of a working system, not the subject of a monthly evaluation. The QA lead doesn't own "zero bugs in production" — they own the regression suite being current, a verdict existing before a release, and defects that escaped to production being reviewed so they close the hole in the system. The infra lead doesn't own "zero incidents" — they own alerts being meaningful, runbooks existing, and MTTR going down.

And this is the answer to the question "how do I make everyone work on product quality rather than on the CTO's KPI": don't translate the outcome downward — ask about the health of the system that produces that outcome. You can "work on" an outcome number in a report; you can't do that to a working system — it either exists or it doesn't.

The evaluation is then phrased through the system as well, in three levels:

Good — the system works and is developing; the outcome is a stable consequence of it.
Satisfactory — the system exists but has gaps: coverage is incomplete, it runs irregularly.
Unsatisfactory — there is no system; the work is reactive, held together by manual labor and heroics.

Notice: "unsatisfactory" is not "bad numbers," it's "no system." The difference is fundamental — and it takes the fear of the stick away.

The construction: standing KPIs plus KPIs of the month

The second finding that made life much easier was to split KPIs into two kinds:

Standing (S) — the hygiene of the role: the same bar every month. Monitoring covers the services, the regression suite is current, RCAs get written for incidents, releases go through a verdict. They don't change, they aren't renegotiated — they simply have to be met.
KPIs of the month (M) — three or four goals set for the month: close a specific piece of tech debt, raise feature coverage, lower MTTR, run a Game Day. They live for one cycle and are set anew at the monthly review.

The standing ones give stability ("what normal means"), the monthly ones give movement ("what we're improving right now"). Without S, monthly goals turn into firefighting; without M, standing ones turn into stagnation.

Metrics: ground them in facts, not in wishes

The selection rules I arrived at:

A metric has to be readable from a live system — from monitoring, the tracker, the app store. A metric the lead counts by hand at the end of the month will be dead within a quarter.
Thresholds are set from facts, not from dreams. Before writing down a response-time threshold, I pulled the real percentiles out of monitoring. It turned out that almost all the services were already inside the corridor and one stood out — and the threshold became honest: "pull up the outlier," not "everyone improve everything." A threshold taken from thin air demotivates worse than no threshold at all.
Few metrics. Three or four standing plus three or four monthly per team is the ceiling. Past that, the evaluation system costs more than it's worth.
Examples of "system → its metrics" pairs by domain: QA — the currency of the regression suite and the trend of defects escaping to production; infrastructure — recovery time and the share of alerts that actually get a response (an alert everyone ignores does more harm than no alert at all); backend — SLOs and error-budget burn; the mobile client — crash-free and ANR, whose thresholds the app store sets for you.

The stabilization pattern: when the goal is far away

The most common reason KPIs "don't land": the goal is objectively unreachable this month. Between the current availability level and the target one there isn't a step but several quarters of work. Making the target level the monthly bar means condemning the lead to an "unsatisfactory" — and to cynicism.

The pattern that works is burn-down plus a ladder toward the horizon:

the absolute goal is a reference point, not a monthly bar: it's honestly written down as the horizon;
the monthly bar is a step: noticeably reduce the number of errors, close their main source, climb one rung of the ladder you laid out from the current level to the horizon;
what gets evaluated is the step: taken — "good," even if the horizon is still far away.

This is also the answer to "how do we agree": the conversation with the lead stops being a haggle about the unreachable and becomes planning for the next rung.

How to present it and how to agree

The procedural part is short, but everything depends on it:

Warn in advance. The leads knew from ordinary conversations, long before the first card, that KPIs were being prepared and that we'd sit down to discuss them soon. It's a cheap step that removes half the tension: the system doesn't fall on them as a surprise, it arrives at an expected moment.
Don't hand down a finished thing. The frame (S/M, the levels, the pattern) is brought by the CTO; filling the card in is the subject of a conversation with the lead. Thresholds are discussed from the actual numbers, out loud, one on one. This is exactly the spoken expectations, in both directions — the lead also says what they need in order for the system to work.
The CTO sets the goals of the month — the final word is theirs, this isn't a democracy; but goals set without a conversation don't work.
A monthly review — by the card, half an hour: how the standing ones are doing, what happened to the goals of the month, what we set for the next one. A red metric is grounds for the question "what's in the way," not for a reprimand. One or two cycles in that mode and the fear of the stick goes away.
Be careful with money. Tying this to a bonus from the first month breaks the whole construction: a metric that has become money stops being a measurement instantly — that's the same Goodhart from the opening, only in his fastest form. First a quarter or two of simply keeping the rhythm and building trust in the numbers.

What came out of it

Not once did I hear a flat "no." That doesn't mean the cards were accepted in silence: every item was worked through one on one — and not once at launch, but continuously, at every review. That is the central point of the process itself: not to hand it down and forget it, but to talk and to agree until the person has their own understanding of why the metric is there. The absence of loud resistance is the result of those conversations, not a sign that nobody cared.

There were corrections along the way, and always of one kind. Either a wording that could be read two ways, or a number that couldn't be reached this month. The first ones we rewrote. The second we replaced with tracking a trend: not "hold the threshold" but "move toward it" — with a check at the next review. Not once did we have to throw a metric out entirely: the argument was always about the threshold, never about the meaning.

What changed noticeably:

ANRs and crashes became the leads' concern, not the CTO's. Exactly what the whole thing was started for: the tracking settled on the teams' side — the lead looks at the numbers and arrives with a plan, instead of waiting for a question from above;
QA stopped needing to be pulled in by hand. Before, the lead had to be prodded both about the release process and about taking part in postmortems. Once the cards existed, the lead and the team started joining in on their own;
responsibility stopped being a matter of guesswork. It became visible who owns what — and that understanding became shared, rather than living "in the CTO's head".

I don't have a case where the scheme wouldn't fit. There is a condition without which it doesn't work: agreeing in a way that leaves everyone understanding the goal. Without that, the cards turn into work for the sake of work — and it won't be the scheme's fault.

Alternatives — and when they're better

An honest section: lead KPIs are not the only instrument.

OKRs — if the task is not "keep the system healthy" but "move toward ambitious goals": inspiring objectives, measurable key results, 70% attainment is normal. Weaker for hygiene, stronger for pushes. Nothing stops you from combining them: the S part as KPIs, the M part in the spirit of OKRs.
Team health checks (a regular self-assessment against a checklist: code, deployment, tests, mood) — softer, without thresholds, good as a quarterly reflection. Not enough if the C-level needs manageability.
Just agreeing on expectations without numbers — works in a small team with strong trust; stops working roughly when there are more than three leads and "keeping it in your head" no longer works.

The choice is simple: the more teams you have and the more expensive failures are, the closer you move to a formal system. But it's always worth starting with the conversation about expectations; KPIs are its written form, not a replacement for it.

Checklist

⬜ The lead owns the system, not the outcome; the outcome is a consequence

⬜ The frame: standing KPIs (hygiene) plus three or four goals of the month; a monthly review

⬜ Three levels of evaluation, through the maturity of the system, not through "red numbers"

⬜ Metrics are read from live systems, thresholds are grounded in facts

⬜ A distant goal is the horizon; the monthly bar is a step (burn-down / one rung)

⬜ The frame is brought by the CTO, the content is a conversation with the lead from actual numbers

⬜ A red metric is the question "what's in the way," not a reprimand

⬜ A number that's unreachable this month is replaced by a trend, not by an "unsatisfactory"

⬜ Don't tie money to the metrics until the system has lived through a couple of quarters

People: Bus Factor, Expectations and Day One

Mansur Fattakhov — Fri, 18 Sep 2026 11:20:01 +0000

Part of the "How to bring order to a project" series — the people track. It comes after infrastructure and architecture not because it matters less, but the opposite: it's the most important and the slowest track. A config can be moved into git in a day; trust, habits and the understanding of "who owns what" take months to rebuild.

Let me repeat the track's thesis from the hub, because it's the main one: code with a bus factor of 1 is not an asset, it's a hostage. And the other way around: a team where knowledge flows freely can fix any code.

Step 1. Bus factor: find the hostages

The source material is the "who knows what" map from the state snapshot. If the "who knows it" column shows the same name for five services, you have five hostages and one irreplaceable person — who, by the way, isn't enjoying it either: they can't take a vacation or move to new work.

The treatment is three moves, in order of rising cost:

Documentation on touch — the cheapest: figured out a piece — leave a passport or a note (this is already built into the architecture track)
Pairing on tasks — a second person on tasks in a "solo" zone; slower, but the knowledge transfers live
Ownership rotation — the most expensive, for critical zones: a service officially gets a second owner

Don't try to push the bus factor to two everywhere — life is too short. Prioritize by the risk map: first what is both critical and lives in one head.

Step 2. Ownership: assemble from facts, don't draw it

The classic mistake is to sit down and draw a beautiful responsibility matrix the way it "should be." A matrix like that dies within a week, because it doesn't match reality.

The working recipe is to assemble it from facts: pull half a year of closed tracker tasks and the git history of the repositories, and see who actually does the work and who actually decides. That's how our RACI matrix across all services was born: for each one — who executes (R), who's accountable for the result (A), who's consulted (C), who's informed (I). Not "as designed" but "as is" — with all the uncomfortable discoveries, like a service whose executor sits in one team while accountability sits nowhere in particular.

From there the matrix works both ways:

a mismatch of "one does it, another answers for it" becomes a conversation and a decision instead of a silent conflict;
every service gets a name next to it — and that name comes from reality.

Updates happen on touch, like everything in this series: the owner changed — the row changes in the same merge request as the passport.

Step 3. Delegate — and spell out expectations

The most personal part of the track. When you're bringing order to a project, it's easy to fall into a trap: you know how to do it right, so you do everything important yourself. The outcome is predictable — you become the very bus factor of 1 you're fighting.

You have to delegate. But delegation has an entry price that often goes unpaid: spelled-out expectations. A familiar scenario: you hand over a task, the person works for a month, brings the result — and you expected something else. Nobody's guilty: you didn't say, they didn't ask, both filled the gaps with assumptions. A month burned, trust dented on both sides.

What to spell out at handover — four things, literally as a list:

The result: what "done" looks like — not the process, the end state. Better in writing, two sentences.
The deadline and checkpoints: not "show me in a month," but "in a week we look at the direction." An early checkpoint is cheap insurance against "I expected something else": turning the work around after a week costs days; after a month it costs a month.
The boundaries of freedom: what the person decides on their own, and what they bring to you. Without this they either ping you about everything, or silently make decisions at your level.
The why: the task's context. A person who knows "why" makes the right micro-decisions where the instructions are silent.

And the symmetric rule: expectations are spelled out in both directions. Ask what the person needs for the task to move — access, context, uninterrupted time. Half of the "didn't get it done" cases turn out, on review, to be "waited three weeks for access and was too shy to nag."

Step 4. Onboarding and one-on-ones

Onboarding is not "handed over a laptop and added to the chats." Two metrics worth measuring it by:

environment in a day — by the end of day one the person runs the project locally;
a production task in a week — within the first week they ship something real, however tiny.

For that to work, the onboarding documents must exist before the hire, and there are exactly three levels — no more needed:

company — what the product is, which teams exist, who leads them, where to look;
engineering — how we work: a task's lifecycle from discussion to deploy, the standards, the decision process;
glossary — the domain dictionary: internal terms and abbreviations "everybody just knows."

And separately — one-on-ones for the early period. Regular short private meetings: weekly for the first month or two, less often later. Not a status update (the tracker exists for that), but "how is it going for you": what's confusing, what's in the way, what surprised you. In their first weeks a fresh person sees your oddities better than any veteran — then they get used to it and that vision is gone. One-on-ones are how you collect it in time. They're also the natural place to re-sync the expectations from step 3 — while a mismatch costs days, not months.

Step 5. Hiring: an honest footnote

Hiring gets one step in this plan, and a short one — not because the topic is small, but because it's a separate big one. The market is full of people; picking the one who will strengthen your particular team is a discipline of its own that doesn't fit into a section.

What does fit is one rule that costs little and gives a lot: write the job post from the project's real tasks, not from a list of technologies. Not "Python, Docker, K8s, 5 years of experience," but "here are three tasks you'll work on in your first quarter." A text like that filters by itself: the people who respond are the ones interested in the tasks, not the ones who matched the keywords. And the interview gets a ready-made script — you talk about those three tasks.

The rest — the funnel, evaluation, probation — deserves its own article; it's in the backlog.

What came out of it

The irreplaceable stopped being irreplaceable — and breathed easier themselves
Every service has a name from reality, not from the org chart
Delegated tasks arrive at the intended result — because the intended result was spelled out up front
A new person is useful in their first week, and their fresh perspective is collected, not lost
Job posts answer "what will I actually do here," and that's who applies

The track's checklist

⬜ Bus factor from the snapshot map; treatment by cost: docs → pairing → rotation

⬜ Responsibility matrix assembled from facts (tracker + git), not drawn

⬜ Every service has a name; "does vs. answers" mismatches resolved

⬜ Delegation = result + checkpoints + boundaries of freedom + the "why"

⬜ Expectations spelled out both ways, in writing

⬜ Onboarding: environment in a day, a production task in a week

⬜ Three documents: company / engineering / glossary — existing before the hire

⬜ One-on-ones early on: weekly for the first months, not status but "how is it going"

⬜ Job posts — from real tasks, not from a list of technologies

The other tracks of the series are on the map.

Originally published at mind.mansur.expert.

Architecture: Write It Down Before Rewriting

Mansur Fattakhov — Wed, 16 Sep 2026 07:13:17 +0000

Part of the "How to bring order to a project" series — the architecture track. Same context: a growing gamedev project, about twenty services on different stacks, an architecture that "grew historically." Nobody designed it badly — nobody designed it at all: services appeared as needs arose, boundaries were drawn by circumstance, decisions lived in chats and heads.

That's a normal phase — up to a point. The signals that the point has arrived:

incidents increasingly happen at the seams between services, not inside them;
every change starts with an argument about "whose area is this anyway";
estimates keep growing, because "we need to be careful there, we don't know what we might break";
the only answer to "why is it built this way?" is "historically."

The track's main thesis: order in architecture starts not with rewriting, but with writing down. Half of the "architecture problems" dissolve once the system is honestly described as it is. The other half turns from "scary to touch" into tasks with a price and a plan.

Step 1. Service passports

The first tool is boring to the point of indecency: every service gets a passport. One page:

# <service>
Why it exists: one sentence.
Area of responsibility: what it does.
Boundaries: what it does NOT do, even when "it would be convenient here."
Stack, port, status (production / deprecated / experiment).
Owner: a name, not "the team in general."
Operations: how it deploys, where the configs are, where it logs.

Three rules that keep passports alive instead of becoming a wiki graveyard:

The owner writes it, not the architect. A passport is "I'm responsible for this and here's how it works," not "management told me to document it."
Half an hour, no more. A passport is not design documentation. If it takes a day, you're writing something else.
Updated on touch. Changed the boundaries or the deploy method — update the passport in the same merge request. Dedicated "documentation days" don't exist; they don't work.

Filling in the passports is diagnostics in itself: you keep catching moments of "wait, why is this even here?" — and those are the right questions, there was just nobody to ask them before. A couple of our services honestly moved to deprecated status during passportization — it turned out habit was the only thing keeping them alive.

Step 2. The system map

The passports add up to a map — two artifacts:

The registry — a table of all services: type, stack, status, owner. Boring, but this is where you see that there aren't "about fifteen" services, but exactly nineteen, three of which are alive only in legends.

The call graph — in text, per service: whom it calls, over which protocol, and why. For example:

shop → accounts service    HTTP REST   ownership check at purchase
shop → game API            HTTP REST   progress levels (callback)
shop ← payment webhooks    HTTPS       events from payment providers

No pretty arrows at the start — a table is enough. And it has a property a slide-deck diagram never will: it lives in git next to the code, so it can be reviewed and diffed. Changed an integration — the map's row changes in the same merge request; a mismatch is caught in review, not a year later.

The rule is the same as in the state snapshot: trust reality, not documentation. A map that shows the desired state is worse than no map — people make wrong decisions with full confidence.

Step 3. Boundaries: where a service ends

The most expensive architecture bugs live not inside services but between them — where two services consider the same thing "theirs." The signs of broken boundaries are recognizable:

two services write to the same database table;
the same business logic is duplicated in two places "for speed";
a service reaches into a neighbor's internals past its API, "because it's simpler";
the first half hour of every incident goes into figuring out whose area it is.

Our classic was the payment flow: historically payments started in the accounts service, then the shop grew its own purchases and webhooks — and the logic sprawled across two homes. Every payment incident opened with archaeology: which of the two services was supposed to handle this event? Duplicated handlers masked the problem right until the day they both handled the same event.

The cure is a boundaries document. For every service, explicitly:

inside: what the service is solely responsible for
outside: what's adjacent but belongs to someone else (and to whom exactly)
interface: how others interact with it — and that's the only door

The order of the fix matters: document first, code second. First the team agrees on paper where payment events live (in our case — webhooks in the shop, accounts only read status via API), and only then the code is brought to the agreement. The reverse doesn't work: code without an agreement sprawls back within a quarter.

The goal isn't beauty but two practical properties: no overlaps (every entity has one owner) and growth control (a new feature first gets an address — which service it goes to and why — instead of settling wherever).

Step 4. Decisions on paper

Architecture "in heads" dies when people leave. Two document genres stuck with us, and the difference between them matters:

RFC — before a change. Half a page, by template:

Problem: what hurts, in numbers or incidents.
Options: 2–3, including "do nothing."
Decision: which one and why.
Cost: work, risks, what about rollback.

The lifecycle is simple: Draft → Review → Approved/Rejected → Implemented. A new RFC is automatically announced in the team channel — the review is open, and objections at the Draft stage cost one comment instead of a production rework. A live example: the proposal to replace direct calls between game services with a message bus went through three review iterations — and the final design (with a journal and delivery guarantees) had almost nothing in common with the first draft. All those iterations happened before the first line of code.

ADRs and research — around a decision. A short record: what was decided, why, what was considered and rejected. Plus research for specific risks — "how to back up the analytics database," "the plan for moving data off the game host." The difference from an RFC: nothing is being proposed here; knowledge is being recorded — so the same research doesn't get redone a year later.

And one rule on top of both genres: before any architecture work — check the index first. Don't reinvent what's already decided; don't start a migration if there's an open document about it.

Step 5. Tech debt with a price tag

Everyone has a tech debt list; it only works with two extra columns:

Debt	What it blocks	Cost to fix
Shop and accounts share payment logic	every payment incident ×2 in time; new providers go into two places	~3 weeks: boundaries + moving the webhooks
No backups for the analytics DB	losing all analytics on a disk failure	research done, ~1 week to implement

"Blocks" is the main column. Debt that blocks nothing is not debt — it's a trait of the system; cross it out and stop carrying it. Debt that makes every feature in a service cost ×2 is the first candidate for work — and now that's visible not only to the tech lead, but to the product manager who sets priorities.

The list's sources are the state snapshot and incident reviews. Debt is paid down two ways: small — by the touch rule (touching a service — leave it cleaner), large — through an RFC, like any architecture change. Once a quarter the list is revised: some items closed by touch, some stopped blocking, some grew into an RFC.

How not to slide into bureaucracy

The honest risk of this track: getting carried away and demanding an RFC for renaming a variable. The antidote is explicit thresholds. For us, an RFC is mandatory when a change:

moves a boundary between services (or creates a new service);
touches data: a shared DB schema, a migration, a storage change;
touches money: payment flows, billing;
is irreversible or expensive to roll back.

Everything else is a regular merge request with a regular review. And the second antidote: all documents live next to the code or in one wiki with a single index, get updated on touch, and fit on a page. A document nobody can find or finish reading — that's what bureaucracy is.

What it buys you: evolution without revolution

With the map, the boundaries and the decision process in place, big rebuilds stop being leaps of faith. From the same project's practice — three rebuilds, all through RFCs and along the boundaries:

The most loaded service was rewritten in another language — entirely behind its own boundary. The API contract was pinned in the passport and the boundaries document, so "rewrite the internals" became a local task: the other seventeen services noticed nothing. Without a pinned contract this would have been "rewriting the system."
The shop was extracted from the "main API" that did everything — first the boundaries document got the line "purchases, cases, storefront — a separate service," then the code moved along that boundary. Extraction by document rather than intuition is also a ready checklist: what moves, what stays, where the interface is.
Data was separated from compute onto different hosts — following a research document with a migration plan, in stages, with rollback points. Not "overnight on Saturday," but a boring sequence of steps, each of which could be stopped.

None of these rebuilds was a "big rewrite." Each was a local operation with a clear price and a rollback. That's the point of the track: architectural order is not a pretty diagram on the wall — it's cheap changes. The diagram is a by-product.

The track's checklist

⬜ A passport for every service: why, area, boundaries, owner, operations

⬜ Owners write the passports; updates on touch, in the same merge request

⬜ Service registry + call graph in text, in git; verified against reality

⬜ Boundaries document: inside / outside / interface; one owner per entity

⬜ Fixing boundaries: agreement on paper first, code second

⬜ RFC before a change (problem → options → decision → cost); open review in the channel

⬜ Explicit RFC thresholds: boundaries, data, money, irreversibility — the rest is a regular MR

⬜ ADRs and research: record knowledge, "index first"

⬜ Tech debt with "what it blocks" and "cost" columns; quarterly revision

⬜ Small debt — by touch; large — through an RFC

The other tracks of the series are on the map.

Originally published at mind.mansur.expert.

Infrastructure and Deployment: Order by Iteration

Mansur Fattakhov — Tue, 15 Sep 2026 07:23:01 +0000

Part of the "How to bring order to a project" series — the infrastructure track. Same starting point: a growing gamedev project, nearly two dozen services on different stacks — Python, Go, C++, a mobile client — and infrastructure that grew faster than it matured. Every service deployed differently, configs lived on servers, and a new service was born by copy-pasting the oldest one.

Attacking this with a "big refactoring" is a road to nowhere: the business won't let you stop shipping features for a month. What works is the iterative approach: small steps, each paying off immediately and breaking nothing.

Step 0. Snapshot and light

Two prerequisites, without which infrastructure changes turn into roulette:

A state snapshot — the service map, who knows what, top risks. How to take one in a week — a separate article.
Observability — logs, metrics, alerts. Also covered. The rule is simple: turn on the lights first, then move the furniture.

Now the actual infrastructure steps, in the order they pay off.

Step 1. CI/CD: templates instead of copy-paste

The classic disease: every service has its own CI config, written in its own era by its own person. You fix the pipeline in one repo — in the other twenty it stays broken.

The recipe that worked for me: a dedicated repository of CI templates. Inside:

core-templates.yml — image builds, test runs, timings
notify-templates.yml — notifications to the team messenger
ready-made pipelines for the typical cases: a basic service, a service with secrets, a cluster deployment

A service plugs it in with one line:

include:
  - project: "infra/ci-templates"
    file: "/pipeline-basic.yml"

And gets the standard conveyor: on a merge request — test image build → tests → candidate build; on the main branch — the production image with a tag and latest. Everything goes through shared templates, so a fix or an improvement in one place rolls out to every service with its next pipeline.

Start with one service — the most frequently deployed one: the template gets polished there, the rest catch up as you touch them.

👉 The side effect matters more than the main one: deployment stops being one person's knowledge. Anyone can press the button in CI — and that's no longer scary, because before the button there are tests and a process that's identical for everyone.

Step 2. Configs into git, servers into code

The second source of chaos: configuration living directly on servers. Edited by hand, versioned nowhere, lost when a server moves.

There are two separate recipes here, and it pays not to mix them:

Static configs — what changes at deploy time: compose files, env templates, environment parameters. They belong in a configuration repository: change → merge request → it's clear who, what and why → rollout through CI.

Runtime configs — what changes the system's behavior on the fly: feature flags, prices, game parameters. These get their own GitOps repository with a schema and validation: values live in YAML, services read them through a small SDK. A product manager changes a parameter with a merge request, not with a ticket to the dev team — and the change history comes for free.

The servers themselves — only through Ansible roles (or any other IaC). The rule is hard: no hand edits on hosts. Not "just for a minute," not "I'll move it into the role later." Every manual edit is a future surprise during a migration or a recovery.

Step 3. Secrets: out of CI, into a vault

A separate pain: secrets smeared across CI variables, env files on servers and personal notes. The recipe I arrived at:

Deploy a secrets manager (Vault, Infisical — the class of tool matters more than the brand) with test/production environments.
CI variables keep exactly five values: the vault address, the machine-identity ID and secret, the project ID and the path. That's it.
Everything else — registry credentials, deploy SSH keys, notification webhooks — the pipeline fetches from the vault itself, per environment.

A technical detail that saved hours of debugging: every secret is written to its own file via base64 -d, byte for byte, with no shell interpretation. Multiline SSH keys and certificates stop breaking on escaping once and for all.

The win isn't only security: rotating a secret is an edit in one place, not archaeology across twenty repositories. And when someone leaves the team, you know exactly what to reissue and where.

Step 4. Deployment: one entrance, different backends

Deployment in the CI templates is standard too, with two backends for different scales:

Simple services live on hosts under docker compose: the deploy job takes the host list and the SSH key from the vault, connects and updates the service. Boring and predictable.
Cluster services ship to Kubernetes via Helm: the repo holds values.yaml + values.production.yaml, the image tag comes from the commit SHA, the job waits for the rollout and logs its progress. Merge requests build a candidate with its own tag — deployable to a test environment with the very same template.

Both roads end the same way: a notification in the team messenger — who deployed what and where. It's cheaper than any "release book": the deploy history assembles itself in the channel.

👉 The general principle: a service has no deployment method of its own. It has parameters (hosts or helm values) — the method is shared. A new deployment method appears in the templates, or it doesn't appear at all.

Step 5. Backups: order isn't there until you can restore

A test that puts a lot into perspective: "the server is dead for good — how many hours until everything runs again?" If the answer starts with "well…" — the track has further to go.

Database backups — on schedule, to a separate location, with an alert on "backup didn't happen"
Recovery is tested by hand at least once a quarter: a backup that has never been restored is a lottery ticket, not a backup
Everything else must be restorable from git: roles bring up the server, CI builds and ships the services, configs arrive from repositories

When this genuinely works, "moving a server" turns from a special operation with a midnight call into a half-day procedure.

Step 6. The reference service template

Once CI is standardized and configs are in git, you can finally pin down "what a proper service looks like." My reference consists of:

Dockerfile — multi-stage, with separate targets for tests and production (the CI templates are built for exactly that)
Repo structure — code, tests, migrations, a README with a two-command local start
JSON logs with a request_id passed through — so the service plugs into centralized logging right away
A healthcheck endpoint — so the orchestrator and monitoring see liveness without magic
The CI templates plugged in — a pipeline out of the box
A service passport — one page: why it exists, its area of responsibility, boundaries, who owns it, how it deploys

The passport is the underrated part. Passports add up to a living map of the system: a service registry with ports, stacks and owners, plus a "who calls whom" graph. With twenty services, that map answers half of the architecture questions before they're asked.

A new service is created from the template in minutes. Old ones are not rewritten en masse: the rule "every touch brings the service closer to the reference" pulls everything alive up to the standard within months — and what's dead gets to die honestly.

What came out of it

Deploying any service is one button in CI, the same process for everyone
Configs and servers are restorable from git — "moving a server" went from special operation to procedure
A new service comes from the template in minutes, with logs, metrics and a pipeline from day one
Secrets rotate in one place; someone leaving the team is a procedure, not a panic
The deploy history assembles itself in the notification channel — who shipped what and where
Infrastructure knowledge stopped living in one head: passports + templates + roles are readable by any team member

The track's checklist

⬜ CI templates in a dedicated repo; the first service plugged in

⬜ Tests are mandatory on merge requests; no deploys bypass CI

⬜ Static configs — in a configuration repository

⬜ Runtime parameters — a GitOps repo with a schema and an SDK

⬜ Servers — only through IaC roles; no hand edits on hosts

⬜ Secrets — in a vault; CI keeps only the connection parameters

⬜ Deployment — shared, from templates (SSH-compose / Helm), with a chat notification

⬜ Database backups on schedule + a quarterly restore test

⬜ Service reference: Dockerfile, structure, JSON logs, healthcheck, passport

⬜ Rule: every touch brings an old service closer to the reference

The next tracks of the series — architecture, people, processes — are on the map.

Originally published at mind.mansur.expert.

Processes: A Living Rhythm with Formal Points

Mansur Fattakhov — Mon, 14 Sep 2026 09:48:48 +0000

The final track of the "How to bring order to a project" series — processes. It came last not because it matters least, but because it leans on everything before it: without observability, status updates degrade into retelling feelings — you can only say "we're fine" while looking at metrics; without reliable deployment, a regular release cadence simply won't happen — every rollout stays manual work everyone wants to postpone.

And straight to the main prejudice. Process is not a boring, faceless instrument of corporate management. A living process changes — it adapts to the team, drops what's useless, grows what's needed: that's how it should be. But for all its liveliness, it defines formal points and artifacts: the task is filed, the decision is written down, the status is spoken out loud. These points pin the work down and keep it from getting lost.

Yes, in the moment it's faster to DM someone than to file a ticket. But the DM is only faster today: a month later that agreement exists nowhere — not in the plan, not in the history, not in the other person's head. A formal point feels like a detour, but over the distance it shortens the path to a delivered result. That's the whole bet: a good process is not more paperwork — it's fewer decisions on the fly and less lost work.

The task lifecycle: one explicit path

Every task in a team should travel one route everyone knows. Ours looked like this:

Discussion
    ↓
Architecture discussion (when the task calls for it)
    ↓
Decision on paper — an RFC or ADR
    ↓
Decision review
    ↓
Planning in the tracker
    ↓
Development: code → merge request → review → deploy
    ↓
Watching production
    ↓
Documentation update

Looks like bureaucracy — in reality, for a small team most steps take minutes: an "architecture discussion" is five minutes at a whiteboard, a "decision on paper" is half a page. And "development" is a single step even though it contains code, MR, review and deploy: once the infrastructure track is done, that's one well-worn conveyor, not four separate events.

The point of the ladder isn't meetings — it's this: no step gets skipped silently. You may consciously decide "no RFC needed here" — you may not fail to notice that a decision was never written down. Two rules from practice:

major architecture changes never skip the RFC step — ever;
work is not done until the documentation is updated. Not "I'll write it later" — not done.

Planning: sprints or kanban is the wrong question

The right question is: what's the nature of your work?

Building a feature with a deadline you can plan out — take sprints: a fixed window, a clear scope, and at the end you can see whether you hit it.
Handling a flow of tasks — support, testing, bugfixes, small improvements — take kanban: tasks stream through columns, throughput matters, not "did we make it by Friday."

Both live peacefully in one team: feature work in sprints, the flow on a kanban board. It's all individual; the tool serves the work, not the other way around. Only two things are universal: a short cycle (a week or two — beyond that plans lie) and a visible backlog — anyone on the team can open it and see what's next.

Status updates and dailies: not optional

The most underrated part of process — and the one where I hold my firmest position: status updates are mandatory, always. Even if the team is two people. Even as plain text in chat. Always.

Without regular sync the same thing happens every time — I've watched it more than once: people don't know about each other's problems, don't know whom to go to, quietly stall for weeks — and everything falls apart. Not because anyone works badly, but because nobody sees the whole picture.

The working mechanics:

A status — always. Short and regular: what I did, what's next, what's blocking me. For a two-person team three lines of text are enough — but every day.
The daily is for discussion, not for reporting to the boss: you hear about a neighbor's problem — you say "I've had something similar, come over."
Problems get solved after, in the right circle. Something half-hour-sized surfaces on the daily — we don't solve it with everyone: we note it, and after the call exactly the two or three people who need it gather. The daily stays short; the problem gets a real deep-dive.

It's the cheapest process of all — ten minutes a day — with the highest return: it's the only one that keeps a finger on the pulse.

Release cadence: predictable and boring

This section is suspiciously short — because releases are a science of their own: windows, frequency, staging, rollbacks, the fear of Friday deploys. It deserves — and will get — a separate article.

For this track one principle is enough: a release should be boring. A clear scheme (mine: candidate tag → staging, release tag → production), identical for every service, with a notification in the channel. If the team's pulse rises before a release, that's not a cadence — it's an arrhythmia, and you fix it in the infrastructure track, not with courage.

Incidents: blameless reviews

When production goes down — and it goes down for everyone — process beats heroism. The minimal skeleton:

pin the timeline: when it started, what coincided (most often — a deploy);
fix first, analyze after — not the other way around;
a review with no blame and one action item: not "we'll be more careful," but a concrete change — an alert, a CI check, a checklist line;
the postmortem goes into a registry; before the next review, first search the history for a similar case.

How to set this up in a small team with one or two people — the minimal artifacts and where to grow from there — will also become a separate article, with a real incident walkthrough.

The meeting audit

The final step of the track is the most enjoyable: open the calendar and honestly answer, for every recurring meeting, what it produces. The rule is simple: a meeting either produces decisions, or syncs people, or dies. "It's historical" is not a third option — it's a diagnosis from the architecture track.

What survives the audit almost always: the daily (it syncs — see above) and incident reviews (they produce decisions). What dies first: the weekly hour-long "all-hands sync" where everyone waits for their minute and stays silent for the other fifty-nine.

The series is complete

With this article, the "How to bring order to a project" map is closed — all six tracks now have their articles:

⬜ → ✅ The state snapshot — a week of fixing nothing

⬜ → ✅ Observability — turn on the lights

⬜ → ✅ Infrastructure and deployment — order by iteration

⬜ → ✅ Architecture — write it down first

⬜ → ✅ People — bus factor and expectations

⬜ → ✅ Processes — you are here

A reminder of what "order has arrived" feels like: deploying is not scary, a new person is useful in their first week, an incident is a procedure rather than a panic, and "why is it like this here?" has a written answer. You'll never reach perfection — it's enough that every week feels a little calmer than the last.

The series is finished; the topics aren't: separate articles on release cadence, small-team incident handling and hiring are coming. Stay around.

Originally published at mind.mansur.expert.

The State Snapshot: Your First Week on a New Project

Mansur Fattakhov — Mon, 14 Sep 2026 07:33:47 +0000

When you join a running project — as a lead, an architect, or part-time — your hands itch to start fixing things right away. There it is, the crooked deploy; the database without indexes; the service nobody has touched since last year. Stop. For the first week you fix nothing at all.

This article covers "Track 0" from the project order map: how to take an honest snapshot of a project's state in one week. I'll show it on a real example — a growing gamedev project I joined to bring order to.

Why a snapshot first, not a fix

Three reasons:

You don't yet know what matters. The first "obvious" problem almost never turns out to be the main one. Fixing it means spending your credit of trust on something secondary.
Any fix without the full picture is a gamble. The system lived for years before you; what looks like a bug may be the prop holding everything up.
The snapshot is a result in itself. A week later you hold something the project has never had: a complete picture. Often it's the first document of its kind in the company's history.

Step 1. Talk to everyone

The first thing I did on the gamedev project was a one-on-one call with every single person: developers, DevOps, support, product. Not a meeting — one-on-one, 30–45 minutes each.

The questions are simple:

What annoys you most about the project?
What's scary to touch, and why?
What would you fix first if it were up to you?
What do you alone know? What happens if you leave for a month?

The discovery that repeats on every project: everybody already knows everything. People carry precise lists of problems in their heads for years — where it hurts, what will fall apart next, why Friday deploys are forbidden. Nobody has ever collected that knowledge in one place. You don't discover problems — you collect them.

The side effect is as valuable as the main one: the team sees that the new person listens first and breaks nothing. It's the cheapest way to earn trust — and it works exactly once, in your first week.

Step 2. The service map

In parallel with the calls — a table. Boring, but honest:

Service	Purpose	Where it lives	Who knows it	How it deploys
…	…	…	…	…

The rule: don't trust the documentation — trust reality. Look at the servers, the cron jobs, what is actually running. On my project a couple of services from the "architecture diagram" turned out to be long dead, while two live ones the diagram knew nothing about surfaced: a script on a host and a bot deployed "just for a minute" a year ago.

In the "who knows it" column you'll keep seeing the same name. That's not a column — that's an alarm.

Step 3. The scary places

The calls and the map produce a special list — the things people describe as "better not touch it." For every item, ask "why" until you hit the bottom:

sometimes the fear hides a real landmine (no tests, no rollback, the only person who knew it has left);
sometimes it's just a legend: it crashed once, scared everyone, and has been avoided ever since.

Separating landmines from legends is half the risk work done. Legends dissolve with a single experiment on staging; landmines go into the top risks.

Step 4. Top 5 risks on one page

The week's finale is one page, written in plain language, no jargon:

If server X dies, the game won't start for anyone, and recovery will take an unknown amount of time, because the backup has never been tested.

Five items like that, sorted by pain. Not fifty — five. Keep the rest in your working notes.

I showed that page to the team before showing it to management. First, they fixed my factual mistakes. Second, the document reached management as "ours," not as "the new guy reporting on everyone." That matters: the snapshot is a tool for working together, not an audit denunciation.

What you have a week later

A service map that matches reality
The scary places list, split into landmines and legends
Top 5 risks the team has signed off on
People's trust — you listened, wrote things down, and broke nothing
Most importantly: a basis for ordering the steps. From here on you fix what actually fires most often, not what caught your eye first

Every other track starts from this snapshot: observability turns the lights on where the map showed darkness, and the infrastructure steps follow the risk list instead of gut feeling.

The week's checklist

⬜ One-on-one calls with everyone — 4 questions, your own notes

⬜ Service map: what / why / where / who knows it / how it deploys

⬜ Verify the map against reality (servers, crons, processes)

⬜ The "scary places" list → split into landmines and legends

⬜ Top 5 risks on one page → show the team → then upward

⬜ Fix nothing. At all. For a week.

Originally published at mind.mansur.expert.

How Developers Can Monitor Production — and Why It Matters

Mansur Fattakhov — Sun, 13 Sep 2026 08:21:47 +0000

When we write code, it often feels like the main thing is to make it work locally. But reality is different: the real life of a service begins not on your laptop, but in production. That’s where it faces load, unpredictable users, and dozens of unexpected situations.

Many developers think of monitoring and logs as “something for DevOps.” Let them look at the graphs and figure out why the service crashed. But the truth is: observability is a developer’s tool first. It helps you quickly understand what went wrong and fix a bug before it turns into a midnight call from support.

In this post I’ll explain:

🧐 why metrics and dashboards are important,
🔧 how developers can actually use them,
🚨 and why without observability you’re basically “flying blind” in production.

What Is Observability (in simple words)

Any service lives in two worlds:

🌍 the code world, where you’re in control with print or breakpoints,
🌍 and the production world, where the system runs under load, with real users and unpredictable inputs.

Observability is how you understand what’s happening inside without looking directly into the code.

The three “pillars” are:

📜 Logs — text outputs (errors, requests, events). Answer: “what happened?”
📊 Metrics — numbers (request counts, response times, memory usage). Answer: “how is it working?”
🧵 Traces — following one request across multiple services. Answer: “why is it working this way?”

Together, they give you a complete picture: find bottlenecks, debug errors, and see how changes affect production.

The Main Tools (and Why These)

There are many monitoring products out there — from Datadog and New Relic to Dynatrace. In practice, most projects I see use the Grafana Labs + Prometheus ecosystem.

⏱ Prometheus
- The de-facto standard for metrics collection.
- Simple model: the service exposes metrics via HTTP, Prometheus “scrapes” them.
- Powerful query language (PromQL).
- Tons of exporters: MySQL, PostgreSQL, Nginx, Docker, etc.
📈 Grafana
- Universal visualization tool.
- Works not only with Prometheus but also with Loki, Elastic, Postgres, and more.
- Great dashboards, alerts, annotations.
- Huge marketplace of ready-to-use dashboards.
📜 Loki
- Centralized logging system with Prometheus-like syntax.
- Much lighter than Elasticsearch.
- Perfectly integrated with Grafana — jump from a metric to the exact log line.
🧵 Tempo / Jaeger
- Distributed tracing.
- Shows the full path of a request across multiple services — essential for microservices.

Why developers love this stack:

✔️ open source — no license cost, easy to start

✔️ low entry barrier — a basic dashboard in a couple of hours

✔️ one ecosystem — everything integrates

✔️ scalable — from pet projects to highload

How Developers Can Actually Use It

Here are some everyday scenarios:

🔎 1. Found a bug you can’t reproduce

User says: “The button doesn’t work.”
Search in Loki by request_id or exception text.
You see the stacktrace → problem identified. 👉 Result: no more “works on my machine.”

⚡ 2. Service slows down after a release

Grafana shows latency doubled after yesterday’s deploy.
Drill down into endpoint metrics → find the culprit.
Check logs for parameters. 👉 Result: you know exactly what broke.

🧮 3. Optimizing SQL queries

Add metrics for query count and duration.
Grafana shows the heaviest queries.
Compare before/after optimization. 👉 Result: real data, not gut feeling.

🚨 4. Reacting to alerts yourself

Rule: “if 5xx > 5% over 5 minutes → send alert.”
Notifications in Slack/Telegram. 👉 Result: developers react instantly, not a day later via support.

🛠️ 5. Your own “developer dashboard”

A few panels: error rates, latency, top endpoints. 👉 Result: all essentials in one place.

A Real Project Example

I joined a growing gaming project (FastAPI + MySQL) as a part-time engineer. It was unstable — clients often complained that the game wouldn’t load.

What I found:

❌ no log collection,
❌ Grafana only had node-exporter metrics nobody watched. The project was flying blind.

Step 1. Logs

Set up Loki with JSON format.
Added request/response logs, duration, endpoint filtering.
Super fast search via Docker plugin.

Step 2. Metrics

Added HTTP latency, DB transaction times, response statuses, concurrent requests.
Finally saw the full load picture.

Step 3. Problem discovery

MySQL queries crashing → fixed with connection pooling.
httpx connection pool overloading → fixed.
Long queries → Percona Monitoring and Management → cleaned up DB load.

Step 4. Alerts

500 errors after release.
RabbitMQ queue overflow.
Network issues.
Payment provider outages → switch to backup quickly.

The result:

✅ from black box → transparent system
✅ developers could debug themselves
✅ support and ops stopped firefighting
✅ downtime minimized → saved money

What About Tracing?

Tracing is important, especially in microservices. But for basics, I use a simpler approach: passing a request_id across all services.

That lets me track a request in Loki end-to-end across the ecosystem. It’s not as powerful as Jaeger or Tempo, but it covers 80% of a developer’s needs.

A Minimal Observability Checklist

If your project has nothing yet, start small — 20% effort for 80% results:

✅ Logs

Centralized (Loki/EFK)
JSON format
Add request_id and duration
Filter by endpoint/error level

✅ Metrics

HTTP latency
Response statuses (2xx/4xx/5xx)
Concurrent requests
DB transaction times
Queue load

✅ Database

MySQL/Postgres exporter
Track slow queries, deadlocks

✅ Alerts

5xx error threshold
Queue overflow
Service outages
DB overload

✅ Team Interface

Base Grafana dashboard: errors, latency, load
A “developer dashboard” with API essentials

👉 With just this, your project stops being a black box. Everything else — tracing, business dashboards, analytics — can be added gradually.

Originally published at mind.mansur.expert.

How to Bring Order to a Project: A Track Map

Mansur Fattakhov — Sun, 13 Sep 2026 08:21:43 +0000

Every now and then I join a project where "everything works, but nobody dares to touch it." Deployment is done by hand from memory, the architecture grew historically, knowledge lives in people's heads, hiring happens by accident. That's normal: this is what almost any project looks like when it grows faster than it matures.

You don't bring order to a project like that in one heroic push. What works is different: split the chaos into tracks, take small steps within each track, and make every step pay off immediately. Below is my complete plan — what to do, in what order, and why. Each track will get its own deep-dive article; this is the map.

How to use this plan

Snapshot first, then light, then changes. Don't fix anything until you can see what's going on.
Tracks run in parallel, at different speeds — push the one that hurts right now.
A step must fit into a week. If it doesn't, it's not a step, it's a project; keep cutting.
Every touch leaves a trace: touched a service — bring it closer to the standard; figured out a piece of the system — write it down.

Track 0. State snapshot

One week, fix nothing — only look and write down.

⬜ Service map: what exists, where it lives, who deploys it and how

⬜ The "scary places" list — things people describe as "better not touch it"

⬜ Who knows what: which knowledge has a single carrier

⬜ Top 5 risks — one page, in plain human language

💬 The main artifact of this stage is not a document but a picture in your head. The document is a way to verify it: show it to the team, let them correct you.

📖 Full article: the state snapshot — your first week on a new project.

Track 1. Observability — turn on the lights

⬜ Centralized, structured logs

⬜ HTTP and database metrics

⬜ 3–5 alerts for the things that actually wake you up at night

⬜ A request_id passed through every service

💬 This track goes first because it makes every other track cheaper: any change is visible, any incident takes minutes to investigate instead of "going by gut feeling."

📖 Full article: how developers can monitor production — and why it matters.

Track 2. Infrastructure and deployment

⬜ CI/CD on the most frequently deployed service

⬜ Configuration in git, not on servers

⬜ A reference service template: structure, Dockerfile, healthcheck, pipeline

⬜ Rule: new services only from the template; old ones catch up as you touch them

💬 The goal of this track is for deployment to stop being an event and stop being one person's knowledge. The success marker: a Friday deploy scares no one.

📖 Full article: infrastructure and deployment — order by iteration.

Track 3. Architecture

⬜ Draw the system "as is" — honestly, without prettifying

⬜ Find the boundaries: what is genuinely a separate service and what got glued together by accident

⬜ Start a decision log — short records, half a page: what was decided and why

⬜ Tech debt — an explicit list with a price tag: what it blocks, what it slows down, what a fix costs

💬 The most common mistake is to start by rewriting. Start by writing down: half of the "architecture problems" dissolve once the system is described as it actually is.

📖 Full article: architecture — write it down before rewriting.

Track 4. People

⬜ Bus factor from the Track 0 map: wherever knowledge lives in one head — pair people up or document first

⬜ Onboarding: working environment within a day, first production task within a week

⬜ Hiring: write the job post from the project's real tasks, not from a list of technologies

⬜ Explicit ownership: every service has a name next to it

💬 Order among people matters more than order in code: code with a bus factor of 1 is not an asset, it's a hostage. And the other way around — a team where knowledge flows freely can fix any code.

📖 Full article: people — bus factor, expectations and day one.

Track 5. Processes

⬜ Planning that survives a week: a short cycle, a visible backlog

⬜ A release rhythm — predictable, boring, documented

⬜ Incidents: blameless reviews with one action item each

⬜ Meetings — audit them: each one either produces decisions or dies

💬 Processes come last not because they don't matter, but because without light (Track 1) and hands on the wheel (Track 2) any process is theater.

How to tell the order has arrived

Not by pretty dashboards. By how the team feels:

deploying is not scary,
a new person is useful in their first week,
an incident is a procedure, not a panic,
the question "why is it like this here?" has a written answer.

You'll never reach perfection — and you don't need to. It's enough that every week feels a little calmer than the last one.

This is the map of the series: links in the tracks will come alive as articles are published. Want a specific track covered sooner — tell me, the queue is flexible.

Originally published at mind.mansur.expert.