DEV Community: 4thwithme

10. Trending products, part 1: sales based models

4thwithme — Mon, 19 Oct 2026 00:03:26 +0000

Every online store has a "Trending now" shelf. Somebody built it in one sprint. Ask what "trending" means in that code and you get a shrug. This post opens it up. We take four simple models that use only past sales. We run them on the same fake data and watch them disagree.

Next week, part 2: we will talk about predictive models. These models predict trends before they happen.

[ TWO FAMILIES ]

Trending product - a product whose sales are growing faster than its own normal. Not the biggest seller. Not the newest. The one that is changing. A bestseller can sell the same amount every day and not be trending at all.

"popular" means big, "trending" means moving

Trend tools fall into two families. This split is my own framing, not a standard.

Reactive models score each product from its recent sales, then rank it. They are cheap, easy to explain and easy to ship. This post.
Predictive models fit a curve to the sales and project it forward. They can tell a seasonal bump from a real climb. Part 2.

Many teams stay reactive on purpose. A 2025 study on a year of e-commerce sales logs found that recent sales beat all-time sales. A weekly-change method did best of all.

[ THE TEST DATA ]

All data here is fake. One seeded script made 120 days of daily sales for five products. Every number below comes from that one run.

five shapes of demand, one seeded dataset

A Tumbler is a real trend. It gains 0.36 sales every day. Its 7-day average grows from 32 to 72.
B Hoodie is seasonal. It sells about 40 a day, then spikes near day 100 for the holidays.
C Gadget is a fad. It takes off on day 55 and peaks at 242 sales on day 70. By day 85 it is back to about 15 a day.
D Tote is small and quiet. It rises slowly.
E Pin sells less than one a day. Its true rate never changes. It is here to cause trouble.

Here is the right answer. A is the only product to plan for. C gets a short flag, not deep stock. B gets flagged with a calendar next to it. E is never flagged.

[ MODEL 1: THE WINDOWED COUNT ]

This is the oldest trick. Add up the last N days and sort.

score(t) = x(t) + x(t−1) + … + x(t−6)

Here x(t) is the units sold on day t. With N = 7, the Gadget scores 1,103 on day 70. The Tumbler scores 374 and the Hoodie 286. It wins by a mile.

It is easy to explain and easy to write in SQL. It has two weak spots.

The cliff. A sale from 6 days ago counts in full. A sale from 8 days ago counts for nothing.
It measures size, not change. The Tumbler is in the top 2 on all 85 days. It is not trending. It is just big. Every "top sellers this week" list works like this.

[ MODEL 2: TIME DECAY ]

Let's fix the cliff. Instead of a hard cut, let each sale fade as it gets older.

score(t) = Σ x(t−a) · e^(−λa)   λ = ln(2) / half-life

A sale that is a days old counts for e^(−λa) of a fresh sale. You pick the half-life. That is the age at which a sale counts half. With a half-life of 7 days, λ is 0.099. A sale from last week counts 0.5. A sale from two weeks ago counts 0.25. This is plain exponential decay.

the window forgets all at once, decay forgets slowly

You never need to scan old rows again. Keep one number per product and update it daily:

S(t) = S(t−1) · e^(−λ) + x(t)

Take yesterday's score and shrink it by 0.906. Then add today's sales. This is an exponential moving average in disguise. On the Gadget's climb:

Day	Sales x(t)	S(t−1) × 0.906	S(t)
68	186	524	710
69	214	643	857
70	242	776	1,019

Or compute it from raw rows:

SELECT product_id,
       SUM(units * EXP(-LN(2) / 7.0 * DATEDIFF(day, sale_date, CURRENT_DATE))) AS score
FROM daily_sales
GROUP BY product_id
ORDER BY score DESC;

Big sites use the same idea. The commonly cited Hacker News formula divides points by (age + 2)^1.8. The old open-source Reddit code adds a time term to the log of the votes. This post compares several such formulas. For a live system, see this Redis tutorial.

Decay still ranks by size, so the Tumbler is still at the top. It has a second problem. On day 90 the Gadget sold only 119 units in 7 days. Its decay score is still 499, four times more. The big days fade slowly, so the dead fad stays on the shelf.

In simple words: A window is a bouncer. You get in if you came this week. You stay out if you did not. Decay is a memory. Last week is a bit blurry. Last month is almost gone. Neither one asks if the crowd is growing.

[ MODEL 3: VELOCITY ]

Now we ask the right question. Is it rising? Compare the recent rate to the usual rate.

velocity = (recent + k) / (baseline + k)

recent is the average daily sales over the last 7 days. baseline is the average over the 28 days before that. A velocity of 2 means "selling twice as fast as normal."

Sellers say Amazon's Movers & Shakers list works on rank change over 24 hours. Amazon does not publish the method, so take that as folklore.

The k matters a lot. Small numbers lie. Look at the Pin. Its true rate never changed. On day 60 it sold 9 units in a week. The 28 days before that averaged 0.6 a day. So the raw ratio is 2.12. It looks like a rocket. It was luck.

adding k = 3 turns 2.12x into 1.19x

Add k = 3 to the top and the bottom. The same week now scores (1.29 + 3) / (0.61 + 3) = 1.19. That is no rocket. For big products, k barely matters. The Gadget on day 66 still scores 6.6. This is additive smoothing. A good k is "how many daily sales before I care."

SELECT product_id,
       (AVG(CASE WHEN days_ago <= 7 THEN units END) + 3)
     / (AVG(CASE WHEN days_ago BETWEEN 8 AND 35 THEN units END) + 3) AS velocity
FROM daily_sales
GROUP BY product_id
HAVING velocity >= 1.5;

Velocity catches the Gadget on day 60, five days after take-off. But it has a blind spot. The Tumbler really is trending, yet its best velocity in 120 days is 1.33. Slow, steady growth looks like nothing when the baseline grows too.

[ MODEL 4: THE Z-SCORE ]

Velocity says how much bigger. The z-score says how surprising.

z = (sales₇ − 7μ) / √(7μ)

μ is the normal daily rate. 7μ is what one week should add up to. Counts like sales wobble by about the square root of their size. This is the Poisson rule of thumb. So √(7μ) is "one unit of normal wobble." The z-score counts how many wobbles away you are.

Take the Gadget on day 60. It sold 127 units in 7 days. Normal is 8.2 a day, so we expect 57.5. One wobble is 7.6. So z = (127 − 57.5) / 7.6 = 9.2. A common alert line is z ≥ 3.

the Gadget peaks at z = 97; the chart cuts it off at 30

It needs no k, and the Pin never crosses 3. In a real catalog, compute μ per category too. Then one booming category does not flag every item in it.

The bad part is the wobble rule. It assumes pure Poisson noise. Real sales are lumpier. There are weekends, promos and viral posts. This extra spread has a name: overdispersion. In our data, the Tumbler's daily spread is about 10. Poisson says 7. So the z-score flags the Tumbler on 24 of 85 days, mostly from noise. If you guess the wobble too low, you get false alarms. Measure it from history, or just raise the threshold.

[ HEAD TO HEAD ]

Now we run all four on the same data. "Flagged" means top 2 of 5 for window and decay, 1.5 or more for velocity, and 3 or more for z.

the Tumbler is the only real trend, and the only product two models flag every single day

First, the Gadget fad. It took off on day 55 and peaked on day 70.

Model	First flagged	Last flagged	Verdict
z ≥ 3	day 59	day 79	fastest in, fast out
Velocity ≥ 1.5	day 60	day 78	fast in, fast out
7-day window	day 64	day 82	9 days late
Decay	day 66	day 89	11 days late, 19 days after the peak

The change-based models win on timing. The size-based ones wait until the fad is big. Then they keep it on the shelf after it is dead.

Now the other two products:

Hoodie (seasonal). Velocity flags it on days 93 to 108. The z-score flags it on days 88 to 110. Both fire as the holiday climb starts. That is correct, but not useful. Nothing tells the model this spike comes back every year.
Tumbler (real trend). The window and decay models rank it top 2 on all 85 days. Velocity never flags it. The z-score flags it on and off for 24 days, mostly from noise.

[ WHAT ALL FOUR CAN'T DO ]

Here is the awkward result. The Gadget fad and the Hoodie spike look the same to all four models while they rise. The Tumbler is the one product that really grew. Its sales doubled. It gets a permanent bestseller badge or no flag at all.

Reactive models only describe the past. They cannot say if a rise will last. Two lessons follow:

Use size models and change models for different shelves. Use window or decay for "popular right now." Use velocity or z for "rising fast." Do not mix them in one list and call it trending.
Never act on one signal. I keep a rule in my notes: wait for three signals to agree before you buy stock. A fast flag from velocity plus a search or social signal is a lead. It is not a decision.

Part 2 looks ahead, so we can finally ask what these four cannot.

[ ELSEWHERE ]

GitHub · LinkedIn · Dev.to · Substack · 4thwithme.dev/blog

May the --force be with you. See you next week.

9. Synthetic customers, part 2: a field guide to fake shoppers you can trust

4thwithme — Mon, 12 Oct 2026 10:27:17 +0000

Part 1 covered the theory. This part is a field guide. It shows how to build synthetic customers without LLMs (large language models), what can go wrong, and what worked for us.

A note first. The real data is not mine to share. So the example store is made up. It sells colored and flavored toilet paper: mint, lavender, bacon, glitter. Every number is invented. The method is real.

[ THE GOAL ]

Say your store has a "similar products" widget. A new version is ready. You want to know if it is better before real customers see it.

A synthetic customer helps. It stands for a group of real customers. You show it a list of products. For each product, it returns the odds of a click, a save, an add to cart, and an order. In part 1 terms, this is the meso scale: one model per group.

There are seven steps. Each has three parts: what to try, what can go wrong, and what worked for us.

[ STEP 1: KNOW YOUR CUSTOMERS ]

What to try. Count orders and revenue per customer. Look at the shape.

made-up numbers, real shape: most customers order once, a few carry the money

Pitfalls.

Most customers bought once. A few heavy buyers carry the revenue. An average hides both.
Divide orders by time as a customer, and a new customer with 3 orders in 5 weeks looks like a heavy buyer. Set a minimum of one year first.

What worked for us. We weight results two ways: by number of customers and by revenue. We always report both. We use the one-year minimum.

[ STEP 2: GROUP THEM ]

What to try. Three options. One model per person. Clustering on many traits. Or a fixed grid on a few simple traits: who they are, how often they buy, how much they spend.

Pitfalls.

One model per person has too little data.
Clusters are hard to name. They can also shift each time you refit.
Too many groups leave some with a handful of customers. Merging them hides your heaviest buyers.

What worked for us. A plain grid. There are 25 customer types, 5 order counts, and 5 order sizes. That is 625 segments. We add 25 for people who have not ordered. Each segment has a readable code, like Hotels_5-10_41-80. We kept thin segments. They lean on a shared model.

one customer type, 25 segments; the corner has 24 customers

[ STEP 3: BUILD THE TRAINING DATA ]

What to try. Use one row per product shown to one customer in one list. Add the product's category, price, and brand. Add the position, the device, and the list length. Label each row yes or no for click, save, cart, and order.

from 900 customers to 1.5 million labeled rows

Orders give the customer its taste: which scents and sizes it buys. Rows give the yes and the no. A model needs both.

Pitfalls.

Keep all clicks and only some non-clicks, and the model learns that everyone clicks. Predicted rates come out several times too high.
Split by time, not at random. Random splits leak the future.
Logs often do not say which products were shown. Rebuild each list with the live system's rule, and say so.
Tracking changes can shift your rates overnight. Plot raw counts per day before you trust a model.
Skipped products must look like what the system could really show. Otherwise the model learns the wrong lesson.

What worked for us. We train on 9 months, calibrate on 3, and test on the next 3. Recent rows count more. Rows from the widget count 5 times more.

[ STEP 4: PICK A MODEL ]

What to try. One model per segment. One shared model. Or both. Always add a baseline that ranks by position only.

Pitfalls.

Position is the strongest signal. If you do not beat the baseline, you built nothing.
Own models overfit small segments. A fancier library does not fix thin data.

What worked for us. Both. A segment with enough clicks uses its own model. A thin one uses the shared model. Both are gradient-boosted trees.

[ STEP 5: FIX THE LEVELS ]

What to try. Calibrate. Shift the odds until the average matches the real rate. Do it per position band.

Pitfalls.

Calibrate on one unusual month and you overshoot. Use several months. Test on months the model never saw.
Calibration never changes the ranking. It only fixes the level.
Raw rates drift with the season. Report the gain over the current version, not raw rates.

What worked for us. Three months, per position band. After that, rates land within about 10% of real.

[ STEP 6: GRADE EACH SYNTHETIC CUSTOMER ]

What to try. One number is not enough. Use five checks and put them on a card.

Ordering. Does it rank the clicked product first? We call this hit@1.
Level. Is the predicted rate close to the real rate?
Mix. Does it favor the same kinds of products as real customers?
Across segments. Does it rank segments the way reality does?
Verdict. Does it pick the same winner as a past A/B test?

one card: who, taste, model, level, checks

Pitfalls.

A threshold tighter than real-world noise fails everyone. Measure the noise first. Split the real clicks into two random halves. See how far apart they are. A synthetic customer closer to reality than the halves are to each other is as good as the data allows.
A customer can rank well and still click far too often. Check level and ranking separately.
The verdict is the strongest check. Many teams skip it.

What worked for us. If a customer fails two checks, it does not vote. About 3 in 10 segments fail, mostly thin ones. Our verdict check still waits for a past A/B result.

[ STEP 7: USE THEM ]

You can estimate three things.

Read one list

Show a synthetic customer a list. It scores each product. It also says why.

one synthetic customer, one list, one reason per score

Here, hotels favor lavender and a bulk pack. The click odds add up to 0.106 clicks per view. The order odds add up to 20.7 orders per 1,000 views. At the segment's $58 median order, that is $1,201 per 1,000 views.

To find a reason, set each input to "unknown", one at a time. The inputs that move the odds most are the reasons.

Compare two versions of a recommender

Run every synthetic customer on the same pages, for each version. Then add up the answers with weights.

version B wins overall, but not for everyone

Version B gains 4% by customer count and 9% by revenue. Hotels gain 17%. Gift buyers lose 3%. One average number hides that. Always look by group.

Pitfalls.

Score each list in the context of its page. If the evaluator only asks "does this customer like this product?", popular items always win.
Do not let the baseline and the evaluator share logic. Otherwise the baseline looks better than it is.
Do not estimate "chance of no click" by multiplying independent odds. Real clicks come in bunches.
Report a range. Redraw the page set 20 times. A gain that flips sign is noise, not a win.

What worked for us. Adding the page's product to the evaluator. And reporting gain by group, not one number.

Try a new product

You need no history. The models read traits like scent family, price, and brand. They do not read product IDs. Put the new product in a list and ask who clicks it.

In the example, event planners click it 31 times per 1,000 views. Hotels click it 6 times. So show it where party buyers browse. This is the what-if part (C2) from part 1. The same trick works for a price change.

[ WHAT IS STILL OPEN ]

Run the verdict check on two versions that were really live. Until then, the numbers are plausible, not validated.

Here is how I plan to use it. Replay two past versions and check the verdict. Screen every new candidate before an A/B test. Send only the winners to a live test. It will not replace A/B tests. It will cut the number of weak ideas that reach them.

Part 1 asked: does the fake shopper act like the real one? My answer is "partly". The ranking is believable. The levels can be calibrated. I know which segments to distrust.

[ REFERENCES ]

Stavinova et al. Synthetic Data-Based Simulators for Recommender Systems: A Survey. 2022.
Bougie, Watanabe. SimUSER. 2025. arXiv:2504.12722
Part 1 has the rest.

[ ELSEWHERE ]

GitHub · LinkedIn · Dev.to · Substack · 4thwithme.dev/blog

May the --force be with you. See you next week.

8. Synthetic customers, part 1: can a fake shopper judge your recommender?

4thwithme — Mon, 05 Oct 2026 23:49:20 +0000

Picture a shop that sells colored and flavored toilet paper. Mint. Lavender. Bacon. You build a new "customers also liked" box. Offline, it scores well. You ship it to half the traffic. Three weeks later, the A/B test (a live split test) shows no difference.

That is the normal story. This post is about a faster check: fake customers. Let them click on your box first. It is part 1 of 2. Part 1 is what the research says. Part 2 is what I built.

The problem: two bad options

A recommender suggests products. To test a new one, you have two options. Both have flaws.

a simulator sits in the middle on purpose

Offline metrics replay old logs. Two examples are nDCG (normalized discounted cumulative gain) and precision@k (how many of the top k picks are right). They are fast and cheap. But a model can win offline and lose online. This is the offline-online gap. A 2022 survey (Stavinova et al.) lists four causes:

The site changed after the logs were made.
The metrics ignore novelty and variety.
Feedback loops are missing. What you show changes what people click next.
The logged data is biased.

Online A/B tests are honest but slow. As post 3 showed, a small gain needs lots of traffic and time. Real customers may see a worse page. Privacy rules also limit the data you can keep.

A simulator sits in the middle. You learn fake users from real data. You let them react to a recommender. You count what they do. You get an answer in hours. No real customer is touched.

5 parts of a simulator

The survey read 18 simulators. They all share five parts, called C1 to C5.

counts from Table 2 of Stavinova et al. (2022), redrawn

All 18 have C1 (make fake data) and C3 (run a recommender on it). About half have C2, the what-if part. "What if prices go up?"

Now look at C4. Only 4 of 18 check the simulator itself. Does the fake world match the real one? Almost nobody asks. Remember this. It is the point of this series.

A simulator learns from logs, so it learns their biases too. The survey lists five. Positivity: people rate what they like. Popularity: popular items get more clicks. Conformity: people copy others. Exposure: people only see what was shown. Position: slot 1 gets clicks for being first. Most simulators only measure these. The survey names SOFA (Simulator for OFfline leArning and evaluation) as one that corrects bias. Position bias returns in part 2. It is the biggest one.

Four ways to fake a user

Each paper fakes a different layer. One fakes the data. Two fake the person. One fakes a chat.

1. Fake data, no fake people

Outbrain researchers (Malenšek et al., 2024) make fully artificial tables. You pick the rules: values per column, distribution shape, how columns interact, how much noise. You wrote the rules, so you know the truth. Then you test if an algorithm finds it.

DeepFM vs logistic regression on 11 engineered datasets (Malenšek et al., arXiv:2412.06809, CC BY 4.0)

They hid feature pairs that matter only together. The rules were logic gates (AND, OR, XOR) and sum of squares. The pairs sat among hundreds of useless features. DeepFM (deep factorization machine, a neural recommender) beat logistic regression in 10 of 11 setups. In config 6, both were near chance, and the simple model won. Real data cannot teach you this, because nobody gives you the true answer.

The catch: this makes data, not shoppers. The paper also never compares its data to real data.

2. A language model plays one shopper

SimUSER (Bougie and Watanabe, Woven by Toyota) gives each fake user an LLM (large language model) brain. It has two steps.

my redrawing of the SimUSER design (the paper's figure is under the authors' copyright)

Step 1: pick a persona. Take about 50 items a real user liked or disliked. The LLM drafts five personas (age, job, Big Five personality traits). Keep the persona that best separates this user from a random other user.

Step 2: run the agent. It has four modules. Persona sets pickiness and habits. Perception reads item thumbnails. Memory holds past ratings and a knowledge graph. The brain reasons, then picks CLICK, NEXT, or EXIT.

The check is the best part. The authors had 55 real A/B tests, scored by pages visited per session. Did the simulator rank the variants like reality? Its Spearman correlation (how well two rankings agree) was the highest. It beat Agent4Rec and RecAgent. Then they tuned a recommender two ways. Tuning for nDCG@10 (nDCG on the top 10) gave results on par with the baseline. Tuning on SimUSER improved engagement in production. The catch: the data is private.

3. Train a small model on real feedback

A click shows what happened, not why. UserMirrorer (Wei et al., 2025) adds the why. Each example is a scene: the user's profile and history, plus the list of items shown. The model picks one item.

one scene: memory on the left, the shown list on the right (Wei et al., arXiv:2508.18142, CC BY-SA 4.0)

four stages, from scenes to a small user simulator (Wei et al., arXiv:2508.18142, CC BY-SA 4.0)

A big LLM writes a reason for each real choice. A small model learns from the choice and the reason. Two tricks keep it cheap. First, keep scenes where a weak and a strong model disagree most. Those teach the most. Second, check each reason against the real choice. Matching reasons become training data for SFT (supervised fine-tuning). Non-matching ones become negative examples for DPO (direct preference optimization). The result is a 3-billion-parameter model. It was tested on eight public datasets. The code and models are open.

4. Chatty fake users

UserSimCRS v2 (Bernard and Balog) covers conversational recommender systems (CRS). It supports two designs. The agenda-based user has clear steps: NLU (natural language understanding), a policy, and NLG (natural language generation). It also has a fixed information need. v2 adds LLMs for reading and writing text. The end-to-end user is one LLM. v2 also adds an LLM judge to score chats. The authors warn that a judge can be biased or gamed.

parts you can inspect, or one LLM that does it all (redrawn from UserSimCRS v2)

Micro, meso, macro

The survey names three scales for fake users:

Micro: learn from one user's history. Most users do not have enough history.
Meso: learn from a group of similar users. One model per group.
Macro: learn from everyone. One model. Most simulators do this.

micro, meso, macro, as the survey defines them

Meso is rare. I picked it for part 2. Shop customers differ a lot. A hotel buys for 40 rooms. A gift buyer buys one roll. One average shopper fits nobody. But one model per person has too little data.

Which one should you use?

It depends on what you have and what you want to test.

You want to test an algorithm against a known truth. Use fake data, like the Outbrain tool.
You want rich behavior, like reactions to pictures and reviews. Use an LLM agent, like SimUSER. It costs tokens on every run.
You have lots of real feedback and want cheap runs. Train a small model, like UserMirrorer.
Your recommender is a chatbot. Use UserSimCRS.

Whatever you pick, ask the C4 question first. How will I check that this fake user acts like a real one? If the answer is "I won't", you have a toy.

What nobody has proven

The survey ends with four open problems:

Recommender rankings on fake and real data sometimes disagree.
There is no standard way to validate a simulator.
There is no head-to-head benchmark.
Few papers show real business gains.

SimUSER's authors add an ethics warning. Fake users can amplify bias. They can also be used to manipulate people.

So the real question is not "can an LLM act like a shopper?" The question is this: do fake customers rank three recommenders in the same order as real customers? That is the only test that matters. It needs real data.

Part 2 is a field guide to building fake customers without LLMs. It has seven steps. Each step says what to try, what can go wrong, and what worked for us. The example store is invented, and its numbers are made up. The method is real.

References

All on arXiv. Figures 3 and 5 are reproduced under their licenses (CC BY 4.0 and CC BY-SA 4.0, both Creative Commons). The rest are my redrawings.

Stavinova et al. Synthetic Data-Based Simulators for Recommender Systems: A Survey. 2022.
Malenšek, Škrlj, Mramor, Demšar. Generating Diverse Synthetic Datasets for Evaluation of Real-life Recommender Systems. 2024. arXiv:2412.06809
Bougie, Watanabe. SimUSER. 2025. arXiv:2504.12722
Wei et al. Mirroring Users. 2025. arXiv:2508.18142
Bernard, Balog. UserSimCRS v2. 2025. arXiv:2512.04588

Elsewhere

GitHub · LinkedIn · Dev.to · Substack · 4thwithme.dev/blog

May the --force be with you. See you next week.

Cron jobs: the tiny line that runs half your backend

4thwithme — Sun, 27 Sep 2026 19:36:51 +0000

Every backend has a few jobs nobody looks at. The nightly backup. The 9 AM report. The script that deletes old files. Each one is a single line of cron. They work fine for months. Then one day you find out the backup hasn't run since March. Nobody noticed. This post is about that one line: what it means, where to run it, and how to hear about it when it stops.

What cron actually is

cron - a program that runs in the background on Unix machines. It starts commands on a schedule. Each scheduled command is a cron job. All the jobs live in one file, the crontab (cron table). You edit it with crontab -e.

the name comes from Chronos, the Greek word for time

Think of it as an alarm clock. But instead of ringing, it runs a program. Cron is old. It first shipped in 1979. Most Linux machines today still run a version based on Paul Vixie's rewrite from 1987. In 2025 the syntax finally got a real written standard, OCPS 1.0. GitHub Actions, Kubernetes, and AWS all use the same kind of line, with small changes.

One thing to know first: cron has no memory. Every minute it wakes up. It checks each line: "should this run right now?" It runs the ones that match. Then it goes back to sleep. It doesn't know what it ran yesterday. If the machine was off at 02:00, the 02:00 job just doesn't happen. Nothing runs it later. (For laptops that sleep a lot, use anacron instead.) Remember this. Most of the tips below exist because of it.

How to read a cron line

read left to right: minute, hour, day of month, month, day of week, then the command

Each field is a filter. Cron checks all five every minute. If all five say yes, the job runs. You only need four symbols for almost everything:

Symbol	Means	Example
`*`	every value	`*` in hour = every hour
`,`	a list	`1,15` = the 1st and the 15th
`-`	a range	`1-5` = Monday to Friday
`/`	a step	`*/15` = 0, 15, 30, 45

There are also shortcuts, like @daily (same as 0 0 * * *) and @hourly. But not every tool supports them. GitHub Actions doesn't. Write all five fields and it works everywhere.

Two examples, read slowly

*/15 9-17 * * 1-5 - every 15 minutes, starting at 9 AM, Monday to Friday.

The trap: 9-17 means "any minute where the hour is 9 to 17". So the whole 17th hour counts. The last run is at 17:45, not 17:00. That's 36 runs a day. Want to stop at exactly 17:00? Use 9-16, then add one more line: 0 17 * * 1-5.

next runs from Mon 2026-10-05: 09:00, 09:15, 09:30 ... 17:45 (checked with croniter)

0 3 1,15 * 5 - you'd read it as "03:00 on the 1st and 15th, but only if it's Friday". Wrong.

If both day fields are set (neither one is *), cron uses OR, not AND. So it runs on the 1st, on the 15th, and every Friday. In October 2026 that's the 1st, 2nd, 9th, 15th, 16th, 23rd, and 30th. Seven runs in one month. You probably expected zero.

the oldest cron surprise, and it's in the rules on purpose

Not sure what a line does? Paste it into crontab.guru. Or print the next few run times with a library: croniter in Python, cron-parser in Node. Do this before you deploy.

Where to run it: GitHub Actions

You don't need a server for this. Add a workflow file to .github/workflows/ with a schedule trigger:

on:
  schedule:
    - cron: '30 5 * * 1-5'        # always quote: a value starting with * breaks YAML
      timezone: "America/New_York" # optional, default is UTC
  workflow_dispatch:               # lets you run it by hand too
jobs:
  report:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: ./scripts/daily-report.sh

What people learn the hard way:

It runs in UTC by default. The timezone key is optional and fairly new.
Every 5 minutes is the fastest you can go.
Runs can be late, or dropped. This happens when GitHub is busy. The top of every hour is the busiest time. So use minute 17, not minute 0.
Only the default branch counts. A schedule on your feature branch never runs.
Public repos with no activity for 60 days get their schedules turned off.

Good for anything that's fine a few minutes late. Bad for anything that must run on time.

Where to run it: Docker

You can put normal cron in a container. It will start. But it fails in three ways, and none of them show an error:

Your job can't see env vars. Cron starts each job with an almost empty env. You set DATABASE_URL on the container, but the job never gets it.
Your logs are empty. Cron sends job output to email or syslog, not to stdout. So docker logs shows nothing, even when the job fails.
Stopping the container can break data. In a container, cron is often the main process (PID 1). When Docker stops the container, cron doesn't wait for jobs to finish. A job can get killed halfway through writing a file.

Most people fix this with supercronic. It's a cron made for containers:

FROM debian:bookworm-slim
# install supercronic binary (see its README for the pinned URL + checksum)
COPY crontab /app/crontab
CMD ["supercronic", "/app/crontab"]

It gives every job the container's env vars. It writes job output to the container log. It shuts down cleanly on SIGTERM. And it won't start a job again while the last run is still going. Add supercronic -test crontab to your CI. It catches a broken line before you deploy.

Don't want to change your images? Try ofelia. It reads schedules from Docker labels. It can run a command inside a container that's already running (job-exec). Or it can start a new container for each run (job-run).

Where to run it: Kubernetes

Kubernetes has cron built in. It's called a CronJob. It doesn't run your code by itself. It's a template. Each time the schedule fires, it creates a new Job. Each Job starts one or more Pods.

CronJob → one Job per run → Pods, with retries inside each Job

apiVersion: batch/v1
kind: CronJob
metadata:
  name: nightly-backup            # max 52 characters
spec:
  schedule: "17 2 * * *"
  timeZone: "Europe/Kyiv"         # GA since v1.27
  concurrencyPolicy: Forbid       # never two at once
  startingDeadlineSeconds: 600    # too late by 10 min? skip it
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 3
  jobTemplate:
    spec:
      backoffLimit: 2             # default is 6 retries
      activeDeadlineSeconds: 3600 # kill it after an hour
      ttlSecondsAfterFinished: 86400
      template:
        spec:
          restartPolicy: Never
          containers:
            - name: backup
              image: myorg/backup:1.4.2
              args: ["--target", "s3://backups"]

The most important field is concurrencyPolicy. It answers one question: the next run is due, but the last one isn't done yet. What now?

Allow piles up, Forbid skips, Replace kills. Choose on purpose. The default is Allow.

Two more things from the docs. First, without timeZone, the schedule uses the time zone of the cluster's controller. Not yours. Second, Kubernetes says a CronJob can create two Jobs for one run. Or none at all. So, in their words, your jobs "should be idempotent."

To test it now, start one run by hand: kubectl create job --from=cronjob/nightly-backup test-run.

On AWS, use EventBridge Scheduler. Its cron has six fields (a year at the end), and one day field must be ?. A normal crontab line gets rejected.

Best practices

Idempotent - running it twice gives the same result as running it once. "Set today's total to X" is idempotent. "Add today's sales to the total" is not. Run that twice and your revenue doubles.

the one rule that makes every other failure harmless

Make every job idempotent. Then retries, double runs, and manual reruns can't hurt you.
Stop overlaps. On a normal server, use flock: */5 * * * * flock -n /run/lock/sync.lock /usr/local/bin/sync.sh. If the lock is taken, the new run just exits.
Set a time limit. Use timeout 2h ./job.sh, or activeDeadlineSeconds in Kubernetes. Without it, one stuck job blocks every run after it.
Use UTC. Local time has daylight saving. In spring, a 02:30 job can get skipped. In autumn, it can run twice. It depends on your cron version. If you must use local time, don't schedule between 01:00 and 03:00.
Don't use minute 0. Everyone does. Spread your jobs out. Or add a small random delay, so a hundred servers don't hit one database at the same second.
Use full paths and set your env. Cron's PATH is tiny (/usr/bin:/bin). Its shell is /bin/sh, not bash. "It works in my terminal" means nothing here.
Escape %. In a crontab, % means a new line. So date +%F breaks without an error. Write date +\%F.
Log to somewhere you'll read. Use >> /var/log/job.log 2>&1, or stdout in a container. But a log is not an alert. Nobody reads the log of a job that never started.
Keep crontabs in git. Someone typed a line into crontab -e on a server in 2021? Nobody will ever find it.
Treat jobs like prod services. Wire up the same APM and error tracking as your app: Sentry, New Relic, Rollbar. Ship logs, traces, and metrics like duration and memory. Then a slow or failing job shows up on the same dashboards as your API.

Know when it didn't run

That last problem is the big one. When a job crashes, you see an error. When it never runs, you see nothing. Maybe the server was down. Maybe cron was off. Maybe someone deleted the line. There's no error to catch. So flip it around: alert on silence.

a dead man's switch: silence is the alarm

Here's how it works. The job sends a ping to a monitor when it's done. The monitor knows the schedule. If no ping comes by the expected time (plus a little extra time), you get an alert. It's one curl at the end of the job:

0 2 * * * /opt/backup.sh && curl -fsS -m 10 --retry 5 -o /dev/null https://hc-ping.com/<your-uuid>

3 services that do this for you

Service	Free plan	Why pick it
Healthchecks.io	20 checks	Open source (BSD). You can self-host it. Knows cron syntax and time zones. Has `/start` and `/fail` pings. Can send the exit code with `/$?`.
Cronitor	5 monitors	Its CLI reads your crontab and wraps each line for you (`cronitor exec`). Also tracks how long each run takes.
Sentry Crons	1 monitor	Already on Sentry? Missed or slow runs show up as issues, right next to that job's errors.

Prices change, so check first. For a side project, free Healthchecks.io is enough. Team already on Sentry? Keep alerts there.

What to actually do

Read each line out loud before you deploy it. Print the next five runs. Use the platform you already have. GitHub Actions if late is okay. supercronic in Docker. A CronJob with Forbid and a deadline in Kubernetes. Make each job safe to run twice. Then add that one curl. It tells you when a job didn't run at all, and that's the failure you'd never see on your own.

Useful links

crontab(5) man page - the official syntax reference
crontab.guru - paste a line, read it in plain English
GitHub Docs: schedule event - limits and delays
Kubernetes: CronJob - every field explained
supercronic - cron for containers
Healthchecks.io: how Debian cron handles DST - the full daylight saving story

Elsewhere

GitHub · LinkedIn · Dev.to · Substack · 4thwithme.dev/blog

May the --force be with you. See you next week.

How to measure color the way your eyes actually see it

4thwithme — Mon, 21 Sep 2026 08:37:25 +0000

Source post: https://4thwithme.dev/blog/color-distance-why-rgb-math-lies/

Take two colors, (0, 64, 0) and (255, 64, 0). The plain math says they're 255 apart. Now compare (255, 64, 0) to (255, 64, 128): only 128 apart, half as much. Trust just the numbers and the second pair looks closer together. Look at them with your own eyes and that can flip completely, because plain math has no idea what your eyes actually see. Good news: people already fixed this. It just took fifty years and four tries. Two real examples with real numbers, later in this post, show exactly how big the gap can be.

[ WHY YOU CAN'T JUST SUBTRACT THE NUMBERS ]

distance = sqrt(
  (R1 - R2)^2 +
  (G1 - G2)^2 +
  (B1 - B2)^2
)

This is the first thing everyone tries, and it's wrong. Your eyes aren't equally good at spotting changes in red, green, and blue. A change of 10 in green is much easier to see than the same change of 10 in blue. Plain math doesn't know that, it treats every kind of "different" the same. It isn't the same.

ΔE (Delta E) - the standard way to measure how far apart two colors are. A score of 1.0 is supposed to be the smallest gap a human eye can notice, the Just Noticeable Difference. Every formula below is an attempt to make that number mean the same thing everywhere in the color space.

[ WHERE THESE NUMBERS ACTUALLY COME FROM ]

Every color space in this post, RGB, CIELAB, Oklab, comes from the same place: tests on real people in the 1920s. Someone looked at a light and mixed red, green, and blue until the mix matched it, across every color the eye can see.

the 1931 chromaticity diagram: every color the human eye can see, mapped from those matching experiments (Sakurambo, Wikimedia Commons, CC BY-SA)

Some matches needed a negative amount of a light, which isn't possible in real life. So the results got turned into a new system, XYZ, where every visible color gets a normal, positive number. Every color space since is built on top of XYZ. But none of it matches what "similar" looks like to a person: XYZ is complete, not built to match perception. That gap is why the next fifty years of formulas exist.

MacAdam ellipses (10x scale): equal steps in this raw space aren't equal steps in perception, some directions need way more distance before you'd notice

[ THE RACE TO FIX IT ]

CIELAB (1976) was the first real attempt: a space where an equal number should mean an equal amount of visible change. It still gets strong blues wrong, so every formula after it patches the last one's mistake:

Formula	Year	What changed
ΔE76	1976	Plain distance in CIELAB. The smallest visible gap lands near 2.3, not the intended 1.0.
CMC l:c	1984	Weighted specifically for the textile industry.
ΔE94	1995	Weights that change by region, from car-paint testing. Still weak on blue-violet.
CIEDE2000	2001	Weights that adapt, plus a fix aimed straight at blue.

Some industries take this dead seriously: car paint has to stay under 0.5 ΔE, printing under 2.0 ΔE. A narrower formula, ΔE_NS (2019), exists just for that near-zero range: the pass/fail check for "are these basically the same color."

[ WHAT'S ACTUALLY INSIDE CIEDE2000 ]

Here's the kid version. Say you have two crayons and want to know how different they look. You could measure three things: which one is lighter or darker, which one is more vivid or more washed-out, and which one leans more red vs. more blue. Add those three differences together (with a little extra twisting for blue, because blue kept fooling the older formulas) and that's basically the whole idea.

ΔE00 = sqrt(
  (ΔL' / (kL·SL))^2 +
  (ΔC' / (kC·SC))^2 +
  (ΔH' / (kH·SH))^2 +
  RT · (ΔC' / (kC·SC)) · (ΔH' / (kH·SH))
)

Scary-looking, but every piece is simple once you name it:

ΔL' - the lightness difference. How much lighter or darker is one color than the other.
ΔC' - the chroma difference. How much more vivid or washed-out one is.
ΔH' - the hue difference. How much the actual color type shifted, more red vs. more blue.
S_L, S_C, S_H - "stretchy rulers." The same real gap counts as bigger or smaller depending on where you are in the color space, because our eyes aren't equally picky everywhere. Near black, a tiny lightness change stands out, so that ruler stretches. In a very vivid color, the same numeric change is harder to spot, so it shrinks.
R_T - the twist. A small correction added just because blues specifically kept breaking the older formulas, it nudges the chroma and hue pieces to work together in that one problem region.
k_L, k_C, k_H - dials you can turn for your specific use case (lighting, industry). Left at 1 almost always.

That's the whole trick: three simple differences, three rulers that stretch and shrink depending on where you're measuring, and one small twist for blue. What makes it hard isn't the idea, it's getting all those moving pieces to line up in code without a sign error. A 2005 paper by Sharma, Wu, and Dalal exists because early implementations, including ones people pointed to as "the right way," had bugs in exactly that plumbing.

Two real pairs, both exactly 30 apart in plain RGB math. Run them through CIEDE2000 and the story flips completely:

🟦 rgb(0,0,255) vs rgb(30,0,255) → ΔE2000 = 0.62. Below the notice-it threshold. Nobody sees a difference.
⬛ rgb(10,10,10) vs rgb(10,10,40) → ΔE2000 = 14.99. Same RGB gap, an obviously different color.

Same 30-point RGB step, roughly 24x difference in how much it actually matters (computed with the colour-science Python library).

Where it breaks - the hue-angle math is easy to flip a sign on, the wraparound at 0°/360° needs its own special handling, not a plain remainder, and the weights genuinely change by region. Copy a CIE94-shaped formula (one formula, no branches) and you get a wrong number back, not an error. The good news: use a library people already tested, and check it against the published test numbers.

[ THE CHEAP TRICK: STAY IN RGB ]

All that trig and region-hopping is worth the trouble when accuracy is the whole point. It stops being worth it the moment you're running the comparison millions of times a second, palette matching, image quantization, because converting every pixel to Lab first is exactly the kind of work that piles up faster at scale. Redmean skips the trip entirely. It never leaves RGB, it just leans harder on red or blue depending on how much red is already in the pair:

r_avg = (R1 + R2) / 2

distance = sqrt(
  (2 + r_avg/256)         * (R1 - R2)^2 +
  4                       * (G1 - G2)^2 +
  (2 + (255-r_avg)/256)   * (B1 - B2)^2
)

It's not the most accurate choice, plain CIELUV usually beats it on average. Its real strength is that it never has a bad day: Luv can go badly wrong on some colors, skin tones especially, and redmean doesn't, for a lot less computer work.

[ OKLAB: A FRESH START, NOT ANOTHER PATCH ]

Redmean and CIEDE2000 are still playing the same game: patch RGB, patch CIELAB, patch the patch. Oklab walks away from the table instead. Björn Ottosson published it in 2020 after running a simple test on CIELAB: hold lightness and saturation fixed, slide through every hue, and watch what happens to the color wheel. Yellow, magenta, and cyan come out visibly brighter than red and blue, a strip that flickers as it spins, not because the colors changed but because the space measuring them is uneven. Oklab was built from real perception data specifically to make that flicker disappear.

same hue sweep, top (HSV/sRGB) visibly flickers in brightness, bottom (Oklab) stays flat (Björn Ottosson)

Oklab / Oklch - XYZ turns into Oklab through two simple matrix multiplications with a cube root in between: cheap, stable, no special cases by region. Make a color brighter or darker and the numbers just scale, nothing else needs fixing.

It's been part of the CSS color spec since 2021, works in every major browser, and is Photoshop's default way to blend colors: the better starting point for gradients and color pickers on the web today, not CIELAB. (Two colors can share the exact same XYZ numbers and still look different under different lighting, that's what a color appearance model like CIECAM02 is built to handle.)

[ WHERE THIS SHOWS UP IN A REAL FEATURE ]

A tool that searches photos by color first has to decide what "the main color" even means. Just counting pixels doesn't work: point it at a beach photo and "blue" always finds the sky, never someone's blue shirt. One real version of this uses plain Lab distance, but turns the lightness weight down to 0.7, so a light and dark version of the same hue don't get scored as too far apart. The color math here was the easy part. The hard part was speeding up the database and showing results as they arrive, instead of making people wait. Swapping in CIEDE2000 wouldn't have made this feature faster. It usually doesn't.

[ WHAT TO ACTUALLY USE ]

Use plain Lab distance by default, it's good enough for almost everything. Reach for CIEDE2000 only when checking against a limit already written in ΔE2000: printing, paint, fabric. Reach for redmean when converting to Lab is genuinely too slow. And for anything visual on the web today, start with Oklab/Oklch: already built into your browser, designed to skip the problems everything above it has been patching since 1976.

[ LIBRARIES THAT DO THIS FOR YOU ]

Don't write CIEDE2000 by hand. Here's what already does it, checked against current docs, not memory:

Python

Library	What it covers
`colour-science`	Broadest: ΔE76/94/2000, CMC, DIN99, HyAB, plus Oklab and CIECAM02/CIECAM16.
`scikit-image`	`skimage.color.deltaE_cie76`, `deltaE_ciede94`, `deltaE_ciede2000`, `deltaE_cmc`. No Oklab.
`coloraide`	Native Oklab/Oklch, `delta_e()` with CIE76, DE2000, HyAB, and an Oklab-based "OK" method.
`colormath`	The old classic, archived since Dec 2023. Use the `colormath2` fork instead.

Node.js

Library	What it covers
`culori`	CIE76/94/2000 difference functions, native Oklab/Oklch parsing and conversion.
`colorjs.io`	By the CSS Color 4 spec editors. `deltaE()` with `76`, `CMC`, `2000`, `Jz`, `deltaEOK`.
`color-diff`	CIEDE2000 specifically, plus palette-mapping (`closest`, `mapPalette`).
`nearest-color`	Plain Euclidean RGB nearest-neighbor, no perceptual conversion, the cheap option from earlier.

One thing to watch for: chroma-js ships a deltaE() too, but it's CMC l:c under the hood, not CIEDE2000, despite the name. Don't use it for perceptual ΔE2000 matching.

[ USEFUL LINKS ]

Björn Ottosson: "A perceptual color space for image processing" - the original Oklab post
Sharma, Wu, Dalal: CIEDE2000 implementation notes - the fix-your-bugs paper
CompuPhase: "Colour metric" - the redmean derivation
Wikipedia: CIE 1931 color space - where this all starts

Originally posted on 4thwithme.dev/blog.

GitHub · LinkedIn · Substack

May the --force be with you. See you next week.

A/B testing: how to test your test

4thwithme — Mon, 14 Sep 2026 08:47:13 +0000

Source post: https://4thwithme.dev/blog/ab-testing-how-to-test-your-test/

Posts 3 and 4 were about the math: what "significant" actually means, and which test design fits which situation. This one is about a question that comes before either: how do you know the test itself is working? A p-value can be perfectly correct and still be answering a question corrupted by a tracking bug, a broken randomizer, or a confound nobody looked for. Here's how to check.

A/A tests

A/A test — run two completely identical experiences against each other and check whether your pipeline reports a difference. There shouldn't be one. If it finds one anyway, the problem isn't your product, it's your testing setup.

It's the cheapest validity check available and it's underused precisely because it feels like it can't fail - you're testing nothing against nothing, what's there to break? Plenty. Randomization bugs, tracking that double-fires for one arm, IDs that leak between groups: none of that shows up until you look. Microsoft's experimentation team documented a platform where repeated A/A tests turned up a false-positive rate as high as 30%, against a nominal 5% - the stats engine itself was miscalibrated, and no amount of correct math downstream would have caught that. The fix isn't a one-time check before launch, it's continuous: route any traffic that isn't allocated to a real test into a standing A/A test, so a regression in the pipeline shows up before it corrupts something you actually care about.

In plain words: two vending machines, stocked with the exact same snacks at the exact same prices. If your sales dashboard tells you machine B is suddenly selling 20% more candy than machine A, the candy isn't the story - one of the machines is broken, or someone's miscounting the coins.

Sample ratio mismatch

Sample ratio mismatch (SRM) — you configured a 50/50 split, but the traffic that actually landed in each group isn't 50/50. A chi-squared goodness-of-fit test on the observed counts tells you whether that gap is noise or a real problem.

The industry convention here is a much stricter bar than the usual 5%: teams flag SRM at p < 0.001, deliberately tighter, because at real traffic volumes even a trivial imbalance clears 0.05 easily, and you'd rather tolerate a few false alarms than miss a broken splitter. Here's what that looks like with actual numbers: 100,000 visitors, intended 50/50, and you actually get 52,000 in control against 48,000 in test. Run the chi-squared test on that and you get χ² = 160, which lands around p ≈ 10⁻³⁶ - not a borderline call, a five-sigma-and-then-some alarm bell. That kind of gap is almost never real randomness; it's bot traffic skewed into one arm, an ID collision, or a redirect that leaks users out of their assigned bucket before they're logged.

a 4% imbalance looks small until the chi-squared test does the math on it

Run this check before you read any other result from the test. A significant SRM doesn't tell you your product change failed, it tells you the comparison itself isn't trustworthy, full stop - fix the split, don't interpret the metric.

In plain words: a teacher splits 100 kids into two identical classrooms by coin flip, expecting roughly 50 and 50. If 62 end up in room A and 38 in room B, you don't go looking for what's different about the kids in room A - you go check the coin.

Simpson's Paradox

Simpson's Paradox — the aggregate result shows B beating A, but every individual segment shows A beating B. The overall number and the segment-level numbers are both correct; they just tell opposite stories, because something is unevenly distributed between the two.

The textbook real-world case is a 1986 study of 700 kidney stone patients comparing two treatments. Overall, Treatment B (a less invasive procedure) had a better success rate than Treatment A, 83% against 78%. But split the same patients by stone size, and Treatment A wins in both the small-stone group and the large-stone group. What happened: doctors preferentially gave the gentler Treatment B to patients with smaller stones, who were easier cases to begin with regardless of treatment. Size was doing the real work; treatment assignment was riding along on top of it.

That case is a selection-bias story, not a randomization bug - nobody split anything wrong, doctors were just choosing based on severity. In an actual A/B test, where assignment is supposed to be random, the same pattern showing up is a stronger signal: something correlated with the outcome ended up unevenly split between your two groups. Check segment-level performance after any test that looks clean in aggregate - by platform, region, new vs. returning, whatever you track - and if every segment disagrees with the topline number, suspect the split before you trust the topline.

In plain words: two delivery drivers. Driver A is faster than Driver B on short routes, and faster than Driver B on long routes too. But overall, across the month, Driver B has the better average time - because Driver B mostly got assigned short routes, and Driver A got stuck with the long ones. Add it all up and the assignment is doing the work, not the driving.

Other checks worth running

A few more from the same family, lower ceremony than the three above but worth having in the rotation:

Guardrail metrics - track a small set of core health metrics (page load time, crash rate, unsubscribe rate) on every test regardless of what it's meant to measure, so a change that "wins" on its target metric while quietly breaking something else gets caught immediately. Example: a checkout redesign lifts conversion 3%, but the page-load guardrail shows +400ms - that's not a false alarm, that's the guardrail doing exactly its job.
Multiple-comparison correction - the more metrics or segments you check on a single test, the more of those checks will clear a 5% bar by chance alone. If you're tracking a dashboard of twenty metrics per test, expect roughly one of them to look "significant" for no real reason - correct your threshold (Bonferroni or similar) or treat any single metric flagged out of a large batch skeptically until it replicates. Example: scroll depth alone spikes to p=0.03 out of twenty tracked metrics where nothing else moved - that's the base rate you'd expect from chance, not a discovery.
Pre-launch bias check - before a test goes live, run the intended split for a short window with no actual treatment applied yet (an A/A period baked into the same pipeline) and confirm it comes back clean. Catches the same class of bug as a standing A/A test, just gated in front of the specific experiment instead of running generally. Example: a 48-hour blank dry run on a pricing test's exact split logic, confirming both arms convert identically before the real price difference goes live.

The checklist

Check	Catches	Run it
A/A test	Tracking, randomization, pipeline bugs	Continuously, on idle traffic
SRM (chi-squared)	Broken splitter, bot traffic, ID leaks	Before reading any other result
Segment breakdown	Simpson's Paradox, hidden confounds	After any "clean" result
Guardrail metrics	A win that breaks something else	On every test, by default
Multiple-comparison correction	False positives from dashboard-scanning	Whenever you track many metrics per test

A statistically significant result off a broken pipeline is just a precise measurement of the wrong thing. These checks are cheap relative to what they protect against - run them before you trust the number, not after someone's already acted on it.

GitHub · LinkedIn · Substack · 4thwithme.dev/blog

May the --force be with you. See you next week.

A/B testing: when the standard test is the wrong tool

4thwithme — Mon, 07 Sep 2026 07:44:07 +0000

Last week was the standard fixed-horizon test: split traffic 50/50, wait for your pre-calculated sample size, look once, done. That's still the right default most of the time. It's not the only tool, though, and treating it as the only one gets expensive once its assumptions stop matching your situation.

Why the default isn't always right

A fixed-horizon test answers one question: is there a detectable difference between two fixed groups, at a preset error tolerance, once enough data has come in. That's the whole job. It doesn't answer "how do I make the most money while this test runs," and it assumes your two groups don't affect each other. Both assumptions can quietly fail.

Here's a real version of why that matters. A subscription business earning roughly $1M a month runs a pricing test: 10% baseline conversion, a 5-week horizon at the standard 5%/80% thresholds. One week in, the test group converts at 15% against the control's 10% - a trend that, if it held, would add roughly $500K to that month's revenue. The textbook says don't stop, the sample size isn't there yet.

That's not a math problem, it's the wrong question. The test was built to answer "is there a detectable difference at a fixed error rate," not "how do I make the most money while this runs." Point it at the second question and it keeps giving you the wrong kind of answer.

Multi-armed bandits

Multi-armed bandit - instead of holding a fixed split for a fixed duration, you continuously shift more traffic toward whichever variant is currently winning, while still sending a smaller slice to the others so you don't lock in a lucky early read. The name comes from a gambler in front of a row of slot machines ("one-armed bandits"), trying to find the one that pays out best while losing as little as possible to the rest.

That tension has a name: the exploration-exploitation tradeoff. Exploit too early and you lock in the wrong winner off a noisy first week; explore forever and you never cash in on knowing which one wins. Studied by Allied scientists in World War II, and found so hard that mathematician Peter Whittle joked it should be dropped over Germany so enemy scientists could waste their time on it too, before Herbert Robbins gave it a serious mathematical treatment in 1952. Three practical strategies still run in production today: epsilon-greedy (mostly exploit, explore at random sometimes), UCB (favor whatever you're least certain about), and Thompson sampling (a Bayesian approach that updates its belief as data comes in).

Run as a bandit, the pricing test above looks different: most traffic goes to whichever price is winning, a smaller slice keeps testing the alternative, and the split keeps moving as the picture updates, instead of freezing at 50/50 for five weeks.

the losing variant never gets shut off entirely, it just gets starved of traffic

The trade-off: a bandit chases money made during the test, not a clean significance result, and usually needs longer to reach the same confidence one calculated sample size gets you. Use it when running a worse variant a little longer costs more than a tidy p-value is worth.

In simple words - Three ice cream stands, and you want to know which one people like best. A regular test sends exactly one-third of people to each stand for a whole month, even once it's obvious stand #2 is winning. A bandit is smarter: it quietly sends more people to stand #2, while still sending a few to the others, just in case they get better later.

Switchback tests

Switchback test - instead of splitting your population into two groups, split time into blocks and alternate the whole population between A and B as blocks pass: this hour everyone gets A, next hour everyone gets B, on a randomized schedule.

The case for this shows up once your users stop being independent of each other. A standard split assumes what happens to user A doesn't leak into user B's experience. That breaks in a marketplace: if half of riders see one price and half see another, drivers respond to whichever price is in front of them, and both groups end up competing for the same pool of drivers. The "control" group is now polluted by the treatment group's effect on shared supply. Alternating the whole market by time block avoids that: "this city under A" versus "the same city under B," not two artificially split groups sharing the same supply.

There's no single textbook citation for this the way "multi-armed bandit" has one; the closest formal relative is what statisticians call crossover design, adapted for markets instead of individual subjects. Uber and Lyft have both published on using it for dynamic-pricing and dispatch experiments, exactly because their two-sided marketplaces make a standard user-level split unreliable.

every time block is its own control for the block right next to it

In simple words - One playground, one set of kids. You can't fairly split them "half get the red slide, half get the blue slide," because they all play together and one group's fun affects the other's. So instead: everyone gets red on Monday, blue on Tuesday, red again on Wednesday - same kids, same playground, just switching what they try on different days.

Hold-out tests

A hold-out test is the simplest design here: keep a slice of your population permanently untouched by the change, and compare how their behavior evolves against everyone else's over time. No alternating, no traffic-shifting, just a group frozen in the "before" state as a long-running reference point. It earns its place for effects a short test can't see - trust, habit, long-term retention - anything that shows up as a slow drift over months, not a clean signal within weeks. The cost: you're deliberately holding back a possibly-better experience from real people, for a long time, just to keep a clean comparison available later.

A real one: Uber's rider-acquisition team saw wild week-to-week swings in Meta ad cost-per-acquisition that looked more like noise than signal, so they ran a 3-month hold-out - just turned Meta ads off for a slice of the market and watched. Nothing happened. Signups held steady, meaning the ads were just taking credit for people who'd have signed up anyway. That freed up roughly $35M a year for channels that actually moved the number - a finding a two-week test would never have had the runway to catch.

Pre/post & synthetic control

Sometimes there's no way to randomize at all, because there's only one of you: one product launching a redesign, one market entering a new pricing policy. You can't hold half a country in a control group. The blunt option is a pre/post comparison - metric before, metric after, call the difference the effect. It's also the weakest design here: everything else moving in that window (seasonality, a competitor's move, the economy) gets mixed into your "effect" with no way to pull it back out.

Synthetic control - when you have exactly one treated unit and no real control group, build a fake one: a weighted blend of other similar untreated units, chosen so the blend tracks your unit's trend closely before the change. After the change, the gap between what actually happened and what the blend predicts is your estimated effect.

This isn't an internet-testing invention, it comes from economics. Alberto Abadie and Javier Gardeazabal introduced it in 2003, building a "synthetic Basque Country" out of a blend of other Spanish regions to estimate what its economy would have done without a decades-long conflict. The method's Wikipedia page is a good primer; the idea carries over cleanly to product work, where "other regions" become other markets, stores, or cohorts that never saw your change.

before the change, the lines sit on top of each other - that's what makes the later gap meaningful

In simple words - One plant, and you want to know if a new fertilizer helped it grow. No second identical plant to compare against. So you blend ten other, similar-but-not-identical plants into one imaginary "average plant" that grew just like yours did before the fertilizer. If your real plant suddenly grows taller than that imaginary one, the gap is probably the fertilizer.

Don't reach for A/B/C/D early

Testing four or five variants at once is tempting when you have a backlog and want an answer fast. Resist it. A clean multi-variant test needs roughly the same sample size as a sequence of pairwise tests, but it's far harder to tell which comparison drove the result, and far easier to p-hack yourself by checking every pair until one clears the bar. A sequence costs calendar time, not data, and buys back your ability to actually interpret what happened.

How to actually choose

No universal method fits every situation - last week's flow solves one problem, nothing beyond that. You'll hit other limits too: overlapping tests, insufficient traffic, a goal about maximizing an outcome instead of detecting a difference. A rough guide:

Your situation	Reach for
Can randomize independently, want a calibrated error rate	Standard fixed-horizon A/B test
Users interact with each other (marketplace, shared supply)	Switchback test
Goal is maximizing outcome during the test, not a calibrated verdict	Multi-armed bandit
Only one treated unit exists, no randomization possible	Pre/post, ideally with synthetic control
Effect is slow, long-term, or compounding	Hold-out test
More than two variants on the table	Decompose into a sequence of A/B tests

None of these replace the fixed-horizon test as a default. Reach for them once you can name the specific way your situation breaks its assumptions - not before.

Elsewhere

GitHub · LinkedIn · Dev.to · Substack · 4thwithme.dev/blog

May the --force be with you. See you next week.

A/B testing: the stat-sig problem

4thwithme — Sun, 30 Aug 2026 22:54:41 +0000

Post source: https://4thwithme.dev/blog/ab-testing-stat-sig-problem/

[ THE SETUP ]

Generic scenario, no company specifics: you run an ecommerce storefront. Someone ships a change, a re-ranked product grid, a new checkout button color, doesn't matter. You split traffic 50/50, wait for data, and want to know: did this actually help, or did it just look like it helped?

That's the whole job of an A/B test. It answers exactly one question: is there a difference between the two groups, yes or no. It does not tell you why. It does not reliably tell you how much the metric will move once you roll out to everyone. It rejects a null hypothesis or it fails to reject it. That's the entire output. Everything else people read into a test result, they're reading in themselves.

[ DEF.1 ]

Null hypothesis (H₀) — the default, skeptical assumption that nothing changed: there is no real difference between A and B. An A/B test is built entirely to argue against this one claim. It either rejects it (evidence a difference exists) or fails to reject it (not enough evidence either way). It never proves the opposite outright, it just runs out of reasons to doubt it.

the assumption every A/B test starts by trying to disprove

[ WHAT "STAT SIG" ACTUALLY MEANS ]

Here's the misunderstanding almost everyone ships with: a p-value is not "the probability that B is better than A." It's the probability of seeing data this extreme (or more extreme) if there were actually no difference at all. It's a statement about the data given no effect, not a statement about the effect given the data. Those sound similar. They are not the same claim, and the difference matters every time you're deciding whether to trust a result.

Put another way: a p-value doesn't tell you "B is probably better." It tells you "if B and A were actually identical, how surprising would this result be?" A low p-value means the result would be surprising under the assumption of no real difference — that assumption starts to look shaky, so it's worth trusting. It does not tell you the odds that B is actually better; that's a different question the test never answers. It's the same logic as saying "if this coin were fair, getting 9 heads in 10 flips would be surprising" — that doesn't prove the coin is rigged, it just means the fair-coin explanation is hard to believe.

[ DEF.2 ]

P-value — the probability of seeing a result at least this extreme, assuming the null hypothesis is true, assuming there's actually no difference between A and B. It is not the probability that B beats A, and it is not the probability that the null hypothesis itself is true. A small p-value just means "this would be a strange coincidence if nothing had actually changed," nothing more.

a statement about the data given no effect, not about the effect given the data

To get to a p-value you first have to set up two competing claims:

Null hypothesis (H₀): there is no difference between A and B. This is always the same claim, every single time.
Alternative hypothesis (H₁): there is a difference.

An A/B test never proves H₁. It only ever rejects H₀ or fails to reject it. Failing to reject H₀ doesn't mean "there's no effect," it means "we didn't detect one at the sensitivity we set up for." That distinction alone would save a lot of bad blog posts about "our test failed, this feature doesn't work."

Then there are two ways to be wrong, and they have genuinely useful mnemonics:

[ DEF.3 ]

Type I error (false positive) — you see a difference that isn't really there. Telling grandpa he's pregnant.
Type II error (false negative) — you miss a difference that is really there. Telling a pregnant woman she isn't.

the two ways an A/B test can lie to you

The industry default is 5% tolerance for Type I (α = 0.05) and 20% tolerance for Type II (β = 0.20, i.e. 80% power). Neither number is handed down by nature. They're both choices you make based on the cost of being wrong. A drug trial might demand α with eight zeroes before the first significant digit. A button-color test where you genuinely don't care much either way could reasonably run at α = 0.20. The 5% convention is a convention, not a law.

Here's how the p-value and alpha actually connect: alpha is the surprise threshold you commit to before you look at any data — your pre-agreed tolerance for crying wolf. The p-value is how surprising your actual result turned out to be. The decision rule is just a comparison: p-value < alpha means "significant." That's it. And alpha isn't just a cutoff, it IS your Type I error rate — a 5% alpha means you've pre-accepted a 5% chance of telling grandpa he's pregnant, across all the times you'd run this test. Type II (missing a real difference) isn't controlled by alpha at all — that's a separate knob, beta, driven mostly by how much data you collect.

Beta works the opposite way from alpha: it's the rate at which you fail to catch a difference that's actually there — telling the pregnant woman she isn't. Beta isn't a tolerance you set directly like alpha; it falls out of how much data you collect, given the effect size you're trying to detect. Small sample, small true effect: beta is high, meaning you'll miss it most of the time and walk away concluding "no difference" when there really was one. More data pulls beta down. Power is just the flip side of the same number: power = 1 − β, so the industry-default β = 0.20 is the same fact stated as "80% power" — an 80% chance of actually seeing the effect if it's real, and a 20% chance of missing it outright.

[ THE FOUR NUMBERS THAT SET THE BAR ]

Before you run anything, four inputs determine how much data you actually need: your alpha threshold, your power, your baseline conversion rate, and your MDE, minimum detectable effect, the smallest lift you actually care about being able to see.

[ DEF.4 ]

MDE (minimum detectable effect) — the smallest lift you actually care about being able to see. It's not a property of the data, it's a choice you make going in: below this size, a real effect might exist and your test still won't reliably catch it. Set it too small and your required sample size balloons past what your traffic can deliver in a reasonable timeframe; set it too large and you'll miss smaller wins that were real.

the smallest lift you've decided is worth being able to see

Change any one of these and your required sample size moves, usually a lot more than intuition suggests. Sample size explodes as your baseline moves away from 50%, or as your MDE shrinks. Concretely, at a 10% baseline conversion rate with the standard 5%/80% thresholds:

Detecting this relative lift	Sample size needed, per arm
10% (10% → 11%)	~14,313
5% (10% → 10.5%)	~56,000
1% (10% → 10.01%)	~1,414,681

FIG.1 — going from "detect a 10% lift" to "detect a 1% lift" costs you 100x the traffic

Going from detecting a 10% lift to detecting a 1% lift costs you roughly one hundred times the traffic. This is why small-traffic products shouldn't be running color-tweak A/B tests: they simply don't have the volume to detect small effects reliably, and running one anyway doesn't make the math work, it just produces a noisy coin flip dressed up as a decision. Where does MDE actually come from in practice? There's no clean formula. In order of how people actually do it: gut feel on plausibility, historical results from similar past tests, or, the more honest method, work backwards from your constraints. How much traffic do you actually have, how many hypotheses are sitting in the backlog, how fast do you need to decide. Set your MDE from what's actually testable in your time and traffic budget, not from an abstract target someone wrote on a roadmap slide.

[ USING THE CALCULATOR: TWO EXAMPLES ]

You don't need to memorize the sample size formula to use any of this. Evan Miller's sample size calculator does the arithmetic for you: plug in your baseline conversion rate and your minimum detectable effect, and it hands you a number per variation. The one setting that trips people up is the toggle between Absolute and Relative. Absolute means "detect a change of this many percentage points" (a 20% baseline with a 5-point absolute MDE means detecting 20% → 25%). Relative means "detect a change of this many percent of the baseline itself" (a 20% baseline with a 10% relative MDE means detecting 20% → 22%, since 10% of 20 is 2). Relative is almost always the more honest way to think about it, because the same absolute point-move means something very different at a 2% baseline than at a 50% one. Here are two examples run through the same tool, same 5%/80% thresholds, to show how differently the same math treats two real spots on a storefront.

Example 1: top of the funnel. Say you're changing the color of the main call-to-action button on the homepage, high-traffic real estate that basically every visitor sees. Baseline click-through is 20%, and you want to detect at least a 10% relative lift, meaning you'd notice a move to roughly 22% or higher.

FIG.2 — 20% baseline, 10% relative MDE → 6,347 per variation

That's 6,347 per variation, 12,694 total. If the homepage does something like 3,000 visits a day, split 50/50, that's 1,500 per arm per day, so you'd clear the bar in well under a week. The strategy here is straightforward because the traffic supports it: pick a real, meaningful MDE, run a standard fixed-horizon test, wait for the number, done.

Example 2: deep in the site, low traffic. Now say the change is on an advanced export feature buried three clicks into account settings, the kind of thing only a small slice of engaged users ever reaches. Baseline conversion on the action you're watching (say, clicking "export") is 3%, and because the effect would need to be large to matter at this baseline, you set a generous 20% relative MDE, detecting a move to roughly 3.6% or higher.

FIG.3 — 3% baseline, 20% relative MDE (already a generous ask) → 13,050 per variation

Even with an MDE more than double the first example's, you need 13,050 per variation, more than double the homepage number, because the low baseline is working against you the whole time. If that settings page gets 300 visits a week, that's 150 per arm per week. Reaching 13,050 per arm at that rate takes roughly 87 weeks, about a year and a half. That's not a test, that's a career milestone.

The two examples share a calculator and a formula. They don't share a strategy. For the homepage button, run the standard test, it's cheap and fast. For the deep feature, a fixed-horizon 50/50 split is very likely the wrong tool entirely: either loosen your MDE further until the sample size matches your actual traffic (accepting you can only detect huge swings), or stop trying to force a classic A/B test onto traffic that can't support one, and reach for one of the alternatives from the next post in this series instead, a long-running hold-out, a qualitative read, or a Bayesian approach that doesn't demand a pre-fixed sample size to say something useful early.

[ THE PROBLEM: PEEKING ]

Here's where the dashboard-refreshing habit turns into an actual statistical problem, not just an impatience problem. Refreshing the page doesn't corrupt the underlying data, the numbers are whatever they are regardless of who's watching, but it does change your stopping rule. "Stop and declare a winner the moment p < 0.05" isn't one test, it's a test you get to retry every time you check.

And "check" has a specific meaning here: computing the p-value and using it to decide whether to stop, not just glancing at a chart. Opening the dashboard to see where the numbers stand is harmless on its own. It's treating that p-value as a decision point, "is this under 0.05, should I call it," that counts as a look.

Your 5% alpha threshold is a guarantee about exactly one such decision point, made after your pre-calculated sample size is reached. Every time you check the result before that point and treat "p < 0.05 right now" as a stopping signal, you're giving yourself another shot at the threshold, another roll of the dice for a false alarm, no new data required. Do that ten times over the course of an experiment and your real false-positive rate isn't 5% anymore. It's roughly 4x that, around 20%.

Here's the simple version of why. A single 5%-threshold look means a 5% chance of a false alarm, and a 95% chance of correctly seeing nothing when there's nothing there. That 95% is your "safe" probability for one look. Now take another look. If those looks were fully independent, both looks correctly showing nothing would need 95% and then 95% again, multiplying down toward zero the more times you check, ten independent looks would already put you near a 40% false-alarm rate.

Real peeking lands lower than that naive multiplication, because your looks aren't independent, each one is checking the same accumulating data as the last, just with a few more rows added. But the direction holds and the size is still large: more looks, more chances to get unlucky, and nobody told the dashboard to warn you which chance you're on.

FIG.4 — your chosen 5% only holds if you look exactly once

Intuitively: a p-value during a running test isn't a fixed property of your data, it's a noisy quantity that jumps around as samples accumulate. It can dip under 0.05 early by pure chance, climb back over, dip again. If you stop the moment it happens to be favorable, you're selectively sampling the noise, not the effect. The p-value only reflects the error rate you actually chose once the pre-calculated full sample size is reached. Everything before that is a preview, not a verdict, no matter how green the dashboard looks.

This doesn't mean the textbook rule, "never look, always wait for full sample size," is the only correct answer. It's correct if you need that specific 5%/20% guarantee, which you usually do in regulated or high-stakes contexts. It's adjustable if you're willing to trade away some of that precision on purpose: agree on a looser threshold up front, or use a testing method actually designed to handle repeated looks (more on that in a minute). What's not adjustable is pretending you didn't peek when you did, and reporting the original 5% anyway.

[ WHY IT KEEPS HAPPENING ]

This isn't a knowledge problem. Most people who work near experimentation can recite "don't peek" if you ask them directly. It keeps happening anyway, because the incentives in the room are almost never aligned with statistical patience.

A test trending toward a strong result after one week, against a five-week pre-registered horizon, puts real pressure on the room: waiting the textbook full duration for rigor has an actual cost in forgone revenue or forgone learning speed. A test trending the wrong direction creates the mirror pressure: cutting losses early feels responsible, and also inflates your error rate exactly the same way a favorable early peek does. Shipping pressure doesn't care which direction the needle is moving. It pushes toward stopping early either way.

The organizational failure mode compounds from here. A single test tells you a difference was detected, not the true magnitude of that difference. Teams that build quarterly OKRs by summing the observed uplifts of "winning" tests are stacking a series of noisy, possibly-inflated point estimates and treating the sum as a forecast. It rarely survives contact with the next two quarters. The statistics didn't fail here. The organization asked the statistics a question they were never built to answer.

[ ALTERNATIVES, BRIEFLY ]

None of this means fixed-horizon testing is broken or that peeking is unforgivable. It means the standard frequentist test answers one specific question, "is there a detectable difference at a preset error tolerance," and if that's not actually the question you're trying to answer, there are other tools built for the question you do have.

Sequential testing and always-valid p-values (mSPRT, group sequential designs) are built specifically to let you look as often as you want without the error-rate blowup, by spending your error budget across looks instead of assuming a single look. Bayesian A/B testing reframes the whole question away from p-values entirely, toward "what's the probability B is actually better, and by how much," and it doesn't require a fixed sample size to interpret at all. Both of these are real, useful, and deserve more than a paragraph each, which is exactly why they're getting their own posts in this series rather than a rushed footnote here.

[ THE MANAGERIAL PROBLEM ]

Even once the math is right, there's a second problem that's arguably harder: communicating a probabilistic result to a stakeholder who wants a yes-or-no answer by end of day.

"We're 87% confident this is an improvement, with a plausible range of 0 to 4%" is a true, useful sentence. It is also not the sentence most rooms want to hear. The fix isn't to round it down into a fake yes-or-no, it's to make the underlying decision structure explicit before you ever run the test, so uncertainty has somewhere to land. A workable hypothesis has four parts: if we do X, then Y will change by Z, because W. X is the change, Y is the metric you're watching, Z is the effect size you actually expect (which is also what feeds your sample size calculation), and W is your causal reasoning for why you expect it. W is the part almost everyone skips, and it's the part that compounds: a single test only tells you that something changed. Your working theory of why is what turns one test into a body of knowledge instead of an isolated anecdote you can't reuse.

[ WHAT WE ACTUALLY DO NOW ]

The practical version of all of this, generalized, not tied to any one team's exact setup:

Pre-register your sample size before you look at results, driven by your actual alpha, power, baseline, and MDE, not by vibes
If you must look early, decide that up front and either use a method designed for it (sequential testing, Bayesian) or explicitly accept a looser real error rate
Run continuous A/A tests on any traffic that isn't allocated to something real, they're nearly free and they catch splitter and tracking bugs before those bugs corrupt an actual test
Write the X/Y/Z/W hypothesis down before launch, so "why" survives even when the test result is ambiguous
Treat a null result as information, not failure, log it and move to the next hypothesis instead of quietly re-running until something turns green

None of this eliminates uncertainty. It just makes sure the uncertainty you're shipping with is the uncertainty you actually chose, instead of one that quietly grew while nobody was looking.

[ USEFUL LINKS ]

Evan Miller's A/B testing sample size calculator — the fastest gut-check on whether you have anywhere near enough traffic before you run anything
"The ASA Statement on p-Values: Context, Process, and Purpose" (Wasserstein & Lazar, The American Statistician, 2016) — the closest thing statistics has to an official correction of the p-value misunderstanding in this post, worth searching up directly
mSPRT / "always-valid p-values" (Johari, Koomen, Pekelis, Walsh) — the paper behind modern sequential testing at scale, more detail in the next post in this series

[ ELSEWHERE ]

GitHub · LinkedIn · Dev.to · Substack · 4thwithme.dev/blog

May the --force be with you. See you next week.

My dev setup

4thwithme — Sun, 23 Aug 2026 22:04:53 +0000

[ HARDWARE ]

[ Ecosystem ]

Let's start from ecosystem. Windows, Linux, and macOS all have their strengths, but my main requirements are:

Stability and reliability, so I can focus on work instead of rebooting or troubleshooting Wi-Fi, Sound, or Bluetooth drivers - so Linux is out.
Well-supported, and well-documented UNIX development environment - so Windows is out.

So macOS it is.

[ Hardware Requirements ]

Let's go to the hardware requirements. I need a machine that can run multiple tools at once without slowing down, and that can handle large codebases and datasets. Supporting Machine Learning and AI workloads is also a plus.

In 2026, to run all the tools I need, 16GB RAM is the absolute minimum.
SSD storage is a must for fast boot times and quick access to files.
A high-resolution display is important for productivity.
GPU support is a plus for machine learning and AI workloads.
Long battery life is important for working on the go.

Right now I work on a MacBook Pro 14" with M4 Pro chip and 24GB RAM, which fulfills all the requirements above. I also have a 4K external monitor, mechanical keyboard, and mouse for a more comfortable and efficient workflow.

Split is roughly 60% external monitor + mechanical keyboard + mouse
40% laptop as-is. No trackball, no split keyboard — I have many sides, but not that many.

[ Karabiner ]

Where the OS actually gets tuned is keyboard shortcuts.
It lets me add custom shortcuts, remap keys, and create complex modifications. For example, I have a rule that allows me to switch between English and Ukrainian keyboard layouts with a single keypress.

Language switching is bound directly. No cycling, no counting presses — you press the key for the language you want and you're already there.

Ctrl+E jumps straight to [E]nglish
Ctrl+U straight to [U]krainian.

FIG.1 — one keybind, one destination — no rotation involved

{
  "description": "Switch to English input source with Control + E",
  "manipulators": [
    {
      "from": {
        "key_code": "e",
        "modifiers": { "mandatory": ["control"] }
      },
      "to": [{ "select_input_source": { "language": "en" } }],
      "type": "basic"
    }
  ]
}

[ Mission Control ]

The same logic runs the desktop layout. I keep six Spaces, each hard-assigned to one thing. Ctrl+1 - Ctrl+6 jump straight to the matching Space. No "next/previous desktop" cycling here either, same reasoning as the language switch: a direct destination beats a direction you have to count.

browser
IDE - my main editor, where I do the bulk of my coding and debugging
terminal - my main workspace, where I run Claude Code, git, and other CLI tools
free space for DB/Redis GUIs and OrbStack - my dev tools, local servers, and containers
Slack - work chat, async comms, and incident response
Obsidian - my 2nd brain, where I keep my notes and plans.

FIG.2 — Ctrl+[number] beats Ctrl+left/right, every single time

Ctrl+A pulls up the Mission Control overview when I actually need to see all six at once. Every window fills its Space at 100%, but I keep them in windowed-fill mode rather than native macOS fullscreen — same usable area, but the menu bar and dock stay one motion away instead of a whole gesture away.

FIG.3 — six desktops, six fixed jobs — the same one is always in the same place

[ TERMINAL ]

My main working tool and my main terminal is Warp. No tmux, no zellij, no herdr, no multiplexer at all, even though I've used a few of them and they all worked fine. But we are living in 2026, not in the wild 2010s. Warp already has tabs, workspaces, and project-grouped sessions natively, with an AI layer on top. I have no nostalgia for multiplexers, sorry ;)

FIG.4 — tabs grouped by project, not by panes I have to remember the layout of (branch names blurred, not a leak)

One deliberate exception: Neovim gets its own terminal app entirely — Rio or Ghostty, kept separate from Warp. Not because Warp can't run it fine, but because I don't want my editor's session sharing chrome, history, and AI panels with everything else I'm doing. It's not the 90s, my machine can afford a second terminal app just so nvim gets a clean box to live in.

[ EDITOR, BRIEFLY ]

Three editors plus one thing that isn't really an editor. We will dive deep into Claude Code and nvim in future posts. In the meantime, here's the rough split of my time:

Neovim gets 20% of my time.
Zed - 15% of time, because it's fast and occasionally I have a mood for it.
My old buddy VS Code - 15%. It was my first IDE and some habits just don't leave.
The other 50% is Claude Code, and for that half of my week I don't open an editor at all. The work happens in the terminal, in diffs, in review — the editor becomes optional infrastructure instead of the default starting point - this is the reason why terminal has to be modern, fast, and AI-enabled.

FIG.5 — nvim, mid-config — fittingly, the plugin on screen is the one that hides my secrets

Almost all my IDEs are configured with the same philosophy: minimal, fast, and keyboard-driven. I don't need a million panels and toolbars, no terminal embedded in the IDE, no DB connectors, no Git GUI, no debugger panels — I have other tools for all of that. I just need a fast editor that can handle large files and projects. If I cannot find a plugin or theme that fits my needs, I write one myself. I have a few of those.

[ AI TOOLING ]

I've been using Claude Code for almost two years now — long enough that opening a traditional IDE feels like the exception, not the default. I run it straight in the terminal, doing the actual work: planning, multi-file changes, the stuff that used to mean opening an IDE and doing it by hand. To stay aware of what a session is actually doing, I use the ccstatusline plugin, which keeps session cost, model, and context usage visible so I'm not flying blind on a long-running task. GitHub Copilot's inline autocomplete still runs alongside it for the moments I'm typing directly — the two aren't competing, they're solving different-sized problems: Copilot finishes your line, Claude Code finishes your ticket.

[ THE ACTUAL WORKFLOW ]

My working day:

Review Claude Code — whatever overnight/async sessions did while I wasn't looking, leaving feedback or new instructions where needed.
Then Slack: blockers, incidents, and whatever business or engineering questions came in overnight.
Then New Relic and Rollbar for anything performance- or error-shaped, plus a daily AI-generated report that's already combed through the logs for me so I'm reading a summary instead of raw noise.
Then PR review. I prefer to do it in the browser default GitHub format. When I see GitHub, my brain switches to "code review mode" and I can focus on the diff, and it is easier for me to think about bottlenecks, edge cases, and the overall quality of the code.
Then JIRA for tickets, planning, and triage.
Then finally I check my own plan for the day written in Obsidian — which is a whole separate post, because "second brain" is a real thing, and it deserves a real writeup.

[ EVOLUTION ]

My setup has been stable for years at its core, but I'm not precious about it — I'll happily give a new tool a real two-to-three month trial to see if it actually brings something new and cool to the workflow. Most of them don't survive contact with what I already have. Raycast got a fair shot and never earned a place — Spotlight already does what I need, and a second launcher just for the sake of having a "better" one wasn't worth the context-switching. Same story with a git GUI — not VS Code's, not LazyVim's — terminal git stays, mostly because I trust what I can see happening over what a panel summarizes for me. My latest churn: Rio → Ghostty → Rio, right back to where I started after giving Ghostty a genuine try. That's the pattern — I'm open to switching, the current setup just keeps winning.

[ ELSEWHERE ]

GitHub · LinkedIn · Dev.to · Substack · 4thwithme.dev/blog

May the --force be with you. See you next week.

Starting my weekly blog (ML, agentic dev)

4thwithme — Mon, 17 Aug 2026 10:01:42 +0000

Post source: https://4thwithme.dev/blog/intro/

Nine years ago I was calculating neutron flux in a reactor core simulation. Today I spend my mornings in 1:1s and my evenings building MVPs, and ML recommendations models. Somewhere between those two sentences is the short version of how I got here. The reactor, for the record, did not explode.

I'm Andrii - an Engineering Manager based in Barcelona, currently leading a team of engineers at a promo-products ecommerce company. My path here was not exactly linear, it started with a physics degree instead of a computer science one: a Master's in Nuclear Power Engineering, a couple of years as a neutron physics engineer running Fortran and Python for reactor calculations, which is where the programming itch actually started, at the same time a few years teaching robotics and programming to kids who debugged with more patience than most senior engineers I've since worked with. Then a full pivot into software, no looking back, mostly because there was nothing to look back at except radiation shielding calculations.

Who I am

If you want the honest ratio: I'm about 70% engineer, 30% manager. I still want to be in the code, still want to know why the model regressed and not just that it did, and I'd rather spend an evening tinkering with a training run or working through the math behind it than sit through another roadmap slide. Management is real work and I take it seriously, but the itch that got me here in the first place is still curiosity about how things actually work, down to the math.

day job

I lead the team responsible for recommendation and search on a storefront that has strong opinions about branded mugs, the kind of place you order 200 branded pens from and never think about again until the next company retreat. Lately my job is less "ship the feature" and more "figure out how a whole engineering org adopts AI tooling without either turning it into a personality or pretending it doesn't exist." Both camps are loud. I try to be the annoying third option that just ships things.

outside the day job

I'm an AI/ML enthusiast in the fullest, most unqualified-to-stop-talking-about-it sense, hands-on with code, and hands-on with hardware too. I like tinkering with models, sitting with the math until it actually clicks instead of just pattern-matching the notation, and reading research papers slower than is efficient because skimming defeats the point. I write Neovim plugins nobody asked for, I'm working through computer vision fundamentals, and I have a Three.js habit that has produced zero shipped products and several very nice spinning cubes.

Building / learning

Computer vision, slowly, deliberately, mostly on Sundays
The math under the models I use daily, not just the API surface
A Neovim plugin nobody asked for (ss.nvim)
An unreasonable Three.js habit with a 0% shipped-product conversion rate
Whatever this blog forces me to actually finish, since public commitment works better than a private todo list

Why this blog

"Cold start" is a recommender-systems term. No data on a new user or item, and you still have to guess something useful, confidently, with nothing to go on. That's also just what most Mondays feel like. So the rule is simple: one honest post a week, no excuses, a threat I'm making primarily to myself, and no quiet retirement of this blog in week six like every New Year's resolution I've ever made.

Expect posts on ML, agentic development, Claude Code, nvim, new tools I'm evaluating, and the occasional controversial take I'll defend confidently for about a week.

Elsewhere

GitHub · LinkedIn · Substack · 4thwithme.dev/blog

May the --force be with you. See you next week.