DEV Community: Keenen Wilkins

Excel Turned a 20% Discount Into 19.999999999999998%. That Hid $2,325.

Keenen Wilkins — Thu, 08 Oct 2026 14:09:49 +0000

The business rule was one line. Any discount of 20% or more is a markdown.

My markdown totals were low, my growth rate was high, every formula in the sheet was correct, and nothing on the screen looked wrong.

What the week was

Week five of i.c.stars was Geek Week. Fifteen tasks against a sales database, each with an expert time attached. Thirty seconds for the shortest, three and a half minutes for the longest. Hitting the time was the only way to pass, so finishing was not the bar. Finishing fast enough was.

During the timed runs: no AI, no mouse, no trackpad, no copy and paste inside Excel. Everything through keyboard shortcuts. You could use whatever you wanted while learning. In the chair, it was you and the keys.

I finished all fifteen inside the expert times by Friday.

The dataset is Northwind, Microsoft's sample database, so anyone can pull it down and run every query in this post. Orders, products, categories, discounts, and the employee who made each sale.

The framing is a company whose sales grew 73% while gross profit did not follow. The CEO suspects discounting is eating the margin and is weighing whether to lay off one of the sales reps. The analysis is supposed to tell her whether she is right.

I spent two years as a sales rep carrying 55 accounts before any of this. I have been the person in that data, discounting to close something before the end of a month. So the question was not abstract to me.

The number that mattered

Not discounts in general. Markdowns specifically, because the business rule splits them: 5 to 9% is low, 10 to 19% is medium, and 20% or higher is a markdown.

A lookup table does the sorting and VLOOKUP applies it to every row. The TRUE at the end is the part I had to look up. It means range lookup, so the table only needs the bottom of each band rather than every possible value:

=VLOOKUP(E2,discountcategory,2,TRUE)

Then a report sums only the rows that came back labeled markdown:

=SUM(IF(Order_Date >= B$1,
     IF(Order_Date <= B$2,
     IF(Category_Name = $A3,
     IF(discount = "markdown", SaleDiscount)))))

Run it and the answer is loud. Markdown dollars last quarter came to about $12,200, up 557% over the same quarter the year before.

The CEO's instinct was right. The answer to her other question is no. A later task ranked discounts by the employee who gave them over the last twelve months, and the three biggest were King, Peacock, and Fuller. Fuller is the Vice President of Sales. When the head of sales is near the top of the list, that is not a rep you fire. That is a pricing problem.

That is the number the whole week is built to produce. And the first version I had was wrong.

Why 20% is not always 20%

Here is one real row from my data. A Beverages line, $14.40 a unit, quantity 45. Sale price $648.00, discount $129.60. That is exactly 20% off.

To get the percentage, I divided the discount by the price:

=D2/B2   ->  0.19999999999999998

Excel stores numbers in binary, and 129.6 has no exact binary form. It is held as 129.599999999999994..., so the division lands a hair under 0.2. Northwind stores Discount as REAL, but that was not what bit me. The division was.

That value is less than 0.20. So VLOOKUP checks it against the table, correctly finds it below the markdown band, and files it as medium. The row is real, the discount is real, and it is missing from the one total the executive team asked for.

It was not one row. 30 of 315 markdown order lines landed under the line, and $2,325 in markdowns got filed as medium. A fresh copy of Northwind gives 31. Different copies round differently.

The fix is one function, rounding the percentage before it reaches the lookup so the comparison happens at the precision the business rule actually cares about:

=ROUND(D2/B2,2)   ->  0.2

And it was wrong in a way I did not expect. Without the fix, the totals came out too low, but the growth rate came out too high: 585% instead of 557%. The early quarters lost proportionally more than the recent ones, so the baseline shrank faster than the top and the growth inflated. One bug, wrong in both directions.

None of this is an Excel bug. It is how binary floating point works everywhere, and it has been written down since 1991: What Every Computer Scientist Should Know About Floating-Point Arithmetic, David Goldberg, ACM Computing Surveys.

I did not catch this one myself. The AI I was learning with flagged it before I ever ran the report: round first, or some 20% discounts come out as medium. A month ago I would have pasted that in and moved on. This time I did not take its word for it. I went back to the data and checked: 30 rows, $2,325, and a growth rate that was too high.

There is no version of this where I read the formula and spot it. The formula is fine. The error lives in the gap between what a number is and how it is stored, and that gap does not appear on screen.

It kept happening

Once I started looking for that shape, the week was full of it.

A letter where a zero should be. One task wanted the discount formatted as a percentage. I wrote 'PO' with a capital O instead of 'P0' with a zero:

FORMAT(od.Discount, 'PO')   -- every row returns the text "PO"
FORMAT(od.Discount, 'P0')   -- 0.15 -> 15%

Not an error. Not a blank. A column full of confident nonsense.

Rows in the wrong order. My results had every correct row and still did not match the expected output. Sorting by category only controls the order of the categories. Inside each one, the database can return rows in any order it likes unless you give it another key. Adding product ID and order ID to the sort made it match. The query was never wrong. It was underspecified.

A number that was true and misleading. One employee showed over 3,000% growth in a quarter. That is accurate. He went from about $500 to about $16,000. When the starting number is tiny, the percentage explodes. The math is correct and the conclusion you would draw from it is not, which is its own category of wrong.

Learning to read the errors

The errors that did announce themselves turned out to be the easy half, once I stopped treating every failure as one event called "it broke."

Before I could read them, every failure cost me minutes of guessing. After, most cost seconds. Nearly all of my practice time went here rather than into learning new formulas.

The ones that scared me were the ones with no error at all.

What the time limit was actually testing

I assumed a thirty second limit was testing whether I knew the function. It is not. Thirty seconds is barely enough time to type it. It tests whether you can do it without stopping to think, which is a different thing and you only get it from repetition.

The useful discovery was that rushing made me slower every single time. Every hurried run produced a mistake that cost more to find than the seconds it saved. I hit the times by doing each step deliberately and correctly until the sequence stopped needing my attention, and the speed showed up on its own after that.

The other half is that most tasks did not start from zero. They started from the last report. One array formula, built once, becomes a different report by swapping a single name with Find and Replace. Sales by category becomes discounts by category becomes discounts by employee becomes sales growth by employee. Four reports, three keystroke sequences.

The no-mouse rule is what forced that. Taking the mouse away removed the option of doing things the slow familiar way, so I had to learn Ctrl+G to jump to a named range, Ctrl+Enter to fill a whole selection at once, Ctrl+H to rewrite every formula on a sheet in one pass. I am now faster with those than I would have been if I had been allowed to point at things.

The part I want on the record

Last week I published a piece about shipping something I could not personally defend, because a model wrote the code and I had not verified it.

This week I could use AI to learn, but not in the chair. And when it told me about the bug that mattered, the one that would have handed an executive team the wrong number about the only thing they asked about, I did not just accept it. I measured it.

That is a smaller claim than "I found the bug," and it is the true one. The thing I am actually practicing is not catching every error myself. It is refusing to pass along a number I have not seen come out of the machine, no matter who handed it to me.

Last month I picked a threshold by reasoning about it instead of measuring it. This week I measured something I was told. That is the direction I want this going.

What I would tell somebody doing this next

The dangerous errors are the ones that produce output. A crash tells you where to look. A column of "PO", a total that is low by exactly the rows that got misfiled, a growth rate of 3,000% that is technically accurate, a 20% discount sitting in the medium band: none of those tell you anything. They sit there looking finished.

So the habit worth building is not writing better formulas. It is running the thing and checking whether the answer is the size you expected, before anyone else sees it.

What is the last number you shipped that was wrong and did not look wrong?

I'm back on the client project now.

Similarity Is Not Authority

Keenen Wilkins — Wed, 30 Sep 2026 19:18:27 +0000

Week four of i.c.stars produced four documents. A defect log. A peer review of somebody else's defect log. Three runbooks. A run of show.

None of them are code. All four turned out to be the same document.

What the testing session actually taught

We were given a portfolio site with bugs planted in it and told to find them and write them up.

The format was given, when, then. Given each nav link opens the page it names, when I click Gallery, then the Gallery page opens. Then the defect log underneath it: steps to reproduce, expected, actual, severity.

My group ran three tests. All three failed, all three high severity. Gallery opened About Me. The third image opened the second. The contact email would not open a message.

Writing that up took longer than finding any of it.

Then the part that made the session worth it. We traded reports with another team and had to go reproduce their findings on the live site and mark each item verified or not verified.

Six of their eight held up. One I could not reproduce at all. One only reproduced on the machine of the person who reported it, and their write-up never said which machine, which browser, or which page they started from.

The verdict I wrote at the bottom of that review: strong at finding bugs, weak at recording them.

That is not an insult. It is the normal failure. Finding a bug feels like the work. Writing it so a stranger can hit it again in ninety seconds is the actual work, and nothing tells you whether you did it except handing it to a stranger.

Three runbooks

Later that week, presentations, and before them, runbooks. Murphy's Law applied to a room. Write down what you do when it breaks, before it breaks.

Setup, happy path, edge case, recovery script. I wrote three.

Slides stop advancing. The clicker dies or animations fire on their own and the deck runs ahead of whoever is talking. Stop clicking. Say "Looks like the slides want to get ahead of me, so I'll talk you through this part." Keep going off the printed outline while a teammate advances by hand.

A handoff breaks. The next speaker misses the cue and the room sits in silence. Bridge without stopping, then hand off at the next slide. The rule underneath it is the one worth keeping: every presenter knows the main point of every section, not just their own.

The laptop or projector dies. "While we get the screen sorted, let me tell you what you would be looking at." Present from the printed outline. And the line that takes actual discipline:

Do not spend more than 60 seconds troubleshooting in front of the room.

Then a preflight list. Deck in three places including a PDF with no animations. A printed outline per presenter. Cue lines agreed in advance. One teammate named as backup on the controls.

A run of show

The fourth document. Every section of the presentation with a named owner and a time block. Not "the team covers the design." A person, a topic, a number of minutes.

And scripted transitions. The exact words, written down, for every handoff between speakers. That felt excessive when I was filling it in. It is the single reason three people who are not presenters got through it sounding like one team. Nobody had to invent a sentence in front of an audience.

My sections were the solution design and the reasoning behind it, then the logic and the wireframes.

The thing I actually contributed to the product

Here is where I should be precise, because it would be easy to imply more than is true.

I did not write the retrieval code. A model wrote most of it, fast, and that is why we had something standing in four weeks instead of twelve.

What I brought was the argument about what the system is not allowed to do.

The problem, stripped of anything about the client: volunteers at a nonprofit need answers about how their organization works. The answers live in documents the organization has already approved. It looks like a search problem. Store the documents, take a question, find the best-matching passage, answer with it.

That framing is wrong in a specific way, and noticing it was my contribution.

Two documents can both apply to the same person. Both official. They disagree. One of them overrides the other, but only in some places and not others, and nothing inside either document records that.

A similarity search cannot see that, ever. It can tell you two passages are about the same subject. It has no way to express that one of them governs and the other does not. Override is a relationship between documents. Similarity is a score on a single document. You cannot get the first from the second by ranking harder.

So the design has two behaviors and no third one. Either the system answers with the document, the section, and the effective date attached, or it stops and hands a briefed summary to a person. Checks run before anything gets written and decide which of the two happens.

The refusal is not a fallback. It is a feature with equal standing. A confident wrong answer about a governing rule can invalidate a decision that real people already made and acted on. A refusal costs somebody ten minutes.

That is the work of a solutions analyst, which is my role on the team. Not writing the function. Deciding what the function must never be allowed to conclude.

Four documents, one idea

Look back at the week.

A test case is written so a stranger can reproduce it. A defect log has severity on it so somebody else can decide what to fix first. A runbook exists so the presentation survives when the person holding the clicker cannot continue. A run of show has named owners so nobody waits to find out whose turn it is.

And the product rule I argued for is the same shape: when the system cannot prove which document governs, it stops and hands the question to a human, with everything that human needs to answer it.

Every deliverable that week was about surviving a handoff. The workshops were not teaching four skills. They were teaching one, from four angles, and the fourth angle was in the product.

Where it actually stands

We presented our initial solution. Slides, in a room, fifteen minutes.

We did not demo. That was my call, and my reason at the time was that this round was a solution presentation rather than a demo, and I wanted to be sure the thing worked for this specific organization before putting it on a screen.

Then one of the judges said presentations should demonstrate, not read.

We used eight and a half minutes of our fifteen. My closing line started with the words "I guess." We did not win the round.

And the part I have not resolved: the prototype runs, the tests pass, and I have not personally verified either of those claims. A model built it and a model checked it. I wrote the rules it has to obey and I have not yet sat down and confirmed it obeys them.

Which is a strange position to argue from in a room. I could defend every design decision in that solution. I am not sure I could have answered a hard question about the code underneath it.

That gap is the actual finding from week four, and it is not one a worksheet covers.

What is coming

I am in Geek Week as I write this. In the office all week, fifteen tasks passed in sequence, three attempts each before you move on and come back. No AI tools. Notes at the table while practicing, nothing during the test.

It is the first week where nothing I produce is produced by anything but me.

If you have built something with a model recently: what did you personally verify, and what did you take on faith because it ran?

The next one covers Geek Week.

I Am a Single Point of Failure

Keenen Wilkins — Mon, 21 Sep 2026 14:32:47 +0000

If someone can't follow it without you, it's a memory, not documentation.

I heard that on a Monday, in a session about writing standard operating procedures. I wrote it down thinking it was about documentation.

By Friday it had described two different failures of mine, and the second one had nothing to do with writing anything down.

The obvious version

Last week I mapped my default work habit as a flowchart and the left branch looped back on itself. I take the work because I already know how to do it, nobody else learns it, and next week I am still the only one who knows.

That line is the same loop in one sentence.

Everything I take on because I already know how to do it lives in my head. It feels like productivity while I am doing it. What it produces is a memory, and a memory leaves the building when I do. The loop was never that I work too hard. The loop is that I keep generating knowledge that has exactly one copy.

The session's worked example was deliberately stupid: how to log into your laptop, written out to the level of which button is the power button and where it sits on the keyboard.

My first reaction was that this is over-explained. That reaction is the problem. "Everybody knows that" holds up right until the person who knows it is out sick.

The same idea pointed at a team

The other framework this week was Role, Signal, Handoff.

Role. What you are specifically responsible for.
Signal. How anyone else knows it is done.
Handoff. How the next person receives it and confirms they got it.

Weak version: "I handle research." "I finished my part."

Strong version: find three credible sources, put the key points on slide 2 by 1 PM, upload it and tag the person who needs it next.

The difference is not effort. It is whether the work is visible from outside your own head.

We have our roles set and our signals defined. Handoff is the part we are still building, which is predictable, because handoff is the only one of the three that requires somebody else to confirm they got it. The first two you can do alone.

The same failure at company size

Friday was the i.c.stars iOpener Innovation Conference. The theme was "KC's AI Boom: Billions Invested, What Comes Next?" Banks, engineering firms, law firms, a school district.

I went in expecting infrastructure and models. Most of what I heard was about documentation.

Traci Wills, who runs enterprise portfolio management and strategy execution at NAIC, put it plainly: the barrier is not access to AI. The access is there. The barrier is organizational readiness. Companies pilot fine and then cannot scale, because there is no clear problem statement, no trusted data, and governance that exists on paper without changing what anyone actually does on a Tuesday.

Then Craig Moore II of Endeavor Heartland said the thing that made me put my pen down:

If you can't do it manually, you're not going to be able to automate it. If you don't have SOPs, if multiple people in your organization cannot speak to the same process the same way, AI is going to be hard for you.

That is Monday's session with a budget attached. A company where one person can describe a process is a company running on memory instead of documentation. Same failure as mine. More zeroes.

Dianna Keen, Managing Director of Technology at VML Enterprise Solutions, gave the version that stuck:

Whatever is true about that system at the time is what you're scaling. If it's bad, you're scaling bad.

Scale is not a fix. Scale is a multiplier on whatever shape is already there.

One shape, three sizes

By Friday afternoon I had seen the same diagram three times in five days.

One person. I already know how to do it, so it never gets written down, so next week I am still the only one who knows.

A team. Roles that sound like "I handle research," so nobody says what done looks like, so the work stalls between people.

An organization. A process only one person can describe, so the pilot works and production never ships.

Something valuable exists, it has exactly one copy, and the fastest path today guarantees you make the same decision again next week.

At every one of those scales, the problem is rarely that the knowledge does not exist. It exists and cannot be reached. Somebody absorbs that friction either way. The only question is whether it lands on the one person who knows or the however-many people who have to go ask them.

The fix is the same at every size and it is boring. Write it down so it survives without you.

The fourth size

Here is the one I did not see coming, and it is smaller than all three.

On Thursday we had a roundtable Q&A with our client. Every team got to ask. I asked one.

Not a throwaway. I am the solutions analyst on my team and I could not tell where this project was supposed to end. The only people who could answer that were sitting right there, so I asked them.

That is the exact thing I told you all I would start doing. I did it inside of a week.

It was also the easiest possible version of it. Clear role, specific gap, question written down in advance. If speaking up were always that structured I would not have a problem.

Friday was not structured.

The table exercise: your table gets a symbolic one billion dollars. Allocate all of it to one of six AI priority areas. Then imagine it is 2031 and it worked. Write the headline.

I contributed. I also choked.

Both are true and the difference between them is the useful part. I said things at my table. They were not the things I was actually thinking. The gap between what I had in my head and what came out of my mouth was wider than it has been in any session so far, and the only variable that changed was who was sitting there.

Then I walked away from the table and networked for the rest of the event with none of that friction. Real conversations. People I want to talk to again.

So I am not quiet. One on one I am fine. What shuts me down is a group of people I have not earned anything with yet, talking about something I have an opinion on.

Which brings it back around to Monday.

An opinion I do not say out loud is knowledge with exactly one copy. It exists. It is in one head. Nobody else can act on it, build on it, or tell me I am wrong about it. Next week I am in another room with the same thought, having the same internal argument, and nothing about the situation has changed.

That is not a personality trait. That is the same loop I drew last week, running at the smallest scale there is.

A memory, not documentation.

Looking ahead to week four

Finish the handoff half of Role, Signal, Handoff, which means actually confirming receipt instead of assuming the work landed.

And find the smaller version of Friday. Not a keynote. A room of eight or ten people where I have an opinion and no assigned role that requires me to speak. That is the rep I am missing, and asking a prepared question at a structured Q&A does not count as having done it.

What is the thing you know how to do that nobody else on your team could run without you?

That is not a skill. That is a single point of failure with your name on it.

Week four goes up Friday.

I Drew My Worst Work Habit as a Flowchart. It Was a Loop.

Keenen Wilkins — Sun, 13 Sep 2026 18:20:08 +0000

Last week I wrote that when a project gets hard and progress feels slow, I take on more work instead of asking for help. I put that in public on a Friday.

By Tuesday the structure of my week was handing me every reason to do it again. By Friday I was sitting in a one on one hearing about a second problem I did not know I had.

The math

Project teams here are built for four or five people. Mine has three.

When a team is short a person, that work does not evenly redistribute. It falls on whoever knows the most about the thing in front of you. On our project right now, for a few pieces, that is me.

So the path of least resistance was obvious. Absorb the extra, move faster in the short term, look productive. It is the exact behavior I had just finished admitting was a weakness, and the situation was practically arguing for it.

What I tried

Load balancing. Deliberately letting my teammates learn the work instead of handling it myself.

I told them what I needed. Not vaguely. I said what support actually looks like for me and why it would help us hit the deliverable. That is harder than it sounds when your instinct is to go do the thing quietly.

I stopped handing over answers. When we got to the core concept behind our solution, I had a head start on it. The old version of me writes it up and presents it finished. Instead I asked them to go learn the base concept so we could build on it together.

We wrote down how we work. A session on trust and team agreements turned what I had been doing informally into something structural. Last week I wrote about trust as an equation. This was the operational version. On a three person team those agreements are not bureaucracy. They are how we hold each other accountable so all three of us grow, and how we cover a role we do not have a person for.

Something did change. They are actively understanding the solution instead of receiving it. When we talk about it now, it is a conversation rather than a briefing.

My week as a flowchart

We covered flowcharts, swimlanes, and UML this week. Mapping a process, showing who owns what, making the handoffs visible.

The whole time I kept thinking that if I diagrammed my own default behavior, it would not look like a line. It would look like a loop.

flowchart TD
    A["Work lands on the team"] --> B{"Am I the one<br/>who knows it best?"}

    B -->|"take it"| C["I do it myself.<br/>Fastest path today."]
    C --> D["Nobody else learns it"]
    D --> E["Next time I am still<br/>the only one who knows"]
    E --> A

    B -->|"balance it"| P["Say out loud<br/>what I need"]
    P --> G["Hand over the concept,<br/>not the answer"]
    G --> H["They learn it with me"]
    H --> I["Next time three people<br/>can carry it"]

The left path loops back to where it started. That is the part I had not seen before I drew it. Taking the work myself is not a one time cost I pay to move faster today. It is a cycle that guarantees I face the same decision again next week, with the same answer waiting, because nothing about the situation changed.

The right path ends.

Then I had my one on one

I went in with my own report ready. Here is the thing I wrote about last week, here is what I have been doing about it, here is where I think I stand. On the development rubric I put myself at proficient with room to grow on emotional intelligence and self-awareness. Patience, and meeting my teammates where they are instead of jumping in with I will just do it, went on my own list as the thing I am actively working on.

I was prepared to talk about all of that.

My program director wanted to talk about something I had not put on the list.

The thing I did not bring up

The growth area she flagged was not the work. It was my voice.

Observant, thoughtful, and tends to hold back. Other people in the room would benefit from hearing more from me, and they are not, because I do not say it.

I sat with that one, because at first it sounds like the opposite of the problem I walked in ready to discuss. How can I be the guy who takes over everything and also the guy who does not speak up?

They are the same thing.

Doing the work quietly and staying quiet in the room are both ways of skipping the exposed part. If I take the task, I never have to say I think we are heading the wrong direction. If I sit on a thought in a session, I never have to find out whether anyone disagrees with it. Both look like humility from the outside. Both are me picking the version of the situation where I cannot be wrong out loud.

Taking over is not confidence. It is the quiet option that happens to produce output.

Seek first to understand, then to be understood.

Stephen Covey, The 7 Habits of Highly Effective People

I had heard that line before. It landed differently this week, and not on the half I expected.

I have been working on the first half all week. Understand first, do not arrive with the answer already formed, let people get there with you.

The second half is the one I keep skipping. Being understood requires actually saying the thing.

The tea

High Tea started this week, which means the job I described last week as something I did not understand yet is now something I have actually done.

It turned out to be less about the guest than I expected.

It is a roundtable. Everybody sits at one table, and you introduce the person next to you and pour their tea. Literally pour it, into their cup. The host opens with the guest, it moves around the table, and the last seat comes back around to introduce the host.

Which means you cannot phone in your introduction. To introduce somebody you have to have asked them something first. The format quietly requires you to go learn a little about the person sitting beside you before you ever sit down.

We sign up and work out as a team who sits where. I keep the document that holds all of it, which this week meant building in a spot for the guest of the day and a sign-up for who helps set up.

Here is the part I did not expect. A format where every single person is handed a turn to talk still does not guarantee everybody gets heard. Making sure each person gets a fair shot, or lands a real question during the session, is an open problem we are still working on.

So I got told Friday morning that I hold back my voice, and then spent Friday afternoon inside a structure purpose built to give everyone one, watching it not entirely solve the problem either.

If a format designed specifically for this still has to be worked at, my own quiet is not going to fix itself by accident.

Nobody asked for the recap one sheeter. I started making them after a session because of what I want them to be good for later. My hope is that six weeks from now somebody in my cycle hits a situation, remembers a guest who talked about exactly that, and can pull up the profile I built instead of trying to remember a conversation from a Thursday in September.

My program director read it as something that could become a running record people come back to, for reference or for motivation. That is close enough to what I was going for that I am going to keep making them.

The hard part is not the writing. It is staying present. I have to catch everything with weight to it while also actually being in the room getting something out of it myself. Those two modes fight each other.

Thursday guest was Dominic Barrett, a Cycle 5 graduate who works as a photographer and a software engineer.

Friday was Shanda McConnell, who runs corporate relations here. Twenty-one years at one firm, then fourteen months of getting to this one. She organizes what she says yes to around an acronym she has been carrying for about a decade: BEE. Build, equip, empower. If something does not fit, she has learned to decline it, even for a season.

The line I wrote down came out of a stretch when she was working sixty and seventy hour weeks and could not physically be everywhere she needed to be. She had to start asking her community for help, and she was honest that it was hard at the time.

Asking for help is a sign of wisdom, not weakness.

I have spent two weeks now writing publicly about a pattern where I take on more work instead of asking for help. On Friday morning I got told I also do not speak up enough. On Friday afternoon a guest sat down and said that.

I did not go looking for it. I just happened to be the one taking notes.

Looking ahead to week three

Two things to carry forward, and only one of them is new.

Keep load balancing, and find out whether it holds when a deadline gets close and the fastest path is to take the work back. Look at the flowchart again. The left branch is not there because I am careless. It is there because it works, right up until it does not.

And start talking. Not more words, just the ones I am currently swallowing. If I have a read on something in a session, say it in the session.

I do not know how either goes. I will report back.

If someone told you that you both take over too much and speak up too little, would you have seen those as the same problem? I did not, until this week.

Week three goes up next Friday.

I Put a Database on the Public Internet. My Flow Logs Knew Before I Did.

Keenen Wilkins — Tue, 08 Sep 2026 17:41:37 +0000

I built a lab in AWS to watch an attack happen. Terraform provisioned a VPC, I put a PostgreSQL database on the public internet on purpose, pointed a script at it, and piped VPC Flow Logs into Splunk so my partner could watch from the SOC side.

Then I wrote an incident report describing a single external threat actor conducting a systematic port scan.

Months later I went back and read my own flow logs line by line. There were twenty-eight source addresses in the sample. I had written about one of them, and it was the wrong one.

The setup

One VPC at 10.0.0.0/16, two subnets across two availability zones, both public. An RDS PostgreSQL instance with the line that makes security people wince:

publicly_accessible = true

and a security group allowing 0.0.0.0/0 inbound on 5432, 22, and 80. Deliberately wrong, in the specific way production gets it wrong.

Flow logs to S3, S3 into Splunk. A Node script ran a dictionary attack against db_admin on 5432, knocked on 22 and 80, and mixed in successful logins with the real password so the SOC side had to separate authorized traffic from attacks. Three different jitter ranges so none of it looked like a metronome.

What was actually in the logs

A 1,000-line sample from that capture:

Source	Lines	What it was
`10.0.1.149`	445	My own traffic, inside the VPC
`194.87.190.127`	245	The simulated bot
`136.35.186.87`	183	A real internet actor
25 other addresses	1 to 3 each	Internet background radiation

876 ACCEPT, 124 REJECT.

The three noisy sources are the ones I designed. The twenty-five quiet ones are the story, and I never looked at them.

The quiet ones were not mine

Here are destination ports that appear in the REJECT lines:

6379   Redis
5902, 5931, 5937, 5959   VNC
10250, 10255   Kubernetes kubelet
137   NetBIOS
8008, 8022, 8081, 8088, 8090, 8103, 8389   assorted alt-HTTP

Nothing in my lab ran Redis. Nothing ran VNC. There was no Kubernetes cluster and no NetBIOS. My script targeted exactly three ports.

Twenty-nine distinct ports show up in the rejections. That is not one attacker sweeping a host. That is the internet's ambient scanning layer finding a fresh public IP and checking whether anything soft is listening. The source addresses back it up: several sequential hosts in one cloud range, a scattering of others across hosting providers known for this.

Every one of those lines arrived because I set one boolean to true. Not one of them appears in the incident report I wrote.

What I put in the ticket instead

Threat Actor IP Address: 10.0.1.179
... an external threat actor is systematically scanning for open database and secure shell ports.

10.0.1.179 sits inside 10.0.1.0/24, one of the two subnets I had defined twelve lines earlier in my own Terraform. RFC 1918 reserves 10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16 for private use, and none of it routes on the public internet. If a private address is in your srcaddr field, whatever produced it was already inside your network.

I had a genuine external actor in the same dataset, 136.35.186.87, doing a real multi-port scan. I escalated the wrong IP and labeled my own traffic as the threat.

Three things flow logs will not tell you

Not who authenticated. A flow log record carries source and destination address, ports, protocol, packet and byte counts, timestamps, and ACCEPT or REJECT. No payload, no usernames, no auth result. My script really was throwing bad passwords, and my analyst notes really did say credential stuffing, but the flow logs were never the evidence for that. A backup job with a broken retry loop looks identical at that layer. Authentication failures live in the RDS PostgreSQL error log, which is an entirely separate pipeline.

Not what a big number means. My ticket led with 15,025 blocked events because it was the largest number on the dashboard. I never worked out which rows produced it. Looking at the sample now, rejections are spread across twenty-nine ports, most of which nothing in my lab ever touched, so that headline figure was mostly counting strangers rather than my simulation. I could have known that in five minutes by grouping by port. Instead I put it in a SEV-1 and moved on, and by the time I thought to check, the account was closed and the data was gone.

Not intent. ACCEPT means the packet passed your security group. It does not mean something answered, and REJECT does not mean you blocked an attack. Both are network-layer facts about your own configuration, not judgments about the traffic.

The part that held up

One line:

publicly_accessible = false

terraform apply, then re-run the simulator:

$ node attack_sim.js
[+] Initiating connection to database...
[-] Error: Connection Timeout. Network unreachable.

Worth being precise about what that flag does, because I got this wrong too. It stops RDS from handing out a publicly resolvable endpoint. It does not move the instance to a different subnet, and my DB subnet group still contained the same two public subnets it always had. My report claimed the database was migrated into a private subnet. The code does no such thing.

The narrower claim is the true one, and it is still worth something: the database is no longer reachable from outside the VPC, the change is version controlled, and anyone can clone the repo and reproduce the before and after.

What I would tell someone building this

Read the long tail. I spent all my attention on the three sources generating volume, because volume is what dashboards are built to show. The twenty-five sources with one line each were the only unplanned thing in the entire dataset, and they were the answer to the question the lab was supposed to ask.

Check whether your attacker is actually external. Look at srcaddr, compare it to your own CIDR blocks. It takes ten seconds and it is embarrassing in a very specific way to skip.

Work out what a number counts before you escalate it. If you cannot say which rows produced a figure, it does not belong in a ticket.

Know what each log source can prove. Flow logs answer who talked to what, on which port, and whether the packet landed. They cannot answer whether someone tried to log in. Different question, different log, different pipeline.

The lab was supposed to demonstrate that exposing a database to the internet gets you found. It did that on day one, in writing, in my own S3 bucket. I just described the wrong attacker.

Terraform, the attack script, and the flow log sample are all here if you want to check my work: aws-incident-response-honeypot

Detection and dashboards by Kenny Barr.

If you have run something like this, what was in your long tail?

The Tea on Teamwork: Week One of My i.c.stars Tech Internship

Keenen Wilkins — Sat, 05 Sep 2026 19:13:10 +0000

I went into my first week at i.c.stars thinking about technology, code, and systems. The week turned out to be about the human layer instead, and the most useful thing I learned was about myself: when a project gets hard and progress feels slow, I take on more work instead of asking for help.

While Team Week was very involved and we spent a lot of time learning about our team dynamic and reflecting on it, I feel like I genuinely got to know my teammates and their approaches to teamwork and collaboration. I realized going into the program that I am a very collaborative person and tend to value my interpersonal relationships more than the task at hand.

Still, some of the self-assessment tools we used really highlighted that blind spot. In many situations taking on more work is a positive attribute. In a team setup it is a major weakness.

Rethinking True Collaboration

We ran through a few workshops during the week that really spoke to me on this topic. One of them was a consulting agency case study. My partner drew a scenario where he was the more experienced colleague doing all the work, because his coworker was resistant to change. The natural impulse there is to take over and do the work for the person who is not prepared.

But in reality, collaboration means setting the expectations, making a plan, and providing the tools for the teammate to bring themselves up to speed.

The Mathematics of Trust

We also had a discussion about trust that came with actual math. Trust is a broad concept, but it comes down to four things. Your credibility, your reliability, and your intimacy, which is the willingness to make other people feel safe with you. Those three stack on top. Underneath all of them sits self-orientation, which is how much of this is really about you.

The single best piece of advice I got that week was that everything above the line is irrelevant if what is below it is not genuine.

In other words, if your primary goal is to get things done by any means necessary, it will come across, and no amount of effort will convince people otherwise that you are trustworthy.

Tackling the Unknowns

We also spent about twenty minutes on the fundamentals of technology transformation: cybersecurity, data analytics, and AI. Twenty minutes is nowhere near enough to learn any of that, which was the point. The workshop was not about the tech. It was about how you handle information you do not have yet.

When trying to absorb a lot of information about a particular topic, it is easy to get overwhelmed and attempt to consume everything at once. Still, realistically, it is better to focus on the knowns and the unknowns, then tackle the unknowns one at a time.

Looking Ahead to Week Two

I also picked up a role this week: tea documentarian. That needs some context if you have not been through this program. High Tea is a daily afternoon session where a business or technology executive sits down with the cohort. They walk through how they actually got where they are, what they are wrestling with right now, and a book that shaped them, and then the floor opens for questions. It is the part of the program that puts nontraditional candidates in a room with people whose calendars are usually closed to us.

My job is to track who comes through and what each visit does for the cycle. We have not had one yet, so I am writing this before I know what I am in for. It is going to test the exact thing I just admitted is my weak spot, because documenting it well means paying attention to everyone else in the room instead of quietly grinding on my own work.

The first week has been exciting and busy, and it has already given me more to think about on the human layer of this program than I expected. I am really looking forward to seeing what else week two has in store.

If you have been the teammate who quietly takes on more work instead of asking for help, what actually got you to stop? I am asking selfishly.

Week two goes up next Friday.