DEV Community: Motzumoto

my bot logged "fetch failed" 450 times a day for a month and the error told me nothing

Motzumoto — Tue, 20 Oct 2026 13:00:00 +0000

for about a month my bot logged TypeError: fetch failed around 450 times a day. i ignored it. it told me nothing.

what was happening

Akiko's AI features call a worker container on the same box. it's slow. summarizing a plain 20 message conversation took 55 seconds, and real ones queue behind other requests and take longer.

i'd given the summarize call a one hour timeout with an AbortController. it didn't matter. Node's fetch (undici) has its own headersTimeout and bodyTimeout of 300 seconds that have nothing to do with your abort signal. anything over five minutes got killed, and the one hour timeout was a story i'd told myself.

why it took a month

the error showed up as a bare fetch failed. i was logging errors with String(err), which drops err.cause. the cause is the only thing that tells ECONNREFUSED apart from UND_ERR_HEADERS_TIMEOUT. about 7,400 errors, all identical, none of them actionable.

there's a helper now that keeps the cause. trust the cause, not the message.

the fix

a module level undici Agent with headersTimeout: 0 and bodyTimeout: 0, passed as the dispatcher on the long calls. any new long running endpoint needs it too, or the default quietly caps it at 300 seconds.
the summarizer model has a 4096 token window and i was feeding it up to 20 raw conversation rows. one request was 5,899 tokens. it now clamps input to about 2,400 tokens, newest first.

a day after the fix, 545 summarize calls, all 200.

one more thing. the worker's /health answers in about a millisecond even when inference is badly backed up. a green healthcheck told me nothing about whether requests were working :3

Akiko's free to add: https://upload.motzumoto.com/devto

i told my bot to create a role and give it to me. it gave me a different one.

Motzumoto — Tue, 13 Oct 2026 13:00:00 +0000

i asked Akiko to create a role called bleh 3 and give it to me. she created bleh 3 and gave me bleh 2.

it wasn't a fluke. every time, she granted the role that was created before the one i asked for.

why

she has tools like create_role and add_role. when i ask for both in one message, the model plans both calls in a single response. so the second call's arguments get written before the first call has run. at that point the newest role ID the model has seen is the previous one, so that's the one it hands to add_role.

bleh 2 got a fresh ID and add_role was handed the ID of the earlier bleh. bleh 3 got a fresh ID and add_role was handed bleh 2's.

running the calls in order doesn't help. the arguments were already written.

the fix

create_role now reports the ID it really made, as structured data and not by parsing a sentence. the engine keeps a list of things created in the current batch. if another role or channel action in the same batch points at something else, it gets refused with the real ID in the error, so the model retries with the right one.

i refuse instead of quietly swapping in the new ID. "create role A, then take role B off me" is a real request that points at an existing role, and redirecting it would do the wrong thing, which is the exact bug i was fixing.

the real problem was bigger

while i was in there i counted her tools. six. none of them could look anything up on a server. no way to ask "what is this role's ID". a model that needs an ID and can't ask doesn't leave it blank. it writes the most believable number it can find.

in one test run it used the ID of a role from a run 48 minutes earlier, a user ID that was just 1234567890123456789, and a channel from 2022 that it never created. it sent a message there, made an invite, and deleted the channel.

i'd written five guards for five different ways of inventing an ID, and each one found a new way. so i stopped guarding and added three lookup tools. guards stop wrong answers. a lookup removes the reason to guess.

if your bot is making up IDs, give it a way to look them up :3

Akiko's free to add: https://upload.motzumoto.com/devto

my bot's system prompt was cut off in the middle on every message

Motzumoto — Tue, 06 Oct 2026 13:00:00 +0000

Akiko's persona file is 10,353 characters. the cap i put on the system prompt was 9,000. i didn't do that math until she started rambling.

the symptom was weird. she'd answer a simple question with a paragraph about rain on a window. separately i wanted to check that premium servers' custom system prompts were actually being applied. and memory seemed to come and go.

one bug

before every request the system prompt gets trimmed to a max length by slicing off the end. the prompt builder assembles roughly 17,000 characters. the persona alone is over the cap, so the cut landed in the middle of the persona.

everything after the cut disappeared on every single turn:

the part of the persona that tells her not to narrate atmosphere
the ground rules
the brevity rule
the block that tells her where she is
the premium server's custom prompt
all injected memory

that's three complaints from one cause. the custom prompt wiring was fine the whole time. it was just always past the cut.

how i found it

i ran the real prompt builder outside the bot and printed where each block started and ended. once you see block offsets sitting next to a 9000 line, it's obvious.

the fix, and the part where it came back

i moved the blocks that have to survive (brevity rule, custom prompt, current context) above the persona so they sit in the protected head, and left the stuff that can be cut at the tail. then i raised the cap from 9000 to 14000 so the full persona fit.

it came back. the persona grew past the new cap, memory got squeezed out again, and i raised the cap a second time, to 21,500. that's when i admitted that raising the number isn't a fix, it's just a longer fuse.

now the head of the prompt has its own budget, and going over it throws an error instead of cutting quietly. if i add to the persona and blow the budget, something fails loudly and i have to cut something. memory gets first claim on whatever room is left over.

truncating a prompt never throws on its own. it just quietly makes your bot a different bot. if you slice a prompt anywhere, print the offsets once, put the things you can't lose at the top, and make going over budget an error :3

Akiko's free to add: https://upload.motzumoto.com/devto

my dashboard had 24 raid settings and the bot ignored 11 of them

Motzumoto — Wed, 30 Sep 2026 23:59:21 +0000

i build Akiko's dashboard and the bot in two separate repos. they deploy separately and share no code. turns out that's a good way to end up with settings that save, show up on the dashboard, and do nothing.

how it happens

the dashboard builds each module's form straight from the database table. every column becomes a field, minus a list of ones i've hidden by hand. the bot then reads whichever columns it feels like.

if the bot stops reading one, nothing complains. the dashboard keeps rendering it, the save goes through, the row updates. the bot just never looks.

nothing ties "the dashboard offers this" to "the bot reads this" except me remembering.

what i found

it started as one automod bug and kept going.

the account age and avatar checks had a punishment box that was free text. anything other than the literal word ban quietly became a kick.
the message filters offered "escalate repeat offenders" and the filter code never mentioned either column.
the link filter and mention spam had infraction columns the dashboard labelled and the code hardcoded over.
raid heat had 24 settings. 11 of them had no reader at all. a few more only changed the colour of the graph.
auto-roles, which i wrote about last time, turned out to be the entire feature.

two of the hints were worse than missing. one said panic mode auto-locks during coordinated raids. it posts an embed.

the check

before writing a description for any setting, i run this per field:

grep -rl "<column_name>" src/features src/core | grep -v test

zero hits means it's decoration. writing a hint for a dead field is the real mistake, because now the bug has documentation.

fixing it was the scary part

wiring up a dead setting is a behaviour change. the raid heat enforcement fields had been sitting on defaults nobody chose, so switching them on would have started handing out timeouts in servers that never asked for them.

i implemented all of it and put enforcement behind a new column that defaults to off.

if you've got a dashboard and a bot in different repos, go grep your columns. i'd bet you've got a few too :3

Akiko's free to add: https://upload.motzumoto.com/devto

stripe told me my webhook was broken. i found the email 17 days later.

Motzumoto — Wed, 30 Sep 2026 23:56:14 +0000

this one's a bit embarrassing. Akiko has a premium tier that runs through Stripe, and for about ten weeks every webhook Stripe sent her dashboard failed. i had no idea.

i only found out because of an email from Stripe that had been sitting unread for seventeen days.

what happened

i retired the old Express backend on the dashboard and moved everything into Next.js. the Stripe keys lived in the root .env. Next runs from the web/ folder under pm2, so it reads web/.env instead, and that file had no Stripe variables at all.

so every delivery hit the webhook route, found no secret, and threw. the catch block turned any error with "Stripe" in the message into a 400. a missing config looked exactly like a bad signature, to Stripe and to me poking at it with curl.

roughly: last event stored June 3. Express retired June 11. first failed delivery July 22. Stripe disabled the endpoint on July 31 after 10 attempts over 9 days. i found it August 16.

there was a second bug in the same file. six variables were written NAME = value with spaces around the equals, which dotenv reads as a variable called "NAME " with a trailing space. even with the right file loaded they'd have come back undefined.

then it got worse

while i was checking what the dead webhook had broken, i looked at how premium ends. it ends one way. a customer.subscription.deleted event flips active to false, and that's the only code in the repo that does. no sweep, nothing else.

so with the webhook dead, nobody's premium could end. it would just run forever.

the data to catch this was right there. expires_at was written in four places and read in none. there was even an index built for a query nobody had written.

the two date columns also disagreed. expires_at and current_period_end are supposed to mirror each other, but old rows never got mirrored. two lapsed subscriptions had a past expires_at and a NULL current_period_end. the AI limits code checked current_period_end, saw NULL, took that to mean a lifetime grant, and gave premium quotas to a subscription that ended in May.

the fix

both gates now deny if either date is in the past. NULL in both still means lifetime, because comps and staff grants are made outside Stripe with no expiry and i didn't want to cut those off. an unparseable date fails open, since a malformed value is my bug and i'd rather not lock out someone who's paying.

i also added a daily sweep that clears active on anything whose own dates have passed. i ran it against a clone of the real table first. it revoked the two lapsed rows, left the lifetime grants alone, and the second run touched nothing.

if you give people access because of an event, also give them an end date the gate actually reads. otherwise one missed webhook turns into free forever.

go read your Stripe emails (i didn't :3)

Akiko's free to add if you want to watch what i break next: https://upload.motzumoto.com/devto

Your Discord bot's member cache is lying to you. Mine never timed out a single member.

Motzumoto — Wed, 30 Sep 2026 23:49:58 +0000

I set largeThreshold: 50 in my Discord bot's client config. The comment next to it said it saved memory with no functional loss. That was wrong, and it took a log search to find out how wrong.

What the setting really does

largeThreshold is the large_threshold field in Discord's gateway IDENTIFY payload. It is the member count above which the gateway stops sending offline members when a guild loads. Above it, a guild counts as "large" and the initial guild payload only carries members who are online.

Nothing fills the gap later. My only full-roster fetch ran from the guild-join event, and that event only fires for brand new guilds. Guilds the bot was already in at startup never got it. So after every restart, the member cache held the online members plus whoever the bot had seen since.

That means this line is not doing what it looks like:

for (const member of guild.members.values()) { ... }

It does not iterate every member. It iterates the members who happen to be online or recently active.

How I found it

I have a verification timeout sweep. If a member does not finish verification in time, the bot acts on them. I searched 200MB of production logs for the line it writes when it actions someone: verification timeout action applied. It appeared zero times. It had never once actioned anyone.

Two of the four servers using the gate were above the threshold, at 532 and 54 members. The 54 member server was only affected because I had set the threshold to 50. At Discord's default of 250 it would have worked.

The fix

Do not treat the cache as a roster. There are two honest options:

Keep a durable record of who needs checking. I added a pending_verification table and made it the source of candidates.
Call the full member fetch explicitly before you iterate, and report a "not cached" count, so that "nobody matched" is distinguishable from "most of the server was never looked at".

The threshold itself is a real memory optimisation and I kept it. The mistake was letting several features read the cache as if it were complete.

The trap inside the trap

The sweep's happy path returns early and quietly when it finds no candidates. So the absence of log output proved nothing for weeks. A check that stays silent when it finds nothing cannot tell you that it is blind.

Takeaways

Before you walk a guild's members, ask whether you need the offline ones. If yes, the cache is the wrong source.
Make sweeps report what they examined, not only what they did.
Comments that claim "no functional loss" deserve a test.

I build Akiko, a Discord bot with AI chat that remembers you, plus economy, music, moderation and a web dashboard. Free to add: https://upload.motzumoto.com/devto

Discord gives your bot exactly 100 slash commands. I hit 100 and found out from a failing test.

Motzumoto — Wed, 30 Sep 2026 23:45:08 +0000

Discord limits an app to 100 global slash commands. Subcommands and subcommand groups do not count toward it. My bot, Akiko, was sitting at exactly 100 when this happened. i've folded a few more commands into groups since, so she's at 95 now.

I learned where the wall is the hard way. I added a /settings command, the total became 101, and a test failed with expected 101 to be less than or equal to 100. In production the same mistake shows up as Discord error code 30032 when you deploy your commands.

The most useful thing in the repo is a boring test

There is a test that builds the real list of commands that would be deployed globally and asserts the count is at most 100. Without it you find out at deploy time, on the live bot. The test's own docstring also lists the escape routes in order, and says to never loosen the bound.

The escape routes

Fold the command into an existing one as a subcommand group. /report_setup became /report setup.
Make it prefix-only, so it only works with the text prefix.
Make it developer-only. Those deploy to a separate control guild instead of globally, and that scope had plenty of room. Global is the binding constraint.

The scope rules are simple: global means visible, not prefix-only and not developer-only.

The permission trap

Discord ignores default_permissions on subcommands. Permissions are set on the top-level command. So when I folded /report_setup (admin only) into /report (used by ordinary members to file reports), I could not require Manage Server on the setup subcommand alone. Putting the stricter permission on the whole command would have locked ordinary members out of /report user.

The check for that one subcommand has to live in the command body.

What I would do differently

Count your commands in a test, not in your head.
Design command names as groups from day one. Every new top-level name spends a scarce slot.
Never assume a permission set on a subcommand is enforced. Check it in code.

I build Akiko, a Discord bot with AI chat that remembers you, plus economy, music, moderation and a web dashboard. Free to add: https://upload.motzumoto.com/devto

My Discord bot's auto-role setting said ON in 127 servers. It worked in 2.

Motzumoto — Wed, 30 Sep 2026 23:44:34 +0000

The dashboard showed the toggle on. The status command said enabled. No member had ever received a role.

I found this while working on something else. A dashboard to-do claimed that auto-roles and my verification gate "both act on join and conflict", and asked me to add a warning about the clash. Before building the warning I went to check how the two interact.

They don't interact, because auto-roles had no code path on join at all. The member-join handler never touched them. The only place that granted auto-roles was the handler that runs when a member passes a verification gate. So the feature worked exactly where a gate was enabled and nowhere else: 2 guilds out of the 127 that had auto-roles switched on with a role selected.

Why nothing noticed

Everything around the feature agreed with itself. The settings page read the row, the status command read the row, and the row said enabled. Nothing checked the outcome: whether a member actually ended up holding the role. A feature can be fully configured and never run.

The fix, and one decision worth stealing

Adding a grant on join is easy. The interesting part is what happens when a gate is on. A verification gate exists so a member holds no roles until they pass it. Granting on join would hand out exactly the roles the gate is there to withhold.

So the join path checks the gate's own enabled flag and steps aside when the gate is on. The gate keeps ownership. The check reads that local flag instead of watching what the gate did, because the gate call is fire-and-forget and its outcome is not visible from the join handler. If the two ever disagree, the member gets nothing. That is deliberate: a missing role is a support message, a role handed to someone the gate meant to stop is an incident.

The same planning function feeds both the join path and a backfill for existing members, so the prechecks cannot drift apart between the two.

What I took from it

Verify the premise of a bug report before building the fix. The spec described a conflict that did not exist. The real bug was a missing branch.
"Enabled" in a settings page proves a row exists, not that the feature runs. Test the outcome.
When two paths can grant the same thing, decide which one fails safe before you write either.

I build Akiko, a Discord bot with AI chat that remembers you, plus economy, music, moderation and a web dashboard. Free to add: https://upload.motzumoto.com/devto

Giving a Discord bot long-term memory, and the two ways it went wrong

Motzumoto — Wed, 30 Sep 2026 23:30:39 +0000

I run a Discord bot called Akiko. The feature I care about most is that she remembers people across servers and DMs. Building it taught me more about what not to store than what to store. Two bugs stand out.

The memory was 97% junk

My first extractor pulled "facts" out of chat with a loose pronoun pattern. It looked fine in testing. In production, when I finally read the table, almost all of it was noise: fragments that matched the pattern but described nothing about the person.

The fix had three parts:

Clean the existing rows down to the ones worth keeping (about 100 survived).
Make the writer classify a candidate before it saves it, instead of saving everything that matched.
Treat "not yet judged" as its own state (NULL) and retry the judgment on an hourly job, so a temporary failure never turns into a permanent bad row.

The lesson: a memory system that only ever adds will bury itself. The write path needs a filter as strict as the read path.

She remembered what people had not said

The extractor also started storing absence as a fact, things like "User has not shared their birthday". That is worse than noise, because recall then surfaces it and the bot sounds like it is keeping a file on what you withheld.

I first tried marking these rows as low priority, but recall never filtered on that flag, so it changed nothing. The real fix was an extractor rule that refuses negative statements, plus a recall-side filter as a second line of defense.

Making it something users can control

Memory is only comfortable if the user can see and change it. Akiko has commands to list, add, delete and export what she has stored. There is also an import that takes a memory export from another AI and condenses it into clean facts, so a new user does not start from zero.

What I would tell anyone building this

Read your memory table in production early. It will not look like your tests.
Filter on the way in, not only on the way out.
Never store the absence of information.
Give users list, delete and export from day one.

Akiko is free to add: https://upload.motzumoto.com/devto
Dashboard: https://akiko.motzumoto.com