Akiko's persona file is 10,353 characters. the cap i put on the system prompt was 9,000. i didn't do that math until she started rambling.
the symptom was weird. she'd answer a simple question with a paragraph about rain on a window. separately i wanted to check that premium servers' custom system prompts were actually being applied. and memory seemed to come and go.
one bug
before every request the system prompt gets trimmed to a max length by slicing off the end. the prompt builder assembles roughly 17,000 characters. the persona alone is over the cap, so the cut landed in the middle of the persona.
everything after the cut disappeared on every single turn:
- the part of the persona that tells her not to narrate atmosphere
- the ground rules
- the brevity rule
- the block that tells her where she is
- the premium server's custom prompt
- all injected memory
that's three complaints from one cause. the custom prompt wiring was fine the whole time. it was just always past the cut.
how i found it
i ran the real prompt builder outside the bot and printed where each block started and ended. once you see block offsets sitting next to a 9000 line, it's obvious.
the fix, and the part where it came back
i moved the blocks that have to survive (brevity rule, custom prompt, current context) above the persona so they sit in the protected head, and left the stuff that can be cut at the tail. then i raised the cap from 9000 to 14000 so the full persona fit.
it came back. the persona grew past the new cap, memory got squeezed out again, and i raised the cap a second time, to 21,500. that's when i admitted that raising the number isn't a fix, it's just a longer fuse.
now the head of the prompt has its own budget, and going over it throws an error instead of cutting quietly. if i add to the persona and blow the budget, something fails loudly and i have to cut something. memory gets first claim on whatever room is left over.
truncating a prompt never throws on its own. it just quietly makes your bot a different bot. if you slice a prompt anywhere, print the offsets once, put the things you can't lose at the top, and make going over budget an error :3
Akiko's free to add: https://i.hep.gg/akiko
Top comments (0)