I sent PostAll to 11 beta users three weeks ago.
I expected feedback about the UI. Maybe some complaints about output quality. A few "would be nice if..." suggestions I could politely defer.
What I got instead: three separate users hitting the same architectural assumption I'd never tested, a feature I was proud of that nobody used, and one piece of feedback that made me rewrite the job queue from scratch.
Here's what actually happened.
The Feedback I Expected vs. What I Got
My mental model of beta feedback: people would try the core flow, generate some content, and tell me what felt awkward.
The reality: two users never got past onboarding. Not because the onboarding was broken — it wasn't. Because their use case didn't map to how I'd structured the product at all.
I'd built PostAll around a "campaign" metaphor: you define a topic, set parameters, generate a batch of articles. Clean. Logical. Made total sense to me.
User #4 (a content agency owner) opened PostAll and spent 10 minutes looking for a way to paste in a list of 60 keywords and just... run them. No campaign setup. No parameter tuning. Just: here are the keywords, go.
User #7 (an e-commerce founder) wanted to generate product descriptions one at a time, from inside a workflow that already existed in another tool. He wasn't looking for a campaign — he was looking for an API endpoint with sensible defaults.
Same product. Completely different mental models. Both of them valid.
The campaign abstraction I'd designed was the right structure for me, building the thing. It was the wrong structure for people who just needed output fast.
The Architecture Change I Didn't Want to Make
Here's what the original job flow looked like:
// Original: campaigns own everything
async function enqueueCampaign(campaignId) {
const campaign = await db.campaigns.findById(campaignId);
const jobs = campaign.keywords.map(keyword => ({
campaignId,
keyword,
parameters: campaign.parameters,
status: 'pending'
}));
await db.jobs.insertMany(jobs);
await queue.add('processCampaign', { campaignId });
}
Every job was a child of a campaign. Made sense for tracking, for billing, for grouping output. But it meant you had to create a campaign before you could generate anything.
After beta feedback, I needed jobs to be first-class:
// Revised: jobs are independent, campaigns are optional grouping
async function enqueueJob(input) {
const { keyword, parameters, campaignId = null, userId } = input;
const job = await db.jobs.create({
userId,
campaignId, // nullable — job works without a campaign
keyword,
parameters: {
...getDefaultParameters(userId), // sensible defaults per user
...parameters // override only what was passed
},
status: 'pending',
createdAt: new Date()
});
await queue.add('processJob', { jobId: job.id }, {
attempts: 3,
backoff: { type: 'exponential', delay: 2000 }
});
return job;
}
This change sounds small. It took two days to implement correctly, because campaign-level analytics were baked into four different places in the codebase that assumed campaignId was always present.
The lesson: the abstraction you build first is usually built for the creator, not the user. The creator needs structure to stay sane. The user needs to get to output as fast as possible.
The Feature Nobody Used
I spent probably 12 hours building a "Content Style Guide" editor — a UI where you could define your brand voice, set tone parameters, add example sentences PostAll should emulate.
In 11 beta users across three weeks: zero used it.
I asked User #2 about it directly. She said: "Oh, I didn't know what that was for. I just typed what I wanted in the prompt box."
The fancier the UI, the higher the cognitive load to understand it. A free-text prompt field has a learning curve of zero. A multi-field style guide editor requires you to understand what "formality score" means, why you'd set it to 7 vs. 8, and whether it'll actually affect output.
I haven't deleted the style guide feature. But I've moved it three levels deeper in the settings, and I added a "Quick Generate" path that bypasses it entirely.
The One Piece of Feedback That Stung
User #9 — a freelance writer who was testing PostAll to see if it would help with client volume — sent me a Loom video. I was expecting a bug report.
Instead: she showed herself generating five articles, then pulling them all into a Google Doc to manually reformat them. Title casing, removing extra line breaks, standardizing the H2 structure. It took her longer than just editing the content.
The output quality was fine. The output format was inconsistent in ways that made it unusable without cleanup.
I'd tested PostAll's output against my own mental model of "good formatting." I hadn't tested it against what someone actually does with the output after it's generated.
The fix wasn't in the prompt. It was in the post-processing layer:
function normalizeArticleOutput(rawContent) {
return rawContent
// Consistent H2 formatting (model sometimes outputs ### instead of ##)
.replace(/^#{3,}\s/gm, '## ')
// Remove double blank lines
.replace(/\n{3,}/g, '\n\n')
// Title case for H2 headers (model is inconsistent)
.replace(/^## (.+)$/gm, (_, title) => `## ${toTitleCase(title)}`)
// Strip trailing whitespace per line
.split('\n').map(line => line.trimEnd()).join('\n')
.trim();
}
Simple. Should have been in v1. Wasn't, because I never watched someone actually use the output in their workflow.
What I Underestimated
Onboarding time. I thought 10 minutes to first generation was fine. Two users spent more than 20 minutes before generating anything, and both of them never came back.
The first generation is the only one that matters for retention. If someone generates garbage on the first try — wrong format, wrong length, doesn't match their mental model of "good content" — the product is dead to them regardless of how good it gets later.
I'm now adding a guided first-run flow that forces a single successful generation before anything else. Not optional. Mandatory.
Where PostAll Is Now
- Job queue redesigned: campaigns are optional, not required
- Quick Generate path ships next week
- Output normalization layer is live
- Style guide editor: still exists, now hidden by default
- Onboarding: being rebuilt
The metrics that matter right now: 4 of 11 beta users have generated content more than once. I want that at 7 of 11 before I expand the beta further.
What surprised you most the first-time real users touched something you built? I keep expecting UI feedback and keep getting architecture feedback. Starting to think that's just how it goes.
Top comments (1)
Running an Android beta for my finance app right now and the pattern matches. I expected feedback on dashboard layout. What I got was three people confused by the same thing: the difference between their bank balance and the spendable number the app calculates. That gap is the whole point of the product, and my first screen explained it with a tooltip nobody opened. The fix was not UI polish, it was reordering onboarding so the first thing you do is enter one bill and watch the number change.
Your point about the first generation being the only one that matters applies to finance apps too. If the first calculated number looks wrong because the user has not entered enough data yet, they decide the app is broken rather than incomplete. I now seed the first session with a worked example for exactly that reason.
The style guide story also rings true. The feature I was proudest of, a toggle that changes how committed savings count against your spendable balance, gets touched by almost nobody. The plain default does the job for most people. Architecture feedback over UI feedback seems to be the rule, not the exception.