I Made My Onboarding Data-Driven. A Hardcoded List Ate Four of Six Answers.

Lifemaxxing AI is a habit app I build with a partner. You answer eight questions when you install it, and it hands you back a plan: a set of daily tasks that feed attribute ratings like Discipline, Wisdom and Confidence. It is Flutter, and the onboarding is the most important surface in the whole app, because those answers are not analytics. They are the input to the thing that generates your plan.

Early on I did what everyone tells you to do. I pulled the questions out of Dart and put them into a JSON file that ships as an asset, so I could reword a question, add an option, or change which tasks an answer produces without touching a widget or shipping a build.

That part worked. What I found this week is that four of the six options on question seven have been silently thrown away before they ever reach the plan generator. Not crashed. Not logged. Quietly replaced with a default.

Here is how a good idea turned into an invisible bug, and what I am doing about it.

The config that generates the plan

assets/onboarding_config.json holds two things: the questions, and the rules that turn answers into tasks.

{
  "id": "q7",
  "question": "Whose lifestyle do you admire most?",
  "type": "selection",
  "options": [
    {"id": "socrates", "text": "Socrates"},
    {"id": "elon_musk", "text": "Elon Musk"},
    {"id": "napoleon", "text": "Napoleon"},
    {"id": "ronaldo", "text": "Ronaldo"},
    {"id": "david_goggins", "text": "David Goggins"},
    {"id": "not_sure", "text": "Not sure"}
  ]
}

And then the assignment rules, which are just a nested map of question id to answer id to task ids:

"taskAssignmentLogic": {
  "mandatoryTasks": ["pushUps", "lessSocialMedia", "running", "coldShowers", "kindness"],
  "conditionalTasks": {
    "q7": {
      "socrates": ["morningMeditation", "journaling", "reading"],
      "elon_musk": ["deepWork", "wakeUpEarly", "planDay"],
      "napoleon": ["planDay", "workout", "wakeUpEarly"],
      "ronaldo": ["workout", "wakeUpEarly", "earlyBedtime"],
      "david_goggins": ["wakeUpEarly", "workout", "journaling"],
      "not_sure": ["wakeUpEarly", "planDay", "journaling"]
    }
  }
}

The whole assignment algorithm is a set union:

static List<String> determineUserTaskIds(Map<String, String> userResponses) {
  final Set<String> userTasks = Set<String>.from(getMandatoryTaskIds());

  userResponses.forEach((questionId, answer) {
    userTasks.addAll(getTaskIdsForAnswer(questionId, answer));
  });

  return userTasks.toList();
}

Five tasks everyone gets, then each answered question contributes zero to three more, and the Set handles the overlap. The overlap is real, not theoretical: wakeUpEarly is reachable from three different questions, so most users get it no matter what they pick.

I still like this design. Onboarding copy changes constantly, the mapping from "what you told me" to "what you should do" is product judgment rather than logic, and product judgment belongs in a file you can edit without a release cycle.

One thing I only noticed while re-reading it: only five of the eight questions map to tasks at all. Gender, "how would you describe your life right now", and "how do you respond when things get hard" collect an answer that produces nothing. That is a defensible choice, they carry the narrative of the onboarding and feed copy, but it should have been a decision I made on purpose rather than one I discovered in my own JSON.

The layer I forgot to move

The app stores answers under semantic keys. Question seven is saved as role_model, question six as addiction, question eight as momentum_feeling. The config addresses them positionally as q6, q7, q8. So there is a mapping function in the middle that converts stored user data back into the question ids the config expects.

That function also carries three guard clauses. At some point an answer from one question ended up in another question's slot, so I added sanity checks. Here is the one for question seven, verbatim:

if (questionResponses.containsKey('q7')) {
  final possibleQ7Answers = ['bryan_johnson', 'barack_obama', 'david_goggins',
                            'wim_hof', 'donald_trump', 'not_sure'];
  if (!possibleQ7Answers.contains(questionResponses['q7'])) {
    // This is likely a mixup - reset to default
    questionResponses['q7'] = 'not_sure';
  }
}

Read that list against the JSON above. They have nothing to do with each other. The config now offers Socrates, Elon Musk, Napoleon, Ronaldo, David Goggins and Not sure. The guard still believes in a lineup I replaced a long time ago.

Git confirms exactly what happened. One commit swapped bryan_johnson and barack_obama for socrates and elon_musk, dropped wim_hof, and rewrote the task lists to match. The JSON moved. The Dart literal did not.

What it actually costs

Four of the six options fall through to not_sure. Only david_goggins and not_sure survive the guard.

So a user who picks Socrates should get morning meditation, journaling and reading. Instead they get wake up early, plan day and journaling, which is the "I don't know" plan. Someone who picks Ronaldo loses their workout and early bedtime and gets the same generic three.

Worse, two tasks fell out of the product entirely. reading and earlyBedtime are only reachable through question seven, so nothing in the automatic path can produce them any more. They are still fully configured in task_config.json, with a display name, a calibration question, progression rates and attribute weights. They just cannot be assigned. A user can still add up to two extra tasks by hand from the additional-tasks sheet, so they are not invisible, but the algorithm cannot reach them.

And none of this logs anything. The guard resets and moves on, because silence was the entire point of the guard.

Why this took so long to find

Three reasons, and I think they generalize past my app.

The compiler stopped being able to help. The moment I moved the options into JSON, 'bryan_johnson' became a perfectly valid Dart string that no longer refers to anything. There is no rename that touches both sides. Data-driven config does not remove coupling, it moves the coupling somewhere your type checker cannot see it.

The guard was designed to be quiet. It exists to absorb a garbage value without bothering the user. That is the correct behavior for the bug it was written for, and exactly the wrong behavior for the bug it became. A guard that discards user input with no trace has no failure signal by construction.

It fails toward something plausible. Nobody gets an error screen. Everybody gets a reasonable plan, just not their plan. A bug that produces a sensible looking wrong answer is the most expensive kind, because there is no moment where anyone thinks to look.

What I am changing

Four things, in order of how much they would have helped.

1. Derive the valid set from the config instead of a literal. The loader already parses every option:

final validIds = OnboardingConfigLoader.getQuestionById('q7')
    ?.options?.map((o) => o.id).toSet() ?? <String>{};

Same for q6 and q8. Once the guard reads the same file as the UI, it cannot drift, and the whole class of bug disappears rather than getting fixed once.

1. Make discards loud. If the guard has to reset a value, that is a log line in debug and a counter in production. Silently dropping user input is only acceptable if someone can find out later that it happened.

2. Test the config, not just the code. One test walks the JSON and asserts that every answer id under conditionalTasks exists in that question's options, and that every task id it references exists in task_config.json. That is maybe fifteen lines and it would have failed on the commit that renamed the options. When you make your data soft, you have to replace the type checker with something, and I replaced it with nothing.

3. Stop deriving storage keys from position. The question manager splits the loaded list in half and calls the first half demographic and the second half lifestyle, with a comment saying the split exists only to keep an older API shape alive. Storage keys come from the index inside those halves. Which means reordering questions in the JSON quietly reassigns keys. The key belongs in the JSON next to the question, not in an index calculation.

The rule I took away

If you move data out of code, every literal left in code that names a value from that data is a second copy of your schema. Copies drift. There are only two honest options: derive the second copy from the first, or write a test that fails the day they disagree.

I have said versions of that sentence in code review to other people. It still took me a rename, a stale whitelist and two orphaned tasks to actually apply it to my own app.

Lifemaxxing AI is at lifemaxxingai.com if you want to see what the onboarding produces. Assume, for now, that it thinks slightly less of your taste in role models than it should.