[4/10]Solving the Dimension Direction Problem: How Positive/Negative Scales Actually Work

Series: Building a Modular Assessment Engine (4/10)

The scale module computes dimension scores from form answers and assigns interpretation tiers. Sounds straightforward. It's not. The "direction" problem — whether high scores mean good or bad — caused more bugs than every other module combined.

The Data Structure

Three levels of hierarchy:

Factor (root)
  └── Scale (dimension)
       ├── id: "workload"
       ├── name: "Workload"
       ├── format: sum | avg | count
       ├── direction: positive | negative
       └── Data[] (score ranges)
            ├── 0-8: "Severe" → "Your workload is critically high..."
            ├── 9-14: "Moderate" → "Some signs of strain..."
            └── 15-999: "Mild" → "Your workload is manageable..."

• format determines how scores are computed: sum (add all field scores), avg (average across fields), count (count specific option selections)
• direction determines semantic meaning: positive (high = good, like "resilience") or negative (high = bad, like "burnout")
• Data[] are the score ranges with labels and descriptions

The Direction Problem

Here's where it gets confusing. Consider two dimensions:

Burnout (negative direction, high = bad):

• Score 5 (low) → Good state, low burnout
• Score 25 (high) → Bad state, severe burnout

Resilience (positive direction, high = good):

• Score 5 (low) → Bad state, low resilience
• Score 25 (high) → Good state, high resilience

The question: when configuring score ranges, do you write them low-to-high or high-to-low? What does "low score" mean for each direction?

Attempt 1: Direction-Aware Configuration

Initially, the SKILL.md told the AI to configure ranges differently based on direction:

• For positive dimensions: low score = worst, high score = best
• For negative dimensions: low score = best, high score = worst

This was logical but confusing in practice. The AI would frequently mix up which direction meant what, especially in mixed-type assessments where some dimensions are positive and others are negative.

Result: roughly 30% of generated apps had at least one dimension with inverted ranges. Users with severe burnout would see "Your burnout is mild" because the ranges were flipped.

Attempt 2: Storage-Time Reversal

The solution that finally worked: unify the configuration convention and let the engine handle the reversal at storage time.

The rule is now dead simple for the AI:

Always configure ranges from lowest score (worst state) to highest score (best state).
The system automatically handles direction reversal internally.

# Burnout (negative direction) — ranges look the same as positive
assess scale add --id burnout --name "Burnout" \
  --format sum --direction negative

assess scale data add --scale burnout \
  --ranges "5-10:Severe:Burnout is high, seek support,11-18:Moderate:Some signs of strain,19-999:Mild:Burnout is low, keep it up"

Wait — "Severe" is at the low-score end? That doesn't make sense for burnout, where low scores mean you're fine.

Here's the trick: the system reverses the range labels when storing negative-direction dimensions. So the stored data becomes:

Stored (after reversal):
  5-10: Mild (low score = good for negative direction)
  11-18: Moderate
  19-999: Severe (high score = bad for negative direction)

The AI doesn't need to think about this. It writes ranges the same way for every dimension: low score = worst label, high score = best label. The engine flips it for negative dimensions.

Why This Works

1. One mental model. The AI always writes "low = worst, high = best." No conditional logic based on direction.
2. Direction is metadata, not configuration logic. --direction negative tells the engine "this is a symptom scale" — it's used for label reversal and report interpretation direction, not for range configuration.
3. The raw score always matches the stored ranges. After reversal, a user who scores 25 on burnout (high) correctly hits "Severe" because the stored ranges have "Severe" at the high end.

The direction Priority Rule

How does the AI decide if a dimension is positive or negative? The priority is:

1. Question wording (highest priority): "I feel exhausted" (symptom, high = worse) → negative. "I can cope with stress" (ability, high = better) → positive.
2. Plan declaration (secondary): Only when question wording is ambiguous.

This matters because the plan agent might declare a dimension as negative, but if the actual questions measure positive constructs ("I feel capable"), the direction should be positive. The SKILL.md explicitly states: when knowledge declaration and question semantics conflict, question semantics win.

The Ranges Continuity Bug

Score ranges must be continuous and non-overlapping:

✅ Correct:   3-6, 7-8, 9-999  (7 = 6+1, 9 = 8+1)
❌ Overlap:   3-6, 3-8, 9-999  (second range starts at 3, overlapping)
❌ Gap:       3-6, 9-999        (7-8 uncovered, users in this range get nothing)

The overlap bug was common. The AI would copy the first range's min as the second range's min instead of incrementing:

# Wrong — second range starts at 3 (copied from first range's min)
assess scale data add --scale workload \
  --ranges "3-6:Severe,3-8:Moderate,9-999:Mild"

# Correct — second range starts at 7 (previous max + 1)
assess scale data add --scale workload \
  --ranges "3-6:Severe,7-8:Moderate,9-999:Mild"

The engine now validates range continuity and rejects overlapping ranges with a clear error message.

The Unreachable Range Bug

Another classic: ranges based on theoretical scores instead of actual possible scores.

Knowledge says: "0-2 = Severe, 3-4 = Moderate, 5-6 = Mild"
But the form fields have min=1 (not 0), so the actual minimum score is 3
→ The "0-2: Severe" range is never reachable
→ Every user falls into "Moderate" or "Mild"
→ The "Severe" tier is dead code

The fix: before setting ranges, the scale skill must read the actual --min and --max values from form fields and compute the real score range:

// In the scale validation logic
int actualMin = 0;
int actualMax = 0;
for (Field f : associatedFields) {
    actualMin += f.getMinScore();
    actualMax += f.getMaxScore();
}
// Ranges must span [actualMin, actualMax]
// If actualMin = 3, ranges starting at 0 are unreachable

The SKILL.md now has a P0 rule:

ranges MUST be based on actual field scores. Read --options :score values from form-fields reference and compute the real min/max. NEVER use theoretical scores from knowledge.

The Sentinel Value

The highest range always uses 999 as its max:

5-10:Severe,11-18:Moderate,19-999:Mild

Why not use the actual maximum? Because floating-point rounding and edge cases. A user who scores 18.5 (from avg format) might not match 18 as a max boundary. 999 as a sentinel ensures the highest tier always catches everything above its minimum.

The engine treats 999 specially — it doesn't validate against the theoretical maximum, it just means "everything above this min."

The count Format Trap

Most dimensions use sum or avg. But count has a specific use case: counting how many times a specific option was selected.

# Count how many "Yes" answers in a checklist
assess scale add --id compliance --name "Compliance" \
  --format count --direction positive

# Each field's "Yes" option has score=1, "No" has score=0
# count format counts fields with score > 0

The trap: count doesn't sum scores. It counts non-zero fields. So if a field has --score 5, count treats it as 1 (non-zero), not 5. This is by design for binary checklists, but the AI would sometimes use count for regular scored questions, producing wrong totals.

The rule: count is for binary (yes/no) statistics only. For regular scoring, always use sum.

Lessons Learned

1. Unify the configuration convention. Don't make the AI think differently for positive vs negative. Let the engine handle the reversal.

2. Validate against actual data, not theoretical declarations. The plan agent might say "scores range 0-6" but the actual form fields produce 3-18. Always read from the source.

3. Range continuity is a validation problem, not an AI problem. Don't trust the AI to get min = previous_max + 1 right every time. Validate and reject.

4. Direction is metadata. It tells the engine how to interpret scores, not how to configure ranges. Keep it separate from the range configuration logic.

---

Next: [Connect Module — AI-driven cover pages and visual styles]