Building an on-device AI journal that does not confuse the prompt with the user's voice

A journaling assistant has a deceptively hard job: it must respond to a person's meaning, not simply react to emotional keywords.

While building Rixin Mind Journal, a private Android journal with on-device AI, I learned that a small local model can sound fluent while still making a basic semantic mistake: treating the system's daily question as if the user had asked it.

For example:

System prompt: "When did you feel most peaceful today?"

User answer: "After everyone went to sleep, I worked alone."

A shallow pipeline may focus on "worked alone" and immediately warn about overwork. A better response first notices what solitude provided: uninterrupted attention, autonomy, relief from social pressure, or simply a quiet room.

That distinction changed the architecture.

1. Keep the system prompt and user answer in separate fields

Do not concatenate everything into an undifferentiated paragraph and hope the model infers the roles.

The analysis input explicitly labels:

• Question asked by the app
• Answer written by the user
• Optional free-form diary text
• Prior patterns, when generating a weekly or monthly report

The instruction also states that the question is context, not a claim or request made by the user.

This sounds obvious, but it eliminated an entire class of confidently wrong responses.

2. Analyze the whole entry before generating prose

Keyword matching is useful as a safety net, but it is not understanding.

"Lazy" can mean shame, fatigue, avoidance, rebellion, depression, humor, or a temporary loss of routine. "Fine" can mean genuine calm or emotional shutdown. "Alone" can mean loneliness or freedom.

The pipeline therefore separates analysis from writing:

1. Identify the concrete situation.
2. Estimate emotional valence and intensity.
3. Look for mixed emotions and negation.
4. Infer the likely need or tension.
5. Select one relevant psychological frame.
6. Generate one small, behaviorally specific next step.
7. Write the final response in a warm voice.

The final prose is not allowed to expose the internal labels or repeat large pieces of the diary.

3. Receive emotion before offering advice

Advice given too early feels like correction.

The response order is deliberately constrained:

1. Recognition: show that the experience was understood.
2. Insight: name one useful pattern, conflict, or need.
3. Action: suggest one small step that could survive an ordinary day.
4. Closing line: end with warmth, not another question the user cannot answer.

This matters for positive emotion too. A reflection engine should not treat every joyful entry as hidden pathology. Relief, pride, affection, playfulness, gratitude, and calm are information about what supports the person.

4. Use expert rules as guardrails, not as the final writer

Early versions relied too heavily on templates. They were safe, but different diary entries produced suspiciously similar responses.

The current approach uses expert rules to control:

• role separation
• emotion-negation checks
• crisis language
• medical boundaries
• response order
• repetition limits
• banned meta phrases such as "the user answered..."
• action specificity

The local model is still responsible for most of the wording. Rules shape the lane; they do not write every sentence.

5. Evaluate semantic fit, not only grammatical quality

A fluent answer can still be useless.

Our test cases cover different ages, jobs, relationships and emotional states. Each output is reviewed against questions such as:

• Did it identify the actual situation?
• Did it preserve positive emotion instead of turning it negative?
• Did it acknowledge mixed feelings?
• Is the advice concrete enough to attempt?
• Could the same response be pasted under a different diary entry?
• Does it sound like a person, or like a classification report?

The most valuable failure metric is replaceability. If an insight could fit twenty unrelated entries, it is too generic.

6. Privacy is part of the product behavior

On-device processing is not only an infrastructure decision. It changes what users are willing to write.

Rixin keeps diary writing local-first and performs AI reflection on the device. It supports guided questions, free-form journaling, daily insights, and longer weekly/monthly letters. The app is for self-reflection, not diagnosis or therapy.

The tradeoff is real: a small local model has less raw capability than a large cloud model. The way forward is not to pretend otherwise, but to narrow the task, provide expert structure, test diverse failure cases, and make the output earn its place.

If you are building any AI product that handles personal writing, I would recommend testing role confusion, negation, mixed emotion, positive emotion, and generic-response reuse before polishing the UI.

Rixin Mind Journal on Google Play:

https://play.google.com/store/apps/details?id=com.rixin.app

I would especially value feedback from Android and local-AI developers on the balance between privacy, model size, and semantic accuracy.