DEV Community

orca_forge
orca_forge

Posted on Originally published at forge.workstyle.tech

Apology Messages Were Tainting the Next Search — A Loop of Failure Strengthening Itself

#llm #rag

📝 Originally published (in Japanese) at forge.workstyle.tech.

The Log of Voice Dialogue Avatar Had This Interval 

👤 This page's content is explained      🤖 The corresponding description was not found.
👤 Can you hear me?                     🤖 (Same apology)
👤 Can you hear me again?               🤖 (Same apology)
👤 I can hear you.                      🤖 (Same apology)
Enter fullscreen mode Exit fullscreen mode

4 consecutive turns returned almost the same apology. The more the user tries to confirm, the more the same answer is returned.

The Cause Was the Improvement I Made Earlier

A little while ago, I made the following improvement: "Detailed" phrases like "Tell me more about that" cannot be searched alone, so add the previous conversation (user's speech + avatar's answer) to the search. The measurement was also effective (0.602 → 0.741).

Looking at the log, it was like this:

[retrieval] Add the previous conversation:
  'Can you hear me? I apologize, but the corresponding description was not found.'
Enter fullscreen mode Exit fullscreen mode

The apology sentence without grounds was also included in the search query.

The apology sentence has no topic. Not only that, but words like "corresponding description" and "another way of saying" pull the search in a completely unrelated direction. That's why the next one is also wrong. And the failed sentence is included in the next query.

The failure was reinforcing itself.

Don't Judge by String

At first, I thought of "detecting and excluding apology sentences". I stopped soon.

Apology phrases differ depending on the job profile. They can be set differently for window, support, and delivery, and can change during operation. Judging by string would create holes every time a new phrase is added.

Instead, I decided based on facts.

if not getattr(self, "_last_turn_grounded", False):
    a = ""     # Don't add answers from turns without grounds
Enter fullscreen mode Exit fullscreen mode

"Whether there was a ground for that turn" is already judged by the system. It doesn't depend on phrases.

⚠️ I shouldn't have written this update in the recording process. Since conversation log recording is best-effort (it doesn't stop the conversation even if it fails), if an exception occurs, the update will also fail. Even if the recording fails, the presence or absence of grounds affects the search quality of the next turn. I moved it to the end of the turn.

⚠️ The apology path also needed to be marked. If I drop it, the apology sentence will be included in the next query.

Another Thing That Went Against the Facts

While investigating the same 4 turns, I noticed another problem. The page text was not held even once during that time.

[page] Use the full text  ← This log didn't appear even once during the 4 turns
Enter fullscreen mode Exit fullscreen mode

I was answering "This page does not contain the corresponding description" without holding the page text. This is against the facts. Whether the visitor can't read it or it's not there after reading is completely different for the visitor.

No text or index → "I'm currently reading the page content. Please wait a moment."
Text exists but no match → "The corresponding description was not found on the page I'm currently viewing."
Enter fullscreen mode Exit fullscreen mode

⚠️ I almost made another mistake. The synthesis audio reuse key didn't distinguish between phrases. Since the standard response reuses audio by caching, if left as is, the audio for "reading" would play when intending to say "not found". I split the key by phrase.

Result 

👤 What's the weather like today?                → 🤖 (Try to answer the page content)
👤 Tell me about the weather.                   → 🤖 This page does not contain weather information, so I apologize.
👤 (Next question)                             → 🤖 (Answer normally)  ← Apology doesn't chain
Enter fullscreen mode Exit fullscreen mode

Generalizable Things

Designs that return output to input amplify when they fail.

Using conversation history as context is a natural design. However, there are two types of "avatar answers":

Answers with grounds     … Contain topics. Useful as context
Apology answers without grounds     … Contain no topics. Rather, distort the search
Enter fullscreen mode Exit fullscreen mode

I should have used only the former. The system already had the material to distinguish (whether there was a ground for that turn), but I was treating answers as one thing.

Similar structures exist elsewhere. Using summaries as input for the next summary, mixing generated data into learning data, using search results as the next search query — all of them "get better when good, worse when bad". A mechanism to stop when bad is needed.

And the judgment to stop should be based on facts, not strings. Phrases change, but "whether there was a ground" doesn't change.

Series: Until the Voice Dialogue Avatar Answers Correctly

This article is the last part of Part 2: Understanding Language.

← Previous: The one line at the end of a huge prompt was ignored four times
→ Next: Not all utterances are questions

Series of 8 articles

Part 1: Stopping Sound

  1. There were two types of events with the same name
  2. The self-echo countermeasure never fired
  3. A finger on the speaker was breaking the echo canceller

Part 2: Understanding Language

  1. The three "its" were different things
  2. The one line at the end of a huge prompt was ignored four times
  3. The apology words were poisoning the search ← Now here

Part 3: Judging

  1. Not all utterances are questions
  2. I thought I was measuring, but I was measuring something else

The notes that led to this insight are summarized in Improving the Response Quality of Voice Dialogue Avatars.

Top comments (1)

Collapse
 
jo-do profile image
Jo Do •

This is a good example of why a retry should not inherit the whole conversational transcript by default. The error explanation is useful to an operator, but it is usually bad search state. I prefer a typed retry envelope: original objective, last tool observation, failure class, changed constraint, and attempt number. Keep apologies and speculative diagnosis out of the next model context unless a policy explicitly asks for them. That also makes repeated failure fingerprints much easier to compare.