DEV Community

Discussion on: The Correct Timestamp Was in My Agent's Prompt. It Still Said We Were Talking.

Collapse
 
capita_harlock profile image
Ricart •

The closing hedge in that clock line, "not proof he was away", may be working against you. Models are shaky with negation (Jang et al. 2022 tested this with negated prompts), so the phrase can still leave "he's here" sitting in the prompt. The line also never says who started the run.

A plain positive statement is a cheap thing to test against the same audit, something like "triggered by scheduler, no user message received this run". I haven't measured it, so treat it as a guess, not a fix.

Collapse
 
kenielzep97 profile image
Self-Correcting Systems •

This aged well. We tested something close to it: one line at the top of the scheduler message, "AUTOMATED SCHEDULER CHECK: sender=scheduler, not Keniel; Keniel is not participating in this turn." Graded blind, the plain audit claimed we were exchanging messages in 7 of 8 answers. With the line it was 1 of 8, or 3 of 8 if you count the unclear ones. Only 8 each, so it's a lead, not a fix, and a bigger run is going now. Your negation point still applies, the line says "not" twice. Would you test a version with no "not" in it at all?