Generating high‑value synthetic questions injects dense teacher supervision into on‑policy distillation, substantially reducing the number of required training questions. The twist is that these queries need no reference answers; their value comes from how much they expose the teacher’s knowledge when a strong model answers them.
Before this work, practitioners relied on real user questions or unoptimized student‑generated prompts, often filtering them with KL divergence to approximate relevance. Vanilla on‑policy distillation (OPD) and KL‑filtered real queries have been the default baselines for math reasoning, code generation, and agentic tasks.
A2D lifts average test scores to the mid‑40s and 60s, outpacing vanilla OPD by several points. “As shown in Table 1, A2D achieves average scores of 46.2, 51.6, and 64.5 for Qwen3‑1.7B, R1‑Distill‑1.5B, and R1‑Distill‑7B, respectively, compared with 43.0, 50.5, and 62.9 for Vanilla OPD.” [1] This gain translates into a 3.2‑point lift on the smallest model and a 1.6‑point lift on the largest, confirming that synthetic high‑value questions can replace real‑bank data without sacrificing performance.
The improvement stems from denser supervision per token rather than sheer quantity of examples. “Signal analyses further show that optimized questions can elicit denser supervision and more total signal from shorter student responses, supporting elicited distributional supervision as an objective for synthetic distillation data.” [1] Shorter answers mean the student consumes fewer tokens while receiving richer teacher feedback, a property that directly speeds learning.
The method assumes a frozen student–teacher pair to compute the surrogate value, so any drift between them during training could degrade the quality of generated questions. Moreover, the evaluation covers only three domains; it remains unclear whether the same gains hold for open‑ended dialogue or multimodal tasks. This suggests an open question: can distributional supervision be reliably estimated without a static teacher in settings where continual adaptation is required?
If synthetic high‑value queries truly provide denser supervision, benchmark pipelines should swap real query banks for the released A2D prompt collections whenever labeled data are scarce. Running standard OPD on the gasolsun/<pair> datasets will give a faster convergence curve and higher final scores without extra annotation effort.
Top comments (0)