[memo]Enhancing Distractor Generation Retrieval Augmented Pretraining and Knowledge Graph Integration

Introduction

• Knowledge graphを使ってdistractorを生成するという手法
• MCQs are widely used to evaluate a learner’s knowledge
• Automatic distractor generation for MCQs
• Task-specific

Contribution

• Improved SoTA DG results
• Experimental evaluation in DG

Related work

Generating and ranking framework

First step: generates candidate distractors

Second step: ranks these candidates

by semantic rules

Task-specific pre training

- RAP: leverages task-specific priors

Knowledge augmented generation

• Leverages knowledge graphs
• Knowledge graphs to improve question answering

Retrieving triplet from KG

• (q,a) → LM → distractors

Triplet ranker

• Encoding QA
• Compute relevancy score
• Supervised triplet classification
• Proposed a binary classification

KAG training

KG integration has shown promising results in question answering tasks

Can treat RAP as a data augmentation mechanism

Cross-domain RAP

Knowledge augmented generation

Experiment

• SciQ

SciQ

• multi-domain multiple-choice question dataset

MCQ dataset

• Cross-domain cloze-style dataset

Conclusion

• Introduced the utilization of task-specific pretraining