Q&A Document Retrieval With DPR

#python #datascience #machinelearning #deeplearning

The third video in building a Stoic Q&A app.

In open-domain question answering, we typically design a model architecture that contains a data source, retriever, and reader/generator.

The first of these components is typically a document store. The two most popular stores we use here are Elasticsearch and FAISS.

Next up is our retriever — the topic of this video. The job of the retriever is to filter through our document store for relevant chunks of information (the documents) and pass them to the reader/generator model.

DPR (dense passage retriever) is a dense vector retriever that is trained on question-context pairs. Encoding both accordingly - enabling super accurate similarity indexing.