2017 SearchQAANewQADatasetAugmentedw

• (Dunn et al., 2017) ⇒ Matthew Dunn, Levent Sagun, Mike Higgins, V. Ugur Guney, Volkan Cirik, and Kyunghyun Cho. (2017). “SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine.” In: ePrint: abs/1704.05179.

Subject Headings: SearchQA Dataset; Reading Comprehension Dataset; QA Dataset.

Cited By

• Google Scholar: ~ 173 Citations, Retrieved: 2020-12-27.

Quotes

Abstract

We publicly release a new large-scale dataset, called SearchQA, for machine comprehension, or question-answering. Unlike recently released datasets, such as DeepMind CNN / DailyMail and SQuAD, the proposed SearchQA was constructed to reflect a full pipeline of general question-answering. That is, we start not from an existing article and generate a question-answer pair, but start from an existing question-answer pair, crawled from J! Archive, and augment it with text snippets retrieved by Google. Following this approach, we built SearchQA, which consists of more than 140k question-answer pairs with each pair having 49.6 snippets on average. Each question-answer-context tuple of the SearchQA comes with additional meta-data such as the snippet's URL, which we believe will be valuable resources for future research. We conduct human evaluation as well as test two baseline methods, one simple word selection and the other deep learning based, on the SearchQA. We show that there is a meaningful gap between the human and machine performances. This suggests that the proposed dataset could well serve as a benchmark for question-answering.

2. SearchQA

Collection A major goal of the new dataset is to build and provide to the public a machine comprehension dataset that better reflects a noisy information retrieval system. In order to achieve this goal, we need to introduce a natural, realistic noise to the context of each question-answer pair. We use a production-level search engine –Google– for this purpose.

(...)

BibTeX

@article{2017_SearchQAANewQADatasetAugmentedw,
  author    = {Matthew Dunn and
               Levent Sagun and
               Mike Higgins and
               V. Ugur Guney and
               Volkan Cirik and
               Kyunghyun Cho},
  title     = {SearchQA: A New Q&A Dataset Augmented with Context from a
               Search Engine},
  journal   = {CoRR},
  volume    = {abs/1704.05179},
  year      = {2017},
  url       = {http://arxiv.org/abs/1704.05179},
  archivePrefix = {arXiv},
  eprint    = {1704.05179},
}

---

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2017 SearchQAANewQADatasetAugmentedw	Kyunghyun Cho Matthew Dunn Levent Sagun Mike Higgins Volkan Cirik V. Ugur Guney			SearchQA: {A} New Q{\&}A Dataset Augmented with Context from a Search Engine						2017