TAC RTE Track
Jump to navigation
Jump to search
See: TAC RTE Track, TAC RTE-7, Text Analysis Conference, 2011, TAC 2011, TAC KBP Track, TAC Summarization Track.
References
- http://www.nist.gov/tac/2011/RTE/
- Given two text fragments called 'Text' and 'Hypothesis', Textual Entailment Recognition is the task of determining whether the meaning of the Hypothesis is entailed (can be inferred) from the Text. The goal of the first RTE Challenge was to provide the NLP community with a benchmark to test progress in recognizing textual entailment, and to compare the achievements of different groups. Since its inception in 2004, the RTE Challenges have promoted research in textual entailment recognition as a generic task that captures major semantic inference needs across many natural language processing applications, such as Question Answering (QA), Information Retrieval (IR), Information Extraction (IE), and multi-document Summarization. ... The RTE-7 tasks focus on recognizing textual entailment in two application settings: Summarization and Knowledge Base Population.
- 1. Main Task (Summarization setting): Given a corpus and a set of "candidate" sentences retrieved by Lucene from that corpus, RTE systems are required to identify all the sentences from among the candidate sentences that entail a given Hypothesis. The RTE-7 Main Task is based on the TAC Update Summarization Task. In the Update Summarization Task, each topic contains two sets of documents ("A" and "B"), where all the "A" documents chronologically precede all the "B" documents. An RTE-7 Main Task "corpus" consists of 10 "A" documents, while Hypotheses are taken from sentences in the "B" documents.
- 2. Novelty Detection Subtask (Summarization setting): In the Novelty Detection variant of the Main Task, systems are required to judge if the information contained in each H (based on text snippets from B summaries) is novel with respect to the information contained in the A documents related to the same topic. If entailing sentences are found for a given H, it means that the content of H is not new; if no entailing sentences are detected, it means that information contained in the H is novel.
- 3. KBP Validation Task (Knowledge Base Population setting): Based on the TAC Knowledge Base Population (KBP) Slot-Filling task, the KBP validation task is to determine whether a given relation (Hypothesis) is supported in an associated document (Text). Each slot fill that is proposed by a system for the KBP Slot-Filling task would create one evaluation item for the RTE-KBP Validation Task: The Hypothesis would be a simple sentence created from the slot fill, while the Text would be the source document that was cited as supporting the slot fill.
- Given two text fragments called 'Text' and 'Hypothesis', Textual Entailment Recognition is the task of determining whether the meaning of the Hypothesis is entailed (can be inferred) from the Text. The goal of the first RTE Challenge was to provide the NLP community with a benchmark to test progress in recognizing textual entailment, and to compare the achievements of different groups. Since its inception in 2004, the RTE Challenges have promoted research in textual entailment recognition as a generic task that captures major semantic inference needs across many natural language processing applications, such as Question Answering (QA), Information Retrieval (IR), Information Extraction (IE), and multi-document Summarization. ... The RTE-7 tasks focus on recognizing textual entailment in two application settings: Summarization and Knowledge Base Population.