2024 OverviewofBenchmarkDatasetsandM
- (Goebel et al., 2024) ⇒ Randy Goebel, Yoshinobu Kano, Mi-Young Kim, Juliano Rabelo, Ken Satoh, and Masaharu Yoshioka. (2024). “Overview of Benchmark Datasets and Methods for the Legal Information Extraction/Entailment Competition (COLIEE) 2024.” In: JSAI International Symposium on Artificial Intelligence. doi:10.1007/978-981-97-3076-6_8
Subject Headings: 11th Competition on Legal Information Extraction and Entailment (COLIEE 2024), Case Law NLP Benchmark, Case Law NLP, Case Law Corpus, Statute Law NLP Benchmark, Statute Law NLP, Statute Law Corpus.
Notes
Cited By
Quotes
Abstract
We summarize the 11th Competition on Legal Information Extraction and Entailment (COLIEE 2024). In this eleventh edition, the competition included four tasks on case law and statute law. The case law component includes an information retrieval task (Task 1), and the confirmation of an entailment relation between an existing case and a selected unseen case (Task 2). The statute law component includes an information retrieval task (Task 3), and an entailment/question-answering task based on retrieved civil code statutes (Task 4). Participation was open to any group based on any approach. Ten different teams participated in the case law competition tasks, most of them in more than one task. We received results from 10 teams for Task 1 (26 runs) and 6 teams for Task 2 (18 runs). On the statute law task, there were 12 different teams participating, most in more than one task. 8 teams submitted a total of 20 runs for Task 3, and 8 teams submitted a total of 23 runs for Task 4. We describe the variety of approaches, our official evaluation, and analysis of our data and submission results.
References
;
| Author | volume | Date Value | title | type | journal | titleUrl | doi | note | year | |
|---|---|---|---|---|---|---|---|---|---|---|
| 2024 OverviewofBenchmarkDatasetsandM | Mi-Young Kim Randy Goebel Ken Satoh Yoshinobu Kano Juliano Rabelo Masaharu Yoshioka | Overview of Benchmark Datasets and Methods for the Legal Information Extraction/Entailment Competition (COLIEE) 2024 | 10.1007/978-981-97-3076-6_8 | 2024 |