2024 OverviewofBenchmarkDatasetsandM

From GM-RKB
Jump to navigation Jump to search

Subject Headings: 11th Competition on Legal Information Extraction and Entailment (COLIEE 2024), Case Law NLP Benchmark, Case Law NLP, Case Law Corpus, Statute Law NLP Benchmark, Statute Law NLP, Statute Law Corpus.

Notes

Cited By

Quotes

Abstract

We summarize the 11th Competition on Legal Information Extraction and Entailment (COLIEE 2024). In this eleventh edition, the competition included four tasks on case law and statute law. The case law component includes an information retrieval task (Task 1), and the confirmation of an entailment relation between an existing case and a selected unseen case (Task 2). The statute law component includes an information retrieval task (Task 3), and an entailment/question-answering task based on retrieved civil code statutes (Task 4). Participation was open to any group based on any approach. Ten different teams participated in the case law competition tasks, most of them in more than one task. We received results from 10 teams for Task 1 (26 runs) and 6 teams for Task 2 (18 runs). On the statute law task, there were 12 different teams participating, most in more than one task. 8 teams submitted a total of 20 runs for Task 3, and 8 teams submitted a total of 23 runs for Task 4. We describe the variety of approaches, our official evaluation, and analysis of our data and submission results.

References

;

 AuthorvolumeDate ValuetitletypejournaltitleUrldoinoteyear
2024 OverviewofBenchmarkDatasetsandMMi-Young Kim
Randy Goebel
Ken Satoh
Yoshinobu Kano
Juliano Rabelo
Masaharu Yoshioka
Overview of Benchmark Datasets and Methods for the Legal Information Extraction/Entailment Competition (COLIEE) 202410.1007/978-981-97-3076-6_82024