Competition on Legal Information Extraction/Entailment (COLIEE)
Jump to navigation
Jump to search
A Competition on Legal Information Extraction/Entailment (COLIEE) is an NLP competition focused on extracting and reasoning with legal information.
- Context:
- It can be organized annually.
- It can feature tasks involving Legal Information Extraction and Legal Textual Entailment.
- It can have participation open to teams worldwide.
- It can use newly annotated Legal Document Data.
- It can benchmark techniques for Legal AI.
- …
- Example(s):
- COLIEE 2018 - Held at JURIX 2018. Included citation prediction and summarization tasks.
- COLIEE 2019 - Held at ICAIL 2019. Added QA and RTE tasks.
- COLIEE 2021 - Held at JURIX 2021. Added legal judgment prediction task.
- COLIEE 2022 [1] - held at JURISIN 2022.
- COLIEE 2023 [2] - held at ICAIL 2023.
- COLIEE 2024 [3] - held at JURISIN 2024.
- …
- Counter-Example(s):
- See: JURISIN, Legal AI, Natural Legal Language Processing, Case Law.
References
2024
- (Goebel et al., 2024) ⇒ Randy Goebel, Yoshinobu Kano, Mi-Young Kim, Juliano Rabelo, Ken Satoh, and Masaharu Yoshioka. (2024). “Overview of Benchmark Datasets and Methods for the Legal Information Extraction/Entailment Competition (COLIEE) 2024.” In: JSAI International Symposium on Artificial Intelligence. doi:10.1007/978-981-97-3076-6_8
- NOTE:
- The competition includes four tasks, two each for case law and statute law.
- Task 1 focuses on information retrieval from case laws to find supporting cases.
- Task 2 entails predicting decisions from previous cases and identifying supporting paragraphs.
- Task 3 involves retrieving relevant civil law articles to answer queries.
- Task 4 focuses on answering yes/no questions based on retrieved civil law articles.
- NOTE:
2022
- (Rabelo et al., 2022) ⇒ J. Rabelo, Randy Goebel, M.Y. Kim, Y. Kano, .... (2022). “Overview and Discussion of the Competition on Legal Information Extraction/Entailment (COLIEE) 2021." In: The Review of Contemporary Law, 45(2), 463-477. [doi:10.1007/s42241-021-00134-x]
- ABSTRACT: We summarize the 8th Competition on Legal Information Extraction and Entailment. In this edition, the competition included five tasks on case law and statute law. The case law component includes an information retrieval Task (Task 1), and the confirmation of an entailment relation between an existing case and an unseen case (Task 2). The statute law component includes an information retrieval Task (Task 3), an entailment/question answering task based on retrieved civil code statutes (Task 4) and an entailment/question answering task without retrieved civil code statutes (Task 5). ....
2019
- (Kano et al., 2019) ⇒ Y. Kano, M.Y. Kim, M. Yoshioka, Y. Lu, J. Rabelo, .... (2019). ["Coliee-2018: Evaluation of the Competition on Legal Information Extraction and Entailment."](https://link.springer.com/chapter/10.1007/978-3-030-32381-3_16) In: New Frontiers in Artificial Intelligence, JURIX 2019, 137–146. [doi:10.1007/978-3-030-32381-3_16]
- QUOTE: "... the evaluation of the 5th Competition on Legal Information Extraction/Entailment 2018 ... of the COLIEE-2018 competition. For the case law, Task 1 retrieves noticed cases (information ..."
2018
- (Kano et al., 2018) ⇒ Yoshinobu Kano, Miyoung Kim, …, R. Goebel, and K. Satoh. (2018). “COLIEE-2018: Evaluation of the Competition on Legal Information Extraction and Entailment.” In: JSAI-isAI Workshops.
2016
- (Kim et al., 2016) ⇒ M.Y. Kim, Randy Goebel, Y. Kano, K. Satoh (2016). ["COLIEE-2016: Evaluation of the Competition on Legal Information Extraction and Entailment."](https://www.researchgate.net/publication/309559477_COLIEE-2016_Evaluation_of_the_Competition_on_Legal_Information_Extraction_and_Entailment) In: JURIX 2016.
- QUOTE: "... competitions on legal information extraction/entailment (COLIEE 2014, COLIEE 2015) on a legal ... experimental effort in the legal information extraction/retrieval field. We held the third ..."