2020 Competition on Legal Information Extraction / Entailment (COLIEE-2020)

A 2020 Competition on Legal Information Extraction / Entailment (COLIEE-2020) is a Competition on Legal Information Extraction / Entailment (COLIEE) (an AI competition that focuses on the legal information retrieval, legal information extraction and legal information entailment) for 2020.

• Context:
  • It can involve multiple tasks aimed at improving the efficiency and accuracy of legal document retrieval and legal case prediction.
  • It can offer a platform for researchers to apply machine learning and natural language processing techniques to legal texts.
  • It can (typically) include:
    • a Relevant Legal Case Identification Task.
    • a Legal Case Decision Prediction Task.
    • a Legal Article Retrieval Task.
    • a Yes/No Legal Question Answering.
  • ...
• Counter-Example(s):
  • ...

See: AI Competition, Legal Document Retrieval.

---

---

References

2021

• (Westermann et al., 2021) ⇒ Hannes Westermann, Jaromir Savelka, and Karim Benyekhlef. (2021). “Paragraph Similarity Scoring and Fine-tuned BERT for Legal Information Retrieval and Entailment.” In: New Frontiers in Artificial Intelligence: JSAI-isAI 2020 Workshops, JURISIN, LENLS 2020 Workshops, Virtual Event, November 15--17, 2020.
  • ABSTRACT: The assessment of the relevance of legal documents and the application of legal rules embodied in legal documents are some of the key processes in the field of law. In this paper, we present our approach to the 2020 Competition on Legal Information Extraction / Entailment (COLIEE-2020), which provides researchers with the opportunity to find ways of accomplishing these complex tasks using computers. Here, we describe the methods used to build the models for the four tasks that are part of the competition and the results of their application. For Task 1, concerning the prediction of whether a base case cites a candidate case, we devise a method for evaluating the similarity between cases based on individual paragraph similarity. This method can be used to reduce the number of candidate cases by 85%, while maintaining over 80% of the cited cases. We then train a Support Vector Machines model to make the final prediction. The model is the best solution submitted for Task 1. We use a similar method for Task 2. For Task 3, we use an approach based on BM25 measure in combination with the identification of similar previously asked questions. For Task 4, we use a transformer model fine-tuned on existing entailment data sets as well as on the provided domain-specific statutory law data set.

2020

• https://sites.ualberta.ca/~rabelo/COLIEE2020/
  • NOTES:
    • The 2020 Competition on Legal Information Extraction / Entailment (COLIEE-2020) was announced, focusing on legal document retrieval and entailment.
    • COLIEE-2020 includes four tasks: two on case law (Task 1 & Task 2) and two on statute law (Task 3 & Task 4), involving legal case retrieval, entailment, and yes/no legal question answering.
    • The competition is associated with the International Workshop on Juris-informatics (JURISIN 2020).
    • Key sponsors include the Alberta Machine Intelligence Institute, University of Alberta, National Institute of Informatics, vLex Canada, Ross Intelligence, and Intellicon.
    • Evaluation of competition results will consider precision, recall, and F-measure for case law tasks, and precision, recall, and F2-measure for statute law tasks, with specific submission formats and evaluation methods outlined.

---