2020 LEGALBERTTheMuppetsStraightOuto

• (Chalkidis et al., 2020) ⇒ Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. (2020). “LEGAL-BERT: The Muppets Straight Out of Law School.” In: arXiv preprint arXiv:2010.02559. doi:10.48550/arXiv.2010.02559

Subject Headings: LEGAL-BERT; Legal-Domain Llm

Cited By

• http://scholar.google.com/scholar?q=%222020%22+LEGAL-BERT%3A+The+Muppets+Straight+Out+of+Law+School

Quotes

Abstract

BERT has achieved impressive performance in several NLP tasks. However, there has been limited investigation on its adaptation guidelines in specialised domains. Here we focus on the legal domain, where we explore several approaches for applying BERT models to downstream legal tasks, evaluating on multiple datasets. Our findings indicate that the previous guidelines for pre-training and fine-tuning, often blindly followed, do not always generalize well in the legal domain. Thus we propose a systematic investigation of the available strategies when applying BERT in specialised domains. These are: (a) use the original BERT out of the box, (b) adapt BERT by additional pre-training on domain-specific corpora, and (c) pre-train BERT from scratch on domain-specific corpora. We also propose a broader hyper-parameter search space when fine-tuning for downstream tasks and we release LEGAL-BERT, a family of BERT models intended to assist legal NLP research, computational law, and legal technology applications.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2020 LEGALBERTTheMuppetsStraightOuto	Ilias Chalkidis Ion Androutsopoulos Nikolaos Aletras Manos Fergadiotis Prodromos Malakasiotis			LEGAL-BERT: The Muppets Straight Out of Law School				10.48550/arXiv.2010.02559		2020