Transformer-based Neural Language Model

A Transformer-based Neural Language Model is a neural LM that is a transformer-based deep NNet.

• Context:
  • It can (typically) be produced by a Transformer-based Neural Language Modeling System (that can solve a Transformer-based Neural Network-based Language Modeling Task).
  • It can range from being a Character-Level Transformer-based Neural Network-based LM to being a Word Token-Level Transformer-based Neural Network-based LM.
  • It can range from being a Small Transformer-based Language Model to being a Large Transformer-based Language Model.
  • It can range from being a Forward Transformer-based Neural Network based Language Model to being a Backward Transformer-based Neural Network based Language Model to being a Bi-Directional Transformer-based Neural Network based Language Model.
  • It can range from being an Encoder-Decoder Transformer-based Neural Language Model to being an Encoder-Only Transformer-based Neural Language Model to being a Decoder-Only Transformer-based Neural Language Model.
  • It can range from being a Custom Transformer-based LM to being a Pre-Trained Transformer-based LM.
  • …
• Example(s):
  • a BERT LM.
  • an ElMO LM.
  • a GPT LM, such as GPT-4.
  • a Turing-NLG LM.
  • …
• Counter-Example(s):
  • an RNN-based LM.
  • a Vision Transformer-based Model.
• See: Neural NLG.

---

---

References

2023

• (Greco & Tagarelli, 2023) ⇒ Candida M Greco, and Andrea Tagarelli. (2023). “Bringing Order Into the Realm of Transformer-based Language Models for Artificial Intelligence and Law.” In: Artif. Intell. Law Journal. doi:10.48550/arXiv.2308.05502
  • QUOTE: ... Transformer-based language models (TLMs) have widely been recognized to be a cutting-edge technology for the successful development of deep-learning-based solutions to problems and applications that require natural language processing and understanding. ...

---