Maximum Likelihood-based Character-Level Language Model (LM) Training Algorithm

Context:
- It can be implemented by a Maximum Likelihood-based Character-Level Language Model Training System.
- It can range from being an Unsmoothed Maximum Likelihood Character Level Language Model to being a Smoothed Maximum Likelihood Character Level Language Model.
Example(s):
- a Python MaxLikelihood-based LM Algorithm, such as in (Goldberg, 2015).
- …
Counter-Example(s):
- a Neural Character-Level LM Algorithm.
- a Word-Level Maximum Likelihood-based Language Model Training Algorithm.
See: n-Gram Character-Level Language Model.

References

(Goldberg, 2015) ⇒ Yoav Goldberg (2015). "The unreasonable effectiveness of Character-level Language Models (and why RNNs are still cool)".
- QUOTE: … I will briefly describe these character-level maximum-likelihood language models, which are much less magical than RNNs and LSTMs, …
  … Mathematically, we would like to learn a function P(c|h). Here, c is a character, h is a n-letters history, and P(c|h) stands for how likely is it to see c after we've seen h.
  Perhaps the simplest approach would be to just count and divide (a.k.a maximum likelihood estimates). We will count the number of times each letter c′ appeared after h, and divide by the total numbers of letters appearing after h. The unsmoothed part means that if we did not see a given letter following h, we will just give it a probability of zero.