National Institute of Standards And Technology (NIST) Metric

A National Institute of Standards And Technology (NIST) Metric is a NLG performance metric managed by NIST that is based on BLEU metric.

• Context:
  • It can be used to benchmarking NLG systems.
  • …
• Example(s):
  • …
• Counter-Example(s):
  • Word Error Rate,
  • F-Measure,
  • METEOR Metric,
  • ROUGE Metric,
  • Position Independent Word Error Rate (PER) Measure,
  • Word Accuracy (WAcc) Measure,
  • BLEU Score.
• See: Recall, Information Retrieval, Speech Recognition, Machine Translation, Levenshtein Distance, Phoneme, Bilingual Evaluation Understudy, Evaluation of Machine Translation, Machine Translation, National Institute of Standards And Technology, n-Gram.

---

---

References

2021

• (Wikipedia, 2021) ⇒ https://en.wikipedia.org/wiki/NIST_(metric) Retrieved:2021-6-20.
  • NIST is a method for evaluating the quality of text which has been translated using machine translation. Its name comes from the US National Institute of Standards and Technology.

    It is based on the BLEU metric, but with some alterations. Where BLEU simply calculates n-gram precision adding equal weight to each one, NIST also calculates how informative a particular n-gram is. That is to say when a correct n-gram is found, the rarer that n-gram is, the more weight it will be given.

    For example, if the bigram "on the" is correctly matched, it will receive lower weight than the correct matching of bigram "interesting calculations", as this is less likely to occur.

    NIST also differs from BLEU in its calculation of the brevity penalty insofar as small variations in translation length do not impact the overall score as much.

---