Word Confusion Network (WCN)
Jump to navigation
Jump to search
A Word Confusion Network (WCN) is a Weighted Directed Graph that is word graph in which each path from the start node to the end node visits all the other nodes.
- AKA: Confusion Network (CN).
- Context:
- It can be generated by a Confusion Network Decoding System.
- It can represent speech recognition or translation hypotheses in a machine translation system.
- Example(s):
- Counter-Example(s):
- See: Word Representation. Neural Encoder-Decoder Network, Natural Language Processing, Speech Recognition, Machine Translation, Directed Acyclic Graph, Open-Source Software, Moses Statistical Machine Translation System.
References
2021
- (Wikipedia, 2021) ⇒ https://en.wikipedia.org/wiki/Confusion_network Retrieved:2021-7-11.
- A confusion network (sometimes called a word confusion network or informally known as a sausage) is a natural language processing method that combines outputs from multiple automatic speech recognition or machine translation systems. Confusion networks are simple linear directed acyclic graphs with the property that each a path from the start node to the end node goes through all the other nodes. The set of words represented by edges between two nodes is called a confusion set. In machine translation, the defining characteristic of confusion networks is that they allow multiple ambiguous inputs, deferring committal translation decisions until later stages of processing. This approach is used in the open source machine translation software Moses and the proprietary translation API in IBM Bluemix Watson.
2021
- (Moses SMT, 2021) ⇒ http://www.statmt.org/moses/?n=Moses.ConfusionNetworks Retrieved:2021-7-11.
- QUOTE: A Confusion Network (CN), also known as a sausage, is a weighted directed graph with the peculiarity that each path from the start node to the end node goes through all the other nodes.
Each edge is labeled with a word and a (posterior) probability. The total probability of all edges between two consecutive nodes sum up to 1. Notice that this is not a strict constraint from the point of view of the decoder; any score can be provided. A path from the start node to the end node is scored by multiplying the scores of its edges. If the previous constrain is satisfied, the product represents the likelihood of the path, and the sum of the likelihood of all paths equals to 1.
- QUOTE: A Confusion Network (CN), also known as a sausage, is a weighted directed graph with the peculiarity that each path from the start node to the end node goes through all the other nodes.
2011
- (Rosti et al., 2011) ⇒ Antti-Veikko Rosti, Eugene Matusov, Jason Smith, Necip Ayan, Jason Eisner, Damianos Karakos, Sanjeev Khudanpur, Gregor Leusch, Zhifei Li, Spyros Matsoukas, Hermann Ney, Richard Schwartz, B. Zhang, and J. Zheng (2011). “Confusion Network Decoding for MT System Combination".In: Handbook of Natural Language Processing and Machine Translation. ISBN: 978-1-4419-7713-7
- QUOTE: A confusion network is a linear graph where all paths travel via all nodes. There may be one or more word arcs between two consecutive nodes. These arcs may be viewed as alternative choices of words in a hypothesis. Thus, a confusion network may encode an exponential number of hypotheses. A word arc may also contain a NULL word which represents an empty word or a deletion.
2008a
- (Leusch et al., 2008) ⇒ Gregor Leusch, Evgeny Matusov, and Hermann Ney (2008). "Complexity of Finding the BLEU-Optimal Hypothesis in a Confusion Network". In: Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP 2008).
- QUOTE: Confusion networks are a simple representation of multiple speech recognition or translation hypotheses in a machine translation system.
...
A confusion network (CN) is a word graph where each edge is labeled with exactly zero or one symbol, and each path from the start node to the end node visits each node of the graph in canonical order. Usually, we represent unlabeled edges by labeling them with the empty word $\epsilon$.
- QUOTE: Confusion networks are a simple representation of multiple speech recognition or translation hypotheses in a machine translation system.
2008b
- (Rosti et al., 2008) ⇒ Antti-Veikko I. Rosti, Bing Zhang, Spyros Matsoukas, and Richard Schwartz (2008)."Incremental Hypothesis Alignment for Building Confusion Networks with Application to Machine Translation System Combination". In: Proceedings of the Third Workshop on Statistical Machine Translation (WMT@ACL 2008).
- QUOTE: Confusion network decoding has been the most successful approach in combining outputs from multiple machine translation (MT) systems in the recent DARPA GALE and NIST Open MT evaluations. Due to the varying word order between outputs from different MT systems, the hypothesis alignment presents the biggest challenge in confusion network decoding. This paper describes an incremental alignment method to build confusion networks based on the translation edit rate (TER) algorithm.
2005
- (Bertoldi & Federico, 2005) ⇒ Nicola Bertoldi, and Marcello Federico (2005). “A New Decoder for Spoken Language Translation Based on Confusion Networks". In: Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU 2005).
- QUOTE: A Confusion Network (CN) $\mathcal{G}$ is a weighted directed graph with a start node, an end node, and word labels over its edges. The CN has the peculiarity that each path from the start node to the end node goes through all the other nodes.