Lexicalized Probabilistic Context Free Grammar
A Lexicalized Probabilistic Context Free Grammar is a PCFG whose grammar rules (PCFG rules) can account for lexical information.
- AKA: LPCFG, Lexicalized PCFG.
- Context:
- head words annotate phrasal nodes
- current state of the art
- Counter-Example(s):
- See: Lexicalized Grammar, Lexicalized HMM.
References
2011
- http://www.cs.columbia.edu/~mcollins/courses/nlp2011/notes/lexpcfgs.pdf
- QUOTE: The basic idea in lexicalized PCFGs will be to replace rules such as: [math]\displaystyle{ S → NP \ VP }[/math] with lexicalized rules such as: [math]\displaystyle{ S(examined) → NP(lawyer) \ VP(examined }[/math]
2007
- (Kakkonen, 2007) ⇒ Tuomo Kakkonen. (2007). “Framework and Resources for Natural Language Evaluation." Academic Dissertation. University of Joensuu.
- In probabilistic grammar formalisms a probability is associated with each rule. For example, probabilistic context-free grammars (PCFGs) can be characterized as CFPSGs that assign to each production the probability of its use. A PCFG (Booth & Thompson 1973) is a 4-tuple [math]\displaystyle{ G = (N, Σ, P, S) }[/math] like a CFPSG (Definition 3-9), except that each rule in P is associated with a probability in which [math]\displaystyle{ α }[/math] will be expanded to β.
- Definition 3-10. Productions in probabilistic context-free phrase structure grammar.
- 1. P = {a ®b a Î N,b Î(ΣÈN)*}.
- 2. There is a probability function p: P→[0,1] such that for each a ÎN,Σ p(a ®b a )= 1.
- Definition 3-11. Production rules in a lexicalized context-free phrase structure grammar.
- P = {a ®b a Î N,b Î(ΣÈN)* a (ΣÈN)*}, where a is a distinguished word in the that is referred to as the anchor.
- A straightforward way of constructing a lexicalized grammar is to take a CFPSG and make many copies of each rule – one for every possible anchor word in each phrase (Schabes et al. 1988, Nederhof & Satta 1994).