2007 SemanticClassInductionAndCoreferenceResolution

• (Ng, 2007) ⇒ Vincent Ng. (2007). “Semantic Class Induction And Coreference Resolution.” In: Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL 2007).

Subject Headings: Entity Mention Coreference Resolution Algorithm, Learning Algorithm, Noun Phrase Coreference Resolution.

Notes

• See: (Ng, 2010).

Cited By

• ~33 http://scholar.google.com/scholar?hl=en&q=%22Semantic+Class+Induction+And+Coreference+Resolution.%22+2007

Quotes

Abstract

This paper examines whether a learning based coreference resolver can be improved using semantic class knowledge that is automatically acquired from a version of the Penn Treebank in which the noun phrases are labeled with their semantic classes. Experiments on the ACE test data show that a resolver that employs such induced semantic class knowledge yields a statistically significant improvement of 2% in F-measure over one that exploits heuristically computed semantic class knowledge. In addition, the induced knowledge improves the accuracy of common noun resolution by 2-6%.

1. Introduction

In the past decade, knowledge-lean approaches have significantly influenced research in noun phrase (NP) coreference resolution — the problem of determining which NPs refer to the same real-world entity in a document. In knowledge-lean approaches, coreference resolvers employ only morpho-syntactic cues as knowledge sources in the resolution process (e.g., Mitkov (1998), Tetreault (2001)). While these approaches have been reasonably successful (see Mitkov (2002)), Kehler et al. (2004). speculate that deeper linguistic knowledge needs to be made available to resolvers in order to reach the next level of performance. In fact, semantics plays a crucially important role in the resolution of common NPs, allowing us to identify the coreference relation between two lexically dissimilar common nouns (e.g., talks and negotiations) and to eliminate George W. Bush from the list of candidate antecedents of the city, for instance. As a result, researchers have re-adopted the once-popular knowledge-rich approach, investigating a variety of semantic knowledge sources for common noun resolution, such as the semantic relations between two NPs (e.g., Ji et al. (2005)), their semantic similarity as computed using WordNet (e.g., Poesio et al. (2004)) or Wikipedia (Ponzetto and Strube, 2006), and the contextual role played by an NP (see Bean and Riloff (2004)).

Another type of semantic knowledge that has been employed by coreference resolvers is the semantic class (SC) of an NP, which can be used to disallow coreference between semantically incompatible NPs. However, learning-based resolvers have not been able to benefit from having an SC agreement feature, presumably because the method used to compute the SC of an NP is too simplistic: while the SC of a proper name is computed fairly accurately using a named entity (NE) recognizer, many resolvers simply assign to a common noun the first (i.e., most frequent) WordNet sense as its SC (e.g., Soon et al. (2001), Markert and Nissim (2005)). It is not easy to measure the accuracy of this heuristic, but the fact that the SC agreement feature is not usedby Soon et al.’s decision tree coreference classifier seems to suggest that the SC values of the NPs are not computed accurately by this first-sense heuristic.

2. Related Work

Mention detection. Many ACE participants have also adopted a corpus-based approach to SC determination that is investigated as part of the mention detection (MD) task (e.g., Florian et al. (2006)). Briefly, the goal of MD is to identify the boundary of a mention, its mention type (e.g., pronoun, name), and its semantic type (e.g., person, location). Unlike them, (1) we do not perform the full MD task, as our goal is to investigate the role of SC knowledge in coreference resolution; and (2) we do not use the ACE training data for acquiring our SC classifier; instead, we use the BBN Entity Type Corpus (Weischedel and Brunstein, 2005), which consists of all the Penn Treebank Wall Street Journal articles with the ACE mentions manually identified and annotated with their SCs. This provides us with a training set that is approximately five times bigger than that of ACE.More importantly, the ACE participants do not evaluate the role of induced SC knowledge in coreference resolution: many of them evaluate coreference performance on perfect mentions (e.g., Luo et al. (2004)); and for those that do report performance on automatically extracted mentions, they do not explain whether or how the induced SC information is used in their coreference algorithms.

Joint probabilistic models of coreference. Recently, there has been a surge of interest in improving coreference resolution by jointly modeling coreference with a related task such as MD (e.g., Daumé and Marcu (2005)). However, joint models typically need to be trained on data that is simultaneously annotated with information required by all of the underlying models. For instance, Daumé and Marcu’s model assumes as input a corpus annotated with both MD and coreference information. On the other hand, we tackle coreference and SC induction separately (rather than jointly), since we train our SC determination model on the BBN Entity Type Corpus, where coreference information is absent.

misc/tbd

Nominal entity tagging is a difficult problem because there is ambiguity involved in both nominal detection and classification. First, the same string can either be a nominal entity1 or a non-nominal entity depending on the context.

The LDC standard defines nominal entities over words as well as the phrases of which they are heads. The words are called head words and the phrases are called the extents.

In our experiments, we trained and tested the model on the data set provided by LDC for the Automatic Content Extraction (ACE) research program, which contains a 110K-hanzi (Chinese characters) training data set and a 135K-hanzi evaluation data set.

5 Conclusions

We have shown that (1) both mention and SCA can be usefully employed to improve the performance of a learning-based coreference system, and (2) employing SC knowledge induced in a supervised manner enables a resolver to achieve better performance than employing SC knowledge computed by Soon et al.'s simple method. In addition, we found that the MUC scoring program is unable to reveal the usefulness of the SCA KS, which, when encoded as a feature, substantially improves the accuracy of common NP resolution. This underscores the importance of reporting both resolution accuracy and clustering-level accuracy when analyzing the performance of a coreference resolver.

References

• D. Bean and Ellen Riloff. (2004). Unsupervised learning of contextual role knowledge for coreference resolution. In: Proceedings of HLT/NAACL, pages 297–304.
• D. M. Bikel, R. Schwartz, and R. M. Weischedel. (1999). An algorithm that learns what's in a name. Machine Learning 34(1–3):211–231.
• C.-C. Chang and C.-J. Lin, (2001). LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/ cjlin/libsvm.
• Michael Collins and Yoram Singer. (1999). Unsupervised models for named entity classification. In: Proceedings of EMNLP/VLC.
• W. Daelemans, J. Zavrel, K. van der Sloot, and A. van den Bosch. (2004). TiMBL: Tilburg Memory Based Learner, version 5.1, Reference Guide. ILK Technical Report.
• H. Daumé III and D. Marcu. (2005). A large-scale exploration of effective global features for a joint entity detection and tracking model. In: Proceedings of HLT/EMNLP, pages 97–104.
• R. Florian, H. Jing, Nanda Kambhatla, and I. Zitouni. (2006). Factorizing complex models: A case study in mention detection. In: Proceedings of COLING/ACL, pages 473–480.
• Marti Hearst. (1992). Automatic acquisition of hyponyms from large text corpora. In: Proceedings of COLING.
• H. Ji, D. Westbrook, and Ralph Grishman. (2005). Using semantic relations to re ne coreference decisions. In: Proceedings of HLT/EMNLP, pages 17–24.
• A. Kehler, Douglas E. Appelt, L. Taylor, and A. Simma. (2004). The (non)utility of predicate-argument frequencies for pronoun interpretation. In: Proceedings of NAACL, pages 289–296.
• Dekang Lin. 1998a. Automatic retrieval and clustering of similar words. In: Proceedings of COLING/ACL, pages 768–774.
• Dekang Lin. 1998b. Dependency-based evaluation of MINIPAR. In: Proceedings of the LREC Workshop on the Evaluation of Parsing Systems, pages 48–56.
• Dekang Lin. 1998c. Using collocation statistics in information extraction. In: Proceedings of MUC-7.
• X. Luo, A. Ittycheriah, H. Jing, Nanda Kambhatla, and S. Roukos. (2004). A mention-synchronous coreference resolution algorithm based on the Bell tree. In: Proceedings of the ACL.
• (Markert and Nissim, 2005) ⇒ Katja Markert, and Malvina Nissim. (2005). “Comparing Knowledge Sources for Nominal Anaphora Resolution.” In: Computational Linguistics 31(3).
• Ruslan Mitkov. (2002). Anaphora Resolution. Longman.
• (Mitkov, 1998) ⇒ Ruslan Mitkov. (1998). “Robust Pronoun Resolution With Limited Knowledge.” In: Proceedings of the 17th International Conference on Computational Linguistics and 36th Annual Meeting of the Association for Computational Linguistics (ACL 1998).
• V. Ng and C. Cardie. (2002). Improving machine learning approaches to coreference resolution. In: Proceedings of the ACL.
• E. W. Noreen. (1989). Computer Intensive Methods for Testing Hypothesis: An Introduction. John Wiley & Sons.
• M. Poesio, R. Mehta, A. Maroudas, and J. Hitzeman. (2004). Learning to resolve bridging references. In: Proceedings of the ACL.
• S. P. Ponzetto and M. Strube. (2006). Exploiting semantic role labeling, WordNet and Wikipedia for coreference resolution. In: Proceedings of HLT/NAACL, pages 192–199.
• J. Ross Quinlan. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA.
• W. M. Soon, H. T. Ng, and D. Lim. (2001). A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521–544.
• J. Tetreault. (2001). A corpus-based evaluation of centering and pronoun resolution. Computational Linguistics, 27(4).
• M. Vilain, J. Burger, J. Aberdeen, D. Connolly, and Lynette Hirschman. (1995). A model-theoretic coreference scoring scheme. In: Proceedings of MUC-6, pages 45–52.
• R. Weischedel and A. Brunstein. (2005). BBN pronoun coreference and entity type corpus. Linguistica Data Consortium.
• X. Yang, G. Zhou, J. Su, and C. L. Tan. (2003). Coreference resolution using competitive learning approach. In: Proceedings of the ACL, pages 176–183.
• David Yarowsky. (1995). Unsupervised word sense disambiguation rivaling supervised methods. In: Proceedings of the ACL.

---

,

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2007 SemanticClassInductionAndCoreferenceResolution	Vincent Ng			Semantic Class Induction And Coreference Resolution			http://acl.ldc.upenn.edu/P/P07/P07-1068.pdf