2001 MetricBasedMethodsFor

Subject Headings: Metric-based Model Selection Algorithm, Model Selection, Regularization, Unlabeled Example.

Notes

(Zhu, 2008) ⇒ Xiaojin Zhu. (2008). “Semi-Supervised Learning Literature Survey (revised edition).” Technical Report 1530, Department of Computer Sciences, University of Wisconsin, Madison.
- Metric-based model selection (Schuurmans & Southey, 2001) is a method to detect hypotheses inconsistency with unlabeled data. We may have two hypotheses which are consistent on [math]\displaystyle{ L }[/math], for example they all have zero training set error. However they may be inconsistent on the much larger U. If so we should reject at least one of them, e.g. the more complex one if we employ Occam’s razor.

We present a general approach to model selection and regularization that exploits unlabeled data to adaptively control hypothesis complexity in supervised learning tasks. The idea is to impose a metric structure on hypotheses by determining the discrepancy between their predictions across the distribution of unlabeled data. We show how this metric can be used to detect untrustworthy training error estimates, and devise novel model selection strategies that exhibit theoretical guarantees against over-fitting (while still avoiding under-fitting). We then extend the approach to derive a general training criterion for supervised learning — yielding an adaptive regularization method that uses unlabeled data to automatically set regularization parameters. This new criterion adjusts its regularization level to the specific set of training data received, and performs well on a variety of regression and conditional density estimation tasks. The only proviso for these methods is that sufficient unlabeled training data be available.

,

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2001 MetricBasedMethodsFor	Dale Schuurmans Finnegan Southey			Metric-based methods for adaptive model selection and regularization		Machine Learning (ML) Subject Area	http://cs.ualberta.ca/~dale/papers/mlj02.pdf	10.1023/A:1013947519741		2001