Latent Factor Models Family
Jump to navigation
Jump to search
A Latent Factor Models Family is a statistical model family that represent entities with latent factors (usually low-dimensional embeddings) and relationships as factor operators (destined to combine factors).
- Context:
- It can be used by a Latent Factor Model Fitting Algorithm, such as a matrix factorization-based learning algorithm.
- It can be instantiated in a Fitted Latent Factors Model.
- It can aim to partition variation in the predictor variables into multiple components that reflect common patterns, and separate these from variation that is idiosyncratic to each variable, or “noise." (West, 2003).
- It can be a Sparse Latent Factor Model Family.
- Example(s):
- Counter-Example(s):
- See: Principal Components Analysis.
References
2003
- (West, 2003) ⇒ Mike West. (2003). “Bayesian factor regression models in the “large p, small n” paradigm." Bayesian Statistics, 7. ISBN:978-0-19-852615-5
- ABSTRACT: I discuss Bayesian factor regression models with many explanatory variables. These models are of particular interest and applicability in problems of prediction, but also for elucidating underlying structure in predictor variables. One key motivating application here is in studies of gene expression in functional genomics. I first discuss empirical factor (principal components) regression, and the use of general classes of shrinkage priors, with an example. These models raise foundational questions for Bayesians, and related practical issues, due to the use of design-dependent priors and the need to recover inferences on the effects of the original, high-dimensional predictors. I then discuss latent factor models for high-dimensional variables, and regression approaches in which low-dimensional latent factors are the predictor variables. These models generalise empirical factor regression, provide for more incisive evaluation of factor structure underlying high-dimensional predictors, and resolve the modelling and practical issues in empirical factor models by casting the latter as limiting special cases. Finally, I turn to questions of prior specification in these models, and introduce sparse latent factor models to induce sparsity in factor loadings matrices. Embedding such sparse latent factor models in factor regressions provides a novel approach to variable selection with very many predictors. The paper concludes with an example of sparse factor analysis of gene expression data and comments about further research. …
… Formal latent factor models aim to partition variation in the predictor variables into multiple components that reflect common patterns, and separate these from variation that is idiosyncratic to each variable, or “noise." Here I note the theoretical structure of standard linear, latent factor models and define a class of factor regression models that naturally relate underlying latent structure in high-dimensional predictors to responses.
- ABSTRACT: I discuss Bayesian factor regression models with many explanatory variables. These models are of particular interest and applicability in problems of prediction, but also for elucidating underlying structure in predictor variables. One key motivating application here is in studies of gene expression in functional genomics. I first discuss empirical factor (principal components) regression, and the use of general classes of shrinkage priors, with an example. These models raise foundational questions for Bayesians, and related practical issues, due to the use of design-dependent priors and the need to recover inferences on the effects of the original, high-dimensional predictors. I then discuss latent factor models for high-dimensional variables, and regression approaches in which low-dimensional latent factors are the predictor variables. These models generalise empirical factor regression, provide for more incisive evaluation of factor structure underlying high-dimensional predictors, and resolve the modelling and practical issues in empirical factor models by casting the latter as limiting special cases. Finally, I turn to questions of prior specification in these models, and introduce sparse latent factor models to induce sparsity in factor loadings matrices. Embedding such sparse latent factor models in factor regressions provides a novel approach to variable selection with very many predictors. The paper concludes with an example of sparse factor analysis of gene expression data and comments about further research. …