Latent Factor Analysis Task

A Latent Factor Analysis Task is an dimensionality compression task that requires the identification of latent variables.

• Context:
  • It can be solved by a Factor Analysis System (that implements a Factor Analysis algorithm).
  • It can range from being a Linear Factor Analysis to being a Non-Linear Factor Analysis.
  • It can range from being an Exploratory Factor Analysis Task to being a Confirmatory Factor Analysis Task.
  • …
• Counter-Example(s):
  • a Feature Selection Task?
  • a Feature Compression Task.
• See: Multivariate Analysis, Factor Model, Low-Rank Approximation, Partial Least Squares Regression, MANOVA.

---

---

References

2015

• (Wikipedia, 2015) ⇒ http://en.wikipedia.org/wiki/factor_analysis Retrieved:2015-2-15.
  • Factor analysis is a statistical method used to describe variability among observed, correlated variables in terms of a potentially lower number of unobserved variables called factors. For example, it is possible that variations in four observed variables mainly reflect the variations in two unobserved variables. Factor analysis searches for such joint variations in response to unobserved latent variables. The observed variables are modelled as linear combinations of the potential factors, plus “error” terms. The information gained about the interdependencies between observed variables can be used later to reduce the set of variables in a dataset. Computationally this technique is equivalent to low-rank approximation of the matrix of observed variables. Factor analysis originated in psychometrics and is used in behavioral sciences, social sciences, marketing, product management, operations research, and other applied sciences that deal with large quantities of data.

     Factor analysis is related to principal component analysis (PCA), but the two are not identical. Latent variable models, including factor analysis, use regression modelling techniques to test hypotheses producing error terms, while PCA is a descriptive statistical technique. There has been significant controversy in the field over the equivalence or otherwise of the two techniques (see exploratory factor analysis versus principal components analysis).

2013

• http://en.wikipedia.org/wiki/Factor_analysis#Type_of_factor_analysis
  • Exploratory factor analysis (EFA) is used to identify complex interrelationships among items and group items that are part of unified concepts. The researcher makes no "a priori" assumptions about relationships among factors.
  • Confirmatory factor analysis (CFA) is a more complex approach that tests the hypothesis that the items are associated with specific factors. CFA uses structural equation modeling to test a measurement model whereby loading on the factors allows for evaluation of relationships between observed variables and unobserved variables. Structural equation modeling approaches can accommodate measurement error, and are less restrictive than least-squares estimation. Hypothesized models are tested against actual data, and the analysis would demonstrate loadings of observed variables on the latent variables (factors), as well as the correlation between the latent variables.

• http://en.wikipedia.org/wiki/Factor_analysis#Applications_in_psychology
  • Factor analysis is used to identify "factors" that explain a variety of results on different tests. For example, intelligence research found that people who get a high score on a test of verbal ability are also good on other tests that require verbal abilities. Researchers explained this by using factor analysis to isolate one factor, often called crystallized intelligence or verbal intelligence, which represents the degree to which someone is able to solve problems involving verbal skills.

    Factor analysis in psychology is most often associated with intelligence research. However, it also has been used to find factors in a broad range of domains such as personality, attitudes, beliefs, etc. It is linked to psychometrics, as it can assess the validity of an instrument by finding if the instrument indeed measures the postulated factors.

• http://en.wikipedia.org/wiki/Factor_analysis#Exploratory_factor_analysis_versus_principal_components_analysis
  • While exploratory factor analysis and principal component analysis are treated as synonymous techniques in some fields of statistics, this has been criticised (e.g. Fabrigar et al., 1999; Suhr, 2009). In factor analysis, the researcher makes the assumption that an underlying causal model exists, whereas PCA is simply a variable reduction technique. Researchers have argued that the distinctions between the two techniques may mean that there are objective benefits for preferring one over the other based on the analytic goal.

2011

• (Fabrigar & Wegener, 2011) ⇒ Leandre R. Fabrigar, and Duane T. Wegener. (2011). “Exploratory Factor Analysis." Oxford University Press. ISBN:0199813515
  • QUOTE: … Factor analysis refers to a set of statistical procedures designed to determine the number of distinct constructs needed to account for the pattern of correlations among a set of measures. Alternatively stated, factor analysis is used to determined the number of distinct constructs assessed by a set of measures. These unobservable constructs presumed to account for the structure of correlations among measures are referred to as factors or more precisely as common factors. The specific statistical procedures comprising factor analysis provide information about the number of common factors underlying a set of measures. They also provide information to aid in interpreting the nature of these factors. The nature of common factors is clarified by providing estimates of the strength and direction of influence each of the common factors exerts on each of the measures being examined. Such estimates of influence are usually referred to as factor loadings. For cases in which the researcher has no clear expectations or relatively incomplete expectations about the underlying structure of correlations, procedures exist to conduct exploratory factor analysis (EFA) or unrestricted factor analysis. In this book we focus on these procedures and refer to them as ERA. When a researcher has a clear prediction about the number of common factors and the specific measures each common factor will influence, procedures are available to conduct confirmatory factor analysis (CFA) or restricted factor analysis (see Bollen, 1989).

2003

• (Pett & Lackey, 2003) ⇒ Marjorie A. Pett, and Nancy R. Lackey. (2003). “Making Sense of Factor Analysis: The Use of Factor Analysis for Instrument Development in Health Care Research." SAGE Publications, ISBN:0761919503
  • QUOTE: Factor analysis is not a single statistical method. Unlike the t test or ANOVA, it is not a test of differences between groups of subjects. Rather, factor analysis represents a complex array of structure-analyzing procedures use to identify the interrelationships amount a large set of observed variables and then, through data reductions, to group a smaller set of these variables into dimension or factors that have common characteristics (Nunnally & Bernstein, 1994).

    What is a factor? Most simply summarized, a factor is a linear combination or cluster or related observed variables that represent a specific underlying dimension of a construct, which is as distinct as possible from the other factors included in the solution (Tabachnick & Fidell, 2001).

1998

• (Johnson & Wichern, 1998) ⇒ Richard A. Johnson, and Dean W. Wichern. (1998). “Applied Multivariate Statistical Analysis, 4th ed." Prentice hall, 1992. ISBN:013834194X
  • QUOTE: The essential purpose of factor analysis is to describe, if possible, the covariance relationships among many variables in terms of a few underlying, but unobservable, random quantities called factors. Basically, the factor model is motivated by the following argument: Suppose variables can be grouped by their correlations. That is, support all variables within a particular group are highly correlated among themselves, by have relatively small correlations within variables in a different group. then it is conceivable that each group of variables represents a single underlying construct, or factor, that is responsible for the observed correlations. …

    … Factor analysis can be considered an extension of principal components analysis. Both can be viewed as attempts to approximate the covariance matrix [math]\displaystyle{ \Sigma }[/math]. however, the approximation based on the factor analysis model is more elaborate. They primary question in factor analysis is whether the data are consistent with a prescribed structure.

---