Factor Analysis
When you measure many things about the same object, some of those measurements are really different views of a smaller set of hidden causes. Factor Analysis is a way to uncover those hidden causes—called latent factors—from the patterns of correlation in the data.
How it works
Factor Analysis assumes each observed variable is built from two parts: a shared signal explained by a few latent factors, and a variable-specific part that is just its own noise or uniqueness. If survey questions about price sensitivity, brand loyalty, and deal-seeking move together, the model can explain that with one or two underlying factors rather than treating each question as fully separate. Mathematically, it models the data as a linear combination of latent factors plus error. This makes it different from PCA: PCA looks for directions that capture maximum variance, while Factor Analysis tries to explain the covariance structure through hidden variables and explicitly models noise.
Why it matters in practice
It is useful when the goal is not just compression, but understanding the hidden structure behind the variables. That matters in unsupervised learning because many downstream choices depend on whether features reflect common drivers or just measurement noise.
- Customer segmentation: reduce dozens of behavioral metrics into latent traits like value-consciousness or engagement before clustering.
- Topic-style structure in text metadata: identify hidden dimensions behind co-occurring tags or document attributes.
- Finance and risk: explain correlations among asset returns using a smaller number of market factors.
- Anomaly detection: detect records that do not fit the usual latent-factor pattern.
What you encounter technically
A key idea is factor loadings, which tell you how strongly each observed variable relates to each latent factor. Choosing the number of factors is crucial: too few hides real structure, too many start fitting noise. In practice, people use tools like scikit-learn's sklearn.decomposition.FactorAnalysis, then inspect loadings and transformed factor scores to build simpler, more interpretable representations of the data.
Factor Analysis is a linear latent-variable method that explains correlations among observed variables using a smaller set of unobserved factors plus variable-specific noise. Unlike PCA, it explicitly models shared variance separately from unique variance. In unsupervised learning, it matters because it reveals hidden structure, reduces dimensionality in a statistically grounded way, and supports interpretable representations for tasks such as psychometrics, genomics, and feature extraction.
Think of a school report card. A student might have many grades, but some hidden traits—like reading ability, focus, or confidence—may be influencing several of those grades at once. Factor Analysis is a way of looking at lots of measured things and guessing the smaller set of hidden influences behind them.
In AI and data analysis, it helps make messy, high-dimensional data easier to understand by reducing many variables into a few underlying factors, meaning broad hidden patterns. This matters because real-world data often contains overlap. Instead of treating every measurement as separate, Factor Analysis tries to reveal the deeper themes shaping what you see.