Putting the cart before the horse, or: “the component is real, no matter what it means” ***

The late Stephen J. Gould, while better known for his evolutionary work, wrote a great book in 1981 called “The mismeasure of Man”, which was essentially a methods criticism about the abuse of Factor Analysis in social and life sciences. I read it a few years ago and recommend it wholeheartedly. Gould got some criticism because of the anthropometric results he criticized and re-worked in the book, but this is only a small detail whose veracity I cannot really comment on. The main conceptual point of the book is far bigger: the beginning of the 20th century, spurred on by the brilliance of several statisticians -Karl Pearson being most notable among them-, used multivariate decomposition techniques like Factor Analysis or PCA extensively on psychometric survey data, often obtained from immigrants coming to the United States through Ellis Island. Early work on the quantification of intellectual ability in France by Alfred Binet, mainly with the goal of identifying weaker children who could benefit from extra help (rather than being written off as “sick”), was now turned on its head into constructing elaborate, and often racist, sorting mechanisms. Test data were factor-analyzed, particular rotation schemes of the factor structure were applied, and people were ranked and evaluated along a few construct dimensions with lasting consequences, much against the original French model. The fact that limited English-language ability, lack of acculturation and test-taking experience, and the recent arrival from a possibly traumatic passage across the Atlantic all would have severely hindered test performance is only the most visibly egregious fact here.

My main criticism for this post similarly applies to the idea of deriving absolute meaning from multivariate decomposition routines in the absence of additional validating information. What Gould criticized is happening on a smaller scale in neuroimaging, fortunately with no bad consequences for society, other than for neuroimaging itself.

The starting point here is the concept of a multivariate decomposition. It does not matter whether it is Independent Component Analysis (ICA), Principal Component Analysis (PCA), or in fact any other technique, since the concept is very general. Imagine a data array Y which assigns different values to locations x in different brain scans s, and the full dependencies are denoted as Y(s,x). Since we are dealing with brain imaging, s and x have the particular assigned roles, but in general, s and x could be different too: s could denote persons, x could denote also batteries of different anthropometric measurements or psychometric tests.

The main point of the multivariate decomposition is to express the data array as a series of different components, or factors according to:

Y(s,x)=w1(s)v1(x)+w2(s)v2(x)+w3(s)v3(x) + …

The goal here is to truncate this series at a number k that is a lot smaller than the real data rank N. This series can also be written very easily as a matrix factorization

Y = V W’

where Y is the data matrix with rows denoting locations in the brain, and columns denoting persons. For V rows would denote locations in the brain, while columns denote the particular components that were retained. W is the matrix of component scores, and now rows denote persons, while columns again denote components; ‘ indicates matrix transposition.

Such decompositions can be very helpful in practice, mainly for the purposes of data reduction. Rather than retaining lots of possibly redundant information in a huge data array, we can focus on the first few components, and possibly capture the majority of the important signal in the data while losing the irrelevant parts.

Different decompositions will impose different constraints on the matrix factorization: PCA imposes orthogonality on both columns of W and V. ICA imposes statistical independence to all possible orders on the columns of either W or V. Other conditions could possibly be imposed altogether in lesser known decomposition schemes.

What is easy to forget now is the following: these decomposition routines work regardless of what data array Y is submitted to them. Even if Y mainly consists of statistical noise devoid of any meaningful signal they will reliably yield orthogonal or statistically independent components by design. This cautions against endowing these components with meaning absent any other corroborating outside information. These components better be statistically robust, that is a minimum requirement, but even more importantly, they need to explain something important or predict a subject variable not contained in original data array Y. This is just common sense – nothing fancy about it. Proliferating components purely by virtue of cool routines, without checking correlation with important subject variables or informing any mechanistic knowledge, is obviously not really science. It’s some kind of “derivative methods’ phenomenology” and might teach you more about the methods being used, rather than the subject of study motivating your data acquisition.

I would venture, somewhat more controversially, one should be careful before endowing such components with meaning, even if they were found to be reasonably robust and replicate between different studies. For brain studies, such replication is often judged purely, and only approximately, on account of topographic component content, i.e. the brain areas that are involved in the components. Before declaring components real they need to explain something (outside the data that they were derived from) with a relationship that was hitherto unappreciated. At a minimum, the components should help predict behavioral or clinical subject performance. Ideally, they should also inform you about the underlying neurobiology. “Meta-analytic eye-balling” underlying judgments of component identity purely on grounds of topography is itself a highly problematic and rampant practice in cognitive neuroscience. Proper replication should include relationships with external variables, not only loosely defined topographic correspondence.

Regrettably, neuroimaging has a good amount of such “liberal arts with computers” research resulting in the reification of questionable, free-floating and highly derivative results, which then spawn further research. After a component has been discovered and replicated along very narrow lines of content validity, the following research program might ensue: what does the component do? Does it correlate with something important, like a clinical variable, i.e. does it change with age or disease, and thus could be used diagnostically?

This is obviously backwards. Techniques and their use should be driven by a research program not the other way around, i.e. their development and use should be guided compellingly by a prior mechanistic question. Figuring out a research program based on a technique’s results puts the cart before the horse. But a lot of carts are pulling horses these days in neuroimaging.

Techniques employed for their own sake, neither to aid biology nor clinical utility, should not take up as much room as they currently do in neuroimaging research. Hopefully a cultural change is on its way already or will at least come very soon.