RIP the clash of cultures: univariate vs. multivariate analysis ***
This post contains a link to a didactic paper on multivariate neuroimaging analysis. This paper is a few years old, but occasionally gets still cited or referenced (actually, to my chagrin, more than some of my other papers I consider heftier…)
Before I give the actual link, I think some historical remarks are in order, since neuroimaging analytics has undergone a profound shift in the last 1.5 decades. Although my account will almost certainly be biased and might conflict with that of other witnesses of that time, I am trying to observe the shift without too much value judgment – some of the shift is unequivocally good, while some other things are not so good. Slightly related, but ultimately different, is the trend for neuroimaging analytics to become ever more derivative and to add layers of complexity and mystery, often for no discernible goal than intellectual ownership. I have complained about this in a different blog post, so will refrain from this side topic here.
When I entered fMRI neuroimaging almost 2 decades ago, data analysis was firmly a univariate affair. (Further, imaging studies were done that from today’s perspective seem ludicrously under-powered and Type-I error doubtlessly was a big problem.) SPM had already been established almost a decade before I entered neuroimaging and was the de-facto standard for analysis. Some researchers were conducting multivariate analysis (Jim Moeller, Stephen Strother, David Eidelberg, Randy McIntosh, Barry Horwitz etc.) but this happened in the PET world of imaging, not so much fMRI. Multivariate analysis was an uncommon occurrence. Reviewers often were suspicious and alleged lack of Type-I error control (“You will always find something!”). Cognitive Neuroscience audiences reared in traditions of regional localization with colorful T-maps were struggling with what multivariate analysis really meant and it whether it was really appropriate. An eternal question in the mind of any agnostic user was “which one of the two is better”, with opinionated univariate and multivariate detractors. At that time, out-of-sample prediction was not so much on researchers’ minds, so one performance metric by which multivariate analysis easily beats univariate analysis was not readily accepted.
People from other fields like Statistics or Engineering at this point might be forgiven to think that this artificial divide is a bit ridiculous. On a visit to Yale University in early 2008, a Biomedical Engineering Professor said as much to me, in a very polite manner. He was correct. (Although I of course benefited from this divide professionally.) Favoring one technique over the other in the abstract, away from the particular research goal and data set in question, is like saying that apples are better than oranges - it makes no sense. The whole “cultural divide” of multivariate and univariate analysis technique is a peculiar phenomenon of Cognitive Neuroscience. Fast forward 10 years and things had utterly changed. It’s hard to pin it to a particular event, but I would guess it happened with the advent of ICA and the FSL and GIFT software packages. Not only did they help dispel the erstwhile skepticism about multivariate analysis, but they led to a methods’ explosion in general (with mixed blessings for the field, but by now you got my drift already…).
Anyways, today challenges to multivariate analyses on the grounds of baseless suspicions are rare. That’s a good thing, definitely. The following didactic note points out a few things I feel might have gotten lost in the rushed wholesale transition to multivariate analysis, some things that are worth pointing out for general analytic literacy. Univariate and multivariate analysis are different and as such work better for different scenarios, i.e. focal activation vs. subtle distributed activation. Prospective application to new data is easier with multivariate analysis, and the biggest overlooked fact: for studies with typical numbers of observations (N<1000), and a number of voxels that’s at least 2 orders of magnitude larger, there HAS to be correlation between voxels, because N is the true data rank. Pretending that the voxels could be independent and imposing harsh inferential penalties (Bonferroni corrections, Field Theory) is a rather inefficient way of discovery. Non-parametric extensions lessen this concern, but the advantage of multivariate analysis for more parsimonious data description and out-of-sample predictions seems to have become universally accepted by now. Focusing on predicting things out of sample has the advantage that it provides very concrete performance metrics in terms of preditive utility and you can easily check which technique works the best. There is still a place for univariate analysis which are still popular, but debates about which one "is better" have fortunately died down.