Adjustment for confounders, it’s not the panacea you think it is ***

Statistical adjustment for confounding variables is a very common technique in neuroscience analytics. I literally do it every day. Ideally, with a carefully controlled experiment, like a randomized clinical trial, one can make sure that nothing else is varying in lock-step with the experimental manipulation of interest. Subjects are randomly assigned to treatment or placebo groups, nullifying any possible confounding correlations with other variables.

Unfortunately, most often in neuroimaging, data are obtained from observational studies, where no such careful control can be exercised. Even if one did their homework and recruited subjects via something like random-market mailing, the ensuing bias in the response of the study participants to the mailed flyer might still present some obvious confounding correlations. I often deal with data where chronological age and intelligence and education of the participants are highly correlated. The elder study population responding to ads for neuroscience research often is highly educated and motivated by things other than money.

The usual trick in the analyst’s tool kit is now a statistical adjustment, i.e. sticking all possible confounding variables, like age or other demographics into the liner regression as additional independent variables, beyond the independent variable of interest. The motivation is that the effect of interest is then automatically purified of nuisance influences, even if collinearities were present amongst the independent variables.

The first reflection in order here is that one of course cannot “control” for anything in an observational study design; “everything else being equal” in this circumstance is an illusion, i.e. it makes a lot of assumptions about the influence of the confounders on the outcome of interest, and pretends that you can achieve the effects of hard work, careful study design and data collection just as easily with a linear-algebra sleight of hand. Not surprisingly, this is too good to be true, and something has to give.

Let us really dig in now. Statistical adjustment is most desirable when the variable of interest is collinear with lots of other subject variables. One very simple observation surprised me several years ago when I was just following the usual approach of “silent regressors” that’s often used in our research. Imagine an outcome Y dependent on two collinear inputs X1 and X2, i.e. X1 and X2 are quite significantly correlated, and we have the following regression model.

Y = X1 *beta1 + X2 * beta2 + intercept + error

So in our example, you have maybe Y=disease probability, x1= age, and x2=dose of exposure to some risk factor. X1 and X2 are most likely correlated here. Like everybody else I thought entering both independent variables simultaneously achieve “control” for collinearity – no problem! Ok, granted, there are some technical consequences of collinearity, i.e. the error estimates on the betas are inflated etc. etc. possibly hampering statistical significance, but in the end if you have a significant beta 2, you can say that “exposure to the risk factor in question contributes to disease beyond age”.

The –to me and probably many other people- surprising insight is now the following: a “contribution” by an independent variable above and beyond some confounder does not imply that the partial model prediction by said independent variable is orthogonal to the confounder. I would bet 90% of researchers are surprised by that; well, I definitely was. In other words: the part of Y explained by X2 is still collinear to X1. Mathematically this is trivial to see: the partial prediction on the basis of the risk factor is just

Partial prediction = X2*beta2 + intercept

which is exactly as collinear with X1 as X2 is. (Correlations are invariant to affine transformations of the variables.) So even though you can enter independent variables simultaneously, you should be aware what exactly that buys you. Unfortunately, the statistical adjustment works best for the scenario that is of least interest: when the confounder and the independent variable are uncorrelated to begin with. In this case, the confounder soaks up some unrelated variance in Y, making the relationship to the independent variable sharper.

So to summarize: when confounders and the independent variable are collinear, even a significant effect of the independent variable beyond the confounder does not imply that the partial prediction is orthogonal to the confounder; but that is what most people implicitly assume. I remember my surprise when plotting the partial prediction against the nuisance variable, “just to make sure I have adequately controlled for the nuisance”. For our above example, the partial prediction of disease risk by the risk factor would still correlate highly with age. That’s probably not what most people have in mind when they are aiming for statistical adjustment. Statistical adjustment thus cannot really beat the real thing, i.e. ensuring in the study design that an experimental variable is uncorrelated with obvious major confounders.

The next more rigorous step that some people take is partialling out the influence of the confounder, and computing a partial correlation between outcome and independent variable. Concretely, this means you first regress both Y and X2 against X1 and compute residuals resY and resX2; the partial correlation is then just the correlation of residuals resY and resX2. This is indeed more rigorous (and often throws out the baby with the bath water), but it is not a very satisfactory solution either. The multi-step procedure with initial partialling is statistically less robust than a one-shot regression, and the meaning of resY and resX2 is not so clear anymore compared with Y and X2.

One last fix around this which results in data loss is to sub-sample the data such that X1 and X2 are uncorrelated; this could be even done repeatedly. However, willingly giving up hard-won data is not ideal either, and sub-sampling is obviously only an option for rich data sets. If you only have 30 observations, sub-sampling might not be an option. Collinear independent variables are thus always a problem however one chooses to deal with them. There is no free lunch: careful experimental design cannot easily be bested by analysis tricks.