Orthogonalization and projection in the null space

Let's take two correlated regressors: reg1 and reg2 = λ * reg1 + cst.

corr(reg1, reg2) = 0.48

We will show in this section how to properly orthogonalize reg2 with respect to reg1, ie. to project reg2 in the null space of reg1. This will insure that our regressors are not correlated.

The GLM solution

Let's compute the part of reg2 explained by reg1:

a = glmfit(reg1, reg2)

We can then remove it from reg2 and get a new regressor orthogonal to reg1:

reg3 = reg2 - a(2) * reg1

corr(reg1, reg3) = 0.00004

The SPM solution

When one have to deal with multiple regressors to be sequentially orthogonalized, it is tempting to use the SPM function spm_orth, designed for this purpose:

regs = spm_orth(reg1, reg2)

reg4 = regs(:,end)

The resulting regressors are indeed orthogonal:

reg1' * reg4 = 4.34 e-11

However, the result might not be what you would expect !

corr(reg1, reg4) = -0.59

This is because spm_orth needs centered regressors to work properly. An easy workaround is either to center your regressors by hand :

regs= spm_orth( reg1 - mean(reg1), reg2)

reg5 = regs(:,end)

or to simply add a constant as a first regressor in spm_orth:

regs= spm_orth([ones(size(reg1)), reg1, reg2])

reg5 = regs(:,end)

And you will in fact get what you want:

corr(reg1, reg5) = 1e-16

Take home message:

Do not forget the constant term when you use spm_orth!