0

I am going to create a model strictly for predictive purposes. Some of my independent variables are highly correlated. When I try to create the regression with all of the variables together, then, I should not trust the p-values? Is it a better idea to check each independent variables separately against the dependent variable, just to make sure that they are significantly related.

User0
  • 287
  • 2
  • 16

1 Answers1

2

You should not use very highly correlated variables in your model, as the result of the regression will be meaningless (Imagine regressing Y on $X_1,...,X_n$ with $X_1=...=X_n$, you're obviously losing the unicity of the coefficient).

Try to perform a principal component analysis and drop the variables that have no explanatory power in sample.

Matt B.
  • 1,246
  • will the fact that some of my variables are categorical violate the assumptions of PCA? – User0 Jul 15 '14 at 12:51
  • I don't know how fluent you are in French, but I think you should look at http://fr.wikipedia.org/wiki/Analyse_factorielle_de_donn%C3%A9es_mixtes (PCA modified for both quantitative and qualitative variables) – Matt B. Jul 15 '14 at 13:33
  • Actually I do speak French! – User0 Jul 15 '14 at 17:02