Thursday, June 5, 2008

Demonstration of numerical issues in LS calculations.

Probably the worst algorithm to perform a least squares estimation is to evaluate

b = (X'X)-1X'y

This is illustrated in http://amsterdamoptimization.com/models/regression/longley2.gms. The solution compares as follows:

 parameter  reference                 LS solver     b=inv(X'X)X'y
B0 -3482258.63459582 -3.48226E+6 65317.000
B1 15.0618722713733 15.06187227 0
B2 -0.358191792925910E-01 -0.03581918 0
B3 -2.02022980381683 -2.02022980 0
B4 -1.03322686717359 -1.03322687 0
B5 -0.511041056535807E-01 -0.05110411 0
B6 1829.15146461355 1.829151E+3 0



A numerically sound method is documented in http://amsterdamoptimization.com/pdf/regression.pdf. The above model can then be coded as: http://amsterdamoptimization.com/models/regression/longley.gms. The results are listed in the column "LS Solver".