Constrained Least Squares solution times

In this post a somewhat larger linearly constrained least squares problem is solved using R. The problem is simply:

\[\begin{align}\min_P&||P s - f||^2\\ & \sum_i p_{i,j}\le 1\\ & p_{i,j} \ge 0 \end{align}\]

To make things easier for the QP solver I implemented the model (in GAMS) as:

\[\begin{align}\min_{P,e}&\sum_i e_i^2\\ & \sum_j p_{i,j} s_j = f_i + e_i\\ &\sum_i p_{i,j} \le 1 \\ &p_{i,j} \ge 0 \end{align}\]

The timings for a problem with \(39 \times 196\) elements in \(P\) are:

R+constrOptim	15 hours
R+auglag	2 hours
GAMS+IPOPT	1.2 seconds
GAMS+Cplex	0.15 seconds

Cplex uses a specialized convex quadratic solver, while IPOPT is a general purpose NLP (nonlinear programming) solver.

Sometimes it helps to choose the right solver for the problem at hand.

References

1. Follow-up on this blog detailing the effects on my reformulation is here: http://yetanothermathprogrammingconsultant.blogspot.com/2016/11/constrained-least-squares-solution.html