Logistic Regression (2): least squares is non-convex

In http://yetanothermathprogrammingconsultant.blogspot.nl/2015/02/logistic-regression.html we implemented a simple maximum likelihood problem which replicates the results from R on a medium size data set (n=400 cases). The question is of course: why can’t we use some least squares objective. The answer is: this problem turns out to be a non-convex problem and thus we may get into trouble.

The can look like:

Of course if you like, we can substitute out the r’s and obtain an unconstrained problem. Note that x and y are data.

Some solvers can solve this, but others have some issues:

Solver	Results
Conopt, Ipopt, Knitro	const -3.94913 gre    0.00211 gpa    0.82281 rank2 -0.69297 rank3 -1.36355 rank4 -1.54693
Minos, Snopt	**** ERRORS/WARNINGS IN EQUATION fit(id1)      1 error(s): exp: FUNC OVERFLOW: x too large (RETURNED 1E299)

Indeed the maximum likelihood version of the model seems easier to solve.