I have a 2 questions, could you please point me into a good direction?
1. Which open source optimization solver do you recommend? (One that can withstand a large and complex problem)
I've been researching online and I've come across with lpsolve, coin-or lp (CLP), and GLPK.
2. I've been having a really tough time to get GLPK to work. I'm new to this field of open source software but I've found out that I need an interface from where I should call the GLPK library. My question is, which application (or interface) should I use?
Thanks in advance!
CBC (https://projects.coin-or.org/Cbc) seems to be the fastest open source MIP solver (see http://plato.asu.edu/ftp/milpc.html). The academic code SCIP is also quite good but is not open source (it requires a license for any non-academic application). Of course the commercial codes are often (much) faster.
The GLPK API is simpler to use than CBC however.
In general I would say the low level solver API are often not a good idea to use for larger, more complex models. For those a modeling language is often more appropriate.
You may want to consider making your application solver independent. Then you can pick the solver most suited for the occasion. Again this is most easily done using a modeling framework.