GAMS/IDE bug

F3 does not repeat a search/replace operation (it becomes a search only). Sometimes it is useful to verify each search/replace and F3 can be used to do this. However, in the GAMS/IDE F3 does not repeat a complete search/replace operation.

Demonstration of numerical issues in LS calculations.

Probably the worst algorithm to perform a least squares estimation is to evaluate

b = (X'X)^-1X'y

This is illustrated in http://amsterdamoptimization.com/models/regression/longley2.gms. The solution compares as follows:

 parameter  reference                 LS solver     b=inv(X'X)X'y
    B0     -3482258.63459582         -3.48226E+6       65317.000
    B1      15.0618722713733         15.06187227          0
    B2     -0.358191792925910E-01    -0.03581918          0
    B3     -2.02022980381683         -2.02022980          0
    B4     -1.03322686717359         -1.03322687          0
    B5     -0.511041056535807E-01    -0.05110411          0
    B6      1829.15146461355         1.829151E+3          0

A numerically sound method is documented in http://amsterdamoptimization.com/pdf/regression.pdf. The above model can then be coded as: http://amsterdamoptimization.com/models/regression/longley.gms. The results are listed in the column "LS Solver".

Excel PivotTable using external data

When storing a GAMS symbol each nonzero in a separate row in Excel, we quickly run out of space on older versions of Excel. Excel before Office 2007 could only handle up to 65,536 rows. To display larger pivot tables, we can write the GAMS symbol to a CSV file and use this CSV file as external data store for the Excel pivot table. This allows for larger data sets than the above mentioned row limit. See also http://www.amsterdamoptimization.com/pdf/toolhelp.pdf.

AMPL defined variables

AMPL has a construct called defined variables that can sometimes reduce the size of an NLP by a large amount. Here are some statistics for a client model (confidential so I cannot share it):

GAMS:

BLOCKS OF EQUATIONS           6     SINGLE EQUATIONS       26,192
BLOCKS OF VARIABLES           5     SINGLE VARIABLES       26,201
NON ZERO ELEMENTS       119,031     NON LINEAR N-Z         45,220
DERIVATIVE POOL               8     CONSTANT POOL           1,541
CODE LENGTH             359,381

AMPL:

Substitution eliminates 26180 variables.
Adjusted problem:
10 variables, all nonlinear
1 constraint, all linear; 10 nonzeros 
1 nonlinear objective; 10 nonzeros.

GAMS has some similar facilities built in the LGO solver, but these are not available as a general feature.

Loops over different data sets

See http://groups.google.com/group/sci.op-research/browse_thread/thread/78456e35ddfc7141. This is actually a question that is often asked. The answer is really dependent on the situation. In this case the single case digests the input during compile time (using $include). This can not be coded directly by placing a loop around it (a loop is runtime). In this case it would probably the easiest the implement the single case as a complete GAMS file passing on the name of the input file and the output file as command line parameters. I.e.

* %input% is input file name (XLS file)
* %output% is output gdx file

$if not set input $set input E:\simuldata1\datag1.xls 
$if not set output $set output E:\simuldata\data1.xls  
 
Set s1  /
$call =xls2gms "i=%input%" o=E:\work\dataif1.inc R=datag1!a2:a12
$include E:\work\dataif1.inc
/;
  
Parameter p(s1) /
$call =xls2gms "i=%input%" o=E:\work\data21.inc R=datag1!j2:k12
$include E:\work\data2l.inc
/;
 
......
 
Execute_Unload "data2.gdx" V1
Execute 'GDXXRW.EXE data2.gdx sq=0 o=%output% var=V1 rng=sheet2!e2 rdim=1';

Note: the $if statements allow the model to be run outside the loop as well.

Now generate the loop in a separate GAMS file:

set i /1*100/;
 
file f /x.cmd/;
 
loop(i,
   put f,"gams singlecase "
         "--input=E:\simuldata" i.tl:0 "\datag" i.tl:0 ".xls "
         "--output=E:\simuldata\data" i.tl:0 ".xls "
         "O=E:\simuldata\singlecase" i.tl:0 ".lst"/;
);
putclose f;
  
execute "x.cmd";

This will generate a batch file:

gams singlecase --input=E:\simuldata1\datag1.xls --output=E:\simuldata\data1.xls O=E:\simuldata\singlecase1.lst
gams singlecase --input=E:\simuldata2\datag2.xls --output=E:\simuldata\data2.xls O=E:\simuldata\singlecase2.lst
gams singlecase --input=E:\simuldata3\datag3.xls --output=E:\simuldata\data3.xls O=E:\simuldata\singlecase3.lst
gams singlecase --input=E:\simuldata4\datag4.xls --output=E:\simuldata\data4.xls O=E:\simuldata\singlecase4.lst
gams singlecase --input=E:\simuldata5\datag5.xls --output=E:\simuldata\data5.xls O=E:\simuldata\singlecase5.lst
....

Instead of writing a batch file it is also possible to directly generate a command line using the infamous put_utility command:

put_utility "shell" / "gams singlecase --input=E:\simuldata" i.tl:0 "\datag" i.tl:0 ".xls --output=E:\simuldata\data" i.tl:0 ".xls O=E:\simuldata\singlecase" i.tl:0 ".lst";

Probably the best is to consult support@gams.com as this is very hairy.

Often it is better to reorganize the data flow and read all data in the beginning, then perform a loop to calculate the results, and then export all results. The input data and results have then an extra index.

Any uglier syntax possible?

To compose a file name for a gdx file, one can do:

set mds /mds1*mds3/;
parameter x;
 
* needed for put_utility
file dummy;
put dummy;
 
* remove all gdx files
execute 'del /q *.gdx';
 
loop(mds,
 
* solve etc here

* say solution is x
    x = ord(mds);
    display x;
 
* filename := 'sol_mds1',  'sol_mds2', ....
    put_utility 'gdxout' / 'sol_' mds.tl:0;
 
* write to gdx file with changed name
    execute_unload x;
);
 
* show what gdx files we now have
execute 'dir *.gdx';
 
* merge gdx files
execute 'gdxmerge sol_*.gdx';
 
* show that it created merged.gdx
execute 'dir *.gdx';
 
* show content of merged.gdx
execute 'gdxdump merged.gdx';

In my experience this is very difficult to explain. The put_utility syntax should really be replaced by something more elegant (not a very high bar).

Notice that we need a put file around even if we don't use it. It should not be too difficult to hide this from the modeler and handle this inside GAMS.

Gamma Distribution CDF

See http://groups.google.com/group/ampl/browse_thread/thread/10861bd79114dfec. In GAMS we have the GAMMAREG function (see http://www.gams.com/~erwin/specfun.pdf) so no need to program this yourself using external functions.

The Gamma CDF from http://en.wikipedia.org/wiki/Gamma_distribution denoted by F(x;k,θ) with

• scale parameter θ
• shape parameter k
• mean: k·θ
• variance: k·θ²

can be expressed in GAMS by:

F(x;k,θ) = GAMMAREG(x/θ,k)

Both first and second derivatives are available for solvers calling this function.

More execute_unload

This seems allowed but has unexpected results.

scalar a /1/; 
execute_unload a;

This will go to a gdx file 'xllink.gdx'???? Be warned: this construct will overwrite any file xllink.gdx even though you did not specify this filename in the GAMS model (luckily chance is pretty small).

GAMS: execute_unload

It would be useful to be able to write to the same GDX file in different steps. I.e. something like:

execute_unload "a.gdx",a;

a(i) = a(i)/1000;
b(i) = c(i) + a(i);

execute_unload "a.gdx","append",b; 

Currently we cannot do this: there is no facility to append.

GAMS/IDE suggestion

The LST viewer can be improved by allowing easy navigation to syntax errors in the listing file. In many cases you don't even need to consult the listing file to fix errors using the IDE, but sometimes it is needed to inspect the listing file. (The previous is an example). This may help to find syntax error messages in the listing file (especially when there are comments with stars so searching for **** is not convenient).