ipopt for very small NLPs

In the paper http://www.palgrave-journals.com/jors/journal/v61/n3/full/jors200977a.html a small NLP model is developed to minimize fuel use for commercial ships by optimizing speed. The GAMS representation of the proposed model can look like:

$ontext

    minimize fuel use by optimizing speed on shipping routes

    Reference:
      Reducing fuel emissions by optimizing speed on
      shipping routes, K Fagerholt, G Laporte and I Norstad,
      Journal of the Operational Research Society (2010) 61, 523 --529

$offtext

set
  i 'segments' /
        s1  Antwerp-Milford Haven
        s2  Milford Haven - Boston
        s3  Boston - Charleston
        s4  Charleston - Algeciras
        s5  Algeciras - Point Lisas
        s6  Point Lisas - Houston
      /
;

table data(i,*)
     distance  early last
s1      510      1    5
s2     2699      9   13
s3      838     11   15
s4     3625     20   24
s5     3437     32   36
s6     2263     35   39

;

scalars
   minspeed 'knots' /14/
   maxspeed 'knots' /20/
;

parameter d(i) 'distance (nautical miles)';
d(i) = data(i,'distance');

positive variables
   v(i) 'speed (knots)'
   t(i) 'time'
;
free variable fuel;

v.lo(i) = minspeed;
v.up(i) = maxspeed;

t.lo(i) = data(i,'early')*24;
t.up(i) = data(i,'last')*24;

*--------------------------------------------------------------------
* speed model
*--------------------------------------------------------------------

equations
  obj1
  time1(i)
;

obj1..  fuel =e= sum(i, d(i)*(0.0036*sqr(v(i))-0.1015*v(i)+0.8848) );

time1(i).. t(i) - t(i-1) =g= d(i)/v(i);

model m1 /obj1,time1/;

*--------------------------------------------------------------------
* time model
*--------------------------------------------------------------------


positive variable
  dt(i) 'sailing time'
;
* prevent division by zero by assuming lowerbound of 1
dt.lo(i) = 1;
equations
  obj2
  time2(i)
;

obj2..  fuel =e= sum(i, d(i)*(0.0036*sqr(d(i)/dt(i))-0.1015*(d(i)/dt(i))+0.8848) );

time2(i).. t(i) - t(i-1) =g= dt(i);

model m2 /obj2,time2/;


solve m2 using nlp minimizing fuel;

The second model has only linear constraints so that may be preferable in general. The waiting time is handled implicitly in this model. I would probably make the model slightly larger by making this more explicit by introducing a positive variable wt(i):

time1(i).. t(i) - t(i-1) =e= d(i)/v(i) + wt(i);

The paper mentions that IPOPT solves these models in about .5 seconds, and suggest that a discretization yielding a shortest path problem is much faster. I would suggest to try also some other NLP solvers, as IPOPT is really geared towards large scale problems. Other solvers may be a lot faster than IPOPT on these small problems. Indeed with CONOPT I see:

S O L V E      S U M M A R Y      MODEL   m1                  OBJECTIVE  fuel      TYPE    NLP                 DIRECTION  MINIMIZE      SOLVER  CONOPT              FROM LINE  65 **** SOLVER STATUS     1 Normal Completion         **** MODEL STATUS      2 Locally Optimal           **** OBJECTIVE VALUE             2266.4832 RESOURCE USAGE, LIMIT          0.000      1000.000 ITERATION COUNT, LIMIT        17    2000000000 EVALUATION ERRORS              0             0

The reported time is 0 seconds. (It is noted that the larger test problems in the paper are a little bit larger than this example, but not by much). Probably a good dense solver like DONLP will do very good on a model like this.

GAMS: writing spreadsheets

When doing real applications the demands on writing reports are sometime very high. Users have special requirements how the reports look like, and they often allow very little deviation from what they want. For this reason I often use GDXXRW with the CLEAR/MERGE option to update a table with solution data. This option will allow us to maintain a carefully designed layout.

The following was a little bit of a puzzle. How to update the spreadsheet below while skipping column E (units). The data is 4 dimensional: Commodity, Variable, Region, Year.

With GDXXRW we cannot use MERGE as that will keep old values in the spreadsheet. The option CLEAR is also not usable as it will wipe out column E. Eventually I found the following to work:

• Create an empty 5 dimensional parameter (rdim=4, cdim=1), so that we can CLEAR the body of the table
• Then use the original 4 dimensional parameter (rdim=3, cdim=1) with a MERGE option to fill the table body.

parameter reportx(*,*,*,*,*,t) 'dummy to be able to clear but keep column with units';
* index 1: commodity or '-'
* index 2: variable name
* index 3: region or '_'
* index 4: scenario
* index 5: unit or ''
* index 6: year

reportx('','','','','',t) = 0;

execute_unload "report.gdx",report,reportx;

execute '=gdxxrw i=report.gdx o=report2.xlsx skipempty=10 par=reportx rng=A4 rdim=5 cdim=1 clear par=report rng=A4 rdim=4 cdim=1 merge trace=2';

When we use the CLEAR/MERGE option, all first rdim columns have to be unique. This is somewhat unfortunate. I want a header now and then, as in:

but I can only use “Production Detail” only once. (I have had similar issues before with things like a “Subtotal”). This case often works as expected. Not always, as can be seen here:

C:\projects\china\dec24\output\chn10>gdxxrw i=report.gdx o=report2.xlsx skipempty=10 par=reportx rng=A4 rdim=5 cdim=1 clear par=report rng=A4 rdim=4 cdim=1 merge trace=2 GDXXRW           BETA  1May10 23.4.0 WIN 17193.17196 VS8 x86/MS Windows Excel version 12.0 Input file : C:\projects\china\dec24\output\chn10\report.gdx Output file: C:\projects\china\dec24\output\chn10\report2.xlsx Type Symbol       Dim     Sheet                Data          RowHeader     ColHeader Par  reportx        6     Sheet1               F5:XFD1000000 A5:E1000000   F4:XFD4 Reading range A5:E1000000 **** Duplicate Row/Column label(s) for symbol reportx:    Duplicate Row: A1062: (Yield, , , , ) Reading range F4:XFD4 Par  report         5     Sheet1               E5:XFD1000000 A5:D1000000   E4:XFD4 Reading range A5:D1000000    Duplicate Row: A1062: (Yield, , , ) Reading range E4:XFD4 Reading range E5:Y3767 Total time = 8065 Ms

May be it depends on whether some cells are empty or contain empty strings (the difference is not really visible in a spreadsheet).

However the same scheme for real rows would also be useful. E.g. in some cases I want data to be repeated, such as “YEAR”: in each row with such a key. just repeat the content.

Here I used the trick to introduce y2,y3,y5,… to make sure multiple year rows are being written. It would be better if GDXXRW would allow me to have multiple rows with row header “year”.

Update:

The “duplicate” errors are really warnings. The reason why some duplicate headers (“Production Detail”) don’t generate a message and others (“Yield”) do, depends on whether these descriptions are also used as UELS (set elements) in the GDX file.

Constraint programming

One of the advantages of CP is that some problems can be formulated in a much more natural form. For instance for this problem: http://yetanothermathprogrammingconsultant.blogspot.com/2010/04/miqp-vs-dynamic-programming.html is it quite useful to use a variable as an index in an array.

/*********************************************  * OPL 6.3 Model  * Author: erwin  * Creation Date: Jun 6, 2010 at 5:31:28 PM  *********************************************/  using CP;  int NC = 239; {int} Chapters = asSet(1..NC);   /* int D = 128; solve smaller version: */ int D=20; {int} Days = asSet(1..D);                      int Verses[Chapters] = [20,24,31,38,22,6,22,38,6,22,36,23,42,30,36,39,55,25,24,22,26,31,32,30,                         25,35,34,18,11,25,54,25,8,22,26,6,30,13,25,22,21,34,16,6,22,32,30,33,                         35,32,14,18,21,9,15,19,35,14,18,77,13,27,27,15,30,18,18,41,27,30,15,                         7,33,21,19,22,29,37,35,12,31,15,20,35,29,26,36,16,39,25,24,39,37,20,                         47,33,38,27,20,62,8,27,32,34,32,46,37,31,29,19,21,39,43,36,30,23,35,18,                         30,17,37,30,14,17,60,38,43,23,41,16,30,47,15,19,26,15,31,54,24,24,41,36,                         25,30,40,37,40,23,24,35,57,36,41,13,36,21,52,17,34,14,37,26,52,41,29,28,                         41,19,38,26,39,31,17,25,30,19,26,33,26,30,26,25,22,19,41,48,34,27,24,20,                         25,39,36,46,29,17,14,18,6,21,33,39,9,2,49,19,29,22,23,24,22,10,41,37,43,                         25,28,19,6,30,27,26,35,34,23,41,31,31,34,4,3,4,3,2,9,48,30,26,34];   dvar int startch[Days] in 1..NC; dvar int numch[Days] in 1..NC; dvar int+ v[Days]; float avg = (sum(c in Chapters) Verses[c])/D;   int verses[1..card(Chapters),1..card(Chapters)]; execute{    var i;    var j;    for(i=1; i<=NC; ++i) {       verses[i][1] = Verses[i];       for(j=2;i+j-1<=NC;++j)           verses[i][j] = verses[i][j-1]+Verses[i+j-1];    }         }   minimize   sum(d in Days) (v[d]-avg)*(v[d]-avg); subject to {   startch[1] == 1;   forall(d in 2..D) startch[d] == startch[d-1] + numch[d-1];   startch[D]+numch[D] == card(Chapters)+1;   forall(d in Days) v[d] == verses[startch[d],numch[d]]; }

Notes:

• This is written in IBM OPL
• GAMS has nothing to offer in this area
• AMPL has some facilities documented in some papers (http://users.iems.northwestern.edu/~4er/WRITINGS/cp.pdf) but I am unaware of this being actually available
• Some of the newer systems (Comet, MS Solver Foundation) have constraint programming support
• Some research codes out there have some support for a form of modeling language input (Minizinc, etc.).

Inventory

A standard formulation to deal with inventory in GAMS can look like:

set t /t1*t5/; positive variables   inv(t) 'inventory at end of period t'   production(t)    sell(t) ; parameter init_inv(t) 'initial inventory' /t1 100/; * Note: for any t > t1 init_inv(t)=0 equation    inv_bal(t) 'inventory balance'; inv_bal(t).. inv(t) =e= inv(t-1) + production(t) - sell(t) + init_inv(t);

Note: GAMS will automatically discard inv(t-1) for the first time period t=t1 (or putting it differently it will set inv(t-1) =0 in that case).

In a project I have to deal with aging of inventory. This looks like a production/inventory model I did for a large cheese manufacturer that had a similar structure: having the cheese stored for a period causes it to age and actually becoming a different (more expensive) product.

The inventory balance equation becomes a little bit more complicated in that case. In its simplest form it can be something like:

set   t /t1*t10/   a 'age classes' /a1*a3/   a1(a) 'first age class' /a1/ ; positive variables   inv(t,a) 'inventory at end of period t'   production(t)   sell(t,a) ; parameter init_inv(t,a) 'initial inventory' /    t1.a1 100    t1.a2 160    t1.a3 150 /; * Note: for any t > t1 init_inv(t,a)=0 equation    inv_bal(t,a) 'inventory balance'; inv_bal(t,a)..    inv(t,a) =e= inv(t-1,a-1)                 + production(t)$a1(a)                 - sell(t,a)                 + init_inv(t,a);

Unsold items are in inv(t,’a3’).

GDX2DBF: Convert Gdx files to dbase .DBF files

This tool is designed to convert GDX files into DBF files. DBF files are used in dBase, xBase, FoxPro and other programs. Also some programs use DBF files to import and export data. A good example is the GIS program ArcView. DBF files are also used in many other GIS applications to store data in conjunction with shape files.

Usage

d:\gdx2dbf>gdx2dbf
GDX2DBF Version 1.1
Usage:
   > GDX2DBF xxx.gdx [xxx.ref] [@inifile] [d=outputdir]

Dumps a GDX file to .DBF files.

Ini file format (default: GDX2DBF.INI):
  [settings]  (section name is required)
  inf=value for +INF (default 9999999.9999)
  mininf=value for -INF (default -9999999.9999)
  eps=value for EPS (default 0)
  na=value for NA (default 0)
  undf=value for UNDF (default 0)
  tablelevel=n (3=dbaseIII+,4=dbaseIV,7=dbaseVII,25=foxpro, default=3)
  numerictype=N,F,O (default=N)
  numericsize=value (default=18)
  numericprecision=value (default=8)
  scalartable=yes/no  (combine scalars in one table, default=yes)
  scalarparameter=value (name of scalartable, default="scalarparameter.dbf")
  scalarvariable=value  (name of scalartable, default="scalarvariable.dbf")
  scalarequation=value  (name of scalartable, default="scalarequation.dbf")

d:\gdx2dbf>

Notes:

• The tool generates a file for each symbol (set, parameter, variable, equation) in the GDX file.
• It can export large amounts of data quickly but it does not have a pretty GUI.
• By default scalar parameters, variables, equations are collected into one table. Otherwise a scalar would generate a file with a single number.
• By default we write numeric data as 'N' data. This is actually a string representation using numericsize and numericprecision. If you want to keep full precision you can store numbers as 8 byte floating point numbers (binary representation) using numerictype=O but this needs tableversion=7.
• To view a DBF file Excel can be used: in Excel 2003: Data|Import External Data|Import Data.
• Don't know if 'F' has an advantage over 'N'.
• I have seen funny things when trying to move large numbers like 1e100 into a small numeric 'N' field. I probably should generate an exception when it does not fit.
• By default the .DBF files are written in the current directory. If you want to write to a different directory you can use the d=xxx parameter.

Detailed description of the options

Command line options

• Name of the GDX file to convert (required)
• Name of the ref file to add domain names (optional). Ref files can be generated using the rf=xxx flag on the GAMS call.
• Name of ini file to read (optional). The format for this flag is @inifilename.
• Examples of the d=xxx parameter:
  • $call '=gdx2dbf trnsport.gdx d=c:\tmp\'
  • $call '=gdx2dbf indus89.gdx indus89.ref d="t m p"'

Ini file settings

• inf can be used to map GAMS INF values to a number in the DBF file. A large positive value that fits in the field width would be appropriate. The default is 9999999.9999.
• mininf can be used to map GAMS -INF values to a number in the DBF file. A large negative value that fits in the field width would be appropriate. The default is -9999999.9999.
• eps can be used to map GAMS EPS values to a number in the DBF file. Often the default of 0 is acceptable.
• na can be used to map GAMS NA values to a number in the DBF file. Often the default of 0 is acceptable. NA's are not often found in GAMS data.
• undf can be used to map GAMS UNDF values to a number in the DBF file. Often the default of 0 is acceptable. UNDF's are not often found in GAMS data.
• tablelevel indicates the DBF version to be used
  • tablelevel=3 (default) generates DBASE III files
  • tablelevel=4 generates DBASE IV files
  • tablelevel=7 generates DBASE VII files. These files can export numeric data in full precision using numerictype=O
  • tablelevel=25 generates FoxPro files
• numerictype indicates the type to be used for numeric values
  • numerictype=N (default) is the standard dbase numeric type. From datatypes we conclude with this type we actually should require numericwidth<18.
  • numerictype=F. This should have numericwidth=20 (from dbase IV).
  • numerictype=O. This will store a numeric value in binary. It has length of 8 bytes.
• numericsize gives the field width for numeric data.
• numericprecision
• scalartable indicates whether scalar data should be collected in separate tables or that they should be written as usual. In that case tables with a single row will be generated.
• scalarparameter is only used if scalartable=yes. In that case it contains the name of the table that will contain all scalar parameters.
• scalarvariable is only used if scalartable=yes. In that case it contains the name of the table that will contain all scalar variables.
• scalarequation is only used if scalartable=yes. In that case it contains the name of the table that will contain all scalar equations.

Test GAMS model

* no params gives help
$call '=gdx2dbf';

* small gdx file
$call '=gamslib trnsport';
$call '=gams trnsport lo=2 gdx=trnsport';
$call '=gdx2dbf trnsport.gdx';

* large gdx + ref file
$call '=gamslib indus89';
$call '=gams indus89 rf=indus89 lo=2 gdx=indus89';
$call '=gdx2dbf indus89.gdx indus89.ref';

This model should show something like:

--- Job testgdx2dbf.gms Start 08/24/07 15:44:30
GAMS Rev 148  Copyright (C) 1987-2007 GAMS Development. All rights reserved
Licensee: Erwin Kalvelagen                               G070509/0001CE-WIN
          GAMS Development Corporation                               DC4572
--- Starting compilation
--- testgdx2dbf.gms(3) 2 Mb
--- call =gdx2dbf
GDX2DBF Version 1.0
Usage:
   > GDX2DBF xxx.gdx [xxx.ref] [@inifile]

Dumps a GDX file to .DBF files.

Ini file format (default: GDX2DBF.INI):
  [settings]  (section name is required)
  inf=value for +INF (default 9999999.9999)
  mininf=value for -INF (default -9999999.9999)
  eps=value for EPS (default 0)
  na=value for NA (default 0)
  undf=value for UNDF (default 0)
  tablelevel=n (3=dbaseIII+,4=dbaseIV,7=dbaseVII,25=foxpro, default=3)
  numerictype=N,F,O (default=N)
  numericsize=value (default=20)
  numericprecision=value (default=8)
  scalartable=yes/no  (combine scalars in one table, default=yes)
  scalarparameter=value (name of scalartable, default="scalarparameter.dbf")
  scalarvariable=value  (name of scalartable, default="scalarvariable.dbf")
  scalarequation=value  (name of scalartable, default="scalarequation.dbf")
--- testgdx2dbf.gms(6) 2 Mb
--- call =gamslib trnsport
Model trnsport.gms retrieved
--- testgdx2dbf.gms(7) 2 Mb
--- call =gams trnsport lo=2 gdx=trnsport
--- testgdx2dbf.gms(8) 2 Mb
--- call =gdx2dbf trnsport.gdx
GDX2DBF Version 1.0
           i.   Write i.dbf                0.00 seconds
           j.   Write j.dbf                0.00 seconds
           a.   Write a.dbf                0.00 seconds
           b.   Write b.dbf                0.00 seconds
           d.   Write d.dbf                0.00 seconds
           f. Added to table ScalarParameter
           c.   Write c.dbf                0.00 seconds
           x.   Write x.dbf                0.00 seconds
           z. Added to table ScalarVariable
        cost. Added to table ScalarEquation
      supply.   Write supply.dbf           0.00 seconds
      demand.   Write demand.dbf           0.00 seconds
Total elapsed time: 0.02 seconds
--- testgdx2dbf.gms(11) 2 Mb
--- call =gamslib indus89
Model indus89.gms retrieved
--- testgdx2dbf.gms(12) 2 Mb
--- call =gams indus89 rf=indus89 lo=2 gdx=indus89
--- testgdx2dbf.gms(13) 2 Mb
--- call =gdx2dbf indus89.gdx indus89.ref
GDX2DBF Version 1.0
Reading 281 symbols, sorting: 0.00 seconds
Reading indus89.ref: 0.00 seconds
           z.   Write z.dbf                0.00 seconds
          pv.   Write pv.dbf               0.00 seconds
         pv1.   Write pv1.dbf              0.00 seconds
         pv2.   Write pv2.dbf              0.00 seconds
         pvz.   Write pvz.dbf              0.00 seconds
          cq.   Write cq.dbf               0.00 seconds
          cc.   Write cc.dbf               0.00 seconds
           c.   Write c.dbf                0.00 seconds
          cf.   Write cf.dbf               0.00 seconds
         cnf.   Write cnf.dbf              0.00 seconds
           t.   Write t.dbf                0.00 seconds
           s.   Write s.dbf                0.00 seconds
           w.   Write w.dbf                0.00 seconds
           g.   Write g.dbf                0.00 seconds
          gf.   Write gf.dbf               0.00 seconds
          gs.   Write gs.dbf               0.00 seconds
          t1.   Write t1.dbf               0.00 seconds
          r1.   Write r1.dbf               0.00 seconds
          dc.   Write dc.dbf               0.00 seconds
          sa.   Write sa.dbf               0.00 seconds
         wce.   Write wce.dbf              0.00 seconds
          m1.   Write m1.dbf               0.00 seconds
           m.   Write m.dbf                0.00 seconds
        wcem.   Write wcem.dbf             0.00 seconds
         sea.   Write sea.dbf              0.00 seconds
        seam.   Write seam.dbf             0.00 seconds
        sea1.   Write sea1.dbf             0.00 seconds
       sea1m.   Write sea1m.dbf            0.00 seconds
          ci.   Write ci.dbf               0.00 seconds
          p2.   Write p2.dbf               0.00 seconds
           a.   Write a.dbf                0.00 seconds
          ai.   Write ai.dbf               0.00 seconds
           q.   Write q.dbf                0.00 seconds
          nt.   Write nt.dbf               0.00 seconds
          is.   Write is.dbf               0.00 seconds
          ps.   Write ps.dbf               0.00 seconds
         isr.   Write isr.dbf              0.00 seconds
    baseyear. Added to table ScalarParameter
        land.   Write land.dbf             0.16 seconds
        tech.   Write tech.dbf             0.02 seconds
     bullock.   Write bullock.dbf          0.06 seconds
       labor.   Write labor.dbf            0.14 seconds
       water.   Write water.dbf            0.11 seconds
     tractor.   Write tractor.dbf          0.05 seconds
       sylds.   Write sylds.dbf            0.03 seconds
        fert.   Write fert.dbf             0.02 seconds
      fertgr.   Write fertgr.dbf           0.00 seconds
    natyield.   Write natyield.dbf         0.00 seconds
     yldprpv.   Write yldprpv.dbf          0.00 seconds
     yldprzs.   Write yldprzs.dbf          0.02 seconds
     yldprzo.   Write yldprzo.dbf          0.00 seconds
    growthcy.   Write growthcy.dbf         0.00 seconds
       weedy.   Write weedy.dbf            0.00 seconds
        graz.   Write graz.dbf             0.00 seconds
       yield.   Write yield.dbf            0.03 seconds
   growthcyf.   Write growthcyf.dbf        0.00 seconds
      iolive.   Write iolive.dbf           0.02 seconds
       sconv.   Write sconv.dbf            0.00 seconds
       repco. Added to table ScalarParameter
          gr. Added to table ScalarParameter
     growthq. Added to table ScalarParameter
          bp.   Write bp.dbf               0.00 seconds
         cnl.   Write cnl.dbf              0.00 seconds
       pvcnl.   Write pvcnl.dbf            0.00 seconds
        gwfg.   Write gwfg.dbf             0.02 seconds
      comdef.   Write comdef.dbf           0.03 seconds
      subdef.   Write subdef.dbf           0.00 seconds
         zsa.   Write zsa.dbf              0.00 seconds
         gwf.   Write gwf.dbf              0.00 seconds
       carea.   Write carea.dbf            0.02 seconds
        evap.   Write evap.dbf             0.02 seconds
        rain.   Write rain.dbf             0.03 seconds
     divpost.   Write divpost.dbf          0.02 seconds
         gwt.   Write gwt.dbf              0.02 seconds
        dep1.   Write dep1.dbf             0.02 seconds
        dep2.   Write dep2.dbf             0.00 seconds
       depth.   Write depth.dbf            0.02 seconds
         efr.   Write efr.dbf              0.03 seconds
      eqevap.   Write eqevap.dbf           0.02 seconds
      subirr.   Write subirr.dbf           0.03 seconds
   subirrfac.   Write subirrfac.dbf        0.00 seconds
         drc. Added to table ScalarParameter
        the1. Added to table ScalarParameter
           n.   Write n.dbf                0.00 seconds
           i.   Write i.dbf                0.00 seconds
          nc.   Write nc.dbf               0.00 seconds
          n1.   Write n1.dbf               0.00 seconds
          nn.   Write nn.dbf               0.02 seconds
          ni.   Write ni.dbf               0.00 seconds
          nb.   Write nb.dbf               0.00 seconds
        ncap.   Write ncap.dbf             0.00 seconds
       lloss.   Write lloss.dbf            0.00 seconds
       lceff.   Write lceff.dbf            0.00 seconds
          cd.   Write cd.dbf               0.00 seconds
     rivercd.   Write rivercd.dbf          0.00 seconds
      riverb.   Write riverb.dbf           0.05 seconds
         s58.   Write s58.dbf              0.00 seconds
    infl5080.   Write infl5080.dbf         0.02 seconds
         tri.   Write tri.dbf              0.02 seconds
      inflow.   Write inflow.dbf           0.00 seconds
        trib.   Write trib.dbf             0.00 seconds
       rrcap.   Write rrcap.dbf            0.00 seconds
      rulelo.   Write rulelo.dbf           0.00 seconds
      ruleup.   Write ruleup.dbf           0.00 seconds
      revapl.   Write revapl.dbf           0.00 seconds
         pow.   Write pow.dbf              0.00 seconds
          pn.   Write pn.dbf               0.00 seconds
           v.   Write v.dbf                0.00 seconds
   powerchar.   Write powerchar.dbf        0.02 seconds
        rcap.   Write rcap.dbf             0.00 seconds
        rep7.   Write rep7.dbf             0.00 seconds
        rep8.   Write rep8.dbf             0.00 seconds
          p3.   Write p3.dbf               0.00 seconds
      prices.   Write prices.dbf           0.00 seconds
   finsdwtpr.   Write finsdwtpr.dbf        0.02 seconds
   ecnsdwtpr.   Write ecnsdwtpr.dbf        0.00 seconds
          p1.   Write p1.dbf               0.00 seconds
         p11.   Write p11.dbf              0.00 seconds
        pri1.   Write pri1.dbf             0.00 seconds
      wageps.   Write wageps.dbf           0.00 seconds
        lstd. Added to table ScalarParameter
       trcap. Added to table ScalarParameter
       twcap. Added to table ScalarParameter
     ntwucap. Added to table ScalarParameter
      twefac. Added to table ScalarParameter
      labfac. Added to table ScalarParameter
      twutil.   Write twutil.dbf           0.00 seconds
     totprod.   Write totprod.dbf          0.00 seconds
    farmcons.   Write farmcons.dbf         0.00 seconds
      demand.   Write demand.dbf           0.00 seconds
        cowf. Added to table ScalarParameter
        buff. Added to table ScalarParameter
       elast.   Write elast.dbf            0.02 seconds
    growthrd.   Write growthrd.dbf         0.00 seconds
   consratio.   Write consratio.dbf        0.00 seconds
      natexp.   Write natexp.dbf           0.00 seconds
    explimit.   Write explimit.dbf         0.00 seconds
  explimitgr. Added to table ScalarParameter
       exppv.   Write exppv.dbf            0.00 seconds
       expzo.   Write expzo.dbf            0.00 seconds
         sr1.   Write sr1.dbf              0.00 seconds
          g1.   Write g1.dbf               0.00 seconds
         zwt.   Write zwt.dbf              0.00 seconds
     eqevapz.   Write eqevapz.dbf          0.00 seconds
     subirrz.   Write subirrz.dbf          0.00 seconds
        efrz.   Write efrz.dbf             0.02 seconds
    resource.   Write resource.dbf         0.02 seconds
       cneff.   Write cneff.dbf            0.00 seconds
       wceff.   Write wceff.dbf            0.02 seconds
       tweff.   Write tweff.dbf            0.03 seconds
      cneffz.   Write cneffz.dbf           0.00 seconds
      tweffz.   Write tweffz.dbf           0.00 seconds
      wceffz.   Write wceffz.dbf           0.02 seconds
      fleffz.   Write fleffz.dbf           0.00 seconds
     canalwz.   Write canalwz.dbf          0.00 seconds
   canalwrtz.   Write canalwrtz.dbf        0.02 seconds
       gwtsa.   Write gwtsa.dbf            0.02 seconds
        gwt1.   Write gwt1.dbf             0.00 seconds
     ratiofs.   Write ratiofs.dbf          0.00 seconds
         ftt.   Write ftt.dbf              0.00 seconds
       res88.   Write res88.dbf            0.02 seconds
    croparea.   Write croparea.dbf         0.00 seconds
   growthres.   Write growthres.dbf        0.00 seconds
    orcharea.   Write orcharea.dbf         0.00 seconds
  orchgrowth.   Write orchgrowth.dbf       0.00 seconds
   scmillcap.   Write scmillcap.dbf        0.00 seconds
        cnl1.   Write cnl1.dbf             0.02 seconds
       postt.   Write postt.dbf            0.00 seconds
     protarb.   Write protarb.dbf          0.00 seconds
         psr.   Write psr.dbf              0.00 seconds
        psr1.   Write psr1.dbf             0.00 seconds
          z1.   Write z1.dbf               0.00 seconds
          cn.   Write cn.dbf               0.00 seconds
         ccn.   Write ccn.dbf              0.02 seconds
          qn.   Write qn.dbf               0.00 seconds
         ncn.   Write ncn.dbf              0.00 seconds
          ce.   Write ce.dbf               0.00 seconds
          cm.   Write cm.dbf               0.00 seconds
          ex.   Write ex.dbf               0.00 seconds
       techc.   Write techc.dbf            0.00 seconds
         tec.   Write tec.dbf              0.00 seconds
         big. Added to table ScalarParameter
       pawat. Added to table ScalarParameter
       pafod. Added to table ScalarParameter
     divnwfp.   Write divnwfp.dbf          0.00 seconds
        rval.   Write rval.dbf             0.00 seconds
      fsalep.   Write fsalep.dbf           0.00 seconds
          pp. Added to table ScalarParameter
        misc.   Write misc.dbf             0.00 seconds
       seedp.   Write seedp.dbf            0.00 seconds
        wage.   Write wage.dbf             0.00 seconds
      miscct.   Write miscct.dbf           0.02 seconds
      esalep.   Write esalep.dbf           0.00 seconds
         epp. Added to table ScalarParameter
       emisc.   Write emisc.dbf            0.00 seconds
      eseedp.   Write eseedp.dbf           0.00 seconds
       ewage.   Write ewage.dbf            0.00 seconds
     emiscct.   Write emiscct.dbf          0.00 seconds
     importp.   Write importp.dbf          0.00 seconds
     exportp.   Write exportp.dbf          0.00 seconds
         wnr.   Write wnr.dbf              0.11 seconds
      tolcnl. Added to table ScalarParameter
       tolpr. Added to table ScalarParameter
     tolnwfp. Added to table ScalarParameter
        beta.   Write beta.dbf             0.02 seconds
       alpha.   Write alpha.dbf            0.00 seconds
       betaf. Added to table ScalarParameter
           p.   Write p.dbf                0.00 seconds
        pmax.   Write pmax.dbf             0.02 seconds
        pmin.   Write pmin.dbf             0.00 seconds
        qmax.   Write qmax.dbf             0.00 seconds
        qmin.   Write qmin.dbf             0.02 seconds
        incr.   Write incr.dbf             0.00 seconds
          ws.   Write ws.dbf               0.08 seconds
          rs.   Write rs.dbf               0.08 seconds
          qs.   Write qs.dbf               0.08 seconds
       endpr.   Write endpr.dbf            0.09 seconds
         cps. Added to table ScalarVariable
       acost.   Write acost.dbf            0.00 seconds
         ppc.   Write ppc.dbf              0.00 seconds
           x.   Write x.dbf                0.06 seconds
      animal.   Write animal.dbf           0.00 seconds
       prodt.   Write prodt.dbf            0.02 seconds
       proda.   Write proda.dbf            0.00 seconds
      import.   Write import.dbf           0.00 seconds
      export.   Write export.dbf           0.00 seconds
     consump.   Write consump.dbf          0.02 seconds
     familyl.   Write familyl.dbf          0.02 seconds
      hiredl.   Write hiredl.dbf           0.00 seconds
         itw.   Write itw.dbf              0.00 seconds
          tw.   Write tw.dbf               0.02 seconds
         itr.   Write itr.dbf              0.00 seconds
          ts.   Write ts.dbf               0.00 seconds
           f.   Write f.dbf                0.03 seconds
       rcont.   Write rcont.dbf            0.03 seconds
    canaldiv.   Write canaldiv.dbf         0.02 seconds
   cnldivsea.   Write cnldivsea.dbf        0.02 seconds
       prsea.   Write prsea.dbf            0.00 seconds
    tcdivsea.   Write tcdivsea.dbf         0.00 seconds
      wdivrz.   Write wdivrz.dbf           0.00 seconds
     slkland.   Write slkland.dbf          0.00 seconds
    slkwater.   Write slkwater.dbf         0.02 seconds
      artfod.   Write artfod.dbf           0.00 seconds
    artwater.   Write artwater.dbf         0.00 seconds
  artwaternd.   Write artwaternd.dbf       0.02 seconds
         nat.   Write nat.dbf              0.09 seconds
        natn. No data.
        objz. Added to table ScalarEquation
       objzn. Added to table ScalarEquation
        objn. Added to table ScalarEquation
       objnn. Added to table ScalarEquation
        cost.   Write cost.dbf             0.00 seconds
        conv.   Write conv.dbf             0.02 seconds
      demnat.   Write demnat.dbf           0.00 seconds
     demnatn. No data.
     ccombal.   Write ccombal.dbf          0.02 seconds
     qcombal.   Write qcombal.dbf          0.00 seconds
     consbal.   Write consbal.dbf          0.02 seconds
      laborc.   Write laborc.dbf           0.00 seconds
      fodder.   Write fodder.dbf           0.00 seconds
     protein.   Write protein.dbf          0.00 seconds
      grnfdr.   Write grnfdr.dbf           0.00 seconds
      bdraft.   Write bdraft.dbf           0.00 seconds
      brepco.   Write brepco.dbf           0.00 seconds
    bullockc. No data.
      tdraft.   Write tdraft.dbf           0.00 seconds
      trcapc.   Write trcapc.dbf           0.02 seconds
      twcapc.   Write twcapc.dbf           0.00 seconds
       landc.   Write landc.dbf            0.02 seconds
   orchareac.   Write orchareac.dbf        0.00 seconds
     scmillc. No data.
   waterbaln.   Write waterbaln.dbf        0.00 seconds
     watalcz.   Write watalcz.dbf          0.02 seconds
     subirrc.   Write subirrc.dbf          0.00 seconds
        nbal.   Write nbal.dbf             0.03 seconds
   watalcsea.   Write watalcsea.dbf        0.00 seconds
      divsea.   Write divsea.dbf           0.00 seconds
   divcnlsea.   Write divcnlsea.dbf        0.02 seconds
   watalcpro.   Write watalcpro.dbf        0.00 seconds
      prseaw.   Write prseaw.dbf           0.00 seconds
     nwfpalc.   Write nwfpalc.dbf          0.00 seconds
Total elapsed time: 2.75 seconds
--- testgdx2dbf.gms(13) 2 Mb
--- Starting execution - empty program
*** Status: Normal completion
--- Job testgdx2dbf.gms Stop 08/24/07 15:44:34 elapsed 0:00:04.282

A large single symbol can be generated as follows:

$ontext

 Test of GDX2DBF. Dumps a large symbol (a million elements)
 to a DBF file.

$offtext

set i /i1*i1000/;
alias (i,j);
parameter p(i,j);
p(i,j) = uniform(-100,100);
execute_unload 'test.gdx',p;

execute '=gdx2dbf test.gdx';

For this example, gdx2dbf is a little bit slower than we would like:

--- Job gdx2dbf.gms Start 08/26/07 22:45:18
GAMS Rev 148  Copyright (C) 1987-2007 GAMS Development. All rights reserved
Licensee: Erwin Kalvelagen                               G070509/0001CE-WIN
          GAMS Development Corporation                               DC4572
--- Starting compilation
--- gdx2dbf.gms(14) 3 Mb
--- Starting execution
--- gdx2dbf.gms(14) 28 Mb
GDX2DBF Version 1.0
           p.   Write p.dbf                41.78 seconds
Total elapsed time: 41.81 seconds
*** Status: Normal completion
--- Job gdx2dbf.gms Stop 08/26/07 22:46:00 elapsed 0:00:42.468

XLS Diff

There are ton of tools around to find differences in two excel spreadsheets. This one seems to work for me: http://www.florencesoft.com/.