Monday, April 29, 2013

Rewriting GAMS code

A first attempt to write some code turned out to be correct, but very slow:
cube('PPX0a.zip','Producer Price',cgroup,cname,y,indicator,
    
'','','-','','','',''
,cty) =
     
sum
((j2c(j,c),cgmap(cgroup,c),cnamemap(c,cname)),
          PPX0(j,cty,y));
After rewriting this fragment to include a loop much better performance was achieved:
loop((cgmap(cgroup,c),cnamemap(c,cname)),
   cube(
'PPX0b.zip','Producer Price'
,cgroup,cname,y,indicator,
       
'','','-','','','',''
,cty) =
        
sum
(j2c(j,c),
             PPX0(j,cty,y));
);
The timings for these assignments are:
----    289 Assignment cube        344.247   344.247 SECS    100 Mb  634708
----    294 Loop                     2.590   346.837 SECS    107 Mb
The second fragment is more than 100x as fast as the original code (from 344 seconds to 2.6 seconds).
In a sense modeling languages are still woefully inadequate and much too primitive. From the first fragment a system like GAMS should be able to understand the meaning of what I want to do and should be able to re-arrange computations to achieve near-optimal performance.
This is a similar to RDBMS systems where query optimizers try to find the most efficient way to execute an SQL query. Although that is more mature technology (not in the least because of larger budgets for larger development teams), even here we see that an experienced database user is able to write much more efficient SQL queries than the average user.