Reorganizing a large parameter in GAMS

To export some fairly substantial data in a certain format I wanted to form:

DownLoadData(downloadfiles,SP_Varname,SP_SubVarName,SP_SubSubVarName,SP_Indicator,                                 SP_Description,SP_Source,SP_Unit,cty,Fwdy) =       sum((mapdownloadfiles(downloadfiles,SP_control),SP_Map,SP_Graph,SP_Options,SP_Type,mapy(FwdY,SP_year)),           cube(SP_Control,                SP_VarName,                SP_SubVarName,                SP_SubSubVarName,                SP_Year,                SP_Indicator,                SP_Source,                SP_Description,                SP_Unit,                SP_Map,                SP_Graph,                SP_Options,                SP_Type,                cty                ) );

The original Cube parameter has 326,755 elements, and this operation basically reshuffles things into a lower-dimensional parameter with some mappings applied. Upon completion the new parameter Download_Data has the same number of elements: 326,755. Unfortunately this operation was very expensive, so much so that I never saw it finish.

The following code does exactly the same in just 0.5 seconds:

loop((mapdownloadfiles(downloadfiles,SP_control),mapy(Download_Year,SP_year)),    Download_Data(downloadfiles,SP_Varname,SP_SubVarName,SP_SubSubVarName,cty,SP_Indicator,                              SP_Description,SP_Source,SP_Unit,Download_Year) =         sum((SP_Map,SP_Graph,SP_Options,SP_Type),           cube(SP_Control,                SP_VarName,                SP_SubVarName,                SP_SubSubVarName,                SP_Year,                SP_Indicator,                SP_Source,                SP_Description,                SP_Unit,                SP_Map,                SP_Graph,                SP_Options,                SP_Type,                cty                ) ); );

GAMS has some built-in facilities to automatically rearrange assignments like these to achieve better performance, but in this case it was not very effective. So human intervention was needed to optimize this assignment.

Amazon not just selling books

Dear Amazon EC2 Customer, We are excited to announce the immediate availability of Cluster GPU Instances for Amazon EC2, a new instance type designed to deliver the power of GPU processing in the cloud. GPUs are increasingly being used to accelerate the performance of many general purpose computing problems. However, for many organizations, GPU processing has been out of reach due to the unique infrastructural challenges and high cost of the technology. Amazon Cluster GPU Instances remove this barrier by providing developers and businesses immediate access to the highly tuned compute performance of GPUs with no upfront investment or long-term commitment. Amazon Cluster GPU Instances provide 22 GB of memory, 33.5 EC2 Compute Units, and utilize the Amazon EC2 Cluster network, which provides high throughput and low latency for High Performance Computing (HPC) and data intensive applications. Each GPU instance features two NVIDIA Tesla® M2050 GPUs, delivering peak performance of more than one trillion double-precision FLOPS. Many workloads can be greatly accelerated by taking advantage of the parallel processing power of hundreds of cores in the new GPU instances. Many industries including oil and gas exploration, graphics rendering and engineering design are using GPU processors to improve the performance of their critical applications. Amazon Cluster GPU Instances extend the options for running HPC workloads in the AWS cloud. Cluster Compute Instances, launched earlier this year, provide the ability to create clusters of instances connected by a low latency, high throughput network. Cluster GPU Instances give customers with HPC workloads an additional option to further customize their high performance clusters in the cloud. For those customers who have applications that can benefit from the parallel computing power of GPUs, Amazon Cluster GPU Instances can often lead to even further efficiency gains over what can be achieved with traditional processors. By leveraging both instance types, HPC customers can tailor their compute cluster to best meet the performance needs of their workloads. For more information on HPC capabilities provided by Amazon EC2, visit aws.amazon.com/ec2/hpc-applications. Learn more about the new Cluster GPU instances for Amazon EC2 and their use in running HPC applications. Sincerely, The Amazon EC2 Team

Unfortunately I don’t have any direct use for this (it would be fun to try this out on some problems). It is interesting that Amazon is so active in the HPC arena.