Some GAMS embedded Python notes

Here are two issues you may want to be aware of. The discussion below is relevant for Windows and not so much for Unix variants.

Using Python Raw strings for directories

Here we use some Python inside an otherwise empty GAMS model:

$onEmbeddedCode Python: dir = r"%system.fp%" print(dir) $offEmbeddedCode

Here I make use of the GAMS predefined compile-time variable system.fp, which indicates the location of the main .gms file of this run. The string %system.fp% will be replaced by the literal value of the file path. In my case this is: c:\users\erwin\downloads\.  I use a so-called raw string because I don't want Python to view \ as an escape character. However, this gives a problem when I run this:

--- Initialize embedded library embpycclib64.dll --- Execute embedded library embpycclib64.dll   File "<string>", line 7     dir = r"C:\Users\erwin\Downloads\"                                      ^ SyntaxError: EOL while scanning string literal Python error! Return code from Python execution: -1*** Error executing embedded code section: *** Check log above

The problem is that raw strings in Python are not really raw strings. Python wants to check for escaped quotes in raw strings, so it does not want a single \ at the end of a string literal. To be more precise, it does not want to have an uneven number of \ at the end of a literal [1].

One possible fix is to add an extra \ to such paths:

dir = r"%system.fp%\"

This will generate a path: C:\Users\erwin\Downloads\\.  The double \\ is not a problem as Windows will interpret this the same as a single \. 

Note: there is a reason why other languages use a more complicated raw string:

• R: r"(C:\Users\erwin\Downloads\)" or even more complex versions in case your string literal contains a )". 
• C++:  R"(C:\Users\erwin\Downloads\)". This again uses )" to terminate the string. We can make this more complicated to make it more difficult for the literal to contain the terminator.

I guess that the Python approach for raw strings is just a bit too simplistic.  

GAMS System Directory

There are different ways to tell Windows where GAMS is located. The first is using a full path when invoking gams.exe. Alternatively, Windows can look along the PATH. But, there is also a registry setting that tells us where GAMS is installed. This can lead to some confusion when using embedded Python tools. Consider the following model:

$call "echo #### GAMS: system directory=[%gams.sysdir%]" $onEmbeddedCode Python: ws = GamsWorkspace() print(f"#### Embedded Python: system directory=[{ws.system_directory}]") $offEmbeddedCode

In theory, the two print statements should deliver the same system directory. However, there are a few scenarios where they are not.  Here is an example. When I run:

        C:\Users\erwin\Downloads>\gams\41\gams.exe testEmbedded.gms

I see:

#### GAMS: system directory=[c:\gams\41\]

#### Embedded Python: system directory=[C:\GAMS\42]

Obviously, this behavior is not what we want. GAMS itself uses a different GAMS version than the embedded Python code. This is likely a very dangerous situation: lots of possibilities of mismatched libraries. In this example, GAMS itself infers the system directory (sysdir) from how it was invoked, while the Python API uses the Windows registry settings. We can make things even worse. If we just use GAMS from a network drive without local installation, we may get:

Exception from Python (line 46): Could not find a GAMS installation, must manually specify system_directory

 

(It is a bit counter-intuitive that while GAMS is running, the Python API can't figure out the correct location of this GAMS system). 

One way to make these problems go away is to use:

ws = GamsWorkspace(system_directory=r"%gams.sysdir%\")

Note again the extra \ (see the discussion in the previous section). Now, I see:

#### GAMS: system directory=[c:\gams\41\]
#### Embedded Python: system directory=[C:\GAMS\41]

 

The system directory is now the same. Except for a small inconsistency in the trailing backslash (I think they should preferably be 100% the same). 

Besides GamsWorkspace, this may affect a few other functions and constructors. I saw these problems occurring when using c = gt.Container() from the gamstransfer package. Again the fix was to use: c = gt.Container(system_directory=r"%gams.sysdir%\"). 

Conclusion

A few issues cropped up when doing some work using embedded Python in GAMS:

• Trailing backslashes in directory names can be a problem when used in Python scripts.
• Some parts of the GAMS Python API have problems finding the correct system directory.  

Some workarounds are presented here.

References

1. Python 3 documentation, Lexical Analysis, String and Bytes literals, https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals