Cython GLPK interface
Home Page: https://equaeghe.github.io/ecyglpki/
License: GNU General Public License v3.0
Cython GLPK interface
Development takes place in branches named
<GLPK library version current.revision>-<Cython version>
Check them out.
Releases correspond to tags with the version string
<major>.<minor>.<patch>
as a name; they are prepared in branches with the same name. The version string follows the semantic versioning conventions.
The glp_[print|read|write]_*-wrapping functions currently only read from and write to files. It would be useful to add functionality so that they can also read and write Python strings (and perhaps even work with Python file handles).
One approach would be to work with temporary files. (This may be the only reasonable approach.) It would be useful to have some second opinions about this.
Currently, terminal output can essentially not be influenced or changed yet from the Python side. It would be useful to let the Python side have more control. This would involve properly wrapping/integrating the functions glp_term_* and perhaps glp_*_tee.
The GLPK functions glp_printf and glp_vprintf ‘write formatted output on terminal’. These functions use va_list arguments, which seem nontrivial to wrap, cf. the corresponding Cython FAQ entry.
Currently, this issue will be deferred to the moment that Cython allows wrapping va_list arguments cleanly.
Add more examples of the code in action.
Possible source: adapt from https://gitorious.org/tutoriais-computacao-cientifica/glpk/source/9cd938d3e5ee00c3ec20ba6f68e3da6bad13238f:
When running the reproduce script below (which causes intopt to raise ValueError), traceback, frame (codename: ecyglpki.Problem.intopt (ecyglpki.c:23524)), frame (codename: _leaky) objects are leaked. (The memory leaked is actually far greater than what the output reports below, it probably does not include C allocated memory).
Examining the referrers tree, frame(_leaky) is reffered to by frame(intopt) and traceback(1). frame(intopt) is reffered to by traceback(2); and traceback(2) is referred to by traceback(1) forming a cycle. Though, Python's GC supposedly handles cycles. If you run the reproduce script with iterations set to 5, you should get a Tk GUI allowing you to browse the referrer tree to the very_leaky frame.
Output (from reproduce script below). First object listing is before running intopt (multiple times). Second is directly after it. Last listing is after running gc.collect:
types | # objects | total size
==================================== | =========== | ============
<class 'dict | 2937 | 2.30 MB
<class 'type | 864 | 858.94 KB
<class 'set | 542 | 185.56 KB
<class 'tuple | 2432 | 165.21 KB
<class 'weakref | 1632 | 127.50 KB
<class 'wrapper_descriptor | 1548 | 120.94 KB
<class 'list | 739 | 111.91 KB
<class 'method_descriptor | 1286 | 90.42 KB
<class 'builtin_function_or_method | 1173 | 82.48 KB
<class 'getset_descriptor | 1131 | 79.52 KB
<class 'collections.OrderedDict | 69 | 75.62 KB
<class 'abc.ABCMeta | 74 | 72.48 KB
function (__init__) | 435 | 57.77 KB
<class 'frozenset | 62 | 37.81 KB
<class 'member_descriptor | 432 | 30.38 KB
types | # objects | total size
============================================================== | =========== | ============
frame (codename: _leaky) | 50000 | 24.80 MB
frame (codename: ecyglpki.Problem.intopt (ecyglpki.c:23524)) | 50000 | 19.07 MB
<class 'traceback | 100000 | 6.10 MB
<class 'dict | 2856 | 2.21 MB
<class 'type | 864 | 858.94 KB
<class 'list | 5460 | 554.85 KB
<class 'set | 542 | 185.56 KB
<class 'tuple | 2348 | 155.85 KB
<class 'weakref | 1632 | 127.50 KB
<class 'wrapper_descriptor | 1548 | 120.94 KB
<class 'method_descriptor | 1286 | 90.42 KB
<class 'builtin_function_or_method | 1173 | 82.48 KB
<class 'getset_descriptor | 1131 | 79.52 KB
<class 'collections.OrderedDict | 69 | 75.62 KB
<class 'abc.ABCMeta | 74 | 72.48 KB
gc.collect
types | # objects | total size
============================================================== | =========== | ============
frame (codename: _leaky) | 50000 | 24.80 MB
frame (codename: ecyglpki.Problem.intopt (ecyglpki.c:23524)) | 50000 | 19.07 MB
<class 'traceback | 100000 | 6.10 MB
<class 'dict | 2856 | 2.21 MB
<class 'type | 864 | 858.94 KB
<class 'list | 5464 | 555.20 KB
<class 'set | 542 | 185.56 KB
<class 'tuple | 2348 | 155.85 KB
<class 'weakref | 1632 | 127.50 KB
<class 'wrapper_descriptor | 1548 | 120.94 KB
<class 'method_descriptor | 1286 | 90.42 KB
<class 'builtin_function_or_method | 1173 | 82.48 KB
<class 'getset_descriptor | 1131 | 79.52 KB
<class 'collections.OrderedDict | 69 | 75.62 KB
<class 'abc.ABCMeta | 74 | 72.48 KB
To reproduce: python script.py the script below:
import numpy as np
import ecyglpki
nan = np.nan
nutrition_target = np.array([[ 0.57142857, 1.42857143],
[ 0.16513761, 1.83486239],
[ 0.57142857, 1.42857143],
[ 0.30188679, 1.69811321],
[ 0.90909091, nan],
[ 0.34586466, 1.65413534],
[ 0.29787234, 1.70212766],
[ 0.24175824, 1.75824176],
[ 0.43137255, 1.56862745],
[ 0.90909091, nan],
[ 0.78947368, 1.21052632],
[ 0.90909091, nan],
[ nan, 2. ],
[ 0.0861244 , 1.9138756 ],
[ 0.26086957, 1.73913043],
[ 0.90909091, nan],
[ nan, 2. ],
[ 0.90909091, nan],
[ 0.90909091, nan],
[ 0.90909091, nan],
[ 0.90909091, nan],
[ 0.02566634, 1.97433366],
[ 0.57142857, 1.42857143],
[ nan, 2. ],
[ 0.90909091, nan],
[ 0.90909091, nan],
[ 0.27160494, 1.72839506],
[ 0.90909091, nan],
[ 0.90909091, nan],
[ 0.90909091, nan],
[ 0.90909091, nan],
[ 0.90909091, nan],
[ 0.98591549, 1.01408451],
[ nan, 2. ],
[ nan, 2. ],
[ nan, 2. ],
[ nan, 2. ],
[ 0.81818182, 1.18181818],
[ 0.72727273, 1.27272727],
[ 0.44444444, 1.55555556],
[ 0.66666667, 1.33333333],
[ 0.66666667, 1.33333333],
[ nan, 2. ],
[ nan, 2. ]])
def _leaky():
problem = ecyglpki.Problem()
problem.add_rows(len(nutrition_target))
problem.add_cols(20)
# Configure rows/nutrients
for i, extrema in enumerate(nutrition_target):
problem.set_row_bnds(i+1, *extrema)
# Solve
int_opt_options = ecyglpki.IntOptControls()
int_opt_options.presolve = True # without this, you have to provide an LP relaxation basis
int_opt_options.msg_lev = 'no' # be quiet, no stdout
try:
problem.intopt(int_opt_options)
except ValueError:
pass
iterations = 50000
def very_leaky():
for _ in range(iterations):
_leaky()
def leaky():
for _ in range(iterations):
problem = ecyglpki.Problem()
problem.add_rows(len(nutrition_target))
problem.add_cols(20)
# Configure rows/nutrients
for i, extrema in enumerate(nutrition_target):
problem.set_row_bnds(i+1, *extrema)
# Solve
int_opt_options = ecyglpki.IntOptControls()
int_opt_options.presolve = True # without this, you have to provide an LP relaxation basis
int_opt_options.msg_lev = 'no' # be quiet, no stdout
try:
problem.intopt(int_opt_options)
except ValueError:
pass
def normal(): # for comparison
x=5
while True:
return np.ones(x)
import gc
from pympler import summary, refbrowser
sum1 = summary.summarize(gc.get_objects())
summary.print_(sum1)
very_leaky() # leaky leaks 1 leaky frame, very_leaky leaks $iterations _leaky frames
sum1 = summary.summarize(gc.get_objects())
summary.print_(sum1)
print('gc.collect')
gc.collect()
sum1 = summary.summarize(gc.get_objects())
summary.print_(sum1)
print('showing referrers tree')
frame = next(obj for obj in gc.get_objects() if type(obj).__name__ == 'frame' and obj.f_code.co_name in ('leaky', 'very_leaky'))
refbrowser.InteractiveBrowser(frame).main() # can take forever if iterations is set higher than 500
input()My CPython version: 3.5.2
pip freeze (not minimal; you probably only need pympler, numpy and ecyglpki):
apipkg==1.4
attrs==16.3.0
click==6.7
colored-traceback==0.2.2
coverage==4.3.1
coverage-pth==0.0.1
decorator==4.0.10
ecyglpki==0.2.0
execnet==1.4.1
networkx==1.11
numpy==1.11.3
pandas==0.19.2
plumbum==1.6.3
py==1.4.32
Pygments==2.1.3
Pympler==0.4.3
pyprof2calltree==1.4.0
pytest==3.0.5
pytest-asyncio==0.5.0
pytest-capturelog==0.7
pytest-cov==2.4.0
pytest-env==0.6.0
pytest-localserver==0.3.6
pytest-mock==1.5.0
pytest-xdist==1.15.0
python-dateutil==2.6.0
pytz==2016.10
six==1.10.0
tabulate==0.7.7
Werkzeug==0.11.15
Hi
Thanks for a great project, I was wondering if anybody had tried building ecyglpki on windows.
Brett
The GLPK functions glp_error to ‘display fatal error message and terminate execution’ and glp_assert to ‘check for logical condition’ are not wrapped yet. They use va_list arguments or are defined using a macro. It is not clear how to wrap them using Cython.
Hi,
I'm having trouble installing the interface
I followed the instructions but I get the following error message
I'm under Ubuntu 14.04 (trusty) kernel 3.13.0-46-generic
and gcc 4.8 (i686-linux-gnu)
a suggestion?
building 'ecyglpki' extension
i686-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.4m -c ecyglpki.c -o build/temp.linux-i686-3.4/ecyglpki.o
ecyglpki.c: In function ‘PyInit_ecyglpki’:
ecyglpki.c:49016:37: error: ‘GLP_BF_LUF’ undeclared (first use in this function)
__pyx_t_2 = __Pyx_PyInt_From_int((GLP_BF_LUF + GLP_BF_FT)); if (unlikely(!__pyx_t_2)) {__pyx_filename = __pyx_f[4]; __pyx_lineno = 27; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
^
ecyglpki.c:49016:37: note: each undeclared identifier is reported only once for each function it appears in
ecyglpki.c:49052:37: error: ‘GLP_BF_BTF’ undeclared (first use in this function)
__pyx_t_2 = __Pyx_PyInt_From_int((GLP_BF_BTF + GLP_BF_BG)); if (unlikely(!__pyx_t_2)) {__pyx_filename = __pyx_f[4]; __pyx_lineno = 30; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
^
error: command 'i686-linux-gnu-gcc' failed with exit status 1
iocp (integer optimization control parameters) callback function assignment is not working with cython functions.
Traceback (most recent call last):
File "./solve", line 5, in <module>
main.solve( )
File "src/main.py", line 340, in src.main.solve (src/main.c:9014)
affectations = args.allocator( args , problem , affectations )
File "src/main.py", line 135, in src.main.dtcntr (src/main.c:4584)
datacenter.solve( D , N , R , P , lp , cb = cb )
File "src/datacenter.py", line 176, in src.datacenter.solve (src/datacenter.c:6889)
if cb : iocp.cb_func = cb
TypeError: Argument 'callback' has incorrect type (expected ecyglpki.CallbackFunction, got cython_function_or_method)
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.