flo-compbio / xlmhg

XL-mHG: A Semiparametric Test for Enrichment in Ranked Lists.

XL-mHG

master
develop

This is an efficient Python/Cython implementation of the semiparametric XL-mHG test for enrichment in ranked lists. The XL-mHG test is an extension of the nonparametric mHG test, which was developed by Dr. Zohar Yakhini and colleagues.

Installation

$ pip install xlmhg

Getting started

The xlmhg package provides two functions (one simple and more more advanced) for performing XL-mHG tests. These functions are documented in the User Manual. Here's a quick example using the "simple" test function:

import xlmhg
stat, cutoff, pval = xlmhg.xlmhg_test(v, X, L)

Where: v is the ranked list of 0's and 1's, represented by a NumPy array of integers, X and L are the XL-mHG parameters, and the return values have the following meanings:

• stat: The XL-mHG test statistic
• cutoff: The cutoff at which XL-mHG test statistic was attained
• pval: The XL-mHG p-value

Documentation

Please refer to the XL-mHG User Manual (hosted on ReadTheDocs).

Citing XL-mHG

If you use the XL-mHG test in your research, please cite Eden et al. (PLoS Comput Biol, 2007) and Wagner (PLoS One, 2015).

Copyright and License

Copyright (c) 2015-2019 Florian Wagner

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

xlmhg's Issues

List length limit

Hi,
I see that the list length is restricted to 36536 but there is no information about this in the documentation. Why would you restrict the length of the list?

Thanks

A lot of tests fail on Windows

This is mostly due to hardcoded reference values, but may also include other problems.

mhg_cython in python 3.9 and above

Hello, thank you very much for developing xlmhg! I found a problem when using xlmhg in python above 3.8. The mhg_cython seems likely can be installed in python 3.9 but everything looks perfect in python 3.8.

Under python 3.9, I got warning:
Warning (xlmhg): Failed to import "mhg_cython" C extension.
Warning (xlmhg): Failed to import the "mhg_cython" C extension.Falling back to the pure Python implementation, which is very slow.

and I got error report if I run xlmhg testing:
File ~/miniconda3/envs/mebocost/lib/python3.9/site-packages/xlmhg/test.py:207, in get_xlmhg_test_result(N, indices, X, L, exact_pval, pval_thresh, escore_pval_thresh, table, use_alg1, tol)
201 raise ValueError('Supplied array for dynamic programming table not'
202 'large enough. It is: %d x %d, but must be at least '
203 '%d x %d ((K+1) x (W+1)).'
204 % (table.shape[0], table.shape[1], K+1, W+1))
206 ### Step 1: Calculate XL-mHG test statistic.
--> 207 stat, cutoff = mhg_cython.get_xlmhg_stat(indices, N, K, X, L, tol)
208 assert 0.0 <= stat <= 1.0
210 # check for special cases

TypeError: get_xlmhg_stat() takes from 3 to 4 positional arguments but 6 were given

Do you have any solution to this? Thank you very much!

Cython code fails to compile on Windows with Python 2.7 (Visual C++ 2008 / 9.0) due to `nanl`

The Visual C++ 2008 compiler does not know nanl (see here).

Failing to build against python-numpy 1.23

xlmhg source from PyPi is failing to build against numpy version 1.23 with following error in archlinux

creating build/temp.linux-x86_64-cpython-310
creating build/temp.linux-x86_64-cpython-310/xlmhg
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fPIC -I/usr/lib/python3.10/site-packages/numpy/core/include -I/usr/include/python3.10 -c xlmhg/mhg_cython.c -o build/temp.linux-x86_64-cpython-310/xlmhg/mhg_cython.o
In file included from /usr/lib/python3.10/site-packages/numpy/core/include/numpy/ndarraytypes.h:1948,
                 from /usr/lib/python3.10/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /usr/lib/python3.10/site-packages/numpy/core/include/numpy/arrayobject.h:5,
                 from xlmhg/mhg_cython.c:599:
/usr/lib/python3.10/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   17 | #warning "Using deprecated NumPy API, disable it with " \
      |  ^~~~~~~
xlmhg/mhg_cython.c: In function ‘__pyx_tp_dealloc_array’:
xlmhg/mhg_cython.c:21202:5: error: lvalue required as increment operand
21202 |     ++Py_REFCNT(o);
      |     ^~
xlmhg/mhg_cython.c:21204:5: error: lvalue required as decrement operand
21204 |     --Py_REFCNT(o);
      |     ^~
xlmhg/mhg_cython.c: In function ‘__pyx_tp_dealloc_memoryview’:
xlmhg/mhg_cython.c:21513:5: error: lvalue required as increment operand
21513 |     ++Py_REFCNT(o);
      |     ^~
xlmhg/mhg_cython.c:21515:5: error: lvalue required as decrement operand
21515 |     --Py_REFCNT(o);
      |     ^~
xlmhg/mhg_cython.c: In function ‘__pyx_tp_dealloc__memoryviewslice’:
xlmhg/mhg_cython.c:21763:5: error: lvalue required as increment operand
21763 |     ++Py_REFCNT(o);
      |     ^~
xlmhg/mhg_cython.c:21765:5: error: lvalue required as decrement operand
21765 |     --Py_REFCNT(o);
      |     ^~
xlmhg/mhg_cython.c: In function ‘__Pyx_ParseOptionalKeywords’:
xlmhg/mhg_cython.c:23285:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
23285 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
      |                     ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:83,
                 from xlmhg/mhg_cython.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
xlmhg/mhg_cython.c:23285:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
23285 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
      |                     ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
  580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
      |                                             ^~~~~~~~~~~~~~~~~~~
xlmhg/mhg_cython.c:23285:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
23285 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
      |                     ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
xlmhg/mhg_cython.c:23285:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
23285 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
      |                     ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
xlmhg/mhg_cython.c:23285:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
23285 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
      |                     ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
  580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
      |                                             ^~~~~~~~~~~~~~~~~~~
xlmhg/mhg_cython.c:23285:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
23285 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
      |                     ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
xlmhg/mhg_cython.c:23301:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
23301 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
      |                         ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
xlmhg/mhg_cython.c:23301:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
23301 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
      |                         ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
  580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
      |                                             ^~~~~~~~~~~~~~~~~~~
xlmhg/mhg_cython.c:23301:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
23301 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
      |                         ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
xlmhg/mhg_cython.c:23301:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
23301 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
      |                         ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
xlmhg/mhg_cython.c:23301:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
23301 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
      |                         ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
  580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
      |                                             ^~~~~~~~~~~~~~~~~~~
xlmhg/mhg_cython.c:23301:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
23301 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
      |                         ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
xlmhg/mhg_cython.c: In function ‘__Pyx_decode_c_string’:
xlmhg/mhg_cython.c:24439:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations]
24439 |         return PyUnicode_FromUnicode(NULL, 0);
      |         ^~~~~~
/usr/include/python3.10/cpython/unicodeobject.h:551:42: note: declared here
  551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
      |                                          ^~~~~~~~~~~~~~~~~~~~~
error: command '/usr/bin/gcc' failed with exit code 1

Add Python 3.6 Windows builds once corresponding Cython binaries become available

Failed to build xlmhg

Hi,

To use COMETSC, I'm trying to install xlmhg 2.4.9, but got the error.
Could you kindly suggest what to do?

Thanks

Add p-value to title of plot.

Hi, I am using your algorithm to check for enrichment (at the top of the list). I currently am saving the item as a PNG and the can't get the p-vlaue to be part of the title... In addition, I am working with the png your provide from the website, is there any way to save the file in a different format ?(tiff)
any ides what I can do?