abess-team / skscope Goto Github PK

In my opinion, it is unnecessary to present the (very) very long table of contents (TOC) element on the first page. No body will be interested in it.

Currently, we cannot install skscope use in python 3.8 via pip? Can we support it on Python 3.8 on at least some platform such as MacOS or Linux?

It seems that the primal-dual active set (PDAS) algorithm is also a (potentially) powerful algorithm for sparsity optimization. It would helpfully enrich our support solvers if it is implemented.

Reference:

• Wen, C., Zhang, A., Quan, S., & Wang, X. (2020). BeSS: An R Package for Best Subset Selection in Linear, Logistic and Cox Proportional Hazards Models. Journal of Statistical Software, 94(4), 1–24. https://doi.org/10.18637/jss.v094.i04

Add some badges in the README.md file (see below). Through them, users will convince our library are well-organized.

[![Python Build](https://github.com/abess-team/abess/actions/workflows/python_test.yml/badge.svg)](https://github.com/abess-team/abess/actions/workflows/python_test.yml)
[![R Build](https://github.com/abess-team/abess/actions/workflows/r_test.yml/badge.svg)](https://github.com/abess-team/abess/actions/workflows/r_test.yml)
[![codecov](https://codecov.io/gh/abess-team/abess/branch/master/graph/badge.svg?token=LK56LHXV00)](https://codecov.io/gh/abess-team/abess)
[![docs](https://readthedocs.org/projects/abess/badge/?version=latest)](https://abess.readthedocs.io/en/latest/?badge=latest)
[![R docs](https://github.com/abess-team/abess/actions/workflows/r_website.yml/badge.svg)](https://abess-team.github.io/abess/)
[![cran](https://img.shields.io/cran/v/abess?logo=R)](https://cran.r-project.org/package=abess)
[![pypi](https://img.shields.io/pypi/v/abess?logo=Pypi)](https://pypi.org/project/abess)
[![Conda version](https://img.shields.io/conda/vn/conda-forge/abess.svg?logo=condaforge)](https://anaconda.org/conda-forge/abess)
[![pyversions](https://img.shields.io/pypi/pyversions/abess)](https://img.shields.io/pypi/pyversions/abess)
[![License](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](http://www.gnu.org/licenses/gpl-3.0)
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/3f6e60a3a3e44699a033159633981b76)](https://www.codacy.com/gh/abess-team/abess/dashboard?utm_source=github.com&utm_medium=referral&utm_content=abess-team/abess&utm_campaign=Badge_Grade)
[![CodeFactor](https://www.codefactor.io/repository/github/abess-team/abess/badge)](https://www.codefactor.io/repository/github/abess-team/abess)
[![Platform](https://anaconda.org/conda-forge/abess/badges/platforms.svg)](https://anaconda.org/conda-forge/abess)
[![Downloads](https://pepy.tech/badge/abess)](https://pepy.tech/project/abess)

@star-dust-ctrl , skscope is open-source now, and thus, readthedocs becomes available to us. Please configure related settings, thx!

At .../docs,
python -m pip install -r requirements.txt
make html

But there is an error:
ImportError: cannot import name 'environmentfilter' from 'jinja2'

It would be interesting to use skscope to address the clustering problem.

Reference:

• Lashkari, D., & Golland, P. (2007). Convex clustering with exemplar-based models. Advances in neural information processing systems, 20.

• Mahdi Soltanolkotabi, Ehsan Elhamifar, Emmanuel J. Candès "Robust subspace clustering," The Annals of Statistics, Ann. Statist. 42(2), 669-699, (April 2014)

In our examples, we often utilize the re-parametrization trick to absorb the equality constraint into the objective function. I think it would be more convenient if we can wrap common re-parametrizations into functions/classes. The reason is three-fold:

• we can avoid re-programming these tricks again and again;

• user can easily use re-parametrizations without programming by themselves;

• we can improve the maintenance for these re-parametrizations.

Can our website cooperates with two following useful tool for conducting the examples in gallery:

• binder
• JupyterLite

Just like the website for scikit-learn
https://scikit-learn.org/stable/auto_examples/miscellaneous/plot_kernel_approximation.html#sphx-glr-auto-examples-miscellaneous-plot-kernel-approximation-py

I notice that the docs of the function convex_solver_nlopt incorrectly appear in docs/build/html/autoapi/base_solver.html and docs/build/html/autoapi/solver.html#classes.

Additionally, the docs of BaseSolver appears in docs/build/html/autoapi/solver.html.

The notebook uses a Latex environment, which is currently not directly supported by Sphinx.

So, shall we avoid using this environment? Or, there are some convenience configuration/plugin/library can support this environment? If we consider the latter one, we have to investigate whether it is supported by the readthedocs.

Related literature about Fast Iterative Shrinkage-Thresholding Algorithm (FISTA): https://www.ceremade.dauphine.fr/~carlier/FISTA

1. Replace seed with random_state? The latter is a convention parameter in scikit-learn.
2. Replace hist with cov_matrix ? See this: https://pyportfolioopt.readthedocs.io/en/latest/_modules/pypfopt/efficient_frontier/efficient_frontier.html#EfficientFrontier.__init__
3. Replace k with s might be better because we use s in the other places when illustrating sparsity constraint optimization.
4. Replace lambda_ with lambda?
5. Have you tested whether it is really compatible with scikit-learn? For example, you may test whether sklearn.model_selection.GridSearchCV can be used for selecting lambda?

It would be interesting to implement sparse regression models listed here: vector generalized linear and additive models

It raises the following error when I import this library.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/zhujin/abess-team/scope/scope/__init__.py", line 16, in <module>
    from .universal import (ConvexSparseSolver)
  File "/Users/zhujin/abess-team/scope/scope/universal.py", line 4, in <module>
    from .pybind_cabess import pywrap_Universal, UniversalModel, init_spdlog
ImportError: dlopen(/Users/zhujin/abess-team/scope/scope/pybind_cabess.cpython-39-darwin.so, 0x0002): tried: '/Users/zhujin/abess-team/scope/scope/pybind_cabess.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/zhujin/abess-team/scope/scope/pybind_cabess.cpython-39-darwin.so' (no such file), '/Users/zhujin/abess-team/scope/scope/pybind_cabess.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

@bbayukari Since skscope is open-source now, we can submit this package to pypi. I also hope this submission can be automatic once we create a tag on github.

When I run the example sparse-precision-matrix.ipynb, the following code block

solver = GraspSolver(
    dimensionality=int(p * (p + 1) / 2),
    sparsity=np.count_nonzero(prec[np.triu_indices(p)]),
    always_select=np.where(np.triu_indices(p)[0] == np.triu_indices(p)[1])[0],
    convex_solver=convex_solver_cvxpy,
)

raises an error as follows

TypeError: __init__() got an unexpected keyword argument 'convex_solver'

It seems that the new version of scope is not compatible with the original parameter convex_solver.

@chenpnn I find that the file [utilities.py](https://app.codecov.io/gh/abess-team/skscope/commit/e7031f68f26eed1fd5301b2c511ca1936dbda380/blob/skscope/utilities.py) is not well covered by python testing.
@bbayukari Shall we our test also cover this utility file?

JAXopt is a differentiable optimizers which can solve general optimization problems such as set constrained or non-smooth optimization. I wonder whether it is possible to extend skscope to such a general sparse solver using JAXopt library.

It would be interesting to implement the objective function for solving the square-root-lasso type method.

Reference: A. Belloni and others, Square-root lasso: pivotal recovery of sparse signals via conic programming, Biometrika, Volume 98, Issue 4, December 2011, Pages 791–806, https://doi.org/10.1093/biomet/asr043

Support a new algorithm "Newton Hard-Thresholding Pursuit". See literature: https://spiral.imperial.ac.uk/bitstream/10044/1/89243/4/19-026.pdf

I try to minimize the least square error with simplex constraint via directly optimizing the normalized absolute parameters
w / jnp.sum(w) and the loss function is shown in the following:

def simplex_solver(X, y, sparsity=None):
    n, p = X.shape

    def custom_objective(w):
        w = jnp.abs(w) / jnp.abs(w).sum()
        loss = jnp.mean((X @ w - y) ** 2)
        return loss

    solver = ScopeSolver(p, sparsity=sparsity)
    w = solver.solve(custom_objective)
    return w / w.sum()

This fails because all nonzero elements in the output are nans, but no warning information raises. Then, I replace the line solver = ScopeSolver(p, sparsity=sparsity) with solver = ScopeSolver(p, sparsity=sparsity, init_params=np.ones(p)/p), it works. This phenomena may be due to the division-by-zero error in w = jnp.abs(w) / jnp.abs(w).sum() since the default initial params is a all-zeros vector.

Therefore, colud the ScopeSolver provide more intermediate information when such warning or error happens?
Thanks!

Conduct the command:

make html

but the following error occurs:

/bin/sh: sphinx-build: command not found
make: *** [html] Error 127

But, actually, sphinx has been installed.

Details about Environment:

• OS: macOS 13.2.1 (22D68)
• Chip: M1
• Python: 3.9.12

I click "edit on github"

but it results in a 404 page:

It seems that main/docs/userguide/index.rst is an incorrect path. It shall be main/docs/source/userguide/index.rst.

Robust variable selection is practically helpful for handling real data because it is insensitive to the outliers that frequently appear in the real world. It is strongly recommended to implement it.

Reference:

• Wang, X., Jiang, Y., Huang, M., & Zhang, H. (2013). Robust variable selection with exponential squared loss. Journal of the American Statistical Association, 108(502), 632-643.

@chenpnn The current implementation is here:
https://skscope.readthedocs.io/en/latest/userguide/examples/SurvivalModels/multivariate-failure-time-model.html, just wrap it.

The scikit-survival has provided an interface for censored-response, you may borrow related ideas from this library to design your API.