Remove research experiments from pysindy about pysindy

[akaptano]

@Jacob-Stevens-Haas Thanks for the very thoughtful post -- I agree with essentially everything here. I wonder if there is a way to have a pysindy repository that is stable (we can strip out all the researchy-topics like the trapping functionality) and a pysindy repository that is for development by researchers. My worry with this is that if every person who makes a new change to some SINDy method creates a new fork/repo of PySINDy, it will be hard to keep PySINDy up-to-date with the latest and best variants. But you're right that we also don't want every person with a half-baked idea and okay coding skills (talking about myself here :)) making updates that break a bunch of other things in the code.

For a historical perspective, the state of the Kutz/Brunton lab before PySINDy was that every new grad student and post-doc wrote their own code full of bugs and conflicts with other codes, so it has been a real boon to getting these SINDy-based methods working without reinventing the wheel (and to me, the "wheel" nowadays should be using weak form SINDy with ensembling...).

On a related noted, I have been wanting to get a version of PySINDy implemented that uses the multi-step prediction error as the loss term, but this requires totally circumnavigating the whole sklearn pipeline, so I have been considering writing a repo based on PySINDy but with all the sklearn stuff stripped out (as well as some of the other "extras" that I think are unnecessary... like multiple trajectory concatenation or non-PDE libraries since the PDELibrary does everything now anyways). This is anyways what @KKahirman and Joe Bakarji had to do for their projects with the multi-step prediction error, but their codes are not compatible currently with PySINDy functionalities. I would really like to do multi-step prediction error with ensembling and weak form and all the other advances. Happy to discuss these topics more here if anyone has more thoughts on this!

[jcallaham]

+1 from me on the proposal. From my end, the main notebook I contributed was the cavity flow one. I'm not sure if a (stripped down) version of that would be good to keep as a "neato" fluid dynamics case (no fancy features or dependencies, just basic SINDy). Should I open a new ticket to discuss that?

In brief, that notebook is not really serving any of the three main purposes you highlighted, so I'd propose either

1. Get rid of it
2. I'll revise and significantly simplify it so it would be fast to run in CI and easy to maintain

[Jacob-Stevens-Haas]

Thanks for your comments guys. To clarify, @akaptano I do think the advanced research methods like Trapping SINDy have value in the repo... best case in point is that I only discovered its issues because it was an attractive method for me as a user in another project 😅 I think we can provide better stability by bumping versions more frequently and faithfully to SemVer. Personally, I'm wary of bumping and releasing a version because I feel uncomfortable with a version release that breaks notebooks, which is part of the reason for this post.

As for multi-step prediction error: I also would love to have that too, and currently feel awkward writing about the heuristic score of "how the discovered system looks". We could have an issue dedicated to that so we can link to use cases and people who have rolled their own implementations. I couldn't see what @KKahirman did, if it's in his public repo.

@jcallaham I think that's a great example in its full glory. It might have been confusing in the issue post, but I can handle a more robust version of (1) that still builds the notebook into the documentation, but as part of a separate repo that depends upon a pinned pysindy version. (2) has a simpler option:

Save it as examples/14_cavity_flow/example.ipynb, save it again as a python file examples/14_cavity_flow/example.py. Then, have a cell up front that determines input data and important constants, e.g. integration tolerances, integration times, data source.

if __name__ == "testing":
    data = {
        "t": np.arange(0, 1, .1),
        "x": np.random.normal(size=(<small array of correct ndim>)
    }
   t_sim = np.arange(0, .5, .1)
    ...
else:
    data = loadmat("data/cavityPOD.mat")
    t_sim = np.arange(0, 300, dt_rom)

The repo currently configures pytest to skip notebook scripts locally, but include them in CI.

[Ohjeah]

Hi, thanks for the pin.

I agree with @Jacob-Stevens-Haas: complex examples should be move to a different repository and each example should have a set of pinned dependencies.

[Jacob-Stevens-Haas]

An aside - for those who have multiple notebooks in a single directory, sphinx chooses a notebook to build into the documentation randomly (well, sphinx's sorting of filenames).

[briandesilva]

I'm very late to the party, but I also agree with the solution that @Jacob-Stevens-Haas has proposed.