Dependency versions are overly strict about eeg-expy

[JohnGriffiths]

Hi Erik.

1. The MNE dependencies could be relaxed by saying version '>=' rather than specific version. Please go ahead and submit a PR with such a change if that fixes your problem and doesn't break anything.

2. Re: dependency management - we are happy to take a look at this, but no guarantees, as we have limited bandwidth for dev activities and poetry is not (yet) standard, so won't necessarily make people's lives simpler. However, if you would like to, please do go ahead and create a branch and PR with alternative dependency management and we will take a look.

3. Yes, help with CI would be much appreciated.

Thanks!

[ErikBjare]

Got a bit carried away and did all of it.

I can understand your concern about (2), but pyproject.toml (standard as of PEP 518) makes it so that poetry is really only needed if you want to update dependency locks. The package can still be installed as usual with pip install .. I don't think it will make people's lives harder, but I'm obviously biased since I already work with poetry in my toolchain.

Please take a look, and don't be afraid to tell me if it's too much: #22

[JadinTredup]

Hi Erik,

I appreciate you taking the initiative. I saw your PR and will give it a test early this week and take a look at it. I agree with Jon's second point but if it doesn't require anything extra to install like you said then it might work out fine. In the mean time, you could always pip install --upgrade mne to upgrade the dependencies and I believe eegnb should function properly with the updated version.

[ErikBjare]

In the mean time, you could always pip install --upgrade mne to upgrade the dependencies and I believe eegnb should function properly with the updated version.

My toolchain already attempted something equivalent, but the catch is that since the setup.py specified a specific version (loaded from requirements.txt), Python will notice and complain about the mne dependency not meeting the version constraint when importing eegnb.

This is why poetry splits dependency versioning into constraints that are put in pyproject.toml (that must be respected, just like before with setup.py) and locks to poetry.lock (which may be ignored, as in the case where the package is installed with pip, but it is recommended to at least use them in CI and the like so that regression bugs can be detected in tests).

[JadinTredup]

Did you attempt the work around from #1 in Johns comment?

[ErikBjare]

@JadinTredup Not sure what work around you are referring to. I don't see one mentioned.

…

On Mon, 9 Nov 2020, 04:41 JadinTredup, ***@***.***> wrote: Did you attempt the work around from #1 <#1> in Johns comment? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#19 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKXDOUH7CMYNV23TR6HDDDSO5QHRANCNFSM4S7BFTXA> .

[JadinTredup]

Where he mentions adding ">=" to the requirements.txt file.

[JadinTredup]

@ErikBjare could you pleae share the output of Python complaining about not having the correct version of MNE when you run pip install --upgrade mne? I just tried manually upgrading MNE on my install and everything is working fine.

[ErikBjare]

Where he mentions adding ">=" to the requirements.txt file.

Oh, I thought you meant something mentioned in the issue #1 you linked.

That would obviously work, but it would mean I'd have to depend on a fork, which is what I'm trying to avoid.

---

could you pleae share the output of Python complaining about not having the correct version of MNE when you run pip install --upgrade mne? I just tried manually upgrading MNE on my install and everything is working fine.

Now that you updated master it works, but if running:

$ pip install git+https://github.com/NeuroTechX/eeg-notebooks@a9a53147f53ebf20d9d1743c1e0456973c09a8ee

Weirdly, I get:

$ eegnb --help
Traceback (most recent call last):
  File "/home/erb/tmp/venv-eegnb2/bin/eegnb", line 5, in <module>
    from eegnb.cli.__main__ import main
ModuleNotFoundError: No module named 'eegnb.cli'

Probably because the eegnb.cli module isn't detected by find_modules() in setup.py, since the module doesn't contain a __init__.py (it works fine in my #22 PR, poetry is just better at detecting modules).

When running pip install mne==0.21.1 it installs fine, but with the non-critical error ERROR: eeg-notebooks 0.2 has requirement mne==0.20.8, but you'll have mne 0.21.1 which is incompatible.

Turns out eegnb still imports fine and I can't reproduce with plain pip, but attempting to install it with poetry would still fail as it can't resolve the dependencies (and I assume the same would be the case for other tools that require a coherent dependency resolution). It's quite possible I've mixed up the behavior of the two.

Installing in the reverse order (mne==0.21.1 -> eegnb@a9a53147f53ebf20d9d1743c1e0456973c09a8ee) will also, naturally, force pip to reinstall the older version. So a simple pip install mne==0.21.1 isn't really a viable workaround since it'll quickly lead to dependency hell.

Example of output when trying to resolve dependencies with poetry:

$ poetry update
Updating dependencies
Resolving dependencies... (11.2s)
  SolverProblemError
  Because eeg-notebooks (0.2 git rev a9a53147f53ebf20d9d1743c1e0456973c09a8ee) depends on mne (0.20.8)
   and erb-thesis depends on mne (^0.21.0), eeg-notebooks is forbidden.
  So, because erb-thesis depends on eeg-notebooks (0.2 git rev a9a53147f53ebf20d9d1743c1e0456973c09a8ee), version solving failed.
  at ~/.local/lib/python3.8/site-packages/poetry/puzzle/solver.py:241 in _solve
      237│             packages = result.packages
      238│         except OverrideNeeded as e:
      239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      240│         except SolveFailure as e:
    → 241│             raise SolverProblemError(e)
      242│
      243│         results = dict(
      244│             depth_first_search(
      245│                 PackageNode(self._package, packages), aggregate_package_nodes

[ErikBjare]

Closing, fixed in 61447c0.