Setting ads_token fails after pip install about bibmanager

[pcubillos]

The problem must be in the installation, since you have a conda environment. pip installations don't always work there. Could you try installing the other way:

git clone https://github.com/pcubillos/bibmanager/
cd bibmanager
python setup.py install

[michaelaye]

Sorry, but I'm using conda and pip together for many years and I have NEVER had an issue of a package not working simply for being inside conda. Please provide me with another known example where something does not work inside conda?

[michaelaye]

there is no git tag for v1.1.5?

[michaelaye]

I installed via:

git clone https://github.com/pcubillos/bibmanager
cd bibmanager
git co 5d903d1c5f79c96f9a3ed1ea9d9cb93a9f9bb4ae
pip install .

and the package works now. So the problem isn't pip but whatever is uploaded to pypi.
Note that python setup.py install isn't the most recommended way of installing anymore, if one wants to be able to cleanly remove an installed package, b/c pip keeps track of all meta-parts as. well.

[michaelaye]

Looks like it's a known issue without a solution, at least I haven't found one yet.

For example here:
https://stackoverflow.com/questions/19569557/pip-not-picking-up-a-custom-install-cmdclass

For testing I have created a testing_post_install_cmd package on the pypi server that is calling a print-out init function and creating a text file init_file.txt in the local directory.
The stackoverflow case described that one also has to cover the egg_info case, but that still doesn't call my init() function via pypi install. Digging further ....

[michaelaye]

So, the problem is not what is uploaded to pypi per se, but simply that a pypi install never runs the post-install command.

[michaelaye]

I'm wondering: could you not simply check at import/run-time, if the init was run, and if not, just run it then? Why is there the requirement that this needs to be run after the pypi install directly?

[michaelaye]

so, binary wheels don't involve running setup.py at all, which is why it is impossible to execute post_install code using bdist_wheel. This is the reason why it only works via GitHub repo install using setup.py, nothing to do with pip and conda. ;)
See my PR for a possible solution and thanks for this awesome project.
Now that I have fixed this for you, maybe you could start a vscode plugin? ;) At least a start so that I can see how the search interface works at all.

[pcubillos]

Hi Michael,
I wanted explicitly to run init() on installation to avoid incompatibility issues in case the database structure changes. Though bibm keeps a plain text file with all the bibtex entries, parsing it to get the authors/titles/years/etc starts to get slow (~a second) as one goes into the hundred/thousand entries. So, bibm also keeps a pickle file with the parsed info to do the indexing/searching faster.
Undoubtedly the database structure will change as one adds new features or refactors the code. So, init() updates the pickled file from the bibtex file (so that I don't worry about unpickling an incompatible file).

I was thinking about this problem yesterday, and thought that I could create some sort of magic number that changes whenever I made a change in the database structure, bibm could quickly check this number whenever the user does any I/O, and run init() if needed. This way I could take your PR (which I highly appreciate!) and we can forget about the post_install issue. What do you think? May there be a better way to do this?

[michaelaye]

I totally get the pickle file for faster search. And I guess you are using pickle to avoid any other dependencies? Note that sqlite is also a Python built-in and serverless, so that could potentially be a better approach?
However, I still don't understand why any update of the existing pickle file absolutely needs to happen directly after pip install/update? You can check version tokens against each other and force the update whenever the run bibmanager version is a newer version than the one stored in the pickle database? This is basically what you suggest in your 2nd paragraph, but no need to invent a new magic number, just check against the package version that is already supported by your package. I'd say to put in this effort is absolutely worth it to attract more users, b/c you will lose potential users if they all have to install via GitHub. pip (and now conda) are still the easiest way of installation.

[pcubillos]

OK, yeah, that makes sense.