bibcure / bibcure

Bibcure helps in boring tasks by keeping your bibfile up to date and normalized...also allows you to easily download all papers inside your bibtex

bibcure (Beta Version)

Bibcure helps in boring tasks by keeping your bibtex file up to date and normalized.

Requirements/Install

Bibcure uses the wonderful Bibtex parser. In this moment we waiting for new release of python-bibtexparser to solve some bugs.

Install it using pip:

$ sudo python /usr/bin/pip install bibcure
# or
$ sudo pip install bibcure
# or
$ sudo pip3 install bibcure  # for Python 3

You can also install from the source: git clone the repository, and install with the setup.py script.

scihub2pdf (beta)

If you want download articles via a DOI number, article title or a bibtex file, using the database of arXiv, libgen or sci-hub, see bibcure/scihub2pdf.

---

Features and how to use

bibcure

Given a bib file...

$ bibcure -i input.bib -o output.bib

• check sure the arXiv items have been published, then update them (requires internet connection),

• complete all fields(url, journal, etc) of all bib items using DOI number (requires internet connection),

• find and create DOI number associated with each bib item which has not DOI field (requires internet connection),

• abbreviate journals names.

arxivcheck

Given an arXiv id...

$ arxivcheck 1601.02785

• check if has been published, and then returns the updated bib (requires internet connection).

Given a title...

$ arxivcheck --title "A useful paper with hopefully unique title published on arxiv"

• search papers related and return a bibtex file for the first item.

You can easily append a bib into a bibfile, just do

$ arxivcheck --title "A useful paper with hopefully unique title published on arxiv" >> file.bib

You also can interact with results, just pass --ask parameter:

$ arxivcheck --ask --title "A useful paper with hopefully unique title published on arxiv"

scihub2pdf

Given a bibtex file

$ scihub2pdf -i input.bib

Given a DOI number...

$ scihub2pdf 10.1038/s41524-017-0032-0

Given an arXiv id...

$ scihub2pdf arxiv:1708.06891

Given a title...

$ scihub2bib --title "A useful paper with hopefully unique title"

or arxiv...

$ scihub2bib --title arxiv:"A useful paper with hopefully unique title"

Location folder as argument:

$ scihub2pdf -i input.bib -l somefolder/

Use libgen instead sci-hub:

$ scihub2pdf --uselibgen -i input.bib

doi2bib

Given a DOI number...

$ doi2bib 10.1038/s41524-017-0032-0

• get bib item given a DOI (requires internet connection)

You can easily append a bib into a bibfile, just do:

$ doi2bib 10.1038/s41524-017-0032-0 >> file.bib

You also can generate a bibtex from a txt file containing a list of DOIs:

$ doi2bib --input file_with_dois.txt --output refs.bib

title2bib

Given a title...

$ title2bib "A useful paper with hopefully unique title"

• search papers related and return a bib for the selected paper (requires internet connection)

You can easily append a bib into a bibfile, just do

$ title2bib "A useful paper with hopefully unique title" --first >> file.bib

You also can generate a bibtex from a txt file containing a list of "titles"

$ title2bib --input file_with_titles.txt --output refs.bib --first

---

Comparison: Sci-Hub vs LibGen

Sci-Hub

• Stable
• Annoying CAPTCHA
• Fast

Libgen

• Unstable
• No CAPTCHA
• Slow

---

License

GNU Affero General Public License v3.0. For more details, see the LICENSE file.

bibcure's Issues

After update arxiv check if in db(for journal abbreviation)

allows script be installed into system

Cannot run on OSX

Hello

I installed bibcure with pip (I tried with both pip and pip3). Everything is OK for the installation ( bibcure-0.2.5.tar.gz). I am using OSX Sierra (10.12.6).

However, when I type 'bibcure' in the terminal, I get:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 429, in get_provider
    module = sys.modules[moduleOrReq]
KeyError: 'journalabbrev'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/bibcure", line 17, in <module>
    from bibcure.database import Db_abbrev
  File "/usr/local/lib/python3.6/site-packages/bibcure/database.py", line 18, in <module>
    "data/db_abbrev.json")
  File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1197, in resource_filename
    return get_provider(package_or_requirement).get_resource_filename(
  File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 431, in get_provider
    __import__(moduleOrReq)
ModuleNotFoundError: No module named 'journalabbrev'

What did I do wrong? It seems that the module journalabbrev is missing, but this module does not seem to exist anymore.

Thanks!

Show info about bib before user interaction

Does not work (python3)

This is the error it returns. I hope it helps.

Traceback (most recent call last):
  File "/Users/michi/anaconda/lib/python3.6/site-packages/bibtexparser/bibdatabase.py", line 102, in expand_string
    self.strings[name])
KeyError: 'sep'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/michi/anaconda/bin/bibcure", line 177, in <module>
    main()
  File "/Users/michi/anaconda/bin/bibcure", line 130, in main
    bibtex = bibtexparser.loads(inputfile.read(), parser=parser)
  File "/Users/michi/anaconda/lib/python3.6/site-packages/bibtexparser/__init__.py", line 48, in loads
    return parser.parse(bibtex_str)
  File "/Users/michi/anaconda/lib/python3.6/site-packages/bibtexparser/bparser.py", line 147, in parse
    self._expr.parseFile(bibtex_file_obj)
  File "/Users/michi/anaconda/lib/python3.6/site-packages/bibtexparser/bibtexexpression.py", line 263, in parseFile
    return self.main_expression.parseFile(file_obj, parseAll=True)
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 2163, in parseFile
    return self.parseString(file_contents, parseAll)
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1622, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 3919, in parseImpl
    return super(ZeroOrMore, self).parseImpl(instring, loc, doActions)
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 3858, in parseImpl
    loc, tmptokens = self_expr_parse( instring, preloc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 3530, in parseImpl
    ret = e._parse( instring, loc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 3395, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 3530, in parseImpl
    ret = e._parse( instring, loc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 3395, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 3395, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 3919, in parseImpl
    return super(ZeroOrMore, self).parseImpl(instring, loc, doActions)
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 3858, in parseImpl
    loc, tmptokens = self_expr_parse( instring, preloc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 3395, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 3717, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 3395, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 3530, in parseImpl
    ret = e._parse( instring, loc, doActions )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1405, in _parseNoCache
    tokens = fn( instring, tokensStart, retTokens )
  File "/Users/michi/anaconda/lib/python3.6/site-packages/pyparsing.py", line 1049, in wrapper
    ret = func(*args[limit[0]:])
  File "/Users/michi/anaconda/lib/python3.6/site-packages/bibtexparser/bibtexexpression.py", line 260, in _string_expr_parse_action
    return self._string_expr_parse_action_fun(s, l, t)
  File "/Users/michi/anaconda/lib/python3.6/site-packages/bibtexparser/bparser.py", line 184, in <lambda>
    BibDataStringExpression.expression_if_needed(t)))
  File "/Users/michi/anaconda/lib/python3.6/site-packages/bibtexparser/bparser.py", line 178, in <lambda>
    maybe_interpolate = lambda expr: as_text(expr)
  File "/Users/michi/anaconda/lib/python3.6/site-packages/bibtexparser/bibdatabase.py", line 223, in as_text
    return text_string_or_expression.get_value()
  File "/Users/michi/anaconda/lib/python3.6/site-packages/bibtexparser/bibdatabase.py", line 184, in get_value
    return ''.join([BibDataString.expand_string(s) for s in self.expr])
  File "/Users/michi/anaconda/lib/python3.6/site-packages/bibtexparser/bibdatabase.py", line 184, in <listcomp>
    return ''.join([BibDataString.expand_string(s) for s in self.expr])
  File "/Users/michi/anaconda/lib/python3.6/site-packages/bibtexparser/bibdatabase.py", line 150, in expand_string
    return string_or_bibdatastring.get_value()
  File "/Users/michi/anaconda/lib/python3.6/site-packages/bibtexparser/bibdatabase.py", line 131, in get_value
    return self._bibdatabase.expand_string(self.name)
  File "/Users/michi/anaconda/lib/python3.6/site-packages/bibtexparser/bibdatabase.py", line 104, in expand_string
    raise(UndefinedString(name))
bibtexparser.bibdatabase.UndefinedString: 'sep'

Resolve C901 -> "main" func. is too complex

Url field renamed as link

"The line is not ending the record. " Strange behavior with quotes and a bib item with only one field

Using

@misc{SM,
    note="Supplemental Material reference"
 }

don't work

Using

@misc{SM,
    note="Supplemental Material reference",
 }

work

Using

@misc{SM,
    note={"Supplemental Material reference"}
 }

work

The source code

import logging
import logging.config
from bibtexparser.bparser import BibTexParser

logger = logging.getLogger(__name__)

logging.config.dictConfig({
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'standard': {
            'format': '%(asctime)s [%(levelname)s] %(name)s %(funcName)s:%(lineno)d: %(message)s'
        },
    },
    'handlers': {
        'default': {
            'level':'DEBUG',
            'formatter': 'standard',
            'class':'logging.StreamHandler',
        },
    },
    'loggers': {
        '': {
            'handlers': ['default'],
            'level': 'DEBUG',
            'formatter': 'standard',
            'propagate': True
        }
    }
})


if __name__ == '__main__':
    bibtex = """@misc{SM,
    note="Supplemental Material reference"
    }
    """

    with open('/tmp/bibtex.bib', 'w') as bibfile:
        bibfile.write(bibtex)


    with open('/tmp/bibtex.bib', 'r') as bibfile:
        bp = BibTexParser(bibfile.read())
        print(bp.get_entry_list())

Do not know how to use

I never used python before, and I want to use it to correct my BibTeX files.
I tried it, but can't find any output.

betterbib

Just a heads-up in case you hadn't seen: betterbib tries to do the same thing as bibcure, more or less.

error when getting dois

Get DOI absent using the title?y(yes, automatic)/m(manual)/n(do nothing)y
Traceback (most recent call last):
  File "/home/user/bin/bibcure", line 177, in <module>
    main()
  File "/home/user/bin/bibcure", line 137, in main
    bibs = update_bibs_get_doi(bibs)
  File "/home/user/lib/python3.7/site-packages/bibcure/title.py", line 26, in update_bibs_get_doi
    bibs[i] = update_bib(bib, get_first)
  File "/home/user/lib/python3.7/site-packages/bibcure/title.py", line 10, in update_bib
    found, bib_string = get_bib_from_title(bib["title"], get_first)
  File "/home/user/lib/python3.7/site-packages/title2bib/crossref.py", line 86, in get_bib_from_title
    found, item = get_from_title(title, get_first)
  File "/home/user/lib/python3.7/site-packages/title2bib/crossref.py", line 50, in get_from_title
    items = r.json()["message"]["items"]
TypeError: list indices must be integers or slices, not str

Abstracts... How to get?

Great lib!
To automatize my daily tasks, I need to get the title and abstract the papers. How can we do it using bibcure?
The title works pretty good, but now abstract is also important to me :-)
Many thanks!
Regards,
Fabio

Improve this horrible section of code

    if len(bibs_arxiv) > 0:
        print "%d bibs are arxiv" % len(bibs_arxiv)
        updated_bibs = bibs_arxiv

    if len(bibs_not_journal) > 0:
        print "%d bibs don't have journal tag " % len(bibs_not_journal)

        updated_bibs += bibs_not_journal

    if len(bibs_abreviated) > 0:
        print "%d bibs are already abbreviated  " % len(bibs_abreviated[0])
        updated_bibs += bibs_abreviated[0]

    if len(bibs_in_db) > 0:
        print "%d bibs can be easily abbreviated " % len(bibs_in_db[0])
        updated_bibs = reduce(
            lambda a, b: a + b,  map(update_bibs_in, bibs_in_db)
        )

    if len(bibs_out_db) > 0:
        print "%d bibs must be manualy abbreviated, " % len(bibs_out_db[0]) + \
            "at least this time "
        updated_bibs += reduce(
            lambda a, b: a + b, map(update_bibs_out, bibs_out_db)
        )

Not working on Windows 10 OS

I'm using Windows 10 Home, version 1903, using Python 3.8.2 and pip 20.2.4. Also, the terminal is the Windows native cmd.

I installed the bibcure lib as indicated:

$ pip install bibcure

However, when using doi2bib command (or any other bibcure command) on cmd, like this:

$ doi2bib 10.1016/j.chaos.2020.110027

I get the following message:

'doi2bib' is not recognized as an internal 
or external command, operable program or batch file.

I've tried to uninstall and re-install again, but the problem persists. Also, I've tried as Issue #13, and it did not work.

Is there extra steps for Windows?

Keys "_text" and "_type" are not being removed

Implement multiprocessing

pdf2bib

Takes a pdf file and outputs a bibtex file
Inspirations
https://github.com/GUSecLab/pdf2bibtex

Conflicting BibTex Keys

Hi,

Sorry if I have missed an easy was to do this.

After appending a .bib file using doi2bib, I run bibcure to tidy up the .bib file. However as part of this tidy it doesn't detect when there are several entries with the same BibTex key. This is a particular problem in my field, which contains some very prolific authors who are capable of publishing multiple 1st author papers a year.

Would this functionality be possible to add this functionality, with the option to append the offending entries with a letter i.e. Author_2017a, Author_2017b etc.?

Another way around it may be to allow for custom bibtex key format? Similar to Author_Word_Year, where word is the first not "A", "The" in the title.

Check spelling

AttributeError: 'bytes' object has no attribute 'encode'

Update bibs using DOI field?y(yes)/n(do nothing)y
Get DOI absent using the title?y(yes, automatic)/m(manual)/n(do nothing)m

Original title: Optimized high-order splitting methods for some classes of
parabolic equations

Traceback (most recent call last):
  File "/usr/bin/bibcure", line 177, in <module>
    main()
  File "/usr/bin/bibcure", line 137, in main
    bibs = update_bibs_get_doi(bibs)
  File "/usr/lib/python3.7/site-packages/bibcure/title.py", line 26, in update_bibs_get_doi
    bibs[i] = update_bib(bib, get_first)
  File "/usr/lib/python3.7/site-packages/bibcure/title.py", line 10, in update_bib
    found, bib_string = get_bib_from_title(bib["title"], get_first)
  File "/usr/lib/python3.7/site-packages/title2bib/crossref.py", line 72, in get_bib_from_title
    found, item = get_from_title(title, get_first)
  File "/usr/lib/python3.7/site-packages/title2bib/crossref.py", line 65, in get_from_title
    found, item = ask_which_is(title, items)
  File "/usr/lib/python3.7/site-packages/title2bib/crossref.py", line 24, in ask_which_is
    unidecode(item["title"]), unidecode(title)))
  File "/usr/lib/python3.7/site-packages/unidecode/__init__.py", line 48, in unidecode_expect_ascii
    bytestring = string.encode('ASCII')
AttributeError: 'bytes' object has no attribute 'encode'

Error upon reading bib file

  File "/home/user/anaconda/envs/bibcure/lib/python3.7/site-packages/bibtexparser/bibtexexpression.py", line 10, in _strip_after_new_lines
    lines = s.splitlines()
TypeError: 'str' object is not callable

Allow update of the database

Journal abbreviations

Hi,

It is sometimes necessary in my field to abbreviate journal titles. For instance,

THE JOURNAL OF PHYSICAL CHEMISTRY C

becomes

J PHYS CHEM C

Is it at the moment possible with bibcure? Would that be an useful feature? I could help with a PR.

A list of abbreviations can be found in the web.