Incorrectly extracts DOI or PMID from bibliography about cermine

[dtkaczyk]

@themeo I believe this bug is corrected by f22c307 Please take a look at the newest version of master branch and test it against your files.

[themeo]

Thanks for the corrections! I think we are almost there. The only issue is that pattern matching seems to not handle well situations when a line break occurs within the doi:

Example of reference:

Bates, D., Mächler, M., Bolker, B., Walker, S., 2015. Fitting linear mixed-effects
models using lme4. J. Stat. Softw. 67 (1), 1–48. http://dx.doi.org/10.18637/jss.
v067.i01.

Which is processed as:

        <mixed-citation>
          <string-name>
            <surname>Bates</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mächler</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bolker</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Walker</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <year>2015</year>
          .
          <article-title>Fitting linear mixed-effects models using lme4</article-title>
          .
          <source>J. Stat. Softw</source>
          .
          <volume>67</volume>
          (
          <issue>1</issue>
          ),
          <fpage>1</fpage>
          -
          <lpage>48</lpage>
          . http://dx.doi.org/
          <pub-id pub-id-type="doi">10.18637/jss</pub-id>
          . v067.
          <year>i01</year>
          .
        </mixed-citation>

Another example:

Elston-Güttler, K.E., Gunter, T.C., Kotz, S.A., 2005. Zooming into L2: Global language
context and adjustment affect processing of interlingual homographs in
sentences. Cognitive Brain Res. 25 (1), 57–70. http://dx.doi.org/10.1016/j.
cogbrainres.2005.04.007.

And this is the processed citation:

        <mixed-citation>
          <string-name>
            <surname>Elston-Güttler</surname>
            ,
            <given-names>K.E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gunter</surname>
            ,
            <given-names>T.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kotz</surname>
            ,
            <given-names>S.A.</given-names>
          </string-name>
          ,
          <year>2005</year>
          .
          <article-title>Zooming into L2: Global language context and adjustment affect processing of interlingual homographs in sentences</article-title>
          .
          <source>Cognitive Brain Res</source>
          .
          <volume>25</volume>
          (
          <issue>1</issue>
          ),
          <fpage>57</fpage>
          -
          <lpage>70</lpage>
          . http://dx.doi.org/
          <pub-id pub-id-type="doi">10.1016/j</pub-id>
          . cogbrainres.
          <year>2005</year>
          .
          <volume>04</volume>
          .007.
        </mixed-citation>

[dtkaczyk]

I am aware of this problem. However, fixing it is not trivial.

Normally line break corresponds to a word separator. Sometimes this is not true, for example if we break words, but then hyphenation is used, so the system can guess that a word exists in two parts and should be concatenated.

In this case, however, there are no such clues, the DOI is sometimes just split in two parts. So when you have a DOI at the end of a line, the first word in the next may or may not be a part of this DOI, and we have to be able to distinguish between these situations, so that we do not accidentally attach a random word to DOI. How can we know this without checking external services? Do you have an idea?

[themeo]

I can see the problem. I presume it is not possible to detect if the doi identifier is adjacent to the right border of the column?

In all cases the doi is a link - sometimes even the doi is hidden in a link that is referenced by a non-doi text, like here: https://www.sugarsync.com/pf/D9346138_07075771_619903
Perhaps the doi could be unambiguously extracted from the content of the link then?

Another solution: Doi is always either the last field in the reference or second to the last (followed by the PMID identifier). So, if there is no PMID label after doi, it should be safe to assume that any text following after a space is still a part of the doi.

[dtkaczyk]

I am afraid we have too small set to assume it always follows some rule, eg. that it is last/second to the last field. However, you can always add a postprocessing step on your side to take care of this.

We do not currently extract links. It would be a nice functionality, but considering the lack of resources, it won't be added anytime soon. I will reopen this issue if the situation changes.