Does not parse arXiv emails with "empty" footer about arxivfilter

[deragent]

Is this the full arxiv-email that you receive?

In my case, there is usually also a footer at the bottom, in the form:

%%%---%%%---%%%---%%%---%%%---%%%---%%%---%%%---%%%---%%%---%%%---%%%---%%%---
For subscribe options to combined physics archives,
e-mail To: [[email protected]](mailto:[email protected]), Subject: subscribe
-----------------------------------------------------------------------------
For help on viewing and making submissions, see http://arxiv.org/help/

And the parser also looks for, and removes this part during the parsing.

[louiskirsch]

This is the footer, it looks a bit different from yours, maybe that is the issue.

------------------------------------------------------------------------------
\\
arXiv:2308.12896
replaced with revised version Tue, 29 Aug 2023 15:57:02 GMT   (2999kb,D)

Title: Beyond Document Page Classification: Design, Datasets, and Challenges
Authors: Jordy Van Landeghem, Sanket Biswas, Matthew B. Blaschko,
 Marie-Francine Moens
Categories: cs.CV cs.CL cs.LG
Comments: 8 pages, under review
\\ ( https://arxiv.org/abs/2308.12896 ,  2999kb)
%%%---%%%---%%%---%%%---%%%---%%%---%%%---%%%---%%%---%%%---%%%---%%%---%%%---

[deragent]

I'll look into this, but this should be an easy fix!

Could you maybe attach a full copy of one of your E-Mails, for testing?

[louiskirsch]

Thanks!
arxiv.txt

[deragent]

So, the E-Mail with the simple footer should no longer lead to an error.

See version 0.3.4: https://pypi.org/project/arxiv-filter/0.3.4/

@louiskirsch: On other thing I noticed in the arxiv.txt file that you posted, is that there are a lot of newlines in unexpected places. For example the author lists are often on multiple lines (see lines 46 - 50).
Is this always the case in your E-mails, or is this due to the way you copied the E-Mail?

The current version of arxiv_filter can not handle such newlines, and thus will not properly parse the author lists in the arxiv.txt file which you posted.

[louiskirsch]

I checked the source of the email, these line breaks (in the authors list) are in the original email I received.

[deragent]

Indeed, I just checked that this is similar in my emails, but there seems to be a difference in the number of spaces at the beginning of a line which continues the previous line. I created a new issue for this: #8

But the original issue should be fixed with the latest release (0.3.4) - closing.