Harvest hangs on UTF-8 errors about metha

[miku]

Thank for the bug report!

is it intentional that harvesting hangs when invalid UTF-8 is encountered

No, not intentional.

If it is possible, it would be nice if harvesting could continue even in the case of UTF-8 errors

Yes, this might be a good idea. Do you have an example endpoint URL, where this problem occurs?

[miku]

@mjlassila, I actually found one example endpoint myself:

$ metha-sync http://firstmonday.org/ojs/index.php/fm/oai
....
2016/11/24 17:29:40 http://firstmonday.org/ojs/index.php/fm/oai?from=2...
2016/11/24 17:29:41 XML syntax error on line 273: invalid UTF-8

Firefox doesn't like it either, XML Parsing Error: not well-formed.

	<dc:creator>Lugano, Giuseppe; University of Jyv�skyl�</dc:creator>
-------------------------------------------------------^

[mjlassila]

Unfortunately the endpoint I'm using is in access restricted network but I'm glad that you found a suitable open endpoint for testing.

[miku]

I cannot reproduce this error on my previous example http://firstmonday.org/ojs/index.php/fm/oai. In case of not well formed XML, it might be OK to reject it. If you do not object, I would close this issue for now.

[mjlassila]

Suits me, I'll create a new issue if I stumble upon this error again.We resolved it by cleaning the source data which had some erroneous characters.

…

On Aug 23, 2017 9:13 AM, "Martin Czygan" ***@***.***> wrote: I cannot reproduce this error on my previous example http://firstmonday.org/ojs/index.php/fm/oai. In case of not well formed XML, it might be OK to reject it. If you do not object, I would close this issue for now. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACWzCPW84uG9UPYtLbVNqMjofR8IM5Sks5sa8MNgaJpZM4K2emO> .

[miku]

Thanks!