General Problem with search_ about litstudy

[stijnh]

Hi FlashFF. Thanks for using Litstudy.

Search, especially for semanticscholar and scopus seems to be extremely slow.

There are two reasons for this. First, these APIs work in two stages: first all the paper IDs are collected for the search query and, second, each paper is requested individually. So if you are search for 10.000 papers, it will take at least 10.000 requests to collect all the results.

Second, the semanticscholar API does not allow too many requests per hour. There are timeouts in the code to limit the number of requests to 100 requests per 5 minutes.

The results are cached though. When you perform a search, litstudy will create a hidden file called .semantischolar.db that contains all the search results so they can be reused later.

• throws errors of not finding Paper Id XYZ
• when returning papers, they are all the same
• throwing Endpoint request timed out.

This sounds like bugs. Do you have reproducible testcase for us?