dataesr / works-magnet

Retrieve the scholarly works of your institution

Works magnet

Retrieve the scholarly works of your institution.

Build for production

The react client app is served by the node server in production.

Install and run app

Run

npm i && npm start

Web App available at http://localhost:5173/ and API at http://localhost:3000/.

Build app

npm run build

Vite build creates a build in /dist folder. This folder has to be moved to the /server folder.

Deployment

The version number follows semver.

To deploy in production, simply run this command from your staging branch :

npm run deploy --level=[patch|minor|major]

⚠️ Obviously, only members of the dataesr organization have rights to push on the repo.

works-magnet's Issues

In affiliation tab, the search on affiliation name is case sensitive

and probably accent sensitive. And probably in the other tabs too.

Hide the "Filter on matching affiliations" button

Hide the "Refresh affiliations" button

Export / import the decided (aka excluded and validated) affiliations

Create a new Tab for datasets ...

... and rename Tab "Works" into "Publications"

Export bug

On click on "save work to file" i got

Option to display only matching affiliations

In the Affiliations tab, option to display only the affiliations that do match one of the criteria, rather than all the affiliations encountered.
(checked by default ?)
And maybe, in the affiliation view, add a column with the number of matching expressions
(ex: if the query is "ecole polytechnique" + "palaiseau" => "ecole polytechnique, route de saclay, palaiseau" has 2 matches

Tab Works, add year and journal columns

Add spinner while loading data

Rename "Restore from file" button into "Restore session" and add tooltip

Normalize OpenAlex types of works

Map everything to barometre type
(use an aggregate on bso-publications + bso-datacite indices to gel all possibles values)
e.g article from OpenAlex ==> journal-article

[perf] Add websocket to return notification at each step

[api][perf] Issue while downloading results for large institution

Use OpenAlex cursor to overcome the size limit of querying OpenAlex

See https://docs.openalex.org/how-to-use-the-api/get-lists-of-entities/paging#cursor-paging

Rename "Save work to file" button into "Save session" and add tooltip

Delete the RoR column

While querying for multiple institutions, only the first one is in bold in the affiliation column in the affiliations tab

Because of https://github.com/dataesr/works-finder/blob/main/server/src/utils/works.js#L230, should iterate over all the normalizedAffiliations. Use reduce function.

Retrieve more than 10k results from BSO

Probably using the search-after param https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html ?

Delete "seve session" and "restore sessions" buttons

Link on the logo

On the logo of the ministry, the link leads to "https://dataesr.github.io/" instead of "https://dataesr.github.io/works-finder/".

Add stepper to help user

https://dataesr.github.io/react-dsfr/?path=/docs/composants-indicateur-d-%C3%A9tapes-stepper--first-step

Optimize UI rendering right after fetching data

In the Affiliations tab, display RoR id in the affiliations name column

If a RoR is associated to an affiliation also use it in the normalized version of the affiliation, and in the displayed name.

In both BSO and OpenAlex, the RoR is (sometimes) given within the affiliation object.

Error 404 on font download

See console errors : downloadable font: download failed (font-family: "Marianne" style:normal weight:700 stretch:100 src index:0): status=2147746065 source: https://dataesr.github.io/fonts/Marianne-Bold.woff2

[merge works] Should we take version number into consideration ie. should we consider multiple versions of a DOI as one or as multiple DOI ?

cf. https://doi.org/10.26434/chemrxiv.12782456 vs. https://doi.org/10.26434/chemrxiv.12782456.v2

Delete the GItHub pages deployment

Add an option to query only datasets in the query part

Use multiple requests for BSO data if more than 10.000 results

• 2 separate requests for bso-publications and bso-datacite ?
• splitted by "published_date" on bso-publications, "created" on bso-datacite ?

=> number of slices should be calculated according to the number of results

Simplify UX

To be discussed with users!

Optimize dedup function in API

• Use reduce func https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/reduce ?
• It might be due to the merge func ?

[optimization] pipeline

1. Query on the 2 datasources (BSO & OpenAlex)
2. Merge all the results in the works array
3. Dedup the works array
4. Group by affiliations
5. Sort works between publications and datasets

[optim] Call the range function before querying the multiple datasources

This function is called here :

• https://github.com/dataesr/works-finder/blob/f432cb656eb5a8e4722405f455d82a4e14b449b1/server/src/utils/works.js#L109-L110
• https://github.com/dataesr/works-finder/blob/f432cb656eb5a8e4722405f455d82a4e14b449b1/server/src/utils/works.js#L207-L208

But it should be called before https://github.com/dataesr/works-finder/blob/f432cb656eb5a8e4722405f455d82a4e14b449b1/server/src/routes/works.routes.js#L16

In affiiliation tab, delete notice

[optimize] Optimize the groupByAffiliations function

Use the reduce function

Add direct Datacite API

For now, we use only the BSO ingested data from Datacite.
We could use the direct Datacite API.
Searching in the fields
creators.affiliation.name / contributors.affiliation.name fields
creators.affiliation.affiliationIdentifier / contributors.affiliation.affiliationIdentifier

https://api.datacite.org/dois?affiliation=true&query=creators.affiliation.name:*%22%C3%A9cole%20polytechnique%22*%20OR%20contributors.affiliation.name:*%22%C3%A9cole%20polytechnique%22*%20OR%20contributors.affiliation.affiliationIdentifier:%22https://ror.org/02s376052%22

Use a backend to use OpenAlex premium API key

Add a progress bar about the remaining affiliations to decide

Add search by funder

Refactor the query builder and all the filters part

And add example for each filter

Tab Works, display the first 3 affiliations ...

... and if needed a "show more" button.
The matched affiliation(s) should be displayed on top.

Delete search by author

[datasets] Add dedup step for datasets

Restore work title column in the publications tab and the datasets tab

Import a list of publications

CSV file containing a list of DOIs, hal-ids, OpenAlex ids ?

Lighten the returned objects (publications, datasets and affiliations)

Rename "Create export file" button into "Export BSO"

Parallel OpenAlex querying

Performance tests for large institutions

We need to make sure the app experience is fine also for (very) large institutions.
Examples to test:

• saclay
• sorbonne
• inrae
• cnrs

Clicks on different buttons should be fast enough (ex: select /unselect "filter on matching affiliations")

May depend on #3 !
#2 has to be done before

Correct the metrics about FOSM total results

... because of #2

Add the patents

• Enable to search on the patents
• Enable to filter on work type (publications, datasets and / or patents)
• Enable to search on fields inventors, applicants and localisations