wcmc-its / reciter-publication-manager Goto Github PK

Here is a complete list of fields that it would be nice if the filter worked on. Some of these are more important than others. For example, targetAuthorInstitutionalAffiliationArticleScopusLabel
and targetAuthorInstitutionalAffiliationArticlePubmedLabel are more important than publicationDateDisplay. So if you would like me to prioritize, please let me know.

pmid
pmcid
publicationDateDisplay
doi
publicationTypeCanonical
scopusDocID
issn
journalTitleISOabbreviation
journalTitleVerbose
articleTitle
lastName [for any author]
firstName [for any author]
emailMatch
identityOrganizationalUnit
articleAffiliation
journalSubfieldScienceMetrixLabel
journalSubfieldDepartment
targetAuthorInstitutionalAffiliationIdentity
targetAuthorInstitutionalAffiliationArticleScopusLabel
targetAuthorInstitutionalAffiliationArticleScopusAffiliationId
targetAuthorInstitutionalAffiliationArticlePubmedLabel
nonTargetAuthorInstitutionalAffiliationMatchKnownInstitution
institutionGrant
articleGrant

As per this mockup
https://www.dropbox.com/s/eatfl0calv9htgt/ReCiter_Design_Nov18-2019.xd?dl=0

Sort should provide options for sort by:

• Score ▼
• Score ▲
• Date ▲
• Date ▼

Could we ensure that the relationships line up if any one is several lines of content, or is this a pain??

Modifications:

• Do not include rounded version of user's headshot. We have to use the rectangular version that is made available via Weill Cornell's API
• Towards the top right, it should say, "Add new record: PubMed[link]"

A user was trying to login to Publication Manager. She had roles of 3 and 4 in admin_users_roles (curator_self and reporter_all).

Of these, the curator_self role was wrong apparently. The user was in the person table but not the person_article table.

I'm not sure why this happened. In any case, we shouldn't prevent such users from logging in, in cases where a user has another role like reporter_all. I suggest that in these cases, instead of showing an error page, we gracefully direct properly authenticated users to the "Find People" page.

Here's the error in the logs:

ReCiter Update Goldstandard api is not reachable: FetchError: invalid json response body at https://reciter.weill.cornell.edu/reciter/goldstandard?goldStandardUpdateFlag=UPDATE reason: Unexpected token < in JSON at position 0
RangeError [ERR_HTTP_INVALID_STATUS_CODE]: Invalid status code: undefined
    at ServerResponse.writeHead (_http_server.js:255:11)
    at ServerResponse.writeHead (/app/node_modules/next/dist/compiled/compression/index.js:46:263)
    at ServerResponse._implicitHeader (_http_server.js:246:8)
    at ServerResponse.end (/app/node_modules/next/dist/compiled/compression/index.js:22:749)
    at ServerResponse.apiRes.end (/app/node_modules/next/dist/server/api-utils/node.js:346:25)
    at sendData (/app/node_modules/next/dist/server/api-utils/node.js:187:9)
    at ServerResponse.apiRes.send (/app/node_modules/next/dist/server/api-utils/node.js:349:31)
    at handler (/app/.next/server/pages/api/reciter/update/goldstandard.js:88:52)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async Object.apiResolver (/app/node_modules/next/dist/server/api-utils/node.js:366:9)
    at async NextNodeServer.runApi (/app/node_modules/next/dist/server/next-server.js:481:9)
    at async Object.fn (/app/node_modules/next/dist/server/next-server.js:741:37)
    at async Router.execute (/app/node_modules/next/dist/server/router.js:252:36)
    at async NextNodeServer.run (/app/node_modules/next/dist/server/base-server.js:365:29)
    at async NextNodeServer.handleRequest (/app/node_modules/next/dist/server/base-server.js:303:20) {
  code: 'ERR_HTTP_INVALID_STATUS_CODE'
}

Occasionally (10% of the time perhaps), I have noticed that I will click on the "Refresh" button, and it will not update the ReCiter database. Additionally, sometimes the search using the PubMed lookup will not go through.

Consistent with best practices, Publication Manager should ensure that attempts to talk to external resources go through. If they don't the system should try again.

this whole function needs to be rewritten, types should not be changing we should either be parsing or stringifying, the logic here is hard to read.
the reason we are using TypeScript in this project is because we want things to be typed so we won't have to check types like is happening in this case. Please type the obj to avoid any issues with type checking

For clarity, I created a new issue but this relates to #11.

Background

User has filtered for a set of scholars and clicked on the "Review All Pending Suggestions" button. This takes the user to the "Review Pending Suggestions" page.

Desired behavior

Right now, the page has a URL of "/individual_suggestions". Instead, the URL should be "/review_suggestions". All URLs should be consistent with the label in the sidebar.

Consistent with the mockup, the "Review Suggestions" line in the sidebar should be highlighted in a lighter blue and that background should extend all the way to the sides of the column.

A user who clicks on "Find Scholar" at the top or "Edit Search" link should go back to the "Find Scholar > Individual" page with any existing search filters in place. (Right now, these are cleared out.)

A user who clicks on "Review suggestions" shouldn't go anywhere. (Right now, they go to Jochen Buck's profile.)

Any identity data that is returned should come from the Redux cache which has already been downloaded.

Instead of "Review Pendings Suggestions", it should say "Review Pending Suggestions" (no "s").

The targetAuthor is not highlighted in light green like they are in the mockup or the individual suggestions page. When I click on "Show evidence behind this suggestion", the column widths are kind of screwy, and inconsistent with the individual suggestions page. To state the obvious, you should probably use the same code here that you do there.

I click on the name of an individual scholar. That fires a blank window. That's good. But then it takes me to the login page. That is not good. :)

Now I click on "Edit Search." This takes me to the "Find Scholar" page. I remove the person type filter I had previously put in place, leaving me only with the "Pediatrics" department. I click the "Review Pending Suggestions" button, but I am shown results for both of those filters even though I had removed one.

Starting over again, I do a search for person-type = "student-md-new-york". I click on the Accept button for some of the records. Nothing happens.

Starting over again, I do a search by department = "Psychiatry", person type = "academic" and then click on the "Review Pending Suggestions" button. This takes 40 seconds and then takes me to a page that has the error message, "Too many results." If it's too many results, why does it seem like it worked? And, if it worked, why would we show the error message at all?? In general, I have no problem with the system taking a while to retrieve data so long as:

• it eventually it works
• we're not doing lookups unnecessarily
• it doesn't crash the system (which I think I did when I tried to load Medicine)

Background

See issue #37 for how all searches / filters should be done against Redux as opposed to calling the API. The previous mockup had a search button and filters. Even without the Redux implementation, this led to some unexpected behavior. For example, things get especially messy when a user tries to both search for a name and apply any of the three filters at the same time. I was going to document all of this, but then I decided it would be easier, both on the user and hopefully the developer, to have a simpler UI.

Updated mockup

https://zpl.io/bP8BBgp (opens in Zeplin if you have it install on your machine). Note how there is now no search button or "advanced search" text in this version. (The filter box is currently missing, but it will be re-added shortly. Including a partial screenshot below.)

URL and sidebar

Please remove the "Individual" option as per the mockup. The "Find Scholar" option should be highlighted. When selected, this should take the user to the URL "/find_scholar" which is the default.

Left: current. Right: mockup.

Desired behavior

The search function should act like a traditional filter. The user types "Joseph" in the input box, and absent any additional keystroke, the number of records changes to around 50.

This updates the result count to 50: "50 scholars". (If there were one scholar, it should say "1 scholar". If there were 0 scholars, it should say, "0 scholars.")

Now the user adds the "Genetic Medicine" filter. This changes the number of records to 1.

As the number of results changes, the jumpbar also changes. (Right now, there are 12 links even if there are no results.) With 50 records, there should be 5 links (first 20, second 20, last 10, and then first and last), and with one user it should not exist. The jumpbar should also not exist if there are no records. Note too how there are "first" and "last" buttons.

Once the number of results is between 1 and 500 scholars, display the "Review All Pending Suggestions."

Additional feedback

• List all the option values in the dropdown menus in alphabetical order.
• The font size of options in the select menu should be 14px.
• The dropdown menu should always read "Department" even if I have selected another organization.
• The "Individual" item in the sidebar should be selected in lighter blue.

The search by multiple person IDs and by person-level parameters are mutually exclusive.

Sample search with 0 results...

de santi sj[au] and li y[au]

just a note. Dont have to change anything but we should in the future check if these env variables are actually available otherwise probally return a 401 unauthorized error

This confuses users when there is only one result. I tried changing the z-index but this didn't fix anything.

As per this mockup
https://weillcornell.invisionapp.com/share/Z6TO9K1KBE9#/screens/405277255

Dropdown menu should just have one option: to log out.

Footer should not be fixed.

Modifications:

• Do not include the logo (as opposed to the logotype)
• Do not include logged in user's headshot

On login, user is directed to http://[domain]/search

This queries the Identity API as it should.

The system should (but does not) store the data in the Redux store, which is stored in the browser.

Changing this avoids the need for running the slow and expensive query each time we return to the list of individuals, search for individuals, or need any metadata about individuals.

The Redux store should be persistent across tabs and windows.

Let me know if you don't like this.

97fb2e144c17055835df9ef46b6af552.ico.zip

This should go in the HTML.

<link rel="apple-touch-icon" sizes="57x57" href="/apple-icon-57x57.png">
<link rel="apple-touch-icon" sizes="60x60" href="/apple-icon-60x60.png">
<link rel="apple-touch-icon" sizes="72x72" href="/apple-icon-72x72.png">
<link rel="apple-touch-icon" sizes="76x76" href="/apple-icon-76x76.png">
<link rel="apple-touch-icon" sizes="114x114" href="/apple-icon-114x114.png">
<link rel="apple-touch-icon" sizes="120x120" href="/apple-icon-120x120.png">
<link rel="apple-touch-icon" sizes="144x144" href="/apple-icon-144x144.png">
<link rel="apple-touch-icon" sizes="152x152" href="/apple-icon-152x152.png">
<link rel="apple-touch-icon" sizes="180x180" href="/apple-icon-180x180.png">
<link rel="icon" type="image/png" sizes="192x192"  href="/android-icon-192x192.png">
<link rel="icon" type="image/png" sizes="32x32" href="/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="96x96" href="/favicon-96x96.png">
<link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png">
<link rel="manifest" href="/manifest.json">
<meta name="msapplication-TileColor" content="#ffffff">
<meta name="msapplication-TileImage" content="/ms-icon-144x144.png">
<meta name="theme-color" content="#ffffff">

Two things:

The sort order of the evidence is: name and then the absolute value of all other data, e.g., 5, 3, -2, 1, 0.2, 0...

Note how the line height in the mockup is consistent between the two columns. (Also note the cell padding.)

This allows users to compare data in the two columns.

See dev vs. mockup

Publication Manager will only suggest the first N articles a person has authored where N is maxArticlesPerPerson.

In cases where the number of pending publication suggestions (where userAssertion=NULL) exceeds maxArticlesPerPerson, display the total count.

Note: This work cannot be done until an additional attribute is being output in Feature Generator. See wcmc-its/ReCiter#399

URL: https://reciter-dev.weill.cornell.edu/app/shk4001

Problem

As you can see here: https://www.dropbox.com/s/by8otdvzv302t0l/scz7002.mov?dl=0

• I accept 174 pubs for scz7002
• I click the refresh link.
• 6 more pubs are suggested
• I accept those and then click the refresh link.
• 0 new pubs are suggested
• I wait a couple seconds and then refresh the browser window
• 2 new pubs are suggested

I have seen this before. I think this may be responsible for certain papers "falling through the cracks."

Fix

I'm not sure what the problem is. Do we want to introduce an additional delay?

I try to login in a second time, and it works just fine. This seems to happen in cases where I haven't logged in in a couple days.

Publication Manager Login Weirdness.mov.zip

We need to squash these errors. We also need less hostile language in cases where they do come up!

App throws an error if there are more than a certain number of results

See this movie: https://www.dropbox.com/s/8e4r6trztr827ut/errorreciter.mov?dl=0

Or try this:

1. Go here:
   https://reciter-dev.weill.cornell.edu/app/amc2056

2. Search for "Choi, Augustine[au]"

3. Get this error:
   Error: Given action "PUBMED_CHANGE_DATA", reducer "pubmedData" returned undefined. To ignore an action, you must explicitly return the previous state. If you want this reducer to hold no value, you can return null instead of undefined.
   Click to reload

Ideally we would only show the top, say, 100.

Search for "Shroof"

https://www.dropbox.com/s/hbm2goo88fyanay/Screen%20Recording%202020-11-28%20at%205.17.54%20PM.mov?dl=0

Tips

https://stackoverflow.com/questions/49217841/react-reducer-giving-error/49217870

This does take a while, so we need a spinner for when this is working.

We need an error message if this attempts to return too many results.

Also, see #21. See Sarbajit for details.

For dal3005, if I click on Rejected, then the filter input, and type "N", the screen goes blank.
https://reciter-dev.weill.cornell.edu/app/dal3005

We can blame this on these two articles, both of which were imported without an article title, no doubt due to the HTML tag in the article title....
29157049
29183177

blankscreen.mov.zip

https://codeburst.io/uncaught-typeerror-cannot-read-property-of-undefined-in-javascript-c81e00f4a5e3

This is the complete list...

31499548
19543463
28651447
27365115
31333836
33071540
31859721
28521375
32227802
31462119
30207193
29460605
30385294
30006084
30258629
19543463
29183177
29157049
32485774
31342494
31236656
28249340
30258629
32147019
33157846
28349634
32147019
33112065
28349634
33157721
29231082
33157724
33157774
33157662
33157794

Some don't even have a title in PubMed such as for 28349634.

For nob9038, 15460678

This should be implemented on any page that has pagination:

• Find Scholar
• Group view
• Individual view (accepted, rejected, pending, add new record)

Mockup here:
https://zpl.io/bP8BBgp

wow this function is long. Ideally would be nice if this broken up into parts but most likely to late now

Switching between tabs or screens shouldn't remove it. But, switching between users should remove it.

The production version of this application uses JSON Web Tokens. JWT is client-based
and should propagate across all open tabs/windows. It has been tested to not log users out unexpectedly.

Something is different in dev. I notice that when I open multiple tabs, some tabs will automatically log out and others will maintain an open session. This suggests to me that a single session is not being maintained across all tabs/windows.

See video example.

https://www.dropbox.com/s/p9qi2a2q634q42f/SessionManagement-2020-03-23.mov?dl=0

Background

• With issue #53, we noted how the app seems to frequently crash. This occurs in both dev and prod.
• We speculated that this was a problem with session management, but @ganga ruled that out.
• Since then, @sarbajitdutta pinpointed a more likely cause: improper exception handling by the Publication Manager app itself.

Reproducing the error

• As an example, consider user = "las2012"
• If we run Feature Generator or Identity APIs on Swagger (https://reciter.weill.cornell.edu/reciter/swagger-ui.html), the system throws a 404 exception. This does not affect uptime.

• Now if we make a request via the dev or prod app like so http://[reciter-publication-manager-URL]]/app/las2012, we see the following error.

https://reciter.weill.cornell.edu/reciter/feature-generator/by/uid?uid=las2012&totalStandardizedArticleScore=4&useGoldStandard=AS_EVIDENCE&analysisRefreshFlag=0&retrievalRefreshFlag=FALSE&filterByFeedback=ALL
POST /users/reciter/validate 200 0.598 ms - 31
undefined:1
The uid provided 'las2012' was not found in the Identity table
^
SyntaxError: Unexpected token T in JSON at position 0
    at JSON.parse (<anonymous>)
    at Request._callback (/usr/src/app/src/api/reciterPublication.js:30:25)
    at Request.self.callback (/usr/src/app/node_modules/request/request.js:185:22)
    at Request.emit (events.js:311:20)
    at Request.<anonymous> (/usr/src/app/node_modules/request/request.js:1154:10)
    at Request.emit (events.js:311:20)
    at IncomingMessage.<anonymous> (/usr/src/app/node_modules/request/request.js:1076:12)
    at Object.onceWrapper (events.js:417:28)
    at IncomingMessage.emit (events.js:323:22)
    at endReadableNT (_stream_readable.js:1204:12)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: `NODE_ENV=production node ./bin/www`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

• As you can see the system is returning a 404 error as opposed to JSON, which crashes npm (package manager). Because the system is not anticipating error exceptions, the entire system crashes via a domino effect.

Suggested change

• In cases of exceptions / errors, Publication Manager should not try to parse JSON.
• Instead the system should only attempt to parse JSON output in cases where a 200 code is returned.
• In all other cases, consistent with best practices, Node should parse any errors and then share the error with the user. This should be done in a way that does not crash npm.

Left is prod, right is dev. Ignore the header. The fonts in dev are too small where we use 14px font most of the time. The padding is different. This is not terrible, just a bit sloppier.

Unless we're purposefully improving the login screen, can we please have it look the same?

As requested by Fred Hutch

I go here:
https://reciter.weill.cornell.edu/curate/skl9004

I accept a publication. (He wrote all of them, so all the suggestions are valid.)

When I click "Refresh Suggestions", the page errors out.

The attempt to talk to the Gold Standard API is not recorded in the logs, nor is the error.
https://www.dropbox.com/scl/fi/gohxjuz99cm9fdlb4ewvf/Refresh-Suggestions-error-2023-08-25.mov?rlkey=5vos9ksokm22i64o94tw8ahs9&dl=0

Further the feedback never made its way into the GoldStandard table. Each time I repeat this, the same thing happens.

For example, in this case, I (paa2013) made the change. Instead, it would appear Dikoma Shungu did (which is inaccurate).

This information will be useful for reporting purposes.

Background

Users reviewing candidate publications would benefit from seeing the top keywords for accepted publications. This gives the user a sense of the types of subjects that someone writes about.

Dependency: wcmc-its/ReCiter#442

Requirements

Display keywords of accepted publications in descending order by count.

Tooltip: "42 accepted articles use this keyword."

I need to try logging in, wait for the error, and then log in again.

Every so often, I provide feedback and I get an error that looks like the below such that I can't provide additional feedback in either prod or dev for this user.

This error sometimes remains for several days. For example, I provided feedback on the 11th, and it was still there on the 12th.

A hallmark of this error is that there is a record in userFeedback for that user like so...

{
  "acceptedPmids": [
    16125823
  ],
  "feedbackDate": "2021-02-11T13:28:18.401Z",
  "rejectedPmids": [
    31308743,
    31305420,
    30624267,
    30262849,
    30218280,
    30082489
  ],
  "uid": "yil4008"
}

I see no other instances of any record in userFeedback besides this one.

If it helps to troubleshoot, I was able to fix this error by deleting this record from DynamoDB. After doing this, I was able to load the user's profile and provide additional feedback without a problem.

Post-error, I will refresh the page and then be logged out. I enter in my accurate credentials, and the system says "Bad credentials."

I’m trying to use this command on ReCiterDB:

SELECT 'x' INTO OUTFILE '/Users/Paul/Desktop/test.rtf';

Please grant my user “admin” the right to execute this command. I believe, but am not positive, this would do it.

GRANT FILE ON *.* TO admin;

This is also needed when we include this button.

https://directory.weill.cornell.edu/api/v1/person/profile/dei2003.png?returnGenericOn404=true
If 404, use backup image like so...

In Chrome, I click on Inspect, and sometimes the elements on the page move around.
https://www.dropbox.com/s/hfqlptr92kfku0y/JavaScriptWarning.mov?dl=0

Another example:
https://www.dropbox.com/s/k8m7wil9h9rsgyg/StrangePageLoad-2020-02-23.mov?dl=0

I noticed in the Javascript console that there are some warnings about not properly using SameSite cookies. Perhaps this explains the error??

28074583 was rejected previously.

das2043 accepted the record as confirmed by this record in admin _feedback_log. He also told me he accepted it.

But it is still appearing as rejected in the UI.