dr01d3r / redcap-em-orca-search

REDCap External Module that is an alternative to the built-in Add/Edit Records

Orca Search (REDCap External Module)

A configurable, searchable, and performant, list dashboard.

Purpose

To enhance searching REDCap records. This module can also be used to create a custom list dashboard.

Disclaimer

This module does not yet support repeating events.

Features

• Configurable field search options
• Configurable field display options
• New Record support for projects with and without auto-numbering enabled
• Only Super Users or project users with explicit "Create Record" rights will see the "Add new record" button
• Result highlighting when searching with partial matching enabled
• Support for searching through repeating forms
• Data Access Group security
  • Users in a DAG will only see results from within their DAG
  • Users not assigned a DAG will see results regardless of DAG
• Long text truncation
  • To prevent dropdowns from extending off screen and table headers causing significant visual issues, field labels, structured data values (select, radio, checkbox), and Dynamic SQL results will be truncated down to 60 characters OR the first new line, whichever one comes first.
  • If truncation causes visual text to be removed, an ellipsis (...) will be appended to the end.
  • NOTE: An ellipsis only applies to multi-line labels if the first line exceeds 60 characters.

Testing & Validation

• REDCap
  • Minimum: v13.1.0 STD (v13.1.5 LTS)
  • Latest: v14.3.7 STD
• PHP
  • Minimum: v7.2.5
  • Latest: v8.1.29

Permissions

• redcap_every_page_top

Configuration

• Enable the module in your project, if it is not already enabled
• Click Configure for Orca Search Module

Options

• Redirect Add/Edit Records to Module
  • Check this option if you would like to "replace" the Add/Edit Records page with this module.
  • When a user clicks the link, they will be directed to the module's search page.
• Limit the search results
  • The results can be restricted to 500, 1000, 2000, 5000, or No Limit. If the search results exceed that number, an error message will display and no results will show.
• Which instances to search through
  • Latest - search only the most recent instances of forms.
  • All - all instances will be considered.
  • Currently, the results table will always display the result from the latest instance, even if the matching value was found on a previous instance.
• Show instance badge
  • A red badge will display in the data table, indicating which instance the displayed value was found.
• Include DAG in results
  • An additional column will display in the results, indicating the Data Access Group that record belongs to.
  • This option only works if the project uses Data Access Groups.
• Prevent an empty search from yielding results
  • Forces users to always provide a value when searching. This can be useful when you want to restrict access to records (i.e. MRN exact match).
• Specify 'Record Home' button location in display fields table (default=Last Column)
  • Allow the record home button to be the first or last column, or not displayed at all
• Search Fields
  • This repeating configuration allows you to select the fields that you want to use as search fields.
  • Fields will display in the dropdown, in the order that exist in the config.
  • Fields that are select, radio, form status, etc., will display a dropdown instead of a text box, for filtering by value.
  • Return Partial Matches
    • This option allows you to do a 'wildcard' search on that field (i.e. searching wis in a state search will return results for Wisconsin)
    • NOTE: This option is ignored if the field type is radio, select, checkbox, or sql
• Display Fields
  • These are the fields you wish to see in the results table
  • Text values will display as-is, while structured data will show the display values.
  • Selecting the form_name_complete fields will display accordingly (Incomplete, Unverified, Complete).
  • Fields will display in the table, in the order they are selected in the config.

Searching a text field using a partial match

Searching a dropdown field

Considerations

• DateTime fields are displayed and searched using the values as they are stored in the database (YYYY-MM-DD).
• If a project uses Data Access Groups, and the record_id field is displayed, I've done my best to ensure that column sorts as expected.
• For projects with significant record counts (10,000+, especially if they use DAGs), the usage of getAutoId() for adding new records will increase the load times of the module.
  • For a baseline example, a project with ~45k records takes 4-8 seconds for getAutoId() to complete.
  • Note: performance for this should see significant improvement in REDCap v9.3.7+
• If you identify any issues, please submit an issue on this GitHub repo or make a post on the forums and tag me (@chris.kadolph)

Future Support

• Multiple Filters
• Use DateTime format from the Data Dictionary rather than raw database value
• Support for displaying the latest instance that matched the search criteria
• Custom configuration page
• Row formatting (i.e. form status indicators)

redcap-em-orca-search's Issues

Check for all required fields and either disable the use of the module or use default values

redcap-em-orca-search/search.php

Line 186 in 9c0ee50

if (empty($config["search_fields"]) || empty($config["display_fields"])) {

let the last search results remain

Hi,

I've been using the search function in our project and noticed that after I check a record and go back, the search results disappear. Any way to keep the last search results there so I don't have to search again?

Thanks!

User rights access not respected

If a user doesn't have access to an instrument and fileds from this instrument are selected to be displayed, this user would see this fields even if he should not.
Is that an expected behaviour?

Orca doesn't work when any Advanced Custom Application Links are Enabled

Chris,
I showed you the issue I was having at REDCapCon in Chicago. I then thought it was caused by a plug-in. But my cohort, David Rogers, dug into it and it appears that Orca will not work if any Custom Application Links of type Advanced are enabled. As soon as we disabled or deleted all the advanced custom application links, Orca started working.

new upgrade crashed the whole server, help!

I downloaded the new version and the upgrade failed. Now the redcap server web interface is down and I can no longer access the control panel, what was changed during the upgrade??

Creating new records, with auto-numbering disabled, ignores field validation on the record id.

https://redcap.vumc.org/community/post.php?id=53849

Incorrect search behavior with certain numeric values

https://community.projectredcap.org/questions/51305/orca-search-ignores-first-selection-value-of-multi.html

Redirect Add/Edit Records to Module

Excellent module, just what i wanted for a new project.
One minor glitch, when i tried to enable "Redirect Add/Edit Records to Module", i receive the following error:
The 'hook_add_edit_records_page' hook did not complete for the 'orca_search' module because of the following error:

Unknown

The current user is a super user, so this module will be automatically disabled.

Disable or remove "Add New Record" button if user does not have Create Record rights

Disable or remove the "Record Home" button if the user has "no access" rights

"Add new record" fails when user is assigned to a DAG

Button navigates to "/redcap_v8.4.3/ExternalModules/NaN" instead of record_home.

Redirect triggers shutdown function, which emails REDCap admin and possibly disables the module

redcap-em-orca-search/OrcaSearch.php

Line 38 in 25f26df

redirect($this->getUrl("search.php"));

[PHP8] Uncaught TypeError: end(): Argument #1 ($array) must be of type array, null given

Testing PHP8 on: REDCap 12.0.25 · PHP 8.0.13 (Linux/Unix OS) · MariaDB 10.6.10

Searching for an existing record throws TypeError, if:

• a field from a repeating form is configured as display field in the search results
• the field value for that repeating field is empty (returns NULL)

Error Message:

REDCap crashed due to an unexpected PHP fatal error!

Error message: Uncaught TypeError: end(): Argument #1 ($array) must be of type array, null given in modules/orca_search_v2.3.5/search.php:313 Stack trace: #0 modules/orca_search_v2.3.5/search.php(313): end() #1 redcap_v12.0.25/ExternalModules/index.php(126): require('...') #2 {main} thrown
File: modules/orca_search_v2.3.5/search.php
Line: 313

The error can be fixed by typecasting on the last value.

New Record button fails due to javascript function "addGoogTrans" in REDCap core being removed in v9.8.0

new version upgrade failed

Here's the error message:

Subject: REDCap External Module Hook Exception - orca_search

[This message was automatically generated by REDCap]

The 'orca_search' module threw the following exception when calling the hook method 'redcap_every_page_top':

Exception: You must supply the following either as a GET parameter or as the last argument to this method: pid in C:\inetpub\wwwroot\redcap\redcap_v8.2.2\ExternalModules\classes\AbstractExternalModule.php:653
Stack trace:
#0 C:\inetpub\wwwroot\redcap\redcap_v8.2.2\ExternalModules\classes\AbstractExternalModule.php(635): ExternalModules\AbstractExternalModule->requireParameter('pid', NULL)
#1 C:\inetpub\wwwroot\redcap\redcap_v8.2.2\ExternalModules\classes\AbstractExternalModule.php(221): ExternalModules\AbstractExternalModule->requireProjectId(NULL)
#2 C:\inetpub\wwwroot\redcap\modules\orca_search_v1.2.2\OrcaSearch.php(34): ExternalModules\AbstractExternalModule->getProjectSetting('replace_add_edi...')
#3 [internal function]: ORCA\OrcaSearch\OrcaSearch->redcap_every_page_top(NULL)
#4 C:\inetpub\wwwroot\redcap\redcap_v8.2.2\ExternalModules\classes\ExternalModules.php(1180): call_user_func_array(Array, Array)
#5 C:\inetpub\wwwroot\redcap\redcap_v8.2.2\ExternalModules\classes\ExternalModules.php(1240): ExternalModules\ExternalModules::startHook('orca_search', 'v1.2.2', Array)
#6 C:\inetpub\wwwroot\redcap\redcap_v8.2.2\Classes\Hooks.php(37): ExternalModules\ExternalModules::callHook('every_page_top', Array)
#7 C:\inetpub\wwwroot\redcap\redcap_v8.2.2\Classes\HtmlPage.php(185): Hooks::call('redcap_every_pa...', Array)
#8 C:\inetpub\wwwroot\redcap\redcap_v8.2.2\ControlCenter\header.php(26): HtmlPage->PrintHeader()
#9 C:\inetpub\wwwroot\redcap\redcap_v8.2.2\ExternalModules\manager\control_center.php(4): require_once('C:\inetpub\wwwr...')
#10 {main}

URL: https://xxx.xxxx.edu/redcap/redcap_v8.2.2/ExternalModules/manager/control_center.php
Server: xxx.xxxx.edu
Module Name: Orca Search Module (orca_search)
Module Author(s): xxxx

After that, i receive the following when try to access the upgrade page again:
500 - Internal server error.
There is a problem with the resource you are looking for, and it cannot be displayed.

Custom labels for fields

Field labels on forms can get long, and can interfere with the the display. Allowing the user to configure custom labels could be an easy solution to that problem.

Checkbox search acts as wildcard instead of exact match

Even with partial match unchecked, the search returns results as if it were.

https://community.projectredcap.org/questions/94151/orca-search-module-results-overinclusive-when-usin.html

I have discovered a bug. When using a checkbox as a searchable field, the search results are overinclusive. When searching for records with a checkbox value of 7, records with checkbox values of 7, 17, 27, 37, 47 etc. are inappropriately included in search results. When searching for records with a checkbox value of 1, records with checkbox values of 1, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 31, 41 etc. are inappropriately included in search results.

I'm using v2.3.3 on REDCap v10.2.0.

Add support for Dynamic (SQL) field type

https://community.projectredcap.org/questions/74184/orca-search-em-search-within-sql-fields.html

The module will currently display the raw saved value of a SQL field, however, when the select statement uses 2 fields to support key/value pairs, the interface is no longer intuitive to the users. Users would expect to search the display value, similar to how select/radio field types are supported.

Additionally, if a mapping of a saved value is not found in the source dataset, it has been requested that no value is shown (to include the key).

Alter the "Replace Add/Edit" config to also allow full replacement

Currently, only the link in the sidebar is updated to take you to the search dashboard. Instead, we could change it to a dropdown, or add another config entry to allow the project to fully replace the add/edit page, forcing users to Orca Search.

Example:
A project owner does not want users to have the ability to search all data across the project and only wants them to search the fields configured within Orca Search.

getAutoId moved in REDCap v10.0.1

A function file form_renderer_functions was removed in v9.8.0 with deprecated support remaining for those functions elsewhere. In v10.0.1, those legacy functions were either removed entirely or refactored elsewhere.

In this case, the new function is \DataEntry::getAutoId(). There may be a new EM framework function for this as well, which may be a more future-proof solution.

Orca does not function properly with REDCap arms

We noticed that Orca does not function properly when using REDCap arms. In our test project, 4 arms exist. When searching, only values for records in the last arm (in casu arm 4) are returned; for all records in other arms, empty rows are returned (see screenshot). Additionally, the link to the Record Home page does not include the arm number; in the example, record 13 belongs to arm 4, but &arm=4 is is missing in the URL.

Feature Request: Ability to move the Open button to the left-hand side of the table

We often have many columns in our search results. Would love to have the ability to move the Open button to the left-hand side of the table so users would not need to scroll to open the record. Or perhaps have the ability to automatically add a hyperlink to the record under the left-most column.

Long field label as a search option

https://community.projectredcap.org/questions/85652/orca-search-em-wrap-text-for-searchable-field.html

PHP8: REDCap crashed due to an unexpected PHP fatal error!

REDCap crashed due to an unexpected PHP fatal error!

Error message: Uncaught TypeError: array_key_exists(): Argument #2 ($array) must be of type array, null given in /var/www/html/redcap/modules/orca_search_v2.3.3/traits/REDCapUtils.php:185 Stack trace: #0 /var/www/html/redcap/modules/orca_search_v2.3.3/traits/REDCapUtils.php(185): array_key_exists() #1 /var/www/html/redcap/modules/orca_search_v2.3.3/search.php(252): ORCA\OrcaSearch\OrcaSearch->getProjectRecordIds() #2 /var/www/html/redcap/redcap_v10.8.0/ExternalModules/index.php(121): require_once('...') #3 {main} thrown
File: /var/www/html/redcap/modules/orca_search_v2.3.3/traits/REDCapUtils.php
Line: 185

would be nice to work with Repeatable instruments

See attachments, only last instance of repeatable instruments is displayed in search result.
It would be nice to display all instances separate line or like the style of "Record Status Dashboard" with a "+" icon to add more instances.
thanks!

Conflict with Send It feature

Even though $project_id is specified, getProjectSetting() requires it to also be specified in the $_GET, and Send It does not provide that.

A hotfix needs to be applied to explicitly provide the optional $project_id parameter to getProjectSetting(). Be sure to look back to previous releases of REDCap to see when this optional parameter was implemented or if it has existed since the beginning.

ORCA Search REDCap

I cant see what output occurs as a result of the following: Can you include a screen shot in the documentation (am using REDCAP 8.3.0)
"Show instance badge
A red badge will display in the data table, indicating which instance the displayed value was found"

This extension is great. I had also hoped to see it operational via a search link on the dashboard page to limit return of dashboard items to those meeting the search criteria.

Thanks great job

Be able to automatically add the search fields to the results grid

Feature request: It would be nice if I could check a box to automatically add a search field to the results grid. Adding fields in projects with lots of fields is so painful, this could ease some of that pain.

Add/Edit replacement does not work when user cannot create new records (i.e. View/Edit)

redcap-em-orca-search/OrcaSearch.php

Line 39 in b7c275e

addEditLinkText: "<?=$lang['bottom_62']?>",

Update the module to check for both phrases that are possible (bottom_62 and bottom_72) for replacement of the link.

Language Options

Hello!

Thanks for the module, loving it. Is there any possible way to configure the language? I.e. the Search Button.

BR
h3n0r1k