catalyst / moodle-tool_cohortdatabase

Tool for sync of cohorts using external db.

https://moodle.org/plugins/tool_cohortdatabase

What is this?

This plugin creates site-level cohorts using a connection to an external database.

Features

• Can be triggered via CLI and/or scheduled task.
• Creates site-level cohorts
• Syncronises users in cohorts
• Optionally auto create users

Installation

1. Install the plugin the same as any standard moodle plugin either via the Moodle plugin directory, or you can use git to clone it into your source:

   git clone [email protected]:catalyst/moodle-tool_cohortdatabase.git admin/tool/cohortdatabase

   Or install via the Moodle plugin directory:

   https://moodle.org/plugins/tool_cohortdatabase

2. Then run the Moodle upgrade

Support

If you have issues please log them in github here

https://github.com/catalyst/moodle-tool_cohortdatabase/issues

Please note our time is limited, so if you need commercial support or want to sponsor a new feature then please contact Catalyst IT

https://www.catalyst.net.nz/contact-us

moodle-tool_cohortdatabase's Issues

Is there a plan to support moodle 3.6

As we plan to upgrade to 3.6 and also would like to use cohort exernal database, may i ask about the status of 3.6 support?

Automatic creation of new users fails when Shibboleth authentication is selected

When createusers_auth is set to 'Shibboleth', user creation fails and returns :

Execute scheduled task: Sync cohorts with external database (tool_cohortdatabase\task\sync)
... used 34 dbqueries
... used 0.32330203056335 seconds
Scheduled task failed: Sync cohorts with external database (tool_cohortdatabase\task\sync),Moodle needs certain Shibboleth attributes which are not present in your case. The attributes are: 'eppn' (''), 'givenName' (''), 'sn' ('') and 'mail' ('')<br />Please contact the webmaster of this server or your Identity Provider.

Help needed: Nothing being sync

Thank you for this plugin.
I use external database auth and the connection works.
In the same database I have a separate table with user cohort info:
USERID, COHORTID

When execution cohort sync via scheduled task and debug enabled, It says sync completed without fetching any cohorts. No cohorts were created in Users=>Cohorts.

Any hints greatly appreciated.

Sam

email on cohort failure could be improved

The email that triggers when more deletions than expected have been deleted could be improved to include the cohort name (not just idnumber.)

and it would be nice if it showed the full log from the cron task - in particular what previous cohorts were affected - eg:
Bulk delete of 23 users from cohortid18471
Bulk delete of 23 users from cohortid18468
Bulk delete of 23 users from cohortid18413
Bulk delete of 23 users from cohortid18469
Bulk delete of 23 users from cohortid18415
Bulk delete of 23 users from cohortid18470

but also including the cohort name.

Error when minrecords is set to 0

When the plugin setting minrecords (that tells how many records should exists, at least, in the database for the plugin to operate) is set to 0, the plugin operate as if the database table was empty.

An error message is displayed, such as :

Failed to sync because the external db returned 0 records and the minimum is 0

This behaviour can be a problem for anybody who want to test the plugin with a table containing only one specific record. (it was my case). A '0' setting could also be used to disable this check for anybody who want that.

Failed to sync

when i run the cron always i got this "Failed to sync because the external db returned 0 records" although the external db has 4 rows

How / where is tool/cohortdatabase:config capability used?

I can't see any obvious place how and where is the capability tool/cohortdatabase:config used. The inline comment in db/access.php is not quite clear to me, too.

Coding errror detected warning and no removal of users

Moodle 3.7
When changing a users cohort in the external database, the warning:

!!! Coding error detected, it must be fixed by a programmer: Invalid context id specified context::instance_by_id() !!!
!!
Error code: codingerror !!
!! Stack trace: * line 5184 of /lib/accesslib.php: coding_exception thrown

• line 334 of /admin/tool/cohortdatabase/classes/sync.php: call to context::instance_by_id()
• line 67 of /admin/tool/cohortdatabase/cli/sync.php: call to tool_cohortdatabase_sync->sync()
  !!

is thrown on the next run of the sync (and only then).

The user is added to the new cohort, but not removed from the old cohort.
I'm still testing the plugin and never succeeded to remove users from cohorts or remove empty cohorts using Moodle 3.7., so I'm not aware what behaviour is expected and what not.

I'm writing up some documentation on https://docs.moodle.org/37/en/Cohort_external_database and translated the plugin to Dutch. Feel free to add / modify.

error writing to database

We get a dmlwriteexeption with this error message.

Scheduled task failed: Synchroniseer site-groepen met een externe databank (tool_cohortdatabase\task\sync),Fout bij het schrijven van de databank
Debug info:
You can't specify target table 'mdl_cohort' for update in FROM clause
DELETE FROM mdl_cohort
                 WHERE id in (SELECT c.id
                                FROM mdl_cohort c
                           LEFT JOIN mdl_cohort_members cm ON cm.cohortid = c.id
                               WHERE component = 'tool_cohortdatabase' AND cm.id is null
                                     AND c.id NOT IN (select customint1 from mdl_enrol where enrol = 'cohort'))
[array (
)]
Backtrace:
* line 291 of /lib/dml/moodle_read_slave_trait.php: call to moodle_database->query_end()
* line 1167 of /lib/dml/mysqli_native_moodle_database.php: call to mysqli_native_moodle_database->query_end()
* line 615 of /admin/tool/cohortdatabase/classes/sync.php: call to mysqli_native_moodle_database->execute()
* line 386 of /admin/tool/cohortdatabase/classes/sync.php: call to tool_cohortdatabase_sync->cleanup()
* line 52 of /admin/tool/cohortdatabase/classes/task/sync.php: call to tool_cohortdatabase_sync->sync()
* line 253 of /lib/cronlib.php: call to tool_cohortdatabase\task\sync->execute()
* line 167 of /admin/cli/scheduled_task.php: call to cron_run_inner_scheduled_task() 

According to https://stackoverflow.com/questions/4429319/you-cant-specify-target-table-for-update-in-from-clause

The query should be something like (look at line 3)

DELETE FROM mdl_cohort
WHERE id in (SELECT c.id
FROM (SELECT * FROM mdl_cohort) AS c
LEFT JOIN mdl_cohort_members cm ON cm.cohortid = c.id
WHERE component = 'tool_cohortdatabase' AND cm.id is null
AND c.id NOT IN (select customint1 from mdl_enrol where enrol = 'cohort'));

which seems to work if executed directly on the DB.

Ability to set user group and sync memberships

Hi,

My old workplace is looking at this plugin as a way of creating cohorts and syncing users into them from their student information system, but they would also like to be able to automatically create group memberships within the same process, e.g. by using the same sort of method you use for the cohort - setting a custom field (or multiple fields probbaly, as they might be in multiple groups) and mapping them to group idnumbers, then syncing users to those groups during the cohort sync. I advised them that the actual creation of groups will have to be done manually beforehand, but they would like this membership sync to happen during the same cohort process, to avoid having to have multiple processes to configure everything.

Is that something which you would be willing to add as a feature? Or, if not, if I extend the plugin for them, would you like me to do a PR so you can include it?

Conn

Email matching on sync is case sensitive

When running "sudo -u www-data /usr/bin/php sync.php -v" we get a load of entries like this:

Starting cohort synchronisation...
Updated 0 cohort names/descriptions
Starting cohort database user sync
Could not find user with email = [email protected]
Could not find user with email = [email protected]

These user emails do exist in the Moodle database, however they have differing case, i.e. User1@ or uSer2@... Is it possible to get the search to do a case insensitive search against emails in the database when looking for users?

How works?

Hi,

How can I configure this?
(No database connection configuration)

Issue with adding users to cohorts that were just created

The code gets a list of cohorts, then adds new cohorts, but when adding new members to a cohort it uses the original lst of cohorts generated before it added new cohorts so users are only added to new cohorts on the 2nd cron run:

these errors in the logs:
PHP Notice: Undefined offset: 6251 in admin/tool/cohortdatabase/classes/sync.php on line 332
PHP Notice: Trying to get property of non-object in lib/classes/event/base.php on line 890

events missing fields - shows errors in logs.

++ Fields list in snapshot record does not match fields list in 'cohort'. Record is missing fields: descriptionformat, component, timecreated, timemodified, visible, theme ++

• line 902 of /lib/classes/event/base.php: call to debugging()
• line 273 of /admin/tool/cohortdatabase/classes/sync.php: call to core\event\base->add_record_snapshot()
• line 53 of /admin/tool/cohortdatabase/classes/task/sync.php: call to tool_cohortdatabase_sync->sync()
• line 252 of /lib/cronlib.php: call to tool_cohortdatabase\task\sync->execute()
• line 120 of /lib/cronlib.php: call to cron_run_inner_scheduled_task()
• line 73 of /lib/cronlib.php: call to cron_run_scheduled_tasks()

Should the CLI sync be documented so that it encourages to run it via cron?

How I see it, the regular execution should run via a scheduled task right? By looking at the phpdoc of the file, one could get a false impression that setting up the execution via the CLI is needed.