Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-73703

LDAP user sync: update users in chunks

XMLWordPrintable

    • MOODLE_310_STABLE, MOODLE_311_STABLE, MOODLE_39_STABLE, MOODLE_404_STABLE
    • MOODLE_404_STABLE
    • MDL-73703-master
    • Hide

      LDAP server creation and Moodle configuration

      1. Run this command

        docker run --rm --env LDAP_ADMIN_USERNAME=admin --env LDAP_ADMIN_PASSWORD=password -p 1389:1389 bitnami/openldap

      2. Add the following entries to your config.php

        define('TEST_AUTH_LDAP_HOST_URL', 'ldap://localhost:1389');
        define('TEST_AUTH_LDAP_BIND_DN', 'cn=admin,dc=example,dc=org');
        define('TEST_AUTH_LDAP_BIND_PW', 'password');
        define('TEST_AUTH_LDAP_DOMAIN', 'dc=example,dc=org');

      3. Log in to Moodle as Admin
      4. Go to Site Admin -> Plugins -> Authentication -> Manage Authentication
      5. Enable 'LDAP server'
      6. Click on the settings link for LDAP server
      7. In the 'Host URL' field, enter in 'ldap://localhost:1389' (without the quotes)
      8. Set Version to 3
      9. Under 'User lookup settings' look for the 'Contexts' field and enter 'ou=users,dc=example,dc=org' (without the quotes)
      10. Under 'User account synchronisation', CONFIRM there is a new field called 'Sync update users chunk size'
      11. Make this value '1' (without the quotes)
      12. Under 'Data mapping', enter into 'Data mapping (First name)' the value of 'givenName' (without quotes)
      13. Change 'Update local (First name)' to 'On every login'
      14. Change 'Update external (First name)' to 'On update'
      15. Leave all other values as the defaults on the page.
      16. Save changes.
      17. Go back to Site Admin -> Plugins -> Authentication -> Manage Authentication and click on LDAP's 'Test settings' link to ensure your connection is working.

      Create LDAP users

      In your CLI, use the following command to create some users. Increment all the values for each user. Do this for three users.

      When prompted for a password, use 'password'. This is the password that was provided in your config.php early on in the setup.

      ldapadd -x -H ldap://localhost:1389 -D "cn=admin,dc=example,dc=org" -W <<EOF
      dn: cn=user01,ou=users,dc=example,dc=org
      objectClass: top
      objectClass: posixAccount
      objectClass: inetOrgPerson
      cn: user01
      sn: Bar1
      uid: user01
      uidNumber: 1001
      gidNumber: 1001
      homeDirectory: /home/user01
      userPassword: password
      EOF
      

      Testing scenario

      Now with all that setup out of the way, we should have Moodle configured to use LDAP authentication and our LDAP server ready to authenticate these users.

      1. Go to Moodle's login screen and log in with each user. (username: user01, password: password)
      2. Repeat this for all users your created (user02, user03)
      3. Run the following CLI command in your Moodle root directory:

        php admin/cli/scheduled_task.php --execute='auth_ldap\task\sync_task'

      4. CONFIRM that you see something like this, with emphasis on the three entries for 'Synced ldap user (1)' x 3. Each of these lines is a separate chunk being synced.

        Execute scheduled task: LDAP users sync job (auth_ldap\task\sync_task)
        ... started 06:03:08. Current memory use 15.7 MB.
        Connecting to LDAP server...
        Creating temporary table tmp_extuser
        ......Got 3 records from LDAP
         Synced ldap users (1)
         Synced ldap users (2)
         Synced ldap users (3)
        No user entries to be added... used 21 dbqueries
        ... used 6.1144211292267 seconds
        Scheduled task complete: LDAP users sync job (auth_ldap\task\sync_task)

      5. CONFIRM that there are no errors for the tasks logged.
      6. Go to Site admin -> Server -> Tasks -> Ad hoc tasks
      7. CONFIRM there are 3 tasks for auth_ldap
      Show
      LDAP server creation and Moodle configuration Run this command docker run --rm --env LDAP_ADMIN_USERNAME=admin --env LDAP_ADMIN_PASSWORD=password -p 1389 : 1389 bitnami/openldap Add the following entries to your config.php define('TEST_AUTH_LDAP_HOST_URL', 'ldap://localhost:1389'); define('TEST_AUTH_LDAP_BIND_DN', 'cn=admin,dc=example,dc=org'); define('TEST_AUTH_LDAP_BIND_PW', 'password'); define('TEST_AUTH_LDAP_DOMAIN', 'dc=example,dc=org'); Log in to Moodle as Admin Go to Site Admin -> Plugins -> Authentication -> Manage Authentication Enable 'LDAP server' Click on the settings link for LDAP server In the 'Host URL' field, enter in 'ldap://localhost:1389' (without the quotes) Set Version to 3 Under 'User lookup settings' look for the 'Contexts' field and enter 'ou=users,dc=example,dc=org' (without the quotes) Under 'User account synchronisation', CONFIRM there is a new field called 'Sync update users chunk size' Make this value '1' (without the quotes) Under 'Data mapping', enter into 'Data mapping (First name)' the value of 'givenName' (without quotes) Change 'Update local (First name)' to 'On every login' Change 'Update external (First name)' to 'On update' Leave all other values as the defaults on the page. Save changes. Go back to Site Admin -> Plugins -> Authentication -> Manage Authentication and click on LDAP's 'Test settings' link to ensure your connection is working. Create LDAP users In your CLI, use the following command to create some users. Increment all the values for each user. Do this for three users. When prompted for a password, use 'password'. This is the password that was provided in your config.php early on in the setup. ldapadd -x -H ldap: //localhost:1389 -D "cn=admin,dc=example,dc=org" -W <<EOF dn: cn=user01,ou=users,dc=example,dc=org objectClass: top objectClass: posixAccount objectClass: inetOrgPerson cn: user01 sn: Bar1 uid: user01 uidNumber: 1001 gidNumber: 1001 homeDirectory: /home/user01 userPassword: password EOF Testing scenario Now with all that setup out of the way, we should have Moodle configured to use LDAP authentication and our LDAP server ready to authenticate these users. Go to Moodle's login screen and log in with each user. (username: user01, password: password) Repeat this for all users your created (user02, user03) Run the following CLI command in your Moodle root directory: php admin/cli/scheduled_task.php --execute= 'auth_ldap\task\sync_task' CONFIRM that you see something like this, with emphasis on the three entries for 'Synced ldap user (1)' x 3. Each of these lines is a separate chunk being synced. Execute scheduled task: LDAP users sync job (auth_ldap\task\sync_task) ... started 06 : 03 : 08 . Current memory use 15.7 MB. Connecting to LDAP server... Creating temporary table tmp_extuser ......Got 3 records from LDAP  Synced ldap users ( 1 )  Synced ldap users ( 2 )  Synced ldap users ( 3 ) No user entries to be added... used 21 dbqueries ... used 6.1144211292267 seconds Scheduled task complete: LDAP users sync job (auth_ldap\task\sync_task) CONFIRM that there are no errors for the tasks logged. Go to Site admin -> Server -> Tasks -> Ad hoc tasks CONFIRM there are 3 tasks for auth_ldap
    • 2
    • Team Hedgehog 2023 Sprint 4.1, Team Hedgehog 2023 Sprint 4.2, Team Hedgehog 2023 Sprint 4.3, Team Hedgehog 2023 Review 4, Team Hedgehog 2024 Sprint 1.1

      sync_users() updates all qualifying users in one big transaction. This may take a very long time for large number of users. This is especially apparent when the LDAP server and the Moodle server are not geographically colocated.

      This patch will:

      1) Update users in chunks of configurable size

      2) When sync_users() is invoked from the sync_task it will delegate each chunk to an ad-hoc task, so the update is run in parallel

      The chunk size is configurable and you can control how many ad-hoc tasks are run using Moodle settings. This allows orgs to tune the processing time and resources.

            david.woloszyn@moodle.com David Woloszyn
            srdjan Srdjan Jankovic
            Meirza Meirza
            Ilya Tregubov Ilya Tregubov
            Kim Jared Lucas Kim Jared Lucas
            Votes:
            0 Vote for this issue
            Watchers:
            15 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 days, 5 hours, 41 minutes
                2d 5h 41m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.