Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-63887

sync_users function in auth_ldap module stuck infinite loop when reached search limit

XMLWordPrintable

    • MOODLE_35_STABLE
    • MOODLE_35_STABLE, MOODLE_36_STABLE
    • MDL-63887_master
    • Hide

      Reset ldap cookie value if limit exceeded.

      Show
      Reset ldap cookie value if limit exceeded.
    • Hide

      OpenLDAP container setup:

      1. Start up an openldap container.

      docker run --hostname ldap.example.org -it -p54600:389 osixia/openldap /bin/bash
      

      2. Now, from another terminal session, shell into the running container:
      First, run 'docker ps' and copy the container id hash
      Then, shell in using:

      docker exec -i -t  CONTAINERHASH /bin/bash
      

      3. Next, we'll need to add 3 users. Change directory to the test dir via:

      cd /container/service/slapd/assets/test/
      

      4. Copy the new-user.ldif file into 2 new files, 'new-user2.ldif' and 'new-user3.ldif'

      5. Modify new-user.ldif (there is vi installed on this container) and change userPassword so that line reads:
      userPassword: secret

      6. Modif new-user2.ldif and:

      • change all instances of 'billy' to 'jimmy'
      • increment the uidnumber by 1

      7. Do the same as above for new-user3.ldif - but use 'sally' as the name.

      8. Now, let's add the users and confirm they are present.

      ldapadd -x -D "cn=admin,dc=example,dc=org" -w admin -f /container/service/slapd/assets/test/new-user.ldif -H ldap://localhost -ZZ
      ldapadd -x -D "cn=admin,dc=example,dc=org" -w admin -f /container/service/slapd/assets/test/new-user2.ldif -H ldap://localhost -ZZ
      ldapadd -x -D "cn=admin,dc=example,dc=org" -w admin -f /container/service/slapd/assets/test/new-user3.ldif -H ldap://localhost -ZZ
      ldapsearch -x -H ldap://localhost -b dc=example,dc=org -D "cn=admin,dc=example,dc=org" -w admin
      

      Verify you see all 3 users returned in the search results before you continue with the test.

      9. Next we'll allow any user to perform a search. Again, create an ldif in the test directory:

      cd /container/service/slapd/assets/test/
      vi access_control_mod.ldif
      

      Put the following contents in the file and save:

      dn: olcDatabase={1}mdb,cn=config
      changetype: modify
      replace: olcAccess
      olcAccess: {0}to attrs=userPassword,shadowLastChange
              by self write
              by dn="cn=admin,dc=example,dc=org" write
              by anonymous auth
              by * none
      olcAccess: {1}to *
              by self read
              by dn="cn=admin,dc=example,dc=org" write
              by * read
      

      Note: the value "olcDatabase={1}mdb,cn=config" might differ on some setups. If the above doesn't work, find your DB by running:

      slapcat -b cn=config | grep 'olcDatabase={'

      and then using the entry for "olcDatabase={1}..." . It might be hdb, not mdb!

      and then run:

      ldapmodify -Y EXTERNAL -H ldapi:/// -f access_control_mod.ldif
      

      To confirm this worked, we'll need to verify you can search as 'billy'.

      ldapsearch -x -H ldap://localhost -b dc=example,dc=org -D "uid=billy,dc=example,dc=org" -w secret
      

      Verify you see the users in the results.

      10. Next, we'll set the search results limit to 2 (down from 500), so we hit it during testing when using a non-admin bind dn.
      Create the ldif file (assuming you're still in the test directory):

      vi search_size_limit.ldif
      

      and put the following contents in the file and save:

      dn: cn=config
      changetype: modify
      replace: olcSizeLimit
      olcSizeLimit: 2
       
      dn: olcDatabase={-1}frontend,cn=config
      changetype: modify
      replace: olcSizeLimit
      olcSizeLimit: 2
      

      and then run:

       ldapmodify -v -Y EXTERNAL -H ldapi:/// -f search_size_limit.ldif
      

      To confirm this worked, search again as 'billy':

      ldapsearch -x -H ldap://localhost -b dc=example,dc=org -D "uid=billy,dc=example,dc=org" -w secret
      

      Verify you see a limited set of users in the results (it might be only the organisation, example.org, and the admin account).
      numentries should read 2. Do not continue unless this is the case.

      Moodle LDAP setup

      1. Go to plugins -> auth -> manage auth and enabled the auth LDAP plugin.

      2. Edit the LDAP settings, setting the following values:

      LDAP server settings:
      host_url: ldap://localhost:54600
      ldap_version: 3
      pagesize: 1

      Bind settings:
      bind_dn: uid=billy,dc=example,dc=org
      bind_pw: secret

      User lookup settings:
      user_type: posixAccount (rfc2307)        Note: 'Default' also works here.
      contexts: dc=example,dc=org
      search_sub: Yes

      3. Save changes and go back to manage authentication

      4. Next to the LDAP plugin, click 'Test settings', and verify this says "Connecting to your LDAP server was successful". If not, check your configuration again.

      5. Log out.

      6. Log in as Sally using:
      sally:secret

      and complete her profile.

      7. Log out.

      Testing

      1. Log in as admin

      2. Edit LDAP settings and set the following:

      Data mapping:
      field_map_firstname: mail
      field_updatelocal_firstname: On every login
      field_updateremote_firstname: Never
      field_lock_firstname: Unlocked

      Save.

      3. (optional) This is the point at which you would be able to confirm the failure, by running the scheduled task below on an unpatched master branch. If you wish to do so, please run the following from the CLI, from :

      cd /var/www/html/YOURINSTANCE   (or wherever it is)
      php admin/tool/task/cli/schedule_task.php --execute=\\auth_ldap\\task\\sync_task
      

      Notice this continually runs? Control + c this process and checkout the patched branch again.

      4. Run the sync task:

      cd /var/www/html/YOURINSTANCE   (or wherever it is)
      php admin/tool/task/cli/schedule_task.php --execute=\\auth_ldap\\task\\sync_task
      

      5. Verify the task completes successfully and that you see the line 'updating user sally' in the output

      6. In Moodle, go to sally's profile.

      7. Verify her firstname has been changed to 'sally@example.com' - i.e. the sync task has run successfully.

      8. Now, log out of Sally's account.

      9. Log in as the admin user and edit the LDAP settings, setting 'pagesize' = 0.

      10. Run the sync_task again, and verify:
      a) You don't see any fatal errors and;
      b) The final lines of the output look something like:

      Did not get any users from LDAP -- error? -- exiting
      ... used 13 dbqueries
      ... used 0.030378103256226 seconds
      Scheduled task complete: LDAP users sync job (auth_ldap\task\sync_task)
      

      Show
      OpenLDAP container setup: 1. Start up an openldap container. docker run --hostname ldap.example.org -it -p54600:389 osixia/openldap /bin/bash 2. Now, from another terminal session, shell into the running container: First, run 'docker ps' and copy the container id hash Then, shell in using: docker exec -i -t CONTAINERHASH /bin/bash 3. Next, we'll need to add 3 users. Change directory to the test dir via: cd /container/service/slapd/assets/test/ 4. Copy the new-user.ldif file into 2 new files, 'new-user2.ldif' and 'new-user3.ldif' 5. Modify new-user.ldif (there is vi installed on this container) and change userPassword so that line reads: userPassword: secret 6. Modif new-user2.ldif and: change all instances of 'billy' to 'jimmy' increment the uidnumber by 1 7. Do the same as above for new-user3.ldif - but use 'sally' as the name. 8. Now, let's add the users and confirm they are present. ldapadd -x -D "cn=admin,dc=example,dc=org" -w admin -f /container/service/slapd/assets/test/new-user.ldif -H ldap://localhost -ZZ ldapadd -x -D "cn=admin,dc=example,dc=org" -w admin -f /container/service/slapd/assets/test/new-user2.ldif -H ldap://localhost -ZZ ldapadd -x -D "cn=admin,dc=example,dc=org" -w admin -f /container/service/slapd/assets/test/new-user3.ldif -H ldap://localhost -ZZ ldapsearch -x -H ldap://localhost -b dc=example,dc=org -D "cn=admin,dc=example,dc=org" -w admin Verify you see all 3 users returned in the search results before you continue with the test. 9. Next we'll allow any user to perform a search. Again, create an ldif in the test directory: cd /container/service/slapd/assets/test/ vi access_control_mod.ldif Put the following contents in the file and save: dn: olcDatabase={1}mdb,cn=config changetype: modify replace: olcAccess olcAccess: {0}to attrs=userPassword,shadowLastChange by self write by dn="cn=admin,dc=example,dc=org" write by anonymous auth by * none olcAccess: {1}to * by self read by dn="cn=admin,dc=example,dc=org" write by * read Note:  the value "olcDatabase={1}mdb,cn=config" might differ on some setups. If the above doesn't work, find your DB by running: slapcat -b cn=config | grep 'olcDatabase={' and then using the entry for "olcDatabase={1}..." . It might be hdb, not mdb! and then run: ldapmodify -Y EXTERNAL -H ldapi:/// -f access_control_mod.ldif To confirm this worked, we'll need to verify you can search as 'billy'. ldapsearch -x -H ldap://localhost -b dc=example,dc=org -D "uid=billy,dc=example,dc=org" -w secret Verify you see the users in the results. 10. Next, we'll set the search results limit to 2 (down from 500), so we hit it during testing when using a non-admin bind dn. Create the ldif file (assuming you're still in the test directory): vi search_size_limit.ldif and put the following contents in the file and save: dn: cn=config changetype: modify replace: olcSizeLimit olcSizeLimit: 2   dn: olcDatabase={-1}frontend,cn=config changetype: modify replace: olcSizeLimit olcSizeLimit: 2 and then run:  ldapmodify -v -Y EXTERNAL -H ldapi:/// -f search_size_limit.ldif To confirm this worked, search again as 'billy': ldapsearch -x -H ldap://localhost -b dc=example,dc=org -D "uid=billy,dc=example,dc=org" -w secret Verify you see a limited set of users in the results (it might be only the organisation, example.org, and the admin account). numentries should read 2. Do not continue unless this is the case. Moodle LDAP setup 1. Go to plugins -> auth -> manage auth and enabled the auth LDAP plugin. 2. Edit the LDAP settings, setting the following values: LDAP server settings: host_url: ldap://localhost:54600 ldap_version: 3 pagesize: 1 Bind settings: bind_dn: uid=billy,dc=example,dc=org bind_pw: secret User lookup settings: user_type: posixAccount (rfc2307)        Note: 'Default' also works here. contexts: dc=example,dc=org search_sub: Yes 3. Save changes and go back to manage authentication 4. Next to the LDAP plugin, click 'Test settings', and verify this says "Connecting to your LDAP server was successful". If not, check your configuration again. 5. Log out. 6. Log in as Sally using: sally:secret and complete her profile. 7. Log out. Testing 1. Log in as admin 2. Edit LDAP settings and set the following: Data mapping: field_map_firstname: mail field_updatelocal_firstname: On every login field_updateremote_firstname: Never field_lock_firstname: Unlocked Save. 3. (optional) This is the point at which you would be able to confirm the failure, by running the scheduled task below on an unpatched master branch. If you wish to do so, please run the following from the CLI, from : cd /var/www/html/YOURINSTANCE (or wherever it is) php admin/tool/task/cli/schedule_task.php --execute=\\auth_ldap\\task\\sync_task Notice this continually runs? Control + c this process and checkout the patched branch again. 4. Run the sync task: cd /var/www/html/YOURINSTANCE (or wherever it is) php admin/tool/task/cli/schedule_task.php --execute=\\auth_ldap\\task\\sync_task 5. Verify the task completes successfully and that you see the line 'updating user sally' in the output 6. In Moodle, go to sally's profile. 7. Verify her firstname has been changed to 'sally@example.com' - i.e. the sync task has run successfully. 8. Now, log out of Sally's account. 9. Log in as the admin user and edit the LDAP settings, setting 'pagesize' = 0. 10. Run the sync_task again, and verify : a) You don't see any fatal errors and; b) The final lines of the output look something like: Did not get any users from LDAP -- error? -- exiting ... used 13 dbqueries ... used 0.030378103256226 seconds Scheduled task complete: LDAP users sync job (auth_ldap\task\sync_task)

      When using OpenLDAP and bind non admin user, auth_ldap's sync_users function stuck loop. Because non admin bind user is affected search size limit (default value: 500). If size limit exceeded ldap_control_paged_result_response function return false but nothing check this return value so enclose process in the loop.

            papillon326 Mitsuru Udagawa
            papillon326 Mitsuru Udagawa
            Mark Nelson Mark Nelson
            Jake Dallimore Jake Dallimore
            Mihail Geshoski Mihail Geshoski
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved:

                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.