Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.2.4
    • Fix Version/s: 2.4
    • Component/s: Enrolments
    • Labels:
      None
    • Testing Instructions:
      Hide

      1/ set up external database enrolment sync
      2/ create invalid enrolment records with following problems - invalid course id, missing user id, invalid user id
      3/ execute CLIsync with --verbose
      4/ verify the appropriate error messages are displayed for each invalid enrolment record (one problem report per record only)

      Show
      1/ set up external database enrolment sync 2/ create invalid enrolment records with following problems - invalid course id, missing user id, invalid user id 3/ execute CLIsync with --verbose 4/ verify the appropriate error messages are displayed for each invalid enrolment record (one problem report per record only)
    • Affected Branches:
      MOODLE_22_STABLE
    • Fixed Branches:
      MOODLE_24_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      w38_MDL-35331_m24_verboseenroldb
    • Rank:
      44003

      Description

      Hi there,

      I noticed that when we went with 2.2.4+, the notification of courses with no enrolments do not show up in the list as we had when we had 1.9.14+.

      In 2.2.4+, we saw that the list of courses only show up when there are enrolments assigned to them. In 1.9.14+ we had both (course that have enrolments and course that don't).

      We prefer the 1.9.14+ info instead ... more detailed and all in one run

      We want to know :

      1. Which courses have synchronized users
      2. Which courses do not exist in moodle that are in our db enrolment datastore
      3. which user accounts are sync with existing moodle users already in moodle
      4. which new user accounts are created during the sync which were not already in moodle.

      The 1.9.14+ tells us all this ... not the 2.2.x version.

      2.2.4+ EXAMPLES:

      Courses WITH NO enrolments attached to courses:
      ----------

      Course synchronisation skipped.
      Starting user enrolment synchronisation...
      ...user enrolment synchronisation finished.

      Course WITH enrolments attached to courses:
      ----------

      Course synchronisation skipped.
      Starting user enrolment synchronisation...
      enrolling: 60124 ==> HOSP 3999: Advanced Sales Skills & Management (Fall 2012) as student
      enrolling: 60123 ==> HOSP 3243: Advanced Sales Skills & Management (WINTER 2010) as student
      enrolling: 60124 ==> HOSP3102 - Fall 2012 as student
      enrolling: 60123 ==> HOSP3102 - Fall 2012 as student
      enrolling: 8189 ==> HOSP 4151 as student
      enrolling: 60124 ==> HOSP 4151 as student
      enrolling: 60123 ==> HOSP 4151 as student
      enrolling: 60123 ==> HOSP 3222 as student
      enrolling: 8189 ==> HOSP 3222 as student
      enrolling: 60124 ==> HOSP 3222 as student
      enrolling: 42514 ==> HOSP 1206 as student
      enrolling: 42514 ==> HOSP 1212 as student
      enrolling: 60126 ==> HOSP1101 as student
      enrolling: 60151 ==> HOSP1101 as student
      enrolling: 60142 ==> HOSP1101 as student
      enrolling: 60126 ==> HOSP 1104 as student
      enrolling: 60151 ==> HOSP 1104 as student
      enrolling: 60142 ==> HOSP 1104 as student
      enrolling: 60126 ==> HOSP 1106 as student
      enrolling: 60261 ==> HOSP 1106 as student
      enrolling: 60151 ==> HOSP 1106 as student
      enrolling: 60142 ==> HOSP 1106 as student
      enrolling: 45196 ==> HOSP 1107 as student
      enrolling: 60151 ==> HOSP 1107 as student
      enrolling: 25468 ==> HOSP 1107 as student
      enrolling: 60142 ==> HOSP 1107 as student
      enrolling: 60126 ==> HOSP 1107 as student
      enrolling: 60261 ==> HOSP 1107 as student
      enrolling: 60126 ==> HOSP 1103 as student
      enrolling: 60151 ==> HOSP 1103 as student
      enrolling: 60142 ==> HOSP 1103 as student
      enrolling: 3866 ==> HOSP 4260 [Kelleher] as student
      enrolling: 3866 ==> HOSP4263_EXCO as student
      enrolling: 3866 ==> HOSP 4264 as student
      enrolling: 3866 ==> HOSP 4153 as student
      enrolling: 54266 ==> PHYS 3001 (70197) as student
      enrolling: 17789 ==> PHYS 3001 (70197) as student
      enrolling: 39862 ==> PHYS 3001 (70197) as student
      enrolling: 15279 ==> PHYS 3001 (70197) as student
      enrolling: 54402 ==> PHYS 3001 (70197) as student
      enrolling: 4151 ==> PHYS 3001 (70197) as student
      enrolling: 55327 ==> PHYS 3001 (70197) as student
      ...user enrolment synchronisation finished.

      ----------
      1.9.14+ EXAMPLES (Preferred as it shows all activity):
      ----------

      === Syncing enrolments for role: admin ===
      === Syncing enrolments for role: coursemgr ===
      === Syncing enrolments for role: editingteacher ===
      === Syncing enrolments for role: teacher ===
      === Syncing enrolments for role: student ===
      Course 20127070153 does not exist, skipping
      Course 20127070154 does not exist, skipping
      Course 20127070197 does not exist, skipping
      Course 20127070203 does not exist, skipping
      Course 20127071039 does not exist, skipping
      Course 20127073587 does not exist, skipping
      Course 20127073588 does not exist, skipping
      Course 20127073597 does not exist, skipping
      Course 20127074155 does not exist, skipping
      Course 20127074156 does not exist, skipping
      Course 20127074157 does not exist, skipping
      Course 20127074158 does not exist, skipping
      Course 20127074159 does not exist, skipping
      Course 20127074160 does not exist, skipping
      Course 20127074178 does not exist, skipping
      Course 20127074179 does not exist, skipping
      Course 20127074180 does not exist, skipping
      Assigned role student to user 42514 in course 309 (HOSP1206 )
      Assigned role student to user 42514 in course 307 (HOSP1212)
      Course 20127074187 does not exist, skipping
      Course 20127074188 does not exist, skipping
      Course 20127074189 does not exist, skipping
      Course 20127074190 does not exist, skipping
      Course 20127074191 does not exist, skipping
      PHP Notice: weird! no user record entry? in /var/www/html/moodle/enrol/database/enrol.php on line 345
      PHP Notice: weird! no user record entry? in /var/www/html/moodle/enrol/database/enrol.php on line 345
      PHP Notice: weird! no user record entry? in /var/www/html/moodle/enrol/database/enrol.php on line 345
      PHP Notice: weird! no user record entry? in /var/www/html/moodle/enrol/database/enrol.php on line 345
      Assigned role student to user 59875 in course 292 (HOSP1101)
      Assigned role student to user 59866 in course 292 (HOSP1101)
      PHP Notice: weird! no user record entry? in /var/www/html/moodle/enrol/database/enrol.php on line 345
      PHP Notice: weird! no user record entry? in /var/www/html/moodle/enrol/database/enrol.php on line 345
      PHP Notice: weird! no user record entry? in /var/www/html/moodle/enrol/database/enrol.php on line 345
      Assigned role student to user 59875 in course 315 (HOSP1104)
      Assigned role student to user 59866 in course 315 (HOSP1104)
      PHP Notice: weird! no user record entry? in /var/www/html/moodle/enrol/database/enrol.php on line 345
      PHP Notice: weird! no user record entry? in /var/www/html/moodle/enrol/database/enrol.php on line 345
      PHP Notice: weird! no user record entry? in /var/www/html/moodle/enrol/database/enrol.php on line 345
      PHP Notice: weird! no user record entry? in /var/www/html/moodle/enrol/database/enrol.php on line 345
      Assigned role student to user 59875 in course 2490 (HOSP1106 )
      Assigned role student to user 59866 in course 2490 (HOSP1106 )
      Assigned role student to user 45196 in course 2491 (HOSP1107 )
      Assigned role student to user 59875 in course 2491 (HOSP1107 )
      Assigned role student to user 25468 in course 2491 (HOSP1107 )
      Assigned role student to user 59866 in course 2491 (HOSP1107 )
      PHP Notice: weird! no user record entry? in /var/www/html/moodle/enrol/database/enrol.php on line 345
      PHP Notice: weird! no user record entry? in /var/www/html/moodle/enrol/database/enrol.php on line 345
      Assigned role student to user 59875 in course 293 (HOSP1103 )
      Assigned role student to user 59866 in course 293 (HOSP1103 )
      Course 20127074197 does not exist, skipping
      Course 20127074202 does not exist, skipping
      Course 20127074203 does not exist, skipping
      Course 20127074204 does not exist, skipping
      Course 20127074205 does not exist, skipping
      Course 20127074206 does not exist, skipping
      Course 20127074213 does not exist, skipping
      Course 20127074214 does not exist, skipping
      Course 20127074215 does not exist, skipping
      Course 20127074216 does not exist, skipping
      Course 20127074217 does not exist, skipping
      Course 20127074218 does not exist, skipping
      Course 20127074236 does not exist, skipping
      Course 20127074237 does not exist, skipping
      Course 20127074238 does not exist, skipping
      Course 20127074239 does not exist, skipping
      Course 20127074240 does not exist, skipping
      Course 20127074241 does not exist, skipping
      Course 20127074338 does not exist, skipping
      Course 20127074770 does not exist, skipping
      Course 20127074772 does not exist, skipping
      Course 20127074773 does not exist, skipping
      Course 20127074774 does not exist, skipping
      Course 20127074775 does not exist, skipping
      Course 20127074776 does not exist, skipping
      Course 20127074777 does not exist, skipping
      Course 20127074778 does not exist, skipping
      Course 20127074779 does not exist, skipping
      Course 20127074780 does not exist, skipping
      Course 20127074781 does not exist, skipping
      Course 20127074785 does not exist, skipping
      Course 20127074786 does not exist, skipping
      === Syncing enrolments for role: guest ===
      === Syncing enrolments for role: user ===
      === Syncing enrolments for role: course_administrator ===
      === Syncing enrolments for role: admin2 ===
      === Syncing enrolments for role: ots_teacher ===
      === Syncing enrolments for role: guest2 ===
      === Syncing enrolments for role: mentor ===
      === Syncing enrolments for role: ace-it ===
      === Syncing enrolments for role: question_sharer ===
      === Syncing enrolments for role: editingteacher_1 ===
      === Syncing enrolments for role: courseidnumbereditorgroup ===
      === Syncing enrolments for role: fprc_member ===

        Activity

        Hide
        Petr Škoda added a comment -

        Hello, I do not understand what information you are missing exactly in the 2.x verbose output - internally it is not the same course, the output is not supposed to be the same. It works fine for me if the external database contains valid data. Please make sure there are no case sensitiveness problems, strange duplicates, non-existent course ids, etc. I am going to add unit tests in 2.4dev soon, the handling of invalid data will be probably improved too.

        Show
        Petr Škoda added a comment - Hello, I do not understand what information you are missing exactly in the 2.x verbose output - internally it is not the same course, the output is not supposed to be the same. It works fine for me if the external database contains valid data. Please make sure there are no case sensitiveness problems, strange duplicates, non-existent course ids, etc. I am going to add unit tests in 2.4dev soon, the handling of invalid data will be probably improved too.
        Hide
        Lawrence N added a comment -

        Hi Petr,

        just as I explained above, when I do an external course enrolment via DB, the results I dumped into a file is different using the same data if I were to run 2.2.4+ vs. 1.9.14

        For example:

        using the same external database enrolment records in our MySQL and run theme both as 1.9.14+ and 2.2.4+ enrolment script:

        In 2.2.4+, if there is no course defined for external enrolment, the 2.2.4+ script won't show that the "course" does not exist in moodle. This is to help us identify that our teachers have not set it up correctly in their 2.2.4+ course.

        In 1.9.14+ however, if there is no course defined in moolde, it will say that "Course XXXXXX does not exist, skipping"

        We just want to make it look like the 1.9.14+ that's all..

        Show
        Lawrence N added a comment - Hi Petr, just as I explained above, when I do an external course enrolment via DB, the results I dumped into a file is different using the same data if I were to run 2.2.4+ vs. 1.9.14 For example: using the same external database enrolment records in our MySQL and run theme both as 1.9.14+ and 2.2.4+ enrolment script: In 2.2.4+, if there is no course defined for external enrolment, the 2.2.4+ script won't show that the "course" does not exist in moodle. This is to help us identify that our teachers have not set it up correctly in their 2.2.4+ course. In 1.9.14+ however, if there is no course defined in moolde, it will say that "Course XXXXXX does not exist, skipping" We just want to make it look like the 1.9.14+ that's all..
        Hide
        Petr Škoda added a comment - - edited

        ok, so you want:
        1/ display a warning when external database contains bogus course information in external user enrolments table - ok, that makes sense

        anything more?

        Show
        Petr Škoda added a comment - - edited ok, so you want: 1/ display a warning when external database contains bogus course information in external user enrolments table - ok, that makes sense anything more?
        Hide
        Lawrence N added a comment - - edited

        Yes Petr.

        Technically, the enrolment table has the enrolment information and is not bogus. It's just that the course in moodle, which we use the course number id as the mapped field does not exist and so it fails.

        Show all enrolments that are synch (already in place) and users that were in that course
        Show all missing/invalid courses that do not exist in moodle during the sync (let us know which ones are not mapped yet)
        Show all new user accounts created from the synch which were only created during the sync (i.e users are not in moodle)

        This is should be it for our 2.2.4+ build

        Thanks

        Show
        Lawrence N added a comment - - edited Yes Petr. Technically, the enrolment table has the enrolment information and is not bogus. It's just that the course in moodle, which we use the course number id as the mapped field does not exist and so it fails. Show all enrolments that are synch (already in place) and users that were in that course Show all missing/invalid courses that do not exist in moodle during the sync (let us know which ones are not mapped yet) Show all new user accounts created from the synch which were only created during the sync (i.e users are not in moodle) This is should be it for our 2.2.4+ build Thanks
        Hide
        Petr Škoda added a comment -

        Show all enrolments that are synch (already in place) and users that were in that course - I disagree with this, it should imo print only changes and problems

        Show all missing/invalid courses that do not exist in moodle during the sync (let us know which ones are not mapped yet) - that is my 1/ about, right?

        Show all new user accounts created from the synch which were only created during the sync (i.e users are not in moodle) - user account are not created in enrolment sync at all, only enrolments are, I will make sure that unknown users ids are printed if that is what you want.

        Show
        Petr Škoda added a comment - Show all enrolments that are synch (already in place) and users that were in that course - I disagree with this, it should imo print only changes and problems Show all missing/invalid courses that do not exist in moodle during the sync (let us know which ones are not mapped yet) - that is my 1/ about, right? Show all new user accounts created from the synch which were only created during the sync (i.e users are not in moodle) - user account are not created in enrolment sync at all, only enrolments are, I will make sure that unknown users ids are printed if that is what you want.
        Hide
        Lawrence N added a comment -

        Petr,

        I did not realize that for 2.2.4+, the enrolment synchronization script does not create the user account if it does not already already exist in the mdl_user table.

        If I'm not mistaken, it does for 1.9.14+ and that's why we love it. If it's not there, it would automatically create it and users don't have to log in at least once for moodle to create the course.

        Show
        Lawrence N added a comment - Petr, I did not realize that for 2.2.4+, the enrolment synchronization script does not create the user account if it does not already already exist in the mdl_user table. If I'm not mistaken, it does for 1.9.14+ and that's why we love it. If it's not there, it would automatically create it and users don't have to log in at least once for moodle to create the course.
        Hide
        Petr Škoda added a comment -

        Enrolment plugins are not supposed to create new accounts, only auth plugins do that.

        Show
        Petr Škoda added a comment - Enrolment plugins are not supposed to create new accounts, only auth plugins do that.
        Hide
        Lawrence N added a comment -

        Ah I understand... it just puts them into the enrolment table and next time should they log in, the accounts would match and thus, the account is created upon first log in.

        Show
        Lawrence N added a comment - Ah I understand... it just puts them into the enrolment table and next time should they log in, the accounts would match and thus, the account is created upon first log in.
        Hide
        Petr Škoda added a comment -

        No, you need 3 external tables - users, course and enrolments. The auth/db handles user creation, it should be executed first. Them from enrol/database the course creation is executed and last the user enrolments.

        Show
        Petr Škoda added a comment - No, you need 3 external tables - users, course and enrolments. The auth/db handles user creation, it should be executed first. Them from enrol/database the course creation is executed and last the user enrolments.
        Hide
        Lawrence N added a comment -

        Yes I know there are 3 tables... I believe even if the use does not exist in the mdl_user table, that user's record id will still be in enrolment table

        Show
        Lawrence N added a comment - Yes I know there are 3 tables... I believe even if the use does not exist in the mdl_user table, that user's record id will still be in enrolment table
        Hide
        Petr Škoda added a comment -

        Absolutely not, the user must be first in user table and only then they can be enrolled.

        Show
        Petr Škoda added a comment - Absolutely not, the user must be first in user table and only then they can be enrolled.
        Hide
        Petr Škoda added a comment -

        This patch should help with diagnostics of database enrol sync in 2.4. It is not supposed to replicate the old 1.9 behaviour, but hopefully all the problems are detected now.

        I decided to not backport this to current stable branches because this is a new feature and there would need to be a lot more testing. I am going to work on a full suite of unit tests in master soon, that should finally detect all problems.

        Thanks for the report!

        Show
        Petr Škoda added a comment - This patch should help with diagnostics of database enrol sync in 2.4. It is not supposed to replicate the old 1.9 behaviour, but hopefully all the problems are detected now. I decided to not backport this to current stable branches because this is a new feature and there would need to be a lot more testing. I am going to work on a full suite of unit tests in master soon, that should finally detect all problems. Thanks for the report!
        Hide
        Lawrence N added a comment -

        Thank you Petr.

        I will implement the patch and have a go at running it.

        Thanks for all your work and attention to this. I / We appreciate it.

        Show
        Lawrence N added a comment - Thank you Petr. I will implement the patch and have a go at running it. Thanks for all your work and attention to this. I / We appreciate it.
        Hide
        Lawrence N added a comment -

        This works great Petr,

        Now only if someone could look at the other serious bug I reported

        http://tracker.moodle.org/browse/MDL-32572

        Show
        Lawrence N added a comment - This works great Petr, Now only if someone could look at the other serious bug I reported http://tracker.moodle.org/browse/MDL-32572
        Hide
        Eloy Lafuente (stronk7) added a comment -

        Looks ok going to integrate it now.

        Offtopic: Just guessing if we could build some PHPUnit support for mtrace(), very similar to the recently introduced support for debugging() @ MDL-35479. That would enable embed cli/cron testing easily.

        Show
        Eloy Lafuente (stronk7) added a comment - Looks ok going to integrate it now. Offtopic: Just guessing if we could build some PHPUnit support for mtrace(), very similar to the recently introduced support for debugging() @ MDL-35479 . That would enable embed cli/cron testing easily.
        Hide
        Eloy Lafuente (stronk7) added a comment -

        Integrated (master only), thanks!

        Show
        Eloy Lafuente (stronk7) added a comment - Integrated (master only), thanks!
        Hide
        Petr Škoda added a comment -

        the mtrace thing should be most probably replaced by renderes - then you could add verbose renderer, silent or phpunit one

        Show
        Petr Škoda added a comment - the mtrace thing should be most probably replaced by renderes - then you could add verbose renderer, silent or phpunit one
        Hide
        Frédéric Massart added a comment -

        Test passed. Cheers!

        Show
        Frédéric Massart added a comment - Test passed. Cheers!
        Hide
        Lawrence N added a comment -

        I also test this myself and it looks good to go as well. Now I can tell which ones were invalid.

        Thanks for looking into this and my other submission (MDL-32572).

        I appreciate how quickly everyone worked to fixed this in a short time.

        Show
        Lawrence N added a comment - I also test this myself and it looks good to go as well. Now I can tell which ones were invalid. Thanks for looking into this and my other submission ( MDL-32572 ). I appreciate how quickly everyone worked to fixed this in a short time.
        Hide
        Petr Škoda added a comment -

        Thanks for testing!

        Show
        Petr Škoda added a comment - Thanks for testing!
        Hide
        Lawrence N added a comment -

        No. THANK YOU Petr & testing team.

        Show
        Lawrence N added a comment - No. THANK YOU Petr & testing team.
        Hide
        Lawrence N added a comment -

        Now let's hope moodle fixes the file picker with JAWS reader...

        http://tracker.moodle.org/browse/MDL-27178

        feel bad for our visually impaired students

        Show
        Lawrence N added a comment - Now let's hope moodle fixes the file picker with JAWS reader... http://tracker.moodle.org/browse/MDL-27178 feel bad for our visually impaired students
        Hide
        Dan Poltawski added a comment -

        Congratulations, you've done it!

        Thanks, this change is now in the latest weekly release!

        Join the crowds of people tomorrow from 8am and download this Moodle release from your local apple store!

        Show
        Dan Poltawski added a comment - Congratulations, you've done it! Thanks, this change is now in the latest weekly release! Join the crowds of people tomorrow from 8am and download this Moodle release from your local apple store!

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: