Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-60915

get_recordset methods load entire result set into memory in Postgres (backport of MDL-60174)

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide

      (from original issue)

      This test covers only the lines where I changed existing code (mainly to make it close recordsets). The idea is to exercise all those places in case some of them are missed by automated testing.

      It requires a Moodle setup that is configured to run Behat and with CLI access. You should probably be using Postgres.

      A) Default run, without any specific configuration (fetchbuffersize = 0)

      For the actual recordset behaviour, these tests will also exercise recordsets, but the unit tests (for this and for code that uses recordets) should be sufficient anyway.

      1. On the command line, run admin/cli/fix_deleted_users.php
        • Confirm script runs without error (it normally gives 2 lines of output and finishes)
      2. Visit admin/tool/messageinbound/index.php
        • Confirm page loads without error
      3. Visit admin/tool/spamcleaner/index.php
      4. Search for the keyword 'special' (this is usually found in the guest user profile)
        • Confirm page displays list of users without error
      5. Set up a test account with manager role at site level
      6. Log in as this test account and go to /cohort/
      7. Click on 'All cohorts' tab
        • Confirm the page displays with no error (it's ok if there aren't any cohorts)
      8. Log back in as administrator
      9. Go to or create an Assignment
      10. Go to the 'Advanced grading' page
      11. Change grading method to 'Marking guide'
      12. Define new grading form from scratch
      13. Fill in the name and the fields for the one existing criterion, then click 'save and make it ready'
      14. As a test student account on the course, submit something to the assignment
      15. Back as the administrator, view all submissions and click 'Grade' next to the test student
      16. Type in a message and score and save the grade
      17. Log in as the test student account and go to the assignment
        • Confirm the grading criteria shows (with the message and score) with no error
      18. Back as administrator, go to the gradebook for the course
      19. Click 'Single view' from the dropdown
        • Confirm page displays with no error (it's OK if it is blank)
      20. From the course settings, choose 'Reset'
      21. Under 'Groups', show more, then choose 'Delete all groups' and 'Delete all groupings'
      22. Press 'Reset course'
        • Confirm the 'Reset course' screen displays with no errors (you do not need to continue past this point)
      23. Go to the reset screen again.
      24. Under 'Groups', show more, then choose 'Remove all groups from groupings'
      25. Press 'Reset course'
        • Confirm the 'Reset course' screen displays with no errors (you do not need to continue past this point)
      26. Go to any course page
        • Confirm the course page displays with no errors (this is checking the load_blocks change)
      27. On the command line, run php admin/tool/behat/cli/util.php --drop
      28. Now run php admin/tool/behat/cli/init.php
        • Confirm the database tables installation completes successfully, ending with the line 'Acceptance tests environment enabled on...' (this checks the message/inbound/manager.php change)
      29. If using Boost, switch to another theme with normal navigation block such as Clean
      30. If you have only one course category, create a second category and put at least one course into it
      31. Go to the course page for a course in one category
      32. The other category should be visible in navigation, but not expanded. Click to expand it
        • Confirm the category expands successfully, showing the courses within it
      33. (You can now set the theme back, if you like.)
      34. Add a glossary to a course, with default settings
      35. Add one entry to the glossary
      36. Do a search for the entry you just added (ie search for its name)
        • Confirm there is no error displayed
      37. Create a new wiki with default settings
      38. Create the first page and save it
      39. Edit the page, change text and save again
      40. Go to Administration tab and choose Delete page versions
      41. Select version 2 and press Delete page versions button
        • Confirm the version was deleted (page goes back to previous text) and no error message
      42. From the course menu, go to the question bank Questions page
      43. Add a question (any type, true/false is easy) and save it
        • Confirm that the page displays OK listing the question you just added, with no error
      44. Hack the URL to change or set the parmater qpage=1, then reload the page
        • Confirm that the page is unchanged i.e. it basically ignored the invalid page number, with no error message
      45. Go to the 'Security overview' report (/report/security/)
      46. Click the 'Backup of user data' link
        • Confirm that the page displays with no error

      B) Custom run, with an extreme fetch buffer in action.

      1. Set $CFG->dboptions['fetchbuffersize'] = 5; in your config.php file. Note this is an extreme value, not suitable for production, but will make the new code to be exercised a lot.
      2. Repeat all the steps in the previous A) section.
      Show
      (from original issue) This test covers only the lines where I changed existing code (mainly to make it close recordsets). The idea is to exercise all those places in case some of them are missed by automated testing. It requires a Moodle setup that is configured to run Behat and with CLI access. You should probably be using Postgres. A) Default run, without any specific configuration (fetchbuffersize = 0) For the actual recordset behaviour, these tests will also exercise recordsets, but the unit tests (for this and for code that uses recordets) should be sufficient anyway. On the command line, run admin/cli/fix_deleted_users.php Confirm script runs without error (it normally gives 2 lines of output and finishes) Visit admin/tool/messageinbound/index.php Confirm page loads without error Visit admin/tool/spamcleaner/index.php Search for the keyword 'special' (this is usually found in the guest user profile) Confirm page displays list of users without error Set up a test account with manager role at site level Log in as this test account and go to /cohort/ Click on 'All cohorts' tab Confirm the page displays with no error (it's ok if there aren't any cohorts) Log back in as administrator Go to or create an Assignment Go to the 'Advanced grading' page Change grading method to 'Marking guide' Define new grading form from scratch Fill in the name and the fields for the one existing criterion, then click 'save and make it ready' As a test student account on the course, submit something to the assignment Back as the administrator, view all submissions and click 'Grade' next to the test student Type in a message and score and save the grade Log in as the test student account and go to the assignment Confirm the grading criteria shows (with the message and score) with no error Back as administrator, go to the gradebook for the course Click 'Single view' from the dropdown Confirm page displays with no error (it's OK if it is blank) From the course settings, choose 'Reset' Under 'Groups', show more, then choose 'Delete all groups' and 'Delete all groupings' Press 'Reset course' Confirm the 'Reset course' screen displays with no errors (you do not need to continue past this point) Go to the reset screen again. Under 'Groups', show more, then choose 'Remove all groups from groupings' Press 'Reset course' Confirm the 'Reset course' screen displays with no errors (you do not need to continue past this point) Go to any course page Confirm the course page displays with no errors (this is checking the load_blocks change) On the command line, run php admin/tool/behat/cli/util.php --drop Now run php admin/tool/behat/cli/init.php Confirm the database tables installation completes successfully, ending with the line 'Acceptance tests environment enabled on...' (this checks the message/inbound/manager.php change) If using Boost, switch to another theme with normal navigation block such as Clean If you have only one course category, create a second category and put at least one course into it Go to the course page for a course in one category The other category should be visible in navigation, but not expanded. Click to expand it Confirm the category expands successfully, showing the courses within it (You can now set the theme back, if you like.) Add a glossary to a course, with default settings Add one entry to the glossary Do a search for the entry you just added (ie search for its name) Confirm there is no error displayed Create a new wiki with default settings Create the first page and save it Edit the page, change text and save again Go to Administration tab and choose Delete page versions Select version 2 and press Delete page versions button Confirm the version was deleted (page goes back to previous text) and no error message From the course menu, go to the question bank Questions page Add a question (any type, true/false is easy) and save it Confirm that the page displays OK listing the question you just added, with no error Hack the URL to change or set the parmater qpage=1, then reload the page Confirm that the page is unchanged i.e. it basically ignored the invalid page number, with no error message Go to the 'Security overview' report (/report/security/) Click the 'Backup of user data' link Confirm that the page displays with no error B) Custom run, with an extreme fetch buffer in action. Set $CFG->dboptions ['fetchbuffersize'] = 5; in your config.php file. Note this is an extreme value, not suitable for production, but will make the new code to be exercised a lot. Repeat all the steps in the previous A) section.
    • Affected Branches:
      MOODLE_33_STABLE, MOODLE_34_STABLE
    • Fixed Branches:
      MOODLE_34_STABLE
    • Pull from Repository:

      Description

      This is a backport of MDL-60174. The rationale for backport is that MDL-60174 is a bug fix, but was slightly risky and had some implications for admins (the change is incompatible with pgbouncer + transaction pooling). We have never said we support pgbouncer, but we know it is used already. Moodle cloud is not affected (uses pgbouncer with session pooling which is fine).

       

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                2 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  9/Jul/18