Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-60174

get_recordset methods load entire result set into memory in Postgres

XMLWordPrintable

    • PostgreSQL
    • MOODLE_32_STABLE, MOODLE_33_STABLE, MOODLE_34_STABLE
    • MOODLE_35_STABLE
    • MDL-60174-master
    • Hide

      This test covers only the lines where I changed existing code (mainly to make it close recordsets). The idea is to exercise all those places in case some of them are missed by automated testing.

      It requires a Moodle setup that is configured to run Behat and with CLI access. You should probably be using Postgres.

      For the actual recordset behaviour, these tests will also exercise recordsets, but the unit tests (for this and for code that uses recordets) should be sufficient anyway.

      1. On the command line, run admin/cli/fix_deleted_users.php
        • Confirm script runs without error (it normally gives 2 lines of output and finishes)
      2. Visit admin/tool/messageinbound/index.php
        • Confirm page loads without error
      3. Visit admin/tool/spamcleaner/index.php
      4. Search for the keyword 'special' (this is usually found in the guest user profile)
        • Confirm page displays list of users without error
      5. Set up a test account with manager role at site level
      6. Log in as this test account and go to /cohort/
      7. Click on 'All cohorts' tab
        • Confirm the page displays with no error (it's ok if there aren't any cohorts)
      8. Log back in as administrator
      9. Go to or create an Assignment
      10. Go to the 'Advanced grading' page
      11. Change grading method to 'Marking guide'
      12. Define new grading form from scratch
      13. Fill in the name and the fields for the one existing criterion, then click 'save and make it ready'
      14. As a test student account on the course, submit something to the assignment
      15. Back as the administrator, view all submissions and click 'Grade' next to the test student
      16. Type in a message and score and save the grade
      17. Log in as the test student account and go to the assignment
        • Confirm the grading criteria shows (with the message and score) with no error
      18. Back as administrator, go to the gradebook for the course
      19. Click 'Single view' from the dropdown
        • Confirm page displays with no error (it's OK if it is blank)
      20. From the course settings, choose 'Reset'
      21. Under 'Groups', show more, then choose 'Delete all groups' and 'Delete all groupings'
      22. Press 'Reset course'
        • Confirm the 'Reset course' screen displays with no errors (you do not need to continue past this point)
      23. Go to the reset screen again.
      24. Under 'Groups', show more, then choose 'Remove all groups from groupings'
      25. Press 'Reset course'
        • Confirm the 'Reset course' screen displays with no errors (you do not need to continue past this point)
      26. Go to any course page
        • Confirm the course page displays with no errors (this is checking the load_blocks change)
      27. On the command line, run php admin/tool/behat/cli/util.php --drop
      28. Now run php admin/tool/behat/cli/init.php
        • Confirm the database tables installation completes successfully, ending with the line 'Acceptance tests environment enabled on...' (this checks the message/inbound/manager.php change)
      29. If using Boost, switch to another theme with normal navigation block such as Clean
      30. If you have only one course category, create a second category and put at least one course into it
      31. Go to the course page for a course in one category
      32. The other category should be visible in navigation, but not expanded. Click to expand it
        • Confirm the category expands successfully, showing the courses within it
      33. (You can now set the theme back, if you like.)
      34. Add a glossary to a course, with default settings
      35. Add one entry to the glossary
      36. Do a search for the entry you just added (ie search for its name)
        • Confirm there is no error displayed
      37. Create a new wiki with default settings
      38. Create the first page and save it
      39. Edit the page, change text and save again
      40. Go to Administration tab and choose Delete page versions
      41. Select version 2 and press Delete page versions button
        • Confirm the version was deleted (page goes back to previous text) and no error message
      42. From the course menu, go to the question bank Questions page
      43. Add a question (any type, true/false is easy) and save it
        • Confirm that the page displays OK listing the question you just added, with no error
      44. Hack the URL to change or set the parmater qpage=1, then reload the page
        • Confirm that the page is unchanged i.e. it basically ignored the invalid page number, with no error message
      45. Go to the 'Security overview' report (/report/security/)
      46. Click the 'Backup of user data' link
        • Confirm that the page displays with no error
      Show
      This test covers only the lines where I changed existing code (mainly to make it close recordsets). The idea is to exercise all those places in case some of them are missed by automated testing. It requires a Moodle setup that is configured to run Behat and with CLI access. You should probably be using Postgres. For the actual recordset behaviour, these tests will also exercise recordsets, but the unit tests (for this and for code that uses recordets) should be sufficient anyway. On the command line, run admin/cli/fix_deleted_users.php Confirm script runs without error (it normally gives 2 lines of output and finishes) Visit admin/tool/messageinbound/index.php Confirm page loads without error Visit admin/tool/spamcleaner/index.php Search for the keyword 'special' (this is usually found in the guest user profile) Confirm page displays list of users without error Set up a test account with manager role at site level Log in as this test account and go to /cohort/ Click on 'All cohorts' tab Confirm the page displays with no error (it's ok if there aren't any cohorts) Log back in as administrator Go to or create an Assignment Go to the 'Advanced grading' page Change grading method to 'Marking guide' Define new grading form from scratch Fill in the name and the fields for the one existing criterion, then click 'save and make it ready' As a test student account on the course, submit something to the assignment Back as the administrator, view all submissions and click 'Grade' next to the test student Type in a message and score and save the grade Log in as the test student account and go to the assignment Confirm the grading criteria shows (with the message and score) with no error Back as administrator, go to the gradebook for the course Click 'Single view' from the dropdown Confirm page displays with no error (it's OK if it is blank) From the course settings, choose 'Reset' Under 'Groups', show more, then choose 'Delete all groups' and 'Delete all groupings' Press 'Reset course' Confirm the 'Reset course' screen displays with no errors (you do not need to continue past this point) Go to the reset screen again. Under 'Groups', show more, then choose 'Remove all groups from groupings' Press 'Reset course' Confirm the 'Reset course' screen displays with no errors (you do not need to continue past this point) Go to any course page Confirm the course page displays with no errors (this is checking the load_blocks change) On the command line, run php admin/tool/behat/cli/util.php --drop Now run php admin/tool/behat/cli/init.php Confirm the database tables installation completes successfully, ending with the line 'Acceptance tests environment enabled on...' (this checks the message/inbound/manager.php change) If using Boost, switch to another theme with normal navigation block such as Clean If you have only one course category, create a second category and put at least one course into it Go to the course page for a course in one category The other category should be visible in navigation, but not expanded. Click to expand it Confirm the category expands successfully, showing the courses within it (You can now set the theme back, if you like.) Add a glossary to a course, with default settings Add one entry to the glossary Do a search for the entry you just added (ie search for its name) Confirm there is no error displayed Create a new wiki with default settings Create the first page and save it Edit the page, change text and save again Go to Administration tab and choose Delete page versions Select version 2 and press Delete page versions button Confirm the version was deleted (page goes back to previous text) and no error message From the course menu, go to the question bank Questions page Add a question (any type, true/false is easy) and save it Confirm that the page displays OK listing the question you just added, with no error Hack the URL to change or set the parmater qpage=1, then reload the page Confirm that the page is unchanged i.e. it basically ignored the invalid page number, with no error message Go to the 'Security overview' report (/report/security/) Click the 'Backup of user data' link Confirm that the page displays with no error

      The Moodle Docs state:

      Where the number of records to be retrieved from DB is high, the get_records_xxx() functions above are far from optimal, because they load all the records in memory at the same time. Under those circumstances, it is highly recommended to use these get_recordset_xxx() functions instead, which use one nice mechanism to iterate over all the target records and save a lot of memory.

       

      This is incorrect. In Postgres the get_recordset functions first load all results from the query returned by the database into PHP memory. An iterator of these results is then created.

      On large tables this causes an out of memory error. 

       

        1. find_naughty_recordset_code.php
          2 kB
        2. omgwtf2.php
          3 kB
        3. recordset_test.php
          2 kB

            quen Sam Marshall
            mattp@catalyst-au.net Matt Porritt
            Ryan Wyllie Ryan Wyllie
            Damyon Wiese Damyon Wiese
            Jake Dallimore Jake Dallimore
            Votes:
            4 Vote for this issue
            Watchers:
            22 Start watching this issue

              Created:
              Updated:
              Resolved:

                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.