Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-40581

backupidsexists and backupidscache could be used more effectively

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Waiting for integration review
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.4.5, 2.5, 3.9.5, 3.10.2, 4.0
    • Fix Version/s: None
    • Component/s: Backup, Performance
    • Testing Instructions:
      Hide
      1. Automated: Pass all PHPUnit tests on ALL databases.

      A) Before patch

      1. Switch to moodle.git branch (or 1 week old integration.git branch, anything not having the patch installed).
      2. Install a site in the selected branch.
      3. Copy the file backup_restore_test.php into backup/tests/ folder, and run the phpunit test  to compare performance between master and new branch.
      4. Run commands (note that it can take a while!):

        $ php admin/tool/phpunit/cli/init.php
        $ vendor/bin/phpunit backup/tests/backup_restore_test.php
        

      5. Annotate results

      B) After patch

      1. Switch to integration.git branch, with the patch applied.
      2. Repeat the steps 2-5 in previous section

      C) Compare results

      1. Put the before and after results face to face and verify that, for all the amplifications (1, 5, 10, 20);
        1. The "Finish backup - time" is always smaller after patch, with differences becoming bigger for growing amplifications.
        2. The "Finish restore - time" is always smaller (better) after patch, with differences becoming bigger for growing amplifications.
        3. The "Memory after backup" is always bigger (worse) after patch, with differences becoming bigger for growing amplifications.
        4. The "Memory after restore" is always bigger (worse) after patch, with differences becoming bigger for growing amplifications.
        5. The total "Time" to run the unit test is smaller (better) after the patch.
        6. The total "Memory" to run the unit test is bigger (worse) after the patch.
      Show
      Automated: Pass all PHPUnit tests on ALL databases. A) Before patch Switch to moodle.git branch (or 1 week old integration.git branch, anything not having the patch installed). Install a site in the selected branch. Copy the file backup_restore_test.php into backup/tests/ folder, and run the phpunit test  to compare performance between master and new branch. Run commands (note that it can take a while!): $ php admin/tool/phpunit/cli/init.php $ vendor/bin/phpunit backup/tests/backup_restore_test.php Annotate results B) After patch Switch to integration.git branch, with the patch applied. Repeat the steps 2-5 in previous section C) Compare results Put the before and after results face to face and verify that , for all the amplifications (1, 5, 10, 20); The "Finish backup - time" is always smaller after patch, with differences becoming bigger for growing amplifications. The "Finish restore - time" is always smaller (better) after patch, with differences becoming bigger for growing amplifications. The "Memory after backup" is always bigger (worse) after patch, with differences becoming bigger for growing amplifications. The "Memory after restore" is always bigger (worse) after patch, with differences becoming bigger for growing amplifications. The total "Time" to run the unit test is smaller (better) after the patch. The total "Memory" to run the unit test is bigger (worse) after the patch.
    • Affected Branches:
      MOODLE_24_STABLE, MOODLE_25_STABLE, MOODLE_310_STABLE, MOODLE_39_STABLE, MOODLE_400_STABLE
    • Pull Master Branch:
      MDL-40581_master_v7

      Description

      The restore process uses backupidsexists and backupidscache to store the existence and data from the database to reduce overhead when importing. The cache was installed as part of MDL-27120. It is slanted to be replaced by MUC.

      This issue will explore the performance benefit/cost of MUC and also improvements to the way these caches are used to reduce database calls.

      This will provide performance improvements for restore and for activity duplication.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              tomotsuyuki Tomo Tsuyuki
              Reporter:
              mr-russ Russell Smith
              Peer reviewer:
              Mark Nelson Mark Nelson
              Participants:
              Component watchers:
              Adrian Greeve, Jake Dallimore, Mathew May, Mihail Geshoski, Peter Dias, Sujith Haridasan, Matteo Scaramuccia, Andrew Lyons, Dongsheng Cai, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              3 Vote for this issue
              Watchers:
              16 Start watching this issue

                Dates

                Created:
                Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 hour, 30 minutes
                  1h 30m