Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-13303

forum perf improvements - meta

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.9
    • Fix Version/s: 1.9
    • Component/s: Forum
    • Labels:
      None
    • Affected Branches:
      MOODLE_19_STABLE
    • Fixed Branches:
      MOODLE_19_STABLE

      Description

      various speed improvements

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

              Hide
              poltawski Dan Poltawski added a comment -

              fixing typo

              Show
              poltawski Dan Poltawski added a comment - fixing typo
              Hide
              skodak Petr Skoda added a comment -

              sending final patch with the rest of cleanup ready for review, needs some more testing before commit

              Show
              skodak Petr Skoda added a comment - sending final patch with the rest of cleanup ready for review, needs some more testing before commit
              Hide
              poltawski Dan Poltawski added a comment -

              Looks good to me,

              before:

              108 users were sent post 6141, 'Re: writing maths equations in moodle'
              108 users were sent post 6151, 'Re: writing maths equations in moodle'
              66 users were sent post 6161, 'Re: Installing new skoool courses'
              108 users were sent post 6171, 'Re: Skoool resources added to CLEO Moodle'
              108 users were sent post 6181, 'Re: Skoool resources added to CLEO Moodle'
              108 users were sent post 6191, 'Re: Skoool resources added to CLEO Moodle'
              108 users were sent post 6201, 'Re: Skoool resources added to CLEO Moodle'
              66 users were sent post 6211, 'uploading xml slideshow into moodle'
              108 users were sent post 6221, 'Re: Skoool resources added to CLEO Moodle'
              Starting digest processing...
              Cleaned old digest records
              ... used 4522 dbqueries
              ... used 10.5584990978 seconds

              After:

              108 users were sent post 6141, 'Re: writing maths equations in moodle'
              108 users were sent post 6151, 'Re: writing maths equations in moodle'
              66 users were sent post 6161, 'Re: Installing new skoool courses'
              108 users were sent post 6171, 'Re: Skoool resources added to CLEO Moodle'
              108 users were sent post 6181, 'Re: Skoool resources added to CLEO Moodle'
              108 users were sent post 6191, 'Re: Skoool resources added to CLEO Moodle'
              108 users were sent post 6201, 'Re: Skoool resources added to CLEO Moodle'
              66 users were sent post 6211, 'uploading xml slideshow into moodle'
              108 users were sent post 6221, 'Re: Skoool resources added to CLEO Moodle'
              Starting digest processing...
              Cleaned old digest records
              ... used 967 dbqueries
              ... used 5.05687093735 seconds

              Show
              poltawski Dan Poltawski added a comment - Looks good to me, before: 108 users were sent post 6141, 'Re: writing maths equations in moodle' 108 users were sent post 6151, 'Re: writing maths equations in moodle' 66 users were sent post 6161, 'Re: Installing new skoool courses' 108 users were sent post 6171, 'Re: Skoool resources added to CLEO Moodle' 108 users were sent post 6181, 'Re: Skoool resources added to CLEO Moodle' 108 users were sent post 6191, 'Re: Skoool resources added to CLEO Moodle' 108 users were sent post 6201, 'Re: Skoool resources added to CLEO Moodle' 66 users were sent post 6211, 'uploading xml slideshow into moodle' 108 users were sent post 6221, 'Re: Skoool resources added to CLEO Moodle' Starting digest processing... Cleaned old digest records ... used 4522 dbqueries ... used 10.5584990978 seconds After: 108 users were sent post 6141, 'Re: writing maths equations in moodle' 108 users were sent post 6151, 'Re: writing maths equations in moodle' 66 users were sent post 6161, 'Re: Installing new skoool courses' 108 users were sent post 6171, 'Re: Skoool resources added to CLEO Moodle' 108 users were sent post 6181, 'Re: Skoool resources added to CLEO Moodle' 108 users were sent post 6191, 'Re: Skoool resources added to CLEO Moodle' 108 users were sent post 6201, 'Re: Skoool resources added to CLEO Moodle' 66 users were sent post 6211, 'uploading xml slideshow into moodle' 108 users were sent post 6221, 'Re: Skoool resources added to CLEO Moodle' Starting digest processing... Cleaned old digest records ... used 967 dbqueries ... used 5.05687093735 seconds
              Hide
              skodak Petr Skoda added a comment - - edited

              what is your smtp session bulk limit?

              the next accesslib patch should cut down some 100 queries more

              Show
              skodak Petr Skoda added a comment - - edited what is your smtp session bulk limit? the next accesslib patch should cut down some 100 queries more
              Hide
              poltawski Dan Poltawski added a comment -

              I was just returning true on the mailer so it wouldn't actually send mail. Don't have any live sites on 1.9

              Show
              poltawski Dan Poltawski added a comment - I was just returning true on the mailer so it wouldn't actually send mail. Don't have any live sites on 1.9
              Hide
              skodak Petr Skoda added a comment -

              me neither

              Show
              skodak Petr Skoda added a comment - me neither
              Hide
              skodak Petr Skoda added a comment -

              sending forum performance patch, includes general accesslib speedup and print user picture fixes too

              Show
              skodak Petr Skoda added a comment - sending forum performance patch, includes general accesslib speedup and print user picture fixes too
              Hide
              poltawski Dan Poltawski added a comment -

              14 reply forum discussion page goes from 164dbq to 47

              Show
              poltawski Dan Poltawski added a comment - 14 reply forum discussion page goes from 164dbq to 47
              Hide
              skodak Petr Skoda added a comment -

              does it have ratings enabled? with ratings on the gain is generally bigger

              Show
              skodak Petr Skoda added a comment - does it have ratings enabled? with ratings on the gain is generally bigger
              Hide
              dougiamas Martin Dougiamas added a comment -

              I tested this on a local server and then tried it on moodle.org (because I can :-P)

              I used this discussion because it's got lots of ratings and posts: http://moodle.org/mod/forum/discuss.php?d=89722 The stats below are from the Moodle perf numbers in the footer.

              Before:

              4.061152 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 407 user: 40 sys: 30 cuser: 0 csys: 0 Load average: 1.35 Record cache hit/miss ratio : 192/4

              Apply the patch: (multiple reloads of the page)

              5.041921 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 504 user: 25 sys: 25 cuser: 0 csys: 0 Load average: 1.44 Record cache hit/miss ratio : 2/5
              5.02335 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 502 user: 23 sys: 25 cuser: 0 csys: 0 Load average: 1.39 Record cache hit/miss ratio : 2/5
              2.795257 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 280 user: 26 sys: 22 cuser: 0 csys: 0 Load average: 1.98 Record cache hit/miss ratio : 2/5
              4.682864 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 469 user: 25 sys: 24 cuser: 0 csys: 0 Load average: 1.89 Record cache hit/miss ratio : 2/5

              Unapply patch again and reload:
              3.886638 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 388 user: 39 sys: 27 cuser: 0 csys: 0 Load average: 1.68 Record cache hit/miss ratio : 192/4
              5.278671 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 528 user: 41 sys: 28 cuser: 0 csys: 0 Load average: 1.91 Record cache hit/miss ratio : 192/4
              2.626606 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 263 user: 39 sys: 29 cuser: 0 csys: 0 Load average: 1.63 Record cache hit/miss ratio : 192/4
              3.703633 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 371 user: 42 sys: 28 cuser: 0 csys: 0 Load average: 1.47 Record cache hit/miss ratio : 192/4

              Redo patch and reload:
              1.357779 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 136 user: 25 sys: 24 cuser: 0 csys: 0 Load average: 1.34 Record cache hit/miss ratio : 2/5
              3.106071 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 311 user: 27 sys: 26 cuser: 0 csys: 0 Load average: 1.66 Record cache hit/miss ratio : 2/5
              4.905793 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 491 user: 24 sys: 25 cuser: 0 csys: 0 Load average: 1.53 Record cache hit/miss ratio : 2/5
              5.331214 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 533 user: 28 sys: 24 cuser: 0 csys: 0 Load average: 2.35 Record cache hit/miss ratio : 2/5

              To me it looks like record cache was doing it's job and covering up for a lot of bad db calls! I guess that's most of the reason for the slightly reduced memory usage. The overall stats are more sensitive to the machine load than number of db queries, even though the drop in queries is significant (1070 -> 410)

              I like the new multiple context stuff and a big thumbs up for fixing up all those calls to print_user_picture. I can't see any new bugs in my limited testing.

              Check this puppy into CVS please! Thanks, Petr!

              Show
              dougiamas Martin Dougiamas added a comment - I tested this on a local server and then tried it on moodle.org (because I can :-P) I used this discussion because it's got lots of ratings and posts: http://moodle.org/mod/forum/discuss.php?d=89722 The stats below are from the Moodle perf numbers in the footer. Before: 4.061152 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 407 user: 40 sys: 30 cuser: 0 csys: 0 Load average: 1.35 Record cache hit/miss ratio : 192/4 Apply the patch: (multiple reloads of the page) 5.041921 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 504 user: 25 sys: 25 cuser: 0 csys: 0 Load average: 1.44 Record cache hit/miss ratio : 2/5 5.02335 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 502 user: 23 sys: 25 cuser: 0 csys: 0 Load average: 1.39 Record cache hit/miss ratio : 2/5 2.795257 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 280 user: 26 sys: 22 cuser: 0 csys: 0 Load average: 1.98 Record cache hit/miss ratio : 2/5 4.682864 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 469 user: 25 sys: 24 cuser: 0 csys: 0 Load average: 1.89 Record cache hit/miss ratio : 2/5 Unapply patch again and reload: 3.886638 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 388 user: 39 sys: 27 cuser: 0 csys: 0 Load average: 1.68 Record cache hit/miss ratio : 192/4 5.278671 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 528 user: 41 sys: 28 cuser: 0 csys: 0 Load average: 1.91 Record cache hit/miss ratio : 192/4 2.626606 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 263 user: 39 sys: 29 cuser: 0 csys: 0 Load average: 1.63 Record cache hit/miss ratio : 192/4 3.703633 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 371 user: 42 sys: 28 cuser: 0 csys: 0 Load average: 1.47 Record cache hit/miss ratio : 192/4 Redo patch and reload: 1.357779 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 136 user: 25 sys: 24 cuser: 0 csys: 0 Load average: 1.34 Record cache hit/miss ratio : 2/5 3.106071 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 311 user: 27 sys: 26 cuser: 0 csys: 0 Load average: 1.66 Record cache hit/miss ratio : 2/5 4.905793 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 491 user: 24 sys: 25 cuser: 0 csys: 0 Load average: 1.53 Record cache hit/miss ratio : 2/5 5.331214 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 533 user: 28 sys: 24 cuser: 0 csys: 0 Load average: 2.35 Record cache hit/miss ratio : 2/5 To me it looks like record cache was doing it's job and covering up for a lot of bad db calls! I guess that's most of the reason for the slightly reduced memory usage. The overall stats are more sensitive to the machine load than number of db queries, even though the drop in queries is significant (1070 -> 410) I like the new multiple context stuff and a big thumbs up for fixing up all those calls to print_user_picture. I can't see any new bugs in my limited testing. Check this puppy into CVS please! Thanks, Petr!
              Hide
              skodak Petr Skoda added a comment - - edited

              unfortunately the record cache works from get_record() only, which I guess is about 1/3 of bad calls

              1290 -> 410 is not bad, but I would expect the resulting number to be a even lower (50 + number of posts in discussion), maybe filters or something else is interfering - I try to improve it more later

              Show
              skodak Petr Skoda added a comment - - edited unfortunately the record cache works from get_record() only, which I guess is about 1/3 of bad calls 1290 -> 410 is not bad, but I would expect the resulting number to be a even lower (50 + number of posts in discussion), maybe filters or something else is interfering - I try to improve it more later
              Hide
              skodak Petr Skoda added a comment -

              committed into cvs

              Show
              skodak Petr Skoda added a comment - committed into cvs
              Hide
              poltawski Dan Poltawski added a comment -

              I think this is causing problems on moodle.org. The first person to post in the using moodle group in a discussion is the only user getting an icon, e.g.:
              http://moodle.org/mod/forum/discuss.php?d=90264

              http://moodle.org/mod/forum/discuss.php?d=89722

              Show
              poltawski Dan Poltawski added a comment - I think this is causing problems on moodle.org. The first person to post in the using moodle group in a discussion is the only user getting an icon, e.g.: http://moodle.org/mod/forum/discuss.php?d=90264 http://moodle.org/mod/forum/discuss.php?d=89722
              Hide
              skodak Petr Skoda added a comment -

              groups bug fixed in cvs, we need somebody to update moodel.org now

              Show
              skodak Petr Skoda added a comment - groups bug fixed in cvs, we need somebody to update moodel.org now
              Hide
              skodak Petr Skoda added a comment -

              should be finished now, please reopen in case of any problem
              thanks for testing

              Show
              skodak Petr Skoda added a comment - should be finished now, please reopen in case of any problem thanks for testing

                People

                • Votes:
                  0 Vote for this issue
                  Watchers:
                  5 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:
                    Fix Release Date:
                    3/Mar/08