Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.9
    • Fix Version/s: 1.9
    • Component/s: Forum
    • Labels:
      None
    • Affected Branches:
      MOODLE_19_STABLE
    • Fixed Branches:
      MOODLE_19_STABLE
    • Rank:
      30616

      Description

      various speed improvements

      1. forum10.patch
        99 kB
        Petr Škoda
      2. forum11.patch
        99 kB
        Petr Škoda
      Progress
      Resolved Sub-Tasks

      Sub-Tasks

      There are no Sub-Tasks for this issue.

        Activity

        Hide
        Dan Poltawski added a comment -

        fixing typo

        Show
        Dan Poltawski added a comment - fixing typo
        Hide
        Petr Škoda added a comment -

        sending final patch with the rest of cleanup ready for review, needs some more testing before commit

        Show
        Petr Škoda added a comment - sending final patch with the rest of cleanup ready for review, needs some more testing before commit
        Hide
        Dan Poltawski added a comment -

        Looks good to me,

        before:

        108 users were sent post 6141, 'Re: writing maths equations in moodle'
        108 users were sent post 6151, 'Re: writing maths equations in moodle'
        66 users were sent post 6161, 'Re: Installing new skoool courses'
        108 users were sent post 6171, 'Re: Skoool resources added to CLEO Moodle'
        108 users were sent post 6181, 'Re: Skoool resources added to CLEO Moodle'
        108 users were sent post 6191, 'Re: Skoool resources added to CLEO Moodle'
        108 users were sent post 6201, 'Re: Skoool resources added to CLEO Moodle'
        66 users were sent post 6211, 'uploading xml slideshow into moodle'
        108 users were sent post 6221, 'Re: Skoool resources added to CLEO Moodle'
        Starting digest processing...
        Cleaned old digest records
        ... used 4522 dbqueries
        ... used 10.5584990978 seconds

        After:

        108 users were sent post 6141, 'Re: writing maths equations in moodle'
        108 users were sent post 6151, 'Re: writing maths equations in moodle'
        66 users were sent post 6161, 'Re: Installing new skoool courses'
        108 users were sent post 6171, 'Re: Skoool resources added to CLEO Moodle'
        108 users were sent post 6181, 'Re: Skoool resources added to CLEO Moodle'
        108 users were sent post 6191, 'Re: Skoool resources added to CLEO Moodle'
        108 users were sent post 6201, 'Re: Skoool resources added to CLEO Moodle'
        66 users were sent post 6211, 'uploading xml slideshow into moodle'
        108 users were sent post 6221, 'Re: Skoool resources added to CLEO Moodle'
        Starting digest processing...
        Cleaned old digest records
        ... used 967 dbqueries
        ... used 5.05687093735 seconds

        Show
        Dan Poltawski added a comment - Looks good to me, before: 108 users were sent post 6141, 'Re: writing maths equations in moodle' 108 users were sent post 6151, 'Re: writing maths equations in moodle' 66 users were sent post 6161, 'Re: Installing new skoool courses' 108 users were sent post 6171, 'Re: Skoool resources added to CLEO Moodle' 108 users were sent post 6181, 'Re: Skoool resources added to CLEO Moodle' 108 users were sent post 6191, 'Re: Skoool resources added to CLEO Moodle' 108 users were sent post 6201, 'Re: Skoool resources added to CLEO Moodle' 66 users were sent post 6211, 'uploading xml slideshow into moodle' 108 users were sent post 6221, 'Re: Skoool resources added to CLEO Moodle' Starting digest processing... Cleaned old digest records ... used 4522 dbqueries ... used 10.5584990978 seconds After: 108 users were sent post 6141, 'Re: writing maths equations in moodle' 108 users were sent post 6151, 'Re: writing maths equations in moodle' 66 users were sent post 6161, 'Re: Installing new skoool courses' 108 users were sent post 6171, 'Re: Skoool resources added to CLEO Moodle' 108 users were sent post 6181, 'Re: Skoool resources added to CLEO Moodle' 108 users were sent post 6191, 'Re: Skoool resources added to CLEO Moodle' 108 users were sent post 6201, 'Re: Skoool resources added to CLEO Moodle' 66 users were sent post 6211, 'uploading xml slideshow into moodle' 108 users were sent post 6221, 'Re: Skoool resources added to CLEO Moodle' Starting digest processing... Cleaned old digest records ... used 967 dbqueries ... used 5.05687093735 seconds
        Hide
        Petr Škoda added a comment - - edited

        what is your smtp session bulk limit?

        the next accesslib patch should cut down some 100 queries more

        Show
        Petr Škoda added a comment - - edited what is your smtp session bulk limit? the next accesslib patch should cut down some 100 queries more
        Hide
        Dan Poltawski added a comment -

        I was just returning true on the mailer so it wouldn't actually send mail. Don't have any live sites on 1.9

        Show
        Dan Poltawski added a comment - I was just returning true on the mailer so it wouldn't actually send mail. Don't have any live sites on 1.9
        Hide
        Petr Škoda added a comment -

        me neither

        Show
        Petr Škoda added a comment - me neither
        Hide
        Petr Škoda added a comment -

        sending forum performance patch, includes general accesslib speedup and print user picture fixes too

        Show
        Petr Škoda added a comment - sending forum performance patch, includes general accesslib speedup and print user picture fixes too
        Hide
        Dan Poltawski added a comment -

        14 reply forum discussion page goes from 164dbq to 47

        Show
        Dan Poltawski added a comment - 14 reply forum discussion page goes from 164dbq to 47
        Hide
        Petr Škoda added a comment -

        does it have ratings enabled? with ratings on the gain is generally bigger

        Show
        Petr Škoda added a comment - does it have ratings enabled? with ratings on the gain is generally bigger
        Hide
        Martin Dougiamas added a comment -

        I tested this on a local server and then tried it on moodle.org (because I can :-P)

        I used this discussion because it's got lots of ratings and posts: http://moodle.org/mod/forum/discuss.php?d=89722 The stats below are from the Moodle perf numbers in the footer.

        Before:

        4.061152 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 407 user: 40 sys: 30 cuser: 0 csys: 0 Load average: 1.35 Record cache hit/miss ratio : 192/4

        Apply the patch: (multiple reloads of the page)

        5.041921 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 504 user: 25 sys: 25 cuser: 0 csys: 0 Load average: 1.44 Record cache hit/miss ratio : 2/5
        5.02335 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 502 user: 23 sys: 25 cuser: 0 csys: 0 Load average: 1.39 Record cache hit/miss ratio : 2/5
        2.795257 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 280 user: 26 sys: 22 cuser: 0 csys: 0 Load average: 1.98 Record cache hit/miss ratio : 2/5
        4.682864 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 469 user: 25 sys: 24 cuser: 0 csys: 0 Load average: 1.89 Record cache hit/miss ratio : 2/5

        Unapply patch again and reload:
        3.886638 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 388 user: 39 sys: 27 cuser: 0 csys: 0 Load average: 1.68 Record cache hit/miss ratio : 192/4
        5.278671 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 528 user: 41 sys: 28 cuser: 0 csys: 0 Load average: 1.91 Record cache hit/miss ratio : 192/4
        2.626606 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 263 user: 39 sys: 29 cuser: 0 csys: 0 Load average: 1.63 Record cache hit/miss ratio : 192/4
        3.703633 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 371 user: 42 sys: 28 cuser: 0 csys: 0 Load average: 1.47 Record cache hit/miss ratio : 192/4

        Redo patch and reload:
        1.357779 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 136 user: 25 sys: 24 cuser: 0 csys: 0 Load average: 1.34 Record cache hit/miss ratio : 2/5
        3.106071 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 311 user: 27 sys: 26 cuser: 0 csys: 0 Load average: 1.66 Record cache hit/miss ratio : 2/5
        4.905793 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 491 user: 24 sys: 25 cuser: 0 csys: 0 Load average: 1.53 Record cache hit/miss ratio : 2/5
        5.331214 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 533 user: 28 sys: 24 cuser: 0 csys: 0 Load average: 2.35 Record cache hit/miss ratio : 2/5

        To me it looks like record cache was doing it's job and covering up for a lot of bad db calls! I guess that's most of the reason for the slightly reduced memory usage. The overall stats are more sensitive to the machine load than number of db queries, even though the drop in queries is significant (1070 -> 410)

        I like the new multiple context stuff and a big thumbs up for fixing up all those calls to print_user_picture. I can't see any new bugs in my limited testing.

        Check this puppy into CVS please! Thanks, Petr!

        Show
        Martin Dougiamas added a comment - I tested this on a local server and then tried it on moodle.org (because I can :-P) I used this discussion because it's got lots of ratings and posts: http://moodle.org/mod/forum/discuss.php?d=89722 The stats below are from the Moodle perf numbers in the footer. Before: 4.061152 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 407 user: 40 sys: 30 cuser: 0 csys: 0 Load average: 1.35 Record cache hit/miss ratio : 192/4 Apply the patch: (multiple reloads of the page) 5.041921 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 504 user: 25 sys: 25 cuser: 0 csys: 0 Load average: 1.44 Record cache hit/miss ratio : 2/5 5.02335 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 502 user: 23 sys: 25 cuser: 0 csys: 0 Load average: 1.39 Record cache hit/miss ratio : 2/5 2.795257 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 280 user: 26 sys: 22 cuser: 0 csys: 0 Load average: 1.98 Record cache hit/miss ratio : 2/5 4.682864 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 469 user: 25 sys: 24 cuser: 0 csys: 0 Load average: 1.89 Record cache hit/miss ratio : 2/5 Unapply patch again and reload: 3.886638 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 388 user: 39 sys: 27 cuser: 0 csys: 0 Load average: 1.68 Record cache hit/miss ratio : 192/4 5.278671 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 528 user: 41 sys: 28 cuser: 0 csys: 0 Load average: 1.91 Record cache hit/miss ratio : 192/4 2.626606 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 263 user: 39 sys: 29 cuser: 0 csys: 0 Load average: 1.63 Record cache hit/miss ratio : 192/4 3.703633 secs RAM: 6.1MB Included 42 files DB queries 1070 Log writes 1 ticks: 371 user: 42 sys: 28 cuser: 0 csys: 0 Load average: 1.47 Record cache hit/miss ratio : 192/4 Redo patch and reload: 1.357779 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 136 user: 25 sys: 24 cuser: 0 csys: 0 Load average: 1.34 Record cache hit/miss ratio : 2/5 3.106071 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 311 user: 27 sys: 26 cuser: 0 csys: 0 Load average: 1.66 Record cache hit/miss ratio : 2/5 4.905793 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 491 user: 24 sys: 25 cuser: 0 csys: 0 Load average: 1.53 Record cache hit/miss ratio : 2/5 5.331214 secs RAM: 6MB Included 42 files DB queries 410 Log writes 1 ticks: 533 user: 28 sys: 24 cuser: 0 csys: 0 Load average: 2.35 Record cache hit/miss ratio : 2/5 To me it looks like record cache was doing it's job and covering up for a lot of bad db calls! I guess that's most of the reason for the slightly reduced memory usage. The overall stats are more sensitive to the machine load than number of db queries, even though the drop in queries is significant (1070 -> 410) I like the new multiple context stuff and a big thumbs up for fixing up all those calls to print_user_picture. I can't see any new bugs in my limited testing. Check this puppy into CVS please! Thanks, Petr!
        Hide
        Petr Škoda added a comment - - edited

        unfortunately the record cache works from get_record() only, which I guess is about 1/3 of bad calls

        1290 -> 410 is not bad, but I would expect the resulting number to be a even lower (50 + number of posts in discussion), maybe filters or something else is interfering - I try to improve it more later

        Show
        Petr Škoda added a comment - - edited unfortunately the record cache works from get_record() only, which I guess is about 1/3 of bad calls 1290 -> 410 is not bad, but I would expect the resulting number to be a even lower (50 + number of posts in discussion), maybe filters or something else is interfering - I try to improve it more later
        Hide
        Petr Škoda added a comment -

        committed into cvs

        Show
        Petr Škoda added a comment - committed into cvs
        Hide
        Dan Poltawski added a comment -

        I think this is causing problems on moodle.org. The first person to post in the using moodle group in a discussion is the only user getting an icon, e.g.:
        http://moodle.org/mod/forum/discuss.php?d=90264

        http://moodle.org/mod/forum/discuss.php?d=89722

        Show
        Dan Poltawski added a comment - I think this is causing problems on moodle.org. The first person to post in the using moodle group in a discussion is the only user getting an icon, e.g.: http://moodle.org/mod/forum/discuss.php?d=90264 http://moodle.org/mod/forum/discuss.php?d=89722
        Hide
        Petr Škoda added a comment -

        groups bug fixed in cvs, we need somebody to update moodel.org now

        Show
        Petr Škoda added a comment - groups bug fixed in cvs, we need somebody to update moodel.org now
        Hide
        Petr Škoda added a comment -

        should be finished now, please reopen in case of any problem
        thanks for testing

        Show
        Petr Škoda added a comment - should be finished now, please reopen in case of any problem thanks for testing

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: