Moodle

recent activity and separate mode cleanup and fixes - meta

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Blocker Blocker
  • Resolution: Fixed
  • Affects Version/s: 1.9
  • Fix Version/s: 1.9
  • Component/s: General
  • Labels:
    None
  • Affected Branches:
    MOODLE_19_STABLE
  • Fixed Branches:
    MOODLE_19_STABLE

Description

meta bug for recent activity performance, fixing and improvements + some other stuff I find while fixing these

  1. chat_groupby2.patch
    14/Feb/08 9:23 PM
    5 kB
    Petr Škoda (skodak)
  2. chat_groupby3.patch
    14/Feb/08 9:44 PM
    5 kB
    Petr Škoda (skodak)
  3. recent_cleanup25.patch
    24/Jan/08 5:35 AM
    274 kB
    Petr Škoda (skodak)

Issue Links

Progress
Resolved Sub-Tasks

Sub-Tasks

Activity

Hide
Petr Škoda (skodak) added a comment - - edited

adding first patch - not finished yet, but it should be suitable for basic perf testing
please note that full manual rebuild_course_cache() is required (not added into main upgrade.php yet)

Show
Petr Škoda (skodak) added a comment - - edited adding first patch - not finished yet, but it should be suitable for basic perf testing please note that full manual rebuild_course_cache() is required (not added into main upgrade.php yet)
Hide
Martin Dougiamas added a comment -

Awesome work, Petr. Would love to see this in 1.9 as long as we get a few testers to confirm that none of the functionality is broken.

Show
Martin Dougiamas added a comment - Awesome work, Petr. Would love to see this in 1.9 as long as we get a few testers to confirm that none of the functionality is broken.
Hide
Dan Poltawski added a comment -

Hope to spend some time testing this after i've had a coffee etc

Show
Dan Poltawski added a comment - Hope to spend some time testing this after i've had a coffee etc
Hide
Dan Poltawski added a comment -

Getting lots of notices:
Notice: Undefined property: stdClass::$id in /var/www/moodle/course/lib.php on line 1278
Notice: Undefined property: stdClass::$groupmode in /var/www/moodle/course/lib.php on line 1284
Notice: Undefined property: stdClass::$groupingid in /var/www/moodle/course/lib.php on line 1285
Notice: Undefined property: stdClass::$groupmembersonly in /var/www/moodle/course/lib.php on line 1286

Show
Dan Poltawski added a comment - Getting lots of notices: Notice: Undefined property: stdClass::$id in /var/www/moodle/course/lib.php on line 1278 Notice: Undefined property: stdClass::$groupmode in /var/www/moodle/course/lib.php on line 1284 Notice: Undefined property: stdClass::$groupingid in /var/www/moodle/course/lib.php on line 1285 Notice: Undefined property: stdClass::$groupmembersonly in /var/www/moodle/course/lib.php on line 1286
Hide
Petr Škoda (skodak) added a comment -

use rebuild_course_cache() to get rid of those, or you can just edit one activity and the course->modinfo will be rebuild for that course

I am testing this with two different checkouts (server/moodle19 and server/moodle19b) where one is patched, you can only edit the courses in the patched one
you can open the coursepage in two tabs and see the difference in query count immediately

Show
Petr Škoda (skodak) added a comment - use rebuild_course_cache() to get rid of those, or you can just edit one activity and the course->modinfo will be rebuild for that course I am testing this with two different checkouts (server/moodle19 and server/moodle19b) where one is patched, you can only edit the courses in the patched one you can open the coursepage in two tabs and see the difference in query count immediately
Hide
Dan Poltawski added a comment -

Sorry! I read your rebuild_course_cache() comment last night but didn't this morning!

Show
Dan Poltawski added a comment - Sorry! I read your rebuild_course_cache() comment last night but didn't this morning!
Hide
Dan Poltawski added a comment -

Trying to complex courses to play with.

On some complex course pages its reducing course page queries from 100 to 70ish

Full report of recent activity reducing by about 100queries

Show
Dan Poltawski added a comment - Trying to complex courses to play with. On some complex course pages its reducing course page queries from 100 to 70ish Full report of recent activity reducing by about 100queries
Hide
Petr Škoda (skodak) added a comment -

I did not start with the full report yet, I would like to review all blocks after the recent activity related work

Show
Petr Škoda (skodak) added a comment - I did not start with the full report yet, I would like to review all blocks after the recent activity related work
Hide
Petr Škoda (skodak) added a comment -

no need for manual rebuild anymore

Show
Petr Škoda (skodak) added a comment - no need for manual rebuild anymore
Hide
Martin Dougiamas added a comment - - edited

I'm running this patch on moodle.org right now. Seems to work OK

BEFORE on http://moodle.org/course/view.php?id=5 :

4.201701 secs
RAM: 11.9MB
Included 129 files
DB queries 374
Log writes 1
ticks: 421
Load average: 1.33
Record cache hit/miss ratio : 0/4

AFTER:

3.331583 secs
RAM: 9.8MB
Included 87 files
DB queries 119
Log writes 1
ticks: 333
Load average: 2.11
Record cache hit/miss ratio : 0/4

(dance)

I think this can go into MOODLE_19_STABLE and HEAD so we can have more testers.

Show
Martin Dougiamas added a comment - - edited I'm running this patch on moodle.org right now. Seems to work OK BEFORE on http://moodle.org/course/view.php?id=5 : 4.201701 secs RAM: 11.9MB Included 129 files DB queries 374 Log writes 1 ticks: 421 Load average: 1.33 Record cache hit/miss ratio : 0/4 AFTER: 3.331583 secs RAM: 9.8MB Included 87 files DB queries 119 Log writes 1 ticks: 333 Load average: 2.11 Record cache hit/miss ratio : 0/4 (dance) I think this can go into MOODLE_19_STABLE and HEAD so we can have more testers.
Hide
Martin Dougiamas added a comment -

Can you please negate the setting for assignment_limitrecentsubmissions?

Currently it's awkward to have default "No" next to "Do not show recent submissions to users without grading permission".

I think it would be better to be assignment_showrecentsubmissions: "Everyone can see notifications of submissions in recent activity reports" and default "Yes".

Show
Martin Dougiamas added a comment - Can you please negate the setting for assignment_limitrecentsubmissions? Currently it's awkward to have default "No" next to "Do not show recent submissions to users without grading permission". I think it would be better to be assignment_showrecentsubmissions: "Everyone can see notifications of submissions in recent activity reports" and default "Yes".
Hide
Martin Dougiamas added a comment -

Another bug. The links to the forum posts have been broken in the patch, eg:

http://moodle.org/mod/forum/discuss.php?d=88599&postid=391908

They are OK in CVS.

Show
Martin Dougiamas added a comment - Another bug. The links to the forum posts have been broken in the patch, eg: http://moodle.org/mod/forum/discuss.php?d=88599&postid=391908 They are OK in CVS.
Hide
Petr Škoda (skodak) added a comment -

sending latest patch, I am not looking for more speedups anymore, just fixing regressions; major fixes in section links and online users menu, some calendar changes and several other queries elsewhere

  • the assignment settings is changed
  • the post links are using #p44444
Show
Petr Škoda (skodak) added a comment - sending latest patch, I am not looking for more speedups anymore, just fixing regressions; major fixes in section links and online users menu, some calendar changes and several other queries elsewhere
  • the assignment settings is changed
  • the post links are using #p44444
Hide
Petr Škoda (skodak) added a comment - - edited

TODO: review/fix all upgrade paths to no use rebuild_course_cache() - do it only once at the end

Show
Petr Škoda (skodak) added a comment - - edited TODO: review/fix all upgrade paths to no use rebuild_course_cache() - do it only once at the end
Hide
Martin Dougiamas added a comment -

Running the latest on moodle.org again - looks good.

(Only the strings for the changed assignment settings need to be updated.)

Would love to see this in CVS soon.

Show
Martin Dougiamas added a comment - Running the latest on moodle.org again - looks good. (Only the strings for the changed assignment settings need to be updated.) Would love to see this in CVS soon.
Hide
Petr Škoda (skodak) added a comment -

oh, renamed the string, forgot to change it; going to add some css to full recent activity and test&test&test and commit today

Show
Petr Škoda (skodak) added a comment - oh, renamed the string, forgot to change it; going to add some css to full recent activity and test&test&test and commit today
Hide
Petr Škoda (skodak) added a comment -

should be mostly finished - My Moodle page needs similar cleanup - going to file it as separate issue

Show
Petr Škoda (skodak) added a comment - should be mostly finished - My Moodle page needs similar cleanup - going to file it as separate issue
Hide
David Mudrak added a comment -

SQL changes in mod/chat/lib.php do not work under PostgreSQL:

ERROR: column "cm.course" must appear in the GROUP BY clause or be used in an aggregate function

SELECT cm.*, ch.name, 'chat' AS modname, MAX(chm.timestamp) AS lasttime FROM mdl_course_modules cm JOIN mdl_modules md ON md.id = cm.module JOIN mdl_chat ch ON ch.id = cm.instance JOIN mdl_chat_messages chm ON chm.chatid = ch.id WHERE chm.timestamp > 1202805600 AND ch.course = 8 AND md.name = 'chat' GROUP BY cm.id ORDER BY chm.timestamp ASC

  • line 686 of lib/dmllib.php: call to debugging()
  • line 944 of lib/dmllib.php: call to get_recordset_sql()
  • line 152 of mod/chat/lib.php: call to get_records_sql()
  • line 932 of course/lib.php: call to chat_print_recent_activity()
  • line 27 of blocks/recent_activity/block_recent_activity.php: call to print_recent_activity()
  • line 240 of blocks/moodleblock.class.php: call to block_recent_activity->get_content()
  • line 264 of blocks/moodleblock.class.php: call to block_base->is_empty()
  • line 338 of lib/blocklib.php: call to block_base->_print_block()
  • line 284 of course/format/topics/format.php: call to blocks_print_group()
  • line 206 of course/view.php: call to require()
Show
David Mudrak added a comment - SQL changes in mod/chat/lib.php do not work under PostgreSQL: ERROR: column "cm.course" must appear in the GROUP BY clause or be used in an aggregate function SELECT cm.*, ch.name, 'chat' AS modname, MAX(chm.timestamp) AS lasttime FROM mdl_course_modules cm JOIN mdl_modules md ON md.id = cm.module JOIN mdl_chat ch ON ch.id = cm.instance JOIN mdl_chat_messages chm ON chm.chatid = ch.id WHERE chm.timestamp > 1202805600 AND ch.course = 8 AND md.name = 'chat' GROUP BY cm.id ORDER BY chm.timestamp ASC
  • line 686 of lib/dmllib.php: call to debugging()
  • line 944 of lib/dmllib.php: call to get_recordset_sql()
  • line 152 of mod/chat/lib.php: call to get_records_sql()
  • line 932 of course/lib.php: call to chat_print_recent_activity()
  • line 27 of blocks/recent_activity/block_recent_activity.php: call to print_recent_activity()
  • line 240 of blocks/moodleblock.class.php: call to block_recent_activity->get_content()
  • line 264 of blocks/moodleblock.class.php: call to block_base->is_empty()
  • line 338 of lib/blocklib.php: call to block_base->_print_block()
  • line 284 of course/format/topics/format.php: call to blocks_print_group()
  • line 206 of course/view.php: call to require()
Hide
David Mudrak added a comment -

The SQL in the chat lib.php is not valid. For more info, see http://www.postgresql.org/docs/7.4/static/sql-select.html#SQL-GROUPBY

Excerpted text:

"...When GROUP BY is present, it is not valid for the SELECT list expressions to refer to ungrouped columns except within aggregate functions, since there would be more than one possible value to return for an ungrouped column."

It is a known difference in PostgreSQL and other RDMS' behaviour.

Show
David Mudrak added a comment - The SQL in the chat lib.php is not valid. For more info, see http://www.postgresql.org/docs/7.4/static/sql-select.html#SQL-GROUPBY Excerpted text: "...When GROUP BY is present, it is not valid for the SELECT list expressions to refer to ungrouped columns except within aggregate functions, since there would be more than one possible value to return for an ungrouped column." It is a known difference in PostgreSQL and other RDMS' behaviour.
Hide
David Mudrak added a comment -

Petr's chat_groupby3.patch works for me. Thanks for the fix.

Show
David Mudrak added a comment - Petr's chat_groupby3.patch works for me. Thanks for the fix.
Hide
Petr Škoda (skodak) added a comment - - edited

fixed in cvs, big thanks for spotting it

Show
Petr Škoda (skodak) added a comment - - edited fixed in cvs, big thanks for spotting it

People

Dates

  • Created:
    Updated:
    Resolved: