Issue Details (XML | Word | Printable)

Key: MDL-16792
Type: Improvement Improvement
Status: Open Open
Priority: Major Major
Assignee: Martín Langhoff
Reporter: Gary Anderson
Votes: 6
Watchers: 9
Operations

Add/Edit UI Mockup to this issue
If you were logged in you would be able to see more operations.
Moodle

Recent Activity block may be causeing big performance hit

Created: 04/Oct/08 06:30 PM   Updated: 09/Nov/08 09:16 PM
Return to search
Component/s: Performance
Affects Version/s: 1.9
Fix Version/s: None

Participants: Amr Hourani, Gary Anderson, José Coelho, Martín Langhoff and Samuli Karevaara
Security Level: None
Affected Branches: MOODLE_19_STABLE


 Description  « Hide
Jos' Coelho reports in http://moodle.org/mod/forum/discuss.php?d=107478 that the queries made by the recent activity block cause a major server load when the log table becomes large.

As this is a default block and many sites will choose to maintain logs for long periods of time, it may be a good idea to optimize this query or otherwise modify how it gets its information to improve performance.

Jose' suggests that the difference in performance was like "night and day".

 All   Comments   Change History   Version Control      Sort Order: Ascending order - Click to sort in descending order
Samuli Karevaara added a comment - 08/Oct/08 10:51 AM
The system used by José was Moodle 1.9.1+ (Build: 20080702). Did Petr work on this since that version?

There was also talk about (or was I sleeping again?) splitting the log table in two, having a smaller "cache" table for the recent activity block and similar "short term history" actions, and then a bigger table for the longer history reports. Maybe the stats cron could move the tail of this recent activity table to the archive log table?

Has anything like this been discussed lately?


José Coelho added a comment - 08/Oct/08 04:26 PM
Instead of a short log table, and a archive log table, why not do a table with only non-View actions. Most of the actions are view actions, that are not needed for the recent activity block.

I am talking without reading the referred talk about this problem, since I don't know its location.

Even with a short log table, we are having about 200.000 page views per day, and even only one week means a short log table of about 1 million rows, that could not work very well if this block is on all courses, and used in each page view.

I don't know the number of non-View operations per day, but the number of forum messages are per day is only 3.000, so probably the non-view actions are at most 5.000 per day, I guess.

The cron could update this table of non-view actions, keeping all the rest of the code without any change, and its records expire at the same time the records in the log table expires.


Amr Hourani added a comment - 09/Nov/08 09:16 PM
Since latest versions of moodle do use mysql 5.1.x or newer, why dont we use mysql partitioning since it is already supported in mysql 5.1 and newer?
it will serve better and effecient, especially for mdl_log because it grows massively everyday. we can simply partition mdl_log by id range.. ie: adding number of partitions according to the number of records our systems can handle effeciently.

simply by adding there lins to the mdl_log creation:
.......
PARTITION BY RANGE(id) (
PARTITION p0 VALUES LESS THAN(10000),
PARTITION p1 VALUES LESS THAN(20000),
PARTITION p2 VALUES LESS THAN(30000),
PARTITION p3 VALUES LESS THAN(40000),
PARTITION p4 VALUES LESS THAN(50000),
PARTITION p5 VALUES LESS THAN(MAXVALUE)
);
these numbers can be changed based on some testing..