Moodle Community Sites

Some improvements to download.moodle.org

Details

  • Type: Improvement Improvement
  • Status: Resolved Resolved
  • Priority: Minor Minor
  • Resolution: Fixed
  • Component/s: download.moodle.org
  • Labels:
    None

Description

Some minor tasks can improve throughput of download.moodle.org:

1) Create better indexes (by url and time). Done!
2) Move old records from the downloads table to a new downloads_history table (older than 2 months).
3) Fetch download stats from the downloads_history table
4) Cache counters somewhere (with a configurable timeout, say 1 min).

Just with 1) we have get a BIG gain in throughput, with 2, 3 & 4, server should be practically instant.

Issue Links

Activity

Hide
Eloy Lafuente (stronk7) added a comment -

Just one doubt:

the 2 months grace period before being moved to downloads_history is for...? Just asking in case the plan is to continue generating monthly stats from the downloads table, instead of the downloads_history table as proposed by 3)

Ciao

Show
Eloy Lafuente (stronk7) added a comment - Just one doubt: the 2 months grace period before being moved to downloads_history is for...? Just asking in case the plan is to continue generating monthly stats from the downloads table, instead of the downloads_history table as proposed by 3) Ciao
Hide
Martin Dougiamas added a comment -

Yeah that was what I was thinking but you're right that it probably would be better generating them from the history table thanks

Show
Martin Dougiamas added a comment - Yeah that was what I was thinking but you're right that it probably would be better generating them from the history table thanks
Hide
Eloy Lafuente (stronk7) added a comment -

I've created these tables:

downloads_history: to accumulate old records (2 days) from the downloads table
downloads_summary: to store monthly totals.

Also, in cron tab, there is one new script, 'moodle-downloads2history2summary' executed daily that is in charge to move records between tables and calculate the summaries.

Next step: Change '/opt/bin/generatestats' to be executed from shell and calculate only the 'alldownloads_cumu' stat (the one being used by moodle.org stats)

Ciao

Show
Eloy Lafuente (stronk7) added a comment - I've created these tables: downloads_history: to accumulate old records (2 days) from the downloads table downloads_summary: to store monthly totals. Also, in cron tab, there is one new script, 'moodle-downloads2history2summary' executed daily that is in charge to move records between tables and calculate the summaries. Next step: Change '/opt/bin/generatestats' to be executed from shell and calculate only the 'alldownloads_cumu' stat (the one being used by moodle.org stats) Ciao
Hide
Eloy Lafuente (stronk7) added a comment -

Now '/opt/bin/generatestats' is generating properly the 'alldownloads_cumu.xml' file (based in the new downloads_summary table) and such file is transferred to moodle.org in order to display updated statistics.

Closing... ciao

Show
Eloy Lafuente (stronk7) added a comment - Now '/opt/bin/generatestats' is generating properly the 'alldownloads_cumu.xml' file (based in the new downloads_summary table) and such file is transferred to moodle.org in order to display updated statistics. Closing... ciao

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: