Issue Details (XML | Word | Printable)

Key: MDL-19711
Type: New Feature New Feature
Status: Open Open
Priority: Major Major
Assignee: Martín Langhoff
Reporter: jai gupta
Votes: 0
Watchers: 4
Operations

Add/Edit UI Mockup to this issue
If you were logged in you would be able to see more operations.
Moodle

Read/Write splitting of DB operations.

Created: 04/Jul/09 04:18 PM   Updated: 06/Jul/09 06:45 AM
Return to search
Component/s: Performance
Affects Version/s: 2.0
Fix Version/s: None

Issue Links:
Dependency
 

Participants: jai gupta, Juan Segarra Montesinos, Martín Langhoff and Petr Skoda
Security Level: None
Affected Branches: MOODLE_20_STABLE


 Description  « Hide
It would give an advantage on clusters if Moodle could allow slave (read-only) databases. Moodle could use master DB server for all write operations and could use slave DB server for all read operations. It will make clustering with Moodle simpler to setup and maintain.

DB layer 2.0 could make it possible (easier) to do.

 All   Comments   Change History   Version Control      Sort Order: Ascending order - Click to sort in descending order
Petr Skoda added a comment - 04/Jul/09 05:30 PM
yes it should be possible to write new database driver for this, but I do not think it will be implemented and maintained in official Moodle distribution; I think that better performance gains should be possible via clever shared memory usage

jai gupta added a comment - 04/Jul/09 08:45 PM
Shared memory is practical in cases where DB size is small compared to budget to buy memory. If shared memory is not clever enough it can actually lead to increase of response time. Separating read and writes operations could help people run Moodle at lower cost.

Drupal will also have native support for master-slave replication in its next release.
http://www.garfieldtech.com/blog/drupal7-database-update
http://buytaert.net/scaling-with-mysql-replication

They also have released a backport for its current version.
http://drupal.org/node/147160

I think it will really make Moodle scalable if it could work with multiple databases at the same time.


Petr Skoda added a comment - 04/Jul/09 09:03 PM
I think in shared memory should be stored data that is expensive to get from database, for example relative small and constant objects that require very complex queries - like guest access data.
Another our problem is repeated database queries such as get_record() inside looks - we still do a lot of these, I think that removing of these problems should have higher priority.

My personal priority is:
1/ eliminate inefficient queries - everybody benefits immediately, no extra set-up needed
2/ implement clever shared memory use - anybody with configured shared memory benefits
3/ read/write separation - probably in contrib only, only special master/slave set-ups would benefit

I believe that it would be much better to spend time on rewriting capability evaluation and enrolments in 2.0 instead of this project, sorry.


jai gupta added a comment - 04/Jul/09 09:17 PM
Right, I understand.

I am not tracking 2.0 status. Google docs spreadsheet shows 9.9 days left for Database API. If I want to take read/write splitting ahead then when would it be a good idea to start. I can start this as soon as all queries in Moodle 2.0 start using the new database API.


Petr Skoda added a comment - 04/Jul/09 09:22 PM
You can start now, the API is 99.9% finished and all core code is already using it for nearly a year.
Eloy is working on more unit tests, they should be in cvs soon. Current plan is to create missing native mssql drivers and fix oracle driver.

jai gupta added a comment - 04/Jul/09 09:25 PM
Great, thanks.

Juan Segarra Montesinos added a comment - 06/Jul/09 01:27 AM
Hi all

Maybe R/W sppliting is not the best for moodle. Basically in places where write and read operations are performed quickly and in asynchronous or semisynchronius configurations (mysql or postgres replication). We tried this with mysql-proxy and read-write sppliting script with semisynchronous replication, the slave delay made some odd results in the database and errors. But it'll be great to have a second opinion and experience

However I think that stats code is the best place that can benefit from R/W sppliting.

It'll be great to track your progress wih this

Regards


jai gupta added a comment - 06/Jul/09 06:45 AM
If mysql-proxy had been working fine that would have eliminated the need for Moodle (or any other app) to support R/W splitting.

http://forge.mysql.com/wiki/MySQL_Proxy_RW_Splitting
http://jan.kneschke.de/2007/8/1/mysql-proxy-learns-r-w-splitting