OK, I'm assigning to Yu for tomorrow. If someone else wants to grab it sooner (while we sleep) please do!

Going to summarize all the changes in a text file now...

Note that there is one point (point 11) that is where all the movement of data must happen (before deleting some DB objets later).

I'm going to generate one file with all the XMLDB steps above (skipping the point 11 for now).

IMPORTANT NOTE: Point 11 in the script must be edited to add the the logic in the upgrade (creation of new records in new tables and so on).

1) Edit the point 11 in the upgrade script above to add all the logic of the upgrade.
2) The resulting script should be copied and pasted somewhere in the upgrade function (I'm not really sure of where exactly), but these conditions should be satisfied:

a) Users installing 1.8.2+ from scratch, should use, exclusively, the install.xml file
b) Users coming from 1.7.x to 1.8.2+ must execute our new script exclusively. Nothing else.
c) Users coming from 1.8.x to 1.8.2+ must execute current code exclusively. Skipping our new script. While this is a mistake, I'm not really sure of what intermediate status are in that process so, perhaps it will be risky to modify it.

3) Tomorrow I'll play with sequences, trying to create some ddllib function able to reset sequence numbers to the desired one. We'll add then that step to the upgrade file trying to repair potentially wrong sequences (it's risky to do that directly in 1.8 - new function, but I really cannot imagine a better situation).

4) Confirm that both the 1.7.x => 1.8.2+ and the 1.8.x => 1.8.2+ upgrades are working and that the final schema is the same (the one currently represented in install.xml). This will help to be sure than the upgrade to 1.9 will work perfectly.

Hope this helps. Going to bed now. I leave 2) for you. Ciao

1) For new installs, this is ok, because groups table will be empty thus there is nothing to move (we can leave this alone, or fix it)
2) For under 1.7, where rolesactive flag is not set. This needs fixing, because usage of groups is expected (call to new function)
3) For 1.7+ -> 1.8. In this case upgrade_group_db() is called, this needs fixing because usage of groups is expected (call to new function)

I propose we fix it like this

Leave 1) alone, except to change the function to remove code handling data migration. It should just read from the xml (group/install.xml) and install the tables. This is for new install only.

Remove call 2) and 3) (not removing this call, but remove the install code within upgrade_group_db()), and replace with the code in core with what Eloy provided, since we know the exact version

if ($result && $oldversion < XXXXXXXXXX) {
}

Note: there is very little code needed to migrate data for point 11. We only need to insert data into course_groups table, before dropping the courseid field.

/// a) get the current groups, foreach one add an entry in groups_courses_groups
if ($oldgroups = get_records('groups')) {
foreach ($oldgroups as $oldgroup) { $rec = new Object(); $rec->courseid = $oldgroup->courseid; $rec->groupid = $oldgroup->id; $rec->timeadded = $oldgroup->timemodified; // I think this is not needed since the field is gone? insert_record('groups_courses_groups', $rec); }
}

That was all I found out.

Though I do not really understand when the following code gets called...

if ($result && $oldversion < 2007012400) {
if (table_exists(new XMLDBTable('groups_temp')) && file_exists($CFG->dirroot.'/group/db/install.xml')) {

The temporary table never gets dropped from 1.7->1.8, so the above code is probably run once, which means groups_transfer_db() is probably called twice during installation. I think the second call doesn't do anything since table would have been populated, but this is probably wrong.

I have tested 1.7 > 1.8 (should be same for 1.6>1.8), and new install. These seem to work. However, I suspect some versions of 1.8->1.8 might break on the rename of password->enrolmentkey. So I think we can check for field existence first.

Also, I am setting groups version to 1 after the groups table installation. I think we can potentially test some groups table existence as well, instead of using this hack.

cvs diff is attached. Please review.

Cheers,

Yu

1) New installations. I have copied install.xml from group/db/ into lib/db/ so new install should get all the new tables. In admin/index.php I also set the groups version manually to the latest 2007012401. This would prevent any groups upgrade code from running later on.

2) Upgrade from installtions prior to 1.8. upgrade_group_db() will be called which in turn calls install_groups_db() because there is no groups version . I have changed install_groups_db() to use xmldb to do proper upgrade and migrate. Upon completion groups version is set to the latest 2007012401.

3) Existing 1.8->1.8. upgrade_groups_db() will be called, and should use existing code xmldb_group_upgrade(). I do not have a copy I can test, but the code is unmodified for this bit. The only question is that I have taken out install.xml to main, and bits of the xmldb_group_upgrade() code looks for install.xml in group/db. Do I need to keep a copy of install.xml there as well, or can I just cvs remove install.xml? I do not understand how this code can be run, as 2007012400 is a dev version? I can keep the existing group/db/install.xml if it is needed.

if ($result && $oldversion < 2007012400) {
if (table_exists(new XMLDBTable('groups_temp')) && file_exists($CFG->dirroot.'/group/db/install.xml')) { /// Need to drop foreign keys/indexes added in last upgrade, drop 'new' tables, then start again!! $result = $result && groups_drop_keys_indexes_db(); $result = $result && groups_revert_db($renametemp=false); $result = $result && install_from_xmldb_file($CFG->dirroot.'/group/db/install.xml'); $result = $result && groups_transfer_db(); }
}

Please review/test.

Cheers,

Yu

I've tested both:

• 1.7 => 1.8.2+ with your patch
• 1.8.2 => 1.9beta

with one site, having one course with 2 groups having 2 users each.

and everything is working fine under MySQL, PostgreSQL and MSSQL now (I had to modify slightly the 1.8 => 1.9 but that's another story).

Under Oracle I'm getting one error I've localized (I think). I'll fix it tomorrow.

So only this needs to be tested:

• 1.8.0 => 1.8.2+ with your patch
• 1.8.1 => 1.8.2+ with your patch

If you can do it (I think that if it works with MySQL and PostgeSQL, it will be safe). It requires to install the initial version (from download.moodle.org, perhaps?) and then, perform the upgrade to 1.8.2+ with your patch.

Later, I'll get this again to:

1) Fix the Oracle issue.
2) Try the sequences thing
3) Perform the pending tests (those that you haven't tested tomorrow).

all this to issue the final +1 ASAP. It's really is looking promising. Cool!

Ciao zzzZZZzzz

Tested

• 1.8.0 => 1.8.2+ with my patch (with groupings)
• 1.8.1 => 1.8.2+ with my patch (no more groupings in this version)

both with new installations. Seems to work. However I didn't have postgres and I tested on MSSQL instead. (One calendar SQL bug under MSSQL, same as the one under HEAD, will file separate bug)

Waiting for your test result and signal to commit =-)

Cheers,

Yu

I'll follow progress with interest. Many thanks
Nick

I'm going to test 1.8 => 1.8.2 too under PG and Oracle, but I guess they'll work.

Also I've the readjust_sequence subtask pending, I hope I'll be able to implement that (although I think is shouldn't stop this to be sent to CVS asap).

Ciao

Cheers,

Yu

1.8.0 ==> 1.8.2 (deleted the groupings)
1.8.1 ==> 1.8.2 (did practically nothing)

Oracle coming soon (although, due to the nature of the actions performed in that two upgrades, I guess it'll work).

so I cannot imagine anything against to vote +1 for this to go to 18_STABLE.

And, perhaps, it would be a good idea to release 1.8.3, now that we have tested all this upgrade paths.

Ciao

admin/index.php
lib/db/install.xml

http://docs.moodle.org/en/Development:Unmerged_files

Ciao