Moodle

Hi Garry thanks for pinging me over IM. I'll try to investigate this ASAP!

Hi Gary,

I've been able to backup & restore one course without problems, both in the same server (reusing existing users) and in another server (creating users).

Can you provide some more info like:

• What user does the backup (admin, teacher in course...).
• Does the course include activity from users (forum posts...).
• What user does the restore (admin, teacher in course...).
• Are you running the restore in the same server (existing users) or in a new server.
• If you run the restore with Admin->Server->Debug set to "DEBUG_DEVELOPER"... do you get PHP notices/warnings/errors ?

I continue investigating... but for now I haven't been able to reproduce this... details above will be welcome to try to reproduce the problem here.

TIA and ciao

For most of the variables I've tried both.

As I said in intial bug report, I get the same problem wiht a new install, with new users.
I first noticed it when trying to restore certain courses from backups on a serever wiht existing users.
backup and restore as a teacher, works the same as the admin .. diffferent actal users

Can you provide some more info like:

• What user does the backup (admin, teacher in course...).
  same user does both, tried wiht teacher and admin. Probblem also affects automatic backups

• Does the course include activity from users (forum posts...).
  assignments mainly, wiht word docs handed in

• What user does the restore (admin, teacher in course...).
  same user does both, tried wiht teacher and admin.
• Are you running the restore in the same server (existing users) or in a new server.

both

• If you run the restore with Admin->Server->Debug set to "DEBUG_DEVELOPER"... do you get PHP notices/warnings/errors ?

done this and saved the pages generated

http://glcweb.org/debug/

I'm surprised you ask about the restore .... examination of the xml file in the backup
does not reveal a users section

backups from older versions of the site ... running 1.8.x will restore ok

(through roles on the same server OR through phpmyadmin on a new server)

Thanks for your efforts

Hi Garry,

can you arbitrarily select one of the users that should be present in that backup, for example "John Doe", then execute this query in your server:

SELECT u.*, m.wwwroot
FROM mdl_user u
JOIN mdl_mnet_host m ON u.mnethostid=m.id

And check if "John Doe" is there?

Just suspecting some sort of anomaly in your mnet status... please tell me if "John Doe" is present in the query results or missing. Also, if missing... can you tell me John Doe's value for the field "mnethostid" in DB? And also, can you tell me the different records you've in the mdl_mnet table?

TIA! I continue checking things here... ciao

Look, I'm not brilliant at sql,
I went into phpmyadmin
I selected the database
I went into the mdl_user table
in the sql window I pasted your query

MySQL returned an empty result set (i.e. zero rows). (Query took 0.0119 sec)

Thanks Garry,

and can you please, execute also these two queries and paste results here:

SELECT mnethostid, COUNT(*)
FROM mdl_user
GROUP BY mnethostid

and

SELECT *
FROM mdl_mnet_host

it seems that some sort of problem in the status of mnet is the primary cause of users not being included in backup. The results of these 2 new queries will confirm that.

TIA!

Copied from PHPMYADMIN screen

Showing rows 0 - 0 (1 total, Query took 0.0004 sec)

SQL query:
SELECT mnethostid, COUNT( * )
FROM mdl_user
GROUP BY mnethostid
LIMIT 0 , 30
=============
mnethostid COUNT( * )
1 161

-------------------------
MySQL returned an empty result set (i.e. zero rows). (Query took 0.0004 sec)

SQL query:
SELECT *
FROM mdl_mnet_host
LIMIT 0 , 30

Hi Garry,

thanks again for the information. The lack of records in the mdl_mnet_host table seems to be the cause for backup not including users.... One test and one question:

The TEST:

1) Execute there queries (posting results here, plz):

SELECT * FROM mdl_config where name = 'mnet_localhost_id';
DELETE FROM mdl_config where name = 'mnet_localhost_id';
SELECT * FROM mdl_config where name = 'mnet_all_hosts_id';
DELETE FROM mdl_config where name = 'mnet_all_hosts_id';
SELECT * FROM mdl_mnet_host;
DELETE FROM mdl_mnet_host;

2) Visit Admin->Mnet->Setting (just visit, don't need to save changes nor anything)

3) Execute these queries (posting results here, plz):

SELECT * FROM mdl_mnet_host;
UPDATE mdl_user SET mnethostid = (SELECT MIN(id) FROM mdl_mnet_host); 
SELECT mnethostid, COUNT(*) FROM mdl_user GROUP BY mnethostid

4) Try backup and check if it includes users.

The QUESTION:

Can you, please, comment here what has been your install/upgrade path of that Moodle site? In theory those records in the mdl_mnet_host table are created automatically during installation and should be there always. Obviously they are missing in your site so, somewhere it must be a point of failure. Knowing about your site upgrading route could help to detect that point and fix it.

Ciao

Sorry for the delay ... teachers can encounter very busy times!

query reults from phpmyadmin
=================================
SQL query:
SELECT *
FROM mdl_config
WHERE name = 'mnet_localhost_id';

LIMIT 0 , 30
[ Edit ] [ Explain SQL ] [ Create PHP Code ] [ Refresh ]

row(s) starting from record #
in mode and repeat headers after cells

Full Texts id name value
Edit Delete 140 mnet_localhost_id 1

================================================
Deleted rows: 1 (Query took 0.0054 sec)
SQL query:
DELETE FROM mdl_config WHERE name = 'mnet_localhost_id';

=========================================================
Showing rows 0 - 0 (1 total, Query took 0.0004 sec)
SQL query:
SELECT *
FROM mdl_config
WHERE name = 'mnet_all_hosts_id';

LIMIT 0 , 30
[ Edit ] [ Explain SQL ] [ Create PHP Code ] [ Refresh ]

row(s) starting from record #
in mode and repeat headers after cells

Full Texts id name value
Edit Delete 141 mnet_all_hosts_id 2
With selected: Check All / Uncheck All With selected: Change Delete Export

==============================================================

Deleted rows: 1 (Query took 0.0003 sec)
SQL query:
DELETE FROM mdl_config WHERE name = 'mnet_all_hosts_id';

[ Edit ] [ Create PHP Code ]

==================================================================
MySQL returned an empty result set (i.e. zero rows). (Query took 0.0003 sec)
SQL query:
SELECT *
FROM mdl_mnet_host;

LIMIT 0 , 30
[ Edit ] [ Explain SQL ] [ Create PHP Code ] [ Refresh ]

=================================================================
Deleted rows: 0 (Query took 0.0002 sec)
SQL query:
DELETE FROM mdl_mnet_host;

[
=========================================================

2) Visit Admin->Mnet->Setting (just visit, don't need to save changes nor anything)
not sure on this one ... visited admin/mnet/index.php ?

=======================

3) Execute these queries (posting results here, plz):

• SELECT * FROM mdl_mnet_host;
  =================================
  Showing rows 0 - 1 (2 total, Query took 0.0003 sec)
  SQL query:
  SELECT *
  FROM mdl_mnet_host;

LIMIT 0 , 30
[ Edit ] [ Explain SQL ] [ Create PHP Code ] [ Refresh ]

row(s) starting from record #
in mode and repeat headers after cells

Sort by key:
Full Texts id deleted wwwroot ip_address name public_key public_key_expires transport portno last_connect_time last_log_id force_theme theme applicationid
Edit Delete 3 0 http://www.glcweb.org/moodle19/moodle 72.167.163.185 0 0 0 0 0 0 NULL 1
Edit Delete 4 0 All Hosts 0 0 0 0 0 0 NULL 1
With selected: Check All / Uncheck All With selected: Change Delete Export

=================================

• UPDATE mdl_user SET mnethostid = (SELECT MIN(id) FROM mdl_mnet_host);

Affected rows: 172 (Query took 0.0343 sec)
SQL query:
UPDATE mdl_user SET mnethostid = ( SELECT MIN( id )
FROM mdl_mnet_host ) ;

[ Edit ] [ Create PHP Code ]
=================================

- SELECT mnethostid, COUNT(*) FROM mdl_user GROUP BY mnethostid

Showing rows 0 - 0 (1 total, Query took 0.0004 sec)
SQL query:
SELECT mnethostid, COUNT( * )
FROM mdl_user
GROUP BY mnethostid
LIMIT 0 , 30
[ Edit ] [ Explain SQL ] [ Create PHP Code ] [ Refresh ]

row(s) starting from record #
in mode and repeat headers after cells

Full Texts

mnethostid 	COUNT(*)

3 172

=================================

4) Try backup and check if it includes users.

OK, now backup and restore is working.

==============================================

The QUESTION:

Can you, please, comment here what has been your install/upgrade path of that Moodle site? In theory those records in the mdl_mnet_host table are created automatically during installation and should be there always. Obviously they are missing in your site so, somewhere it must be a point of failure. Knowing about your site upgrading route could help to detect that point and fix it.

My upgrade path

I have not kept detailed records

I ahve installed and reinstalled several times since about 2005

My initial installl for the current site was probably via fantastico

I do vaguely recall ahiving some issues when moving to utf-8 encoding ....

sorry i can't be more specific than that.

I'm still mystified as to why a clean install also gave me the same error ...

I am VERY grateful for the fix, thanks a lot!

Hi Garry,

thanks for feedback! B-)

So, definitively... it seems that missing mnet_host records in your site was the primary problem causing backup to malfunction. And knowing the origin is good, for sure.

I've been playing these days trying to reproduce that (missing mnet_host records) in my test server, performing all sort of installations and upgrades... but I haven't been able to do that.

I any case, I think we must follow a 2-ways approach to solve this:

1) Modify backup and make it "tolerant" to missing mnet_host records. Fix the effect.
2) Fix missing mnet_host records. Fix the cause.

About 1) I've already modified backup and now it's able to backup users no matter of the status of the mnet_host records. That change should be available in next weekly build.

And about 2) I need some feedback here from mnet_gurus. I'm not sure if those missing mnet_host records only affect to the user table or there are more relations out there.

What I'm planning is something like this (in upgrade):

• Check we have the 2 mandatory hosts in mnet_host table (localhost and allhosts). Create if necessary.
• Check CFG->mnet_localhost_id is pointing to localhost record. Fix if necessary.
• Check all users having non-existing mnhostid and assign CFG->mnet_localhost_id to them.

How does this sound? TIA!

I assume you are talking mainly to others? that is mostly gibberish to me!

thanks vvery much

Hehe, yes Garry!

Summarizing... I've fixed backup to work always... but I'm asking (to Nigel and Peter, from NZ - mnet gurus) about the plan above to fix those mnet problems.

Problems that... in your case... were fixed manually with the queries you executed.

So yes, you can ignore this, hehe. Ciao

Hi Eloy, how can I fix the backup so it works always? What do I have to change and where? I am having the same problems you are writing above but I do not know how to apply the solution you mention. Thank you very much.

Hi JL,

the best way to do so is to upgrade your Moodle server to latest 1.9.x+ version, that way you'll get a lot of bug and security fixes (like this) already solved.

Alternatively, you can try to replace the current backup/backuplib.php file in your server with the last one available in 1.9.x+ (don't forget to save the replaced one in a safe place just in case you need to restore it).

That's basically, all, ciao

Re-copying the plan here for last-minute confirmation from moodle/mnet gurus. No objections = I'll implement it tomorrow.

The plan:

• Check we have the 2 mandatory hosts in mnet_host table (localhost and allhosts). Create if necessary.
• Check CFG->mnet_localhost_id is pointing to localhost record. Fix if necessary.
• Check all users having non-existing mnhostid and assign CFG->mnet_localhost_id to them.

Ciao

No objections here.. just a note that long term, I believe Moodle wants all of that mess to go away (I think we discussed this before elsewhere?)

Sorry for the bug-spam folks, I just edited a lot of comments to and code tags, so that ( * ) in SQL did not get turned into a pretty . Now a real comment:

I'm not an mnet expert, so I don't want to vote. However, I have two concerns that I would like answered:

1.

I don't understand why we have bugs relating to mnet_hosts. Surely on original install/upgrade, those two records and CFG->mnet_localhost_id are created, and nothing should ever change them. That does not seem like complicated logic - we should be able to implement it bug free.

Therefore, I would be very reluctant to do anything that just silently fixes things. If those records are getting screwed up, we have bugs and we want Moodle to scream as soon as this problem is detected so we can track down the real bugs. (However, I know we have had bugs related to this for a while)

Therefore, my -1 for doing this every time we do a backup.

My +1 for doing this once in an upgrade on the 1.9 and HEAD branches.
(Make a fix_mnet_hosts function in upgradelib.php, so if this problem reccurs for someone, we can give them a simple script that just calls the function to fix the problem on their site, like sometimes we have to tell people to run build_context_paths.)

Only 31 matches for regex \$DB->.*\bmnet_host\b in HEAD. If there is a bug, it should be findable.

2. I am worried about a potential problem with the 3rd bullet. A scenario something like:
i. User originally gets an account on our server via mnet.
ii. That remote host is removed from the config, but the user record, isn't.
iii. Upgrade script runs, and suddenly that user becomes a local user.
iv. User gets permissions they did not have before, and uses that to hack our Moodle.

I please reassure my that this risk is imaginary.

As far as I recall, Eloy, Peterb and myself discussed doing this once in an upgrade, like you suggest, including putting it in a function for later. Eloy may/may not remember this, but it makes sense to me anyway.

While there may be a bug and it may be findable, the effort involved probably isn't worth it given that we want to remove mnet_localhost_id and those two records anyway.

About the case described by Tim (that really can happen), some comments:

1) Note that I'm not changing the "auth" column at all. So those uses will continue being "mnet" users, so they won't be able to login unless the admin change their auth to manual. Users won't be able to login suddenly after the upgrade, as far as mnet forbids local-login (always return false) for any "mnet" user. In the other side "manual" users with their mnethotid faked will continue working too (as manual doesn't check that field). So no changes expected.

2) In the worst case (user is able to login) he/she will have the roles/permissions as given originally, i.e. that won't be able to do anything different nor escalate perms and so.

3) I't only a DB consistency check to have everything pointing to existing records. I've tried tons of install/upgrade routes and never have ended with any mesh in the mnethostsid field. But obviously, at some point, it has been inestable enough to generate those "phantom" ids.

So, given that, I think I'm going to follow the procedure as described above unless somebody found any problem in the next hours.

Ah, and yes, Nigel, I remember we commented about that time ago (to be applied in HEAD). But it was a chat session, not sure if that has ended documented somewhere. I really hate those 2 "virtual" hosts. Having them out + 0 in users table seems to be far better for me (as agreed).

I just want to leave things consistent before those changes arrive to HEAD. TIA!

Ciao

+1

Just have found this: MDL-18704 will look to it later

Hi,

I've added the upgrade_fix_incorrect_mnethostids() to lib/db/upgradelib.php

it looks for localhost & all_hosts records in mnet_table, creating them if needed, deleting any old record if existed but was incorrect and finally, updating all the users with incorrect mnethostid to the local one.

Note the update of users cannot be done in one simple statement (thanks, bloody MySQL), so its' performed within one recordset, updating by PK, so should be quick (but not quickest).

I haven't implemented the upgrade step yet. Will do that next monday. Please take a look to it.

TIA and ciao

Resolving as fixed.

Now the upgrade_fix_incorrect_mnethostids() function (lib/db/upgradelib.php) will be executed both in 19_STABLE and HEAD and should fix missing mandatory mnet_host records + orphan users.

Ciao