|
[
Permalink
| « Hide
]
Eloy Lafuente (stronk7) added a comment - 22/Feb/07 02:18 AM
Oops, I hope this should be only for 1.9 (if approved). Deleting 1.8 as target.
Oh, yep. I the "hash" name isn't the correct one although I talked about "unique hash". I've attached one simple example of calculating user storage area based one userid.
Ciao for example :
ID= 123 hash_fix_length=6 userid_to_path(ID=123)-> /0/0/0/1/2/3/ ID= 0 ( admin ) hash_fix_length=6 userid_to_path(ID=0)-> /0/0/0/0/0/0/ ID=999999 hash_fix_length=6 userid_to_path(ID=0)-> /9/9/9/9/9/9/ one millon users, or also we can use hexadecimal notation for ID I see if I can make a draft of the other points Is this something we should consider moving to the database?
Profile images I mean... this would still be useful for user files, sure, in the new local repository.
I really think that it's better to have all the user files stored in FS for now (it would be easy to make avatars invisible for users once the local repository is working).
Then, someday, we should be able to abstract all the current file system accesses and create, in a pluggable fashion, all sort of storing systems (file system, local or external DB, other CMS...over mnet...). That's my point of view. Although it would be easy to store them in DB, just analysing if it's the moment to do that. Ciao I'd be a little concerned that the file system starts to look like Blackboards and gets confusing to navigate in the FS explorer.
How about a halfway that caps the number of sub folders to 1000 but remains easy to navigate? If you took the ID 1123 and replaced the last 3 numbers with zeros, or rounded it down, and created another string with 999 added to it you get '1000' and '1999', concatenate these and add the ID after a slash you'd have '/1000-1999/1123/'. This would be infinitely scalable, and I think a lot easier to nativage 'by eye'. The attached patch performs what Red Morris suggests: 1000 users per directory at most, with an easier to navigate directory tree. It uses a 3 level deep structure (which allows for up to 1,000,000,000 users
So a user with an id of 123,456,789 would have a private directory like: $CFG->dataroot/users/123000000-123999999/123456000-123456999/123456789 and a user with an id of 1,234 a directory like: $CFG->dataroot/users/0-999999/1000-1999/1234 As the code only uses integer multiplications and divisions, it should be faster than dealing with non-decimal bases, spliting and rejoining strings, etc. Saludos. Iñaki. A little complex .... how about just:
/user/xxxx/yyyy 1000 containing folders 1-1000 OK, so this is suddenly of major importance because I just discovered the ext3 limit is 32000 directories and moodle.org hit that limit some months ago!
Nicolas, can you do this please? On reflection I think it's even simpler if the first-level names are the LOWER bound of the 1000 range (first one will be 0) So, here are the steps: 1) make_user_directory($userid=45678) which will create moodledata/user/45000/45678 and RETURN the full path '45000/45678' Yu and Petr, can you just check this plan and make sure I haven't forgotten anything?
I am, Tim, but it's further down on the list now.
All implemented and tested
Is this fully working? I just updated my 1.83+ site from MOODLE_18_STABLE and the avatars are shown correctly in the Online users block, but when I click on a user, the avatar shown is the standard smiley face moodle avatar.
I guess (because I haven't seen any change in code), that both backup & restore aren't handling this properly. More exactly these functions:
Ciao I suppose we could implement slices of 10,000 instead of 1,000. It would allow for 320 million users, a figure which will most likely never be reached, and still fit within the 32,000 directories hard limit within each slice.
Backup ... restore .... ?
I tested the latest stable (1.83+) for today, and I'm still having the same problem.
With the pix.php file, version 1.16.2.1, that was modified for this bug, it doesn't load any user avatars, all avatars are displayed as the default Moodle smiley face icon. I rolled back to pix.php version 1.16 and the avatars are properly loaded from the moodledata folder. Backup restore complete in head, coming up in 1.9 and 1.8 (just waiting for CVS update before I commit)
After updating backup restore files in 1.8.3+ there is no change at all.
Nicolas, can you please attend to 1.8 urgently ... it's supposed to be stable but people are reporting problems now ...
Currently the upgrade to 1.8.3+ fails to copy the folders from "moodledata/users" to "moodledata/user/0". We upgraded several sites both on Fedora 6 and Debian Sarge and none copied the user's image folders on the new place.
In all upgrades it has been needed to do it manually. Yep,
fix is in MOODLE_18_STABLE cvs and please do not commit trailing whitespace
Anyway what happens in 1.9/2.0, is the upgrade executed again? What if somebody deletes the picture, would it be copied again from users to user? And please fix the version numbers in MOODLE_19_STABLE the /version.php and lib/db/upgrade.php does not match! Also please keep HEAD and MOODLE_19_STABLE versions in sync for a while, this is important in case somebody accidentally upgrades to 2.0 and it also prevents problems with upgrade in 2.0 (we must not execute the upgrade code both in 1.9 and 2.0).
In 1.8.3+ (2007021532) it works fine now. Thanks!
The fix seems not working for sites already migrated to 1.8.3+ prior to the fix availability: sites where profile images has been compromised by the new user/0 folder still does not shows the images.
Sorry about the issues. The version mismatch is now solved in 1.9, and 2.0 is in sync with it. However, applying this particular upgrade twice will not cause any problems: the files in the old users directory that are now missing will not cause the new ones to be changed or deleted. The upgrade will simply have no effect.
Nicolas, just to shorten the upgrade time from 1.8.3+ to 1.9 ... can you add some code to the upgrade in 1.9 and HEAD so that if the new 'user' folder already exists then the "users" upgrade is simply skipped?
From Andrea's post above, it seems that sites that have executed the previous upgrade have the "0" folder not created (nor those target user dirs copied).
Would be a good idea to add one more upgrade block in order to check if the "0" dir has been created and copy missing target users there? I know it's "only" 1/1000 of users, hehe, but right now their images are in the "limbo", if I'm not wrong. Just one idea, if effectively, those "0" dirs weren't created and filled for early upgraders. Ciao Nicholas - Are there any plans to merge these changes into 17STABLE? I was reading http://moodle.org/mod/forum/discuss.php?d=82610
Please no more non-critical patching in 1.7.x and earlier, we should work on 1.9dev fixing instead
reopening - the restore related code does not work
should be fixed in cvs - tested both new and old backups
A new script, admin/fixuserpix.php, has been added to 1.8 and 1.9. It can either be run in the browser (if logged as admin user) or from CLI (with admin user credentials).
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||