Final push to get Wiki 2.0 in HEAD

First of all, thanks for working on 2.0.

Following is my feedbacks, I am using the code from git://potato.lsi.upc.edu/wiki20, the latest commit was 15th, april.
Commit 1ad992555b1bb74f98a97224c030a0c48b2bd98d

From user point of view:

1. Admin needs to add blocks to create new pages, should we have a few blocks enabled by default (adding them in wiki_add_instace)? The functionalities of module shouldn't depend on blocks.
2. why use pageid to identify wiki page instead of page name?
3. There are many blocks for wiki, IMHO, for sure that's very flexable, but personally, I more like mediawiki's blocks: toolbox, search, navigation, it is easier to setup, we can use capabilities and settings to disable/enable specific items, just my personal opoinion, don't mind me if I am wrong

From developer point of view

File layout

There are quite a few library files: weblib.php locallib.php lib.php and pagelib.php, we discussed this issue before: http://tracker.moodle.org/browse/MDL-21786?focusedCommentId=82198&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_82198

We should store functions and classes used by mod/wiki locally in locallib.php, lib.php stores functions required by core code. externallib.php stores funcitons used by webservices. renderer.php stores rederer subclass.

functions in weblib.php can be converted to a subclass of plugin_renderer_base and move to renderer.php. editors/html.php and editors/wiki_editor.php functions can be converted to methods of renderer class as well

And we will have text parser in moodle core later and probably a wiki type editor.

Coding style issues

1. We don't use CalmelCase for class, parser/*
2. adding moodle license boilerplate to all files
3. if/else block should put a space before and after the control statement in brackets
4. Should use an indent of 4 spaces with no tab characters, for example, pagelib.php line 166, is using tab character
5. boolean value should use lower case: true, false and null
6. put a space between equel sign and letters.
7. put a space between comma and variable, for example pagelib.php line 441
8. "_" prefix not used for private methods in 2.0 anymore, see Petr's comment here: http://tracker.moodle.org/browse/MDL-21958
9. remove global $USER $PAGE from top of every page.
10. $PAGE->__get('course') => $PAGE->course pagelib.php
11. $PAGE->__get('activityrecord') => $PAGE->activityrecord
12. pagelib.php should not use var to decarle class varible, please use private, protected or public instead, for example, page_wiki class and page_wiki_create class in pagelib.php

Please refer moodle coding style: http://docs.moodle.org/en/Development:Coding_style

Javscript:

1. Should organize js files better
2. wiki/editors/wiki/buttons.js is out-dated
3. move wiki/editors/wiki/images/* to mod/wiki/pix/
4. It will be good if wiki module could use yui3, it will help minimise js code and make code easier to maintain
5. improve mod/wiki/diff/diff.js

Please refer moodle javascript guideline: http://docs.moodle.org/en/Development:JavaScript_guidelines

Use New Moodle APIs

1. Rating API
2. Use renderer system: http://docs.moodle.org/en/Development:Output_renderers
3. Use print_paging_bar function? pagelib.php line 1033

Migrating

1. works on fresh installation but doesn't work if it overwrite the existing moodle 2 installation, should use table_exists and field_exists to test before adding tables and fields
2. use upgrade_set_timeout, the upgrade script could be a long process, so it is neccessary to set timeout
3. use get_recordset_sql instead get_records when dealing with a large amount of records, and do remember to close the resultset after finished
4. locks expired time is hard-coded locallib.php wiki_delete_old_locks
5. unused $THEME varible in db/migration/wiki_migrate_definition.php

Misc

1. $DB->insert_record $DB->update_record return boolean value for backward compatibility, please don't depend on this return value, dml uses exception to handle errors, for example: locallib.php line 93 wiki_add_subwiki function and wiki_save_page function, and more in mod/wiki/lib.php
2. redirect function should be used before printing header out, see http://tracker.moodle.org/browse/MDL-21957
3. wiki/styles.css will be loaded automatically, pagelib.php line 78,
4. please don't use $_SESSION, use moodle's $SESSION
5. found a few hard-coded css style in html element, please don't hard-coded css style, always use class name
6. pagelib.php line 372, and html tag should be lowercased
7. Seems there are some strings haven't ben translated
8. function wiki_add_progress, what is this function for?
9. Capabilities are defined, but not in use
10. context level must be checked before use get_coursemodule_from_id
11. parser/utils.php, function h() can be replaced by html_writer:tag()
12. parser/parser.php require_once should move to top of the page
13. move all pictures to pix/
14. editors/wiki_editor.php should be xhtml strict,

    & should be replaced with &

15. db/access.php admin role should be manager add RISK_SPAM riskbitmask to write captype
16. diff.php should use html_writer::table not $OUTPUT->table()
17. html_table class don't have set_classes method

Thanks Dongsheng.

It's obviously been difficult for outsiders to keep up with Moodle APIs and practices, especially since some have changed in 2.0.

I really want to get this into Moodle 2.0 beta, obviously, though it's only 14 days away. I think the way forward is:

1) Dongsheng to work full time now on wiki using contrib CVS so that Pigui can monitor/test it and flag any major regressions: http://cvs.moodle.org/contrib/plugins/mod/wiki/

• fix issues with use of core API and sessions etc
• fix issues with upgrade and test well to make sure it works from 1.9
• move search and index (anything else) into the main interface (central page and navigation) to make it usable without blocks

2) By the beginning of the week we should have something that can go into HEAD, so let's put it there and replace the old wiki code.

3) All next week let's all work very hard on remaining interface issues and possibly text formats to make sure we can release a Beta with the new Wiki in it.

I going to commit last version of wiki 2.0 into moodle repo and start working in these issues.

I have no write access to /contrib/plugins/blocks to remove old wiki blocks and commit the new ones.

About:
"From user point of view:
Admin needs to add blocks to create new pages, should we have a few blocks enabled by default (adding them in wiki_add_instace)? The functionalities of module shouldn't depend on blocks."
That's not true. The "Wiki create page block" is a block that has very much success among nwiki users, but it is not necessary to create page. We don't have it on the old wiki, do we?
The way to create a new page is to refer the new page with [[the page name between double brackets]] or in CamelCase if enabled. The rendered page will automatically link to the page creation link in red color after the page name. Just like mediawiki and all wikis do.
the wiki create page, is just a add on that allos to create pages to the users not iniciated in wikis. And also, provides the list of wanted pages, making it easier to know wht pages need to be written. its a nice add on, that's why we did it :-D , but is not necessary.

Hi Ludo

Thanks for the explanation

I understand we could refer the new page by using interlink syntax in wiki content, but sometimes, users want to create pages not related to existing pages, yes, the old wiki don't have this feature either, but it uses page name in url, so we could type http://moodle.local/mod/wiki/view.php?id=2&page=apple to create a new page named "apple". wiki 2.0 uses pageid, so it is impossible to do such thing.

Not sure if it is just me: the first confusing thing I had when I started to use mediawiki is I don't know how to create a new wiki page! Then I learned we should create new page by typing a new url it isn't quite friendly to wiki newbie, Personally I quite like confluence, it has a good looking 'add page' button, it is a small change but makes this wiki welcome.

About Blocks..

• Wiki index can be squeezed into Navigation
• Search can be placed at the right of the breadcrums bar just like in Forum Search
  If its ok we can get into it rigth now

The "create new page" block could be placed into a new tab. There the list of wanted pages could be available to choose from, or the user could enter the pagename at the risk of making the page orphan.

Functionalities on other blocks.

All the features implemented in blocks (as agreed with petr) are only read only information that the wiki provides.
Proposal: we can add a tab named "toolbox" (like the box in moodle docs) and put there links to all the posible reports provided by blocks : orphaned pages, wanted pages, page list, updated pages, contributions by user, etc ... in time (somewhere in the future) some this reports could also have an RSS, because what you would put into a Wiki RSS is what we display in blocks today.

This can be done AFTER wiki gets into HEAD, because blocks do NOT implement features, just read only reports.

About the upgrade.php

I am not sure why dividing upgrade code into a few blocks by version numbers, are you trying to make different steps of migrating? We usually don't do this in moodle, just bump the version number once, and do everything in one if/else block will be ok.

Do you use xmldb editor, Ludo? we usually use xmldb editor to generate creating/renaming/droping table php code from there, and copy the generate code directly into the upgrade.php

The smaller the upgrade blocks the better, we had several problems with upgrade blocks (1.8 groups for example) that got stuck in the middle and could not be restarted, this also mostly removes the need for table exists and filed exists calls - they may not be necessary, because developers usually use then when they need to bump version repeatedly in the main upgrade while waiting for cvs commit confirmation/review.

Hi,

one of our developers is working on Coding Style issues #1, #5 and #8 and Misc issues #4, #11 and #12.

Now I'm going to talk to another guy to fix more issues.

Hi, Jordi and Ludo

I made a few changes on upgrade.php script, I am not wiki expert, so I attach a patch here for review.

the upgrade script still use small blocks, like Petr said, I bumped the version number as well.

Notes on changes:

1. created upgradelib.php and created a few functions there to migrating versions/installing new tables based on your work
2. create a brand new wiki table, and migrate wiki instances from wiki_old table
3. The migrating history sql seems to have some problems, we need this function support 1.9 and 2.0 both
   • wiki.summary doesn't exist in the wiki in head, and should use wiki.intro in head
   • 'LEFT OUTER JOIN {wiki}

     w ' should be 'LEFT OUTER JOIN

     {wiki_old}

     w'

4. I am not sure if we should delete old wiki tables?

I'm going to try and review your patch right now.

I've seen that you renamed the wiki table, I did the same and Petr told me that it was not a good idea...

I'll be back in few mins.

Hi Dongsheng,

I've found some bugs after applying your patch. How do you want me to proceed? Renaming wiki tables (and reseting ids) has caused some colateral damage...

Hi Jordi

What are the bugs are you talking about? Is it caused by renaming tables? If we don't rename wiki table, we need to change the table structure, and keep consistent with install.xml, not sure which one is better, but I saw workshop module renamed tables. Petr, can you comment this issue?

Hi, Jordi

I removed weblib.php according [Moodle Libraries Organization Structure|http://docs.moodle.org/en/Development:Libraries_Organization#Structure
]

All functions moved to locallib.php except wiki_print_link, because it can be replace by html_writer::link, I already replaced all wiki_print_link in cvs.

Hi Dongsheng,
I see you added a combo with the wiki page list, like in ewiki. It is not a good idea to have it. Gives a wrong way to browse the wiki (we have the wikilinks and the Index in Navigation for that) and can get really mesy with a medium sized wiki witha few dozens of pages. I told Jordi to take it out.

Guys, please be very specific in your comments here, give as much details as you can. There's very little time and the timezone difference is going to kill us here.

My las comment is about a combobox on the top right of the view page ... a minor issue.

been testing several ours, and the wiki is becoming stable. some minors issues that should not affect us this week
this afternoom I will heavy test groups and groupings

I have added all require_capability(). I'm going to add all needed has_capability() right now.

Hi, Ludo and Jordi

Before wiki goes to head, we must keep the database structure stable, can we safely remove wiki_rating table from install.xml and start to use moodle rating api?

About testing, can you also test the migrating from 1.9 and head, let me know if you find any problem, so we can make the upgrade process smoothly, this is very important.

After the discussion in HQ, we decide to keep wiki tabs consistant throughout out every page, currently, page diff and comment editing page have different tabs set, which is strange, because 'current version' == 'view' and 'return to history' == history tab, I made some changes on print_tabs function in pagelib.php to make active tab linked, so user can click history tab.

Also I created diff function in render, so the theme designer can create better diff view, not committed yet, I will commit tomorrow with other minor changes.

Another note here:
we need to display moodle navigation block in very page, to do so, add $cm to require_course_login function, I added a few links to navigation so user can edit/comment/view history from here.

Ratings to moodle rating api... sure thing. we can even disable by now all the ratngs thi g and out it nside later. ratings are related to grading and we already commented with martin that this could be added later

>After the discussion in HQ, we decide to keep wiki tabs consistant throughout out every
>page, currently, page diff and comment editing page have different tabs set, which is
>strange, because 'current version' == 'view' and 'return to history' == history tab, I made
>some changes on print_tabs function in pagelib.php to make active tab linked, so user can
>click history tab.

I beliebe we should keep it as it is (was): In Diff View you need to know what are you looking at, so you need a "diff" tab. If you use View is confusing because you are analyzing two versions of the page... when you say "View" it can mean "view first version in compared diff" , "view second version compared in diff", or "view current version"... not all users will understand the same. he had questions about it with students looong ago. Edit would be more confusing...

I have to disagree there, Ludo. The way the tabs change feels very odd, and having a "Back to X" tab is unique.

I think the best solution is to keep the main tab as "History" (since we ARE actually still in history) and add information under that to make it clear we are looking at diffs and choose between DIff views and versions.

Okidoki boss ,
in this case we should put a "DIFF" label/title under the tab bar, and for sanity shake disable the "Edit" tab - make it gray or whatever.

Hi all,

Are you sure to migrate student wikis into assigments? It would be weird...

What are we going to do with teacher wikis then?

I'm able to add old wiki types to wiki 2.0 in a couple of hours. What do you want me to do? Migration will be easy and painless (for developers, teachers and students)?

If we are able to support the old student types in wiki 2.0 then I really think we should have them. Migrating to other things was only a last resort because I thought student wikis would be too hard to get done in the time available.

The more I looked at those modes yesterday the more useful they looked.

On monday morning first meeting, Jordi an I will estimate how it takes to put student wikis to work.

Hi, Ludo

Question about the version 0 of page, is it supposed to be empty all time?

Because there is no version 0 in wiki of moodle 1.9, after migrating, wiki 2.0 history page will failed to get author by version 0 record.

Hi Donsheng,
yes version 0 is suposed to be empty. Is like this because we can create the page without waiting for the user to commit the edit version. Fixes problems of concurrency and simplifies processes.

ok then, I added a few line to insert a version 0 record during migrating

Latest status of wiki 2.0

• most coding style issues has been fixed
• migrating now works for 1.9 and head.
  TODO:
  • Test it with medium/large size wiki data
  • Move student wiki to online assignment module

• Removed rating table
  TODO
  • Use moodle rating api

• Created wiki renderer, moved most used html component to renderer so theme designer could override it.
• All js files merged into module.js except wikieditor js file
  TODO
  the wiki editor will eventually use moodle core editor (we will provide wiki type editor), so the build-in wiki editor will be replaced later.

UI

• Use consistant tabs in every page
• Added actions to moodle navigation blocks
• Added search
• Added create page form

About Ratings: Let's take out for the moment the Ratings, during the beta phase we will recode ratings and will finish grades without pressure.

Student Wiki. Pigui has analyzed the problem, and we can get student wikis working today. So we don't have to do fancy migrations. Current migration will work for student wikis with minor adjustments. working on it.

UI suggestions:

• Combo box with page list should be removed.
• Comment tab (as discussed on the wiki forum over the years) if no comments should be in red color, should display the number of comments
• Other finesse will be polished without pressure once in head

Hi, Ludo

Combo box with page list should be removed

I will discuss this with Martin, it is ok for small wiki, for medium/large wiki, we should use something else.

Comment tab (as discussed on the wiki forum over the years) if no comments should be in red color, should display the number of comments

Good idea, I will look into this issue.

Hi Dongsheng

combo -> is really ugly , really

Comment tab -> jordi (pigui) commited it several times and it was removed each time , we will push it again

combo thing, expanded... main reason is that the combo is not the way the wiki should be browsed. To use it for small / medium wikis... ejem, and when the wiki grows in page number we take it out and then the users this its a bug ? If we are pushing the index tree in navigation ... it should be consistent IMO

Hi, Ludo

Comment tab -> jordi (pigui) commited it several times and it was removed each time , we will push it again

Apologize if I removed it, I was trying to clean up the code that doesn't work and fix conflict.

combo thing, expanded... main reason is that the combo is not the way the wiki should be browsed. To use it for small / medium wikis... ejem, and when the wiki grows in page number we take it out and then the users this its a bug ? If we are pushing the index tree in navigation ... it should be consistent IMO

So you suggest we put the index in navigation tab with linked pages list?

Hi Donsheng,
to move our "index" block into navigation (moodle 2 navigation block) was one of the things we said to do with Martin, there should also be the orphaned pages and the wanted pages list, because is the "navigation" information for the wiki, and since is hierarchical, will display nicelly for small and huge wikis. This information could be also "replicated" inside the "Navigation" tab ... along with the other information we display in blocks

• user's contributed pages
• latest updates
• page list (alphabetical, usefull to create wiki based glosaries)
  etc
  and when propertly implemented
• ratings
  But this can be done later, once wiki is in head

Hi all,

I working on wiki types. Few seconds ago I commited some code that needs wiki to be re-installed becouse of some changes at database schema. I will keep 'wtype' field from ewiki and I've added some other fields.

I'll be working on this next hours.

Hi, Jordi and Ludo

Is there any news on student wiki? We are about to add wiki to HEAD

Hi all,

I have just commited some changes into cvs. Now wiki supports collaborative and invidividual wikis.

TODO:

• Adapt migration to this new feature
• Print some selects to navigate between wikis
• Add some code to view.php to retrieve the correct page when somebody switches from subwiki.

Thats all.

ps: You will need to re-install the wiki due some changes on database schema.

The migration must work before it can go to head ... how long to do it?

MIGRATION TODO

1) At step 4 wiki entries must migrate to subwiki using wikiid, groupid and USERID.
2) Wikis that are using wiki sintax are not being migrated correctly. Old parser makes stange things....
3) Migrate wiki files and attachments
4) Fill wiki_links table

Thats all.

Hi, Jordi

2) Wikis that are using wiki sintax are not being migrated correctly. Old parser makes stange things....

What is the problem exactly? Can you assign me some TODOs I could help, so we will put it in head on time?

Advise needed.

During migration I need to overwrite student privileges and remove edit page capability in case of old teacher wikis.

It is going to be a good idea to do it as follows:

$studentroles = $DB->get_records('role', array('archetype' => 'student'));
foreach (role and wiki_context)
role_change_permission($studentrole->id, $context, 'mod/wiki:editpage', CAP_PROHIBIT);

Thanks.

hi, Jordi

It looks ok because we didn't change the context of the module.

Great, I commited that changes.

Jordi

Why you added another upgrade.php?

And I got this when upgrading from 1.9

Debug info: Duplicate entry '2-0' for key 'mdl_wikisubw_wikgro_uix'
INSERT into mdl_wiki_subwikis (wikiid, groupid, userid) SELECT e.wikiid, e.groupid, e.userid FROM mdl_wiki_entries_old e
[array (
)]
Stack trace:
line 381 of /lib/dml/moodle_database.php: dml_write_exception thrown
line 545 of /lib/dml/mysqli_native_moodle_database.php: call to moodle_database->query_end()
line 158 of /mod/wiki/db/upgrade.php: call to mysqli_native_moodle_database->execute()
line 510 of /lib/upgradelib.php: call to xmldb_wiki_upgrade()
line 294 of /lib/upgradelib.php: call to upgrade_plugins_modules()
line 1266 of /lib/upgradelib.php: call to upgrade_plugins()
line 268 of /admin/index.php: call to upgrade_noncore()

What...? Where?

mod/wiki/upgrade.php
it supposed to be mod/wiki/db/upgrade.php

Ooops... Sorry, I was an error.

And I got this when upgrading from 1.9

Debug info: Duplicate entry '2-0' for key 'mdl_wikisubw_wikgro_uix'
INSERT into mdl_wiki_subwikis (wikiid, groupid, userid) SELECT e.wikiid, e.groupid, e.userid FROM mdl_wiki_entries_old e
[array (
)]

How did you get this error? e.wikiid, e.groupid, e.userid supposed to be unique.

I suppose that I'm not testing all possible combinations. Can you attach here your wikis backup?

I fixed it, you updated the install.xml but didn't update the php code for db table keys.

I just found more problems, going to work on it later today, can I contact you by jabber, I ready added you, but seems you were not online.

EPIC FAIL!!!

Thanks a lot.

Hi all,

I commited some changes at upgrade/migration.

All old wikis are migrating to new wiki 2.0 structure.

ps: Only files and attatchments issues left to finish migration.

Hi all,

I have commited a patch that renames navigation tab to map tab.
This new tab shows all information that appeared in wiki blocks (navigation, index, page list, orphan pages).

Hi, Jordi

The major problem I had yesterday is the upgrade.php cannot migrate student wiki from 1.9.

The wiki pages with share title will have same content after upgrade, the userid is missing from the upgrade sql.

Please review my fix, it works here, let me know if you find any problem.

Thanks.

Just committed file migrating upgrade section, got it working.
Here is an issue:
In old wiki, images are embedded by [internal:://image.jpg], during upgrade, this link is converted to a strange link: <img src="?binary=internal://image.jpg" alt="internal://image.jpg" />
ewiki does some strange thing here, I have to change ewiki_link_regex_callback function to convert it to file api url.

so the question is is it ok to hack ewiki engine? And do we use this engine except upgrading?

Oh yes, it's totally OK to hack the ewiki engine to improve upgrades. We shouldn't use it at all in the wiki after the upgrade. Great!

Sure, the old ewiki parser is just used to convert ewiki pages into html wiki pages , so we don' t need to use this parser ever again.

Hi Ludo

Tried upgrading old developer wiki (from moodle.org) today, found some problems of processing interlinks, files, all fixed, can you help do a final test, then I will commit to head today.

Thanks.

Wiki 2.0 in head now!

Small problem - I upgraded a clean site (no wikis in it, but it upgraded with no errors) then added a new wiki then tried to add first page and had an error on this URL, just after I had to set the initial page title but the text field was locked.
http://localhost/head/mod/wiki/create.php?action=create&swid=1

Notice: Undefined property: stdClass::$pagetitle in /web/head/mod/wiki/pagelib.php on line 914
Call Stack

1. Time Memory Function Location
   1 0.0004 84540 {main}

   ( ) ../create.php:0
   2 0.2368 20290644 page_wiki_create->create_page( ) ../create.php:91

Debug info: Column 'title' cannot be null
INSERT INTO mdl_wiki_pages (subwikiid,title,cachedcontent,timecreated,timemodified,timerendered,userid,pageviews,readonly) VALUES(?,?,?,?,?,?,?,?,?)
[array (
0 => '1',
1 => NULL,
2 => '',
3 => 1272803451,
4 => 1272803451,
5 => 1272803451,
6 => '2',
7 => 0,
8 => 0,
)]
Stack trace:

• line 381 of /lib/dml/moodle_database.php: dml_write_exception thrown
• line 716 of /lib/dml/mysqli_native_moodle_database.php: call to moodle_database->query_end()
• line 759 of /lib/dml/mysqli_native_moodle_database.php: call to mysqli_native_moodle_database->insert_record_raw()
• line 357 of /mod/wiki/locallib.php: call to mysqli_native_moodle_database->insert_record()
• line 914 of /mod/wiki/pagelib.php: call to wiki_create_page()
• line 91 of /mod/wiki/create.php: call to page_wiki_create->create_page()

By the way, as Dongsheng said, this is in head finally, so hurrah for the milestone

Please work on the the version in HEAD from now on (don't add new blocks yet though). Jordi has write CVS access.

My bad, forgot this case, working on the fix...

I disabled creole and nwiki temporarily, because:
1. We will have text format lib in moodle core, the format will be handle by moodle core eventually.
2. we store links in text by specially format, something like: %%PLUGINFILE%%/filename.ext, but the wiki editor doesn't do this

And the old wiki tables are kept in moodle table, will be removed until we can make sure upgrade 100% work

Hooray!!!

Resolved this issue, the new bus of wiki should create new issues

Thanks everyone.

Great work everybody! Congrats!

Thanks Guys!
Now starts the fun part!

Hi Jordi,

just saw this change: http://cvs.moodle.org/moodle/mod/wiki/backup/moodle2/backup_wiki_stepslib.php?r1=1.1&r2=1.2

and while all the "tabs" and so are ok, I think that you should revert the declaration of all those backup_nested_element() - at the beginning - to their old multi-line appearance. In fact that is the correct way to declare arrays with a lot of elements (for readability).

Ciao