Non-core contributed modules

Add hint capability to preg question

Details

  • Type: New Feature New Feature
  • Status: Development in progress Development in progress
  • Priority: Critical Critical
  • Resolution: Unresolved
  • Affects Version/s: 1.9.9
  • Fix Version/s: 2.0.10
  • Component/s: Question type: Preg
  • Labels:
    None
  • Difficulty:
    Difficult
  • Affected Branches:
    MOODLE_19_STABLE
  • Fixed Branches:
    MOODLE_20_STABLE

Description

Adding DFA regular expression matching we colud achieve hint capability similar to the regex question type, but with much better implementation.

Attachments

  1. Text File
    moodle_exception.txt
    14/Aug/10 11:44 PM
    1 kB
    Dmitriy Kolesov
  1. screenshot-1.jpg
    390 kB
    11/Aug/10 5:53 AM

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Dmitriy Kolesov added a comment -

I'm add hint capability, using determined finite automate to match string and regular expirience.
It's support: character classes, negative character classes, \d\D\w\W\s\S\t character classes,
metasymbol dot, empty in alternative, concatenation, alternative, greed quantificators (* + ? {})
grouping and true forward assert

Please test it.

Show
Dmitriy Kolesov added a comment - I'm add hint capability, using determined finite automate to match string and regular expirience. It's support: character classes, negative character classes, \d\D\w\W\s\S\t character classes, metasymbol dot, empty in alternative, concatenation, alternative, greed quantificators (* + ? {}) grouping and true forward assert Please test it.
Hide
Permalink
Oleg Sychev added a comment -

Dmitriy, could you now rewrite unit-tests for overall question so that they'll use normal regular expression syntax instead of prefix form? This would give people nice examples how to write tests. Please carefully comment at least one of them...

Show
Oleg Sychev added a comment - Dmitriy, could you now rewrite unit-tests for overall question so that they'll use normal regular expression syntax instead of prefix form? This would give people nice examples how to write tests. Please carefully comment at least one of them...
Hide
Permalink
Dmitriy Kolesov added a comment -

Yes. I will rewrite overall tests and carefully comment one of them.

Show
Dmitriy Kolesov added a comment - Yes. I will rewrite overall tests and carefully comment one of them.
Hide
Permalink
Dmitriy Kolesov added a comment -

It's done.

Show
Dmitriy Kolesov added a comment - It's done.
Hide
Permalink
Joseph Rézeau added a comment -

Dmitriy,
Could you please upgrade this plugin's files in http://cvs.moodle.org/contrib/plugins/question/type/preg/
to reflect the recent changes you made?
At the moment, in CONTRIB the version for preg is 2008111100 but in the zip file you attached in this tracker issue, it is 2010080800.
I use the ECLIPSE environment to upgrade my local moodle test site by CVS and I need the files in contrib to be the very latest available.
Thanks in advance,
Joseph

Show
Joseph Rézeau added a comment - Dmitriy, Could you please upgrade this plugin's files in http://cvs.moodle.org/contrib/plugins/question/type/preg/ to reflect the recent changes you made? At the moment, in CONTRIB the version for preg is 2008111100 but in the zip file you attached in this tracker issue, it is 2010080800. I use the ECLIPSE environment to upgrade my local moodle test site by CVS and I need the files in contrib to be the very latest available. Thanks in advance, Joseph
Hide
Permalink
Oleg Sychev added a comment -

Dmitriy:

1. Writing regex to the file to allow JLex read it is a bit insane (and with such specific path to the file as you use it won't work on most systems). This project http://code.google.com/p/phpstringstream/ could help to avoid these problems (see http://code.google.com/p/phpstringstream/wiki/Usage for examples). Please rewrite the code to avoid using files (and convert unit-tests from files too). It would be good to place the constructor and the code for regex parsing in preg_matcher class, since they are identical for any preg matcher (validation is not).This is critical since you thing with harcoded file path just don't run for many people.

2. Prefix notation was used temporary before we had actual parser. It will be best to remove it at all from all unit-tests (and function to parse this notation too), otherwise the function should be thoroughtly tested and a manual for notation written. I suggest converting all unit-test to normal regular expression notation and deleting this temporary code.

3. You have a good collection of tests written as functional tests to old C++ program. It will be good to convert them to the unit-tests for dfa_preg_matcher now.

4. Please rename 'get_result' to the 'match' (correcting unit-tests as needed)

5. It would be better to create directory 'src' and move all source files that doesn't need to run application there (i.e. preg_lexer.lex, parser_preg.y and parser_preg.out etc). It would be good also to create a subdirectories there for JLexPHP (you can place compiled version, since Java files compiled once could be run anywhere) and LemonPHP you are using with an instruction how to run them on preg_lexer.lex and parser_preg.y files), in case something should be modified in scaner or parser.

Show
Oleg Sychev added a comment - Dmitriy: 1. Writing regex to the file to allow JLex read it is a bit insane (and with such specific path to the file as you use it won't work on most systems). This project http://code.google.com/p/phpstringstream/ could help to avoid these problems (see http://code.google.com/p/phpstringstream/wiki/Usage for examples). Please rewrite the code to avoid using files (and convert unit-tests from files too). It would be good to place the constructor and the code for regex parsing in preg_matcher class, since they are identical for any preg matcher (validation is not).This is critical since you thing with harcoded file path just don't run for many people. 2. Prefix notation was used temporary before we had actual parser. It will be best to remove it at all from all unit-tests (and function to parse this notation too), otherwise the function should be thoroughtly tested and a manual for notation written. I suggest converting all unit-test to normal regular expression notation and deleting this temporary code. 3. You have a good collection of tests written as functional tests to old C++ program. It will be good to convert them to the unit-tests for dfa_preg_matcher now. 4. Please rename 'get_result' to the 'match' (correcting unit-tests as needed) 5. It would be better to create directory 'src' and move all source files that doesn't need to run application there (i.e. preg_lexer.lex, parser_preg.y and parser_preg.out etc). It would be good also to create a subdirectories there for JLexPHP (you can place compiled version, since Java files compiled once could be run anywhere) and LemonPHP you are using with an instruction how to run them on preg_lexer.lex and parser_preg.y files), in case something should be modified in scaner or parser.
Hide
Permalink
Oleg Sychev added a comment -

Jospeph - I'll place the code in the CVS after Dmitriy will fix problems described above, especially 1, which may prevent question from working for you unless you use Denwer.

The code will be placed to the HEAD branch, but for now it will work for Moodle 1.9. I'll start porting it to 2.0 soon.

Show
Oleg Sychev added a comment - Jospeph - I'll place the code in the CVS after Dmitriy will fix problems described above, especially 1, which may prevent question from working for you unless you use Denwer. The code will be placed to the HEAD branch, but for now it will work for Moodle 1.9. I'll start porting it to 2.0 soon.
Hide
Permalink
Oleg Sychev added a comment -

Dmitriy, it would be good to place public/protected control for the functions in you classes. It is now in coding guidelines for Moodle 2.0

Show
Oleg Sychev added a comment - Dmitriy, it would be good to place public/protected control for the functions in you classes. It is now in coding guidelines for Moodle 2.0
Hide
Permalink
Joseph Rézeau added a comment -

Dmitry (and Oleg).
Just installed version 2010080800 of preg question type on my moodl e1.9.9 local test site.

1- When I run question/type/preg/simpletest I get :
3/3 test cases complete: 134 passes, 161 fails and 2552 exceptions.
I have no idea what this means (other than preg is probably not working as it should).

2.- LInks to online Help do not work:
Help file "preg.html" could not be found!
Help file "exactmatch.html" could not be found!

3.- When trying out preg, I get this error php notice:
Notice: Trying to get property of non-object in C:\server\htdocs\moodle19dev\moodle\question\type\preg\questiontype.php on line 137

4.- When switching Hint on, when trying question, I get:
Notice: Undefined property: stdClass::$next in C:\server\htdocs\moodle19dev\moodle\question\type\preg\questiontype.php on line 98
Notice: Trying to get property of non-object in C:\server\htdocs\moodle19dev\moodle\question\type\preg\questiontype.php on line 137
Fatal error: Call to a member function get_result() on a non-object in C:\server\htdocs\moodle19dev\moodle\question\type\preg\questiontype.php on line 54

How is the "hint" facility supposed to work?

5.- In my REGEXP question type it is possible to detect the absence of necessary words in the student's answer. Can this (important) feature be achieved in your PREG question type? Please provide real world examples.

Looking forward to a full-working version soon,

Joseph

Show
Joseph Rézeau added a comment - Dmitry (and Oleg). Just installed version 2010080800 of preg question type on my moodl e1.9.9 local test site. 1- When I run question/type/preg/simpletest I get : 3/3 test cases complete: 134 passes, 161 fails and 2552 exceptions. I have no idea what this means (other than preg is probably not working as it should). 2.- LInks to online Help do not work: Help file "preg.html" could not be found! Help file "exactmatch.html" could not be found! 3.- When trying out preg, I get this error php notice: Notice: Trying to get property of non-object in C:\server\htdocs\moodle19dev\moodle\question\type\preg\questiontype.php on line 137 4.- When switching Hint on, when trying question, I get: Notice: Undefined property: stdClass::$next in C:\server\htdocs\moodle19dev\moodle\question\type\preg\questiontype.php on line 98 Notice: Trying to get property of non-object in C:\server\htdocs\moodle19dev\moodle\question\type\preg\questiontype.php on line 137 Fatal error: Call to a member function get_result() on a non-object in C:\server\htdocs\moodle19dev\moodle\question\type\preg\questiontype.php on line 54 How is the "hint" facility supposed to work? 5.- In my REGEXP question type it is possible to detect the absence of necessary words in the student's answer. Can this (important) feature be achieved in your PREG question type? Please provide real world examples. Looking forward to a full-working version soon, Joseph
Hide
Permalink
Joseph Rézeau added a comment -

Oleg: ... which may prevent question from working for you unless you use Denwer.
What is Denwer?

Show
Joseph Rézeau added a comment - Oleg: ... which may prevent question from working for you unless you use Denwer. What is Denwer?
Hide
Permalink
Oleg Sychev added a comment -

Dmitry - please look that all unit-tests passed correctly before downloading next version (and there are no working with files for regexes there anymore, it is quite tricky on the servers!). There is a chance that tests failed for Joseph due to file handling.

I am going to rewrite integration of matcher with the question, but first need a matcher class in good shape from you.

Joseph - 1,3&4 may be result of file handling issues, which should be eliminated soon.

Hint facility is supposed to work like in you question, adding one character to the correct part of the answer.

I think that detecting a missing strings is possible with carefully created regexes for responses without missing string, which should be placed before correct regex. However, I think that this is not really the case for using regular expressions, there are better ways to do this.

I am working on autofeedback question, that will be able to detect and write a messages (without revealing an actual words) about:

  • missing words,
  • superfluous words,
  • misplaced words,
  • typos in the words.

That will be much better than just missing words detection.

Denwer is a useful test-server program, that contains apache+mysql+php preconfigured for testing purposes (windows only), very easy to install. Dmitry hardcoded some Denwer-specific file paths in the program, which may cause some parts to malfunction.

Show
Oleg Sychev added a comment - Dmitry - please look that all unit-tests passed correctly before downloading next version (and there are no working with files for regexes there anymore, it is quite tricky on the servers!). There is a chance that tests failed for Joseph due to file handling. I am going to rewrite integration of matcher with the question, but first need a matcher class in good shape from you. Joseph - 1,3&4 may be result of file handling issues, which should be eliminated soon. Hint facility is supposed to work like in you question, adding one character to the correct part of the answer. I think that detecting a missing strings is possible with carefully created regexes for responses without missing string, which should be placed before correct regex. However, I think that this is not really the case for using regular expressions, there are better ways to do this. I am working on autofeedback question, that will be able to detect and write a messages (without revealing an actual words) about:
  • missing words,
  • superfluous words,
  • misplaced words,
  • typos in the words.
That will be much better than just missing words detection. Denwer is a useful test-server program, that contains apache+mysql+php preconfigured for testing purposes (windows only), very easy to install. Dmitry hardcoded some Denwer-specific file paths in the program, which may cause some parts to malfunction.
Hide
Permalink
Joseph Rézeau added a comment -

Oleg,
Thanks for your reply.
As you say, testing PREG should not rely on Denwer (or other specific software) to be installed. It should all work within Moodle itself.
At the moment it is not clear who is doing what on PREG (between Oleg and Dmitry).

Oleg
I think that detecting a missing strings is possible with carefully created regexes for responses without missing string, which should be placed before correct regex. However, I think that this is not really the case for using regular expressions, there are better ways to do this.

Sorry, but I do not understand what you mean.
For me the proof of concept of PREG is tthat it would allow all the features of my REGEXP question type plus new features. Can you please provide examples (based on my French flag colors question, see REGEXP online help) demonstrating this?

Looking forward to new developments, (please commit all changes to Moodle contrib server).

Joseph

Show
Joseph Rézeau added a comment - Oleg, Thanks for your reply. As you say, testing PREG should not rely on Denwer (or other specific software) to be installed. It should all work within Moodle itself. At the moment it is not clear who is doing what on PREG (between Oleg and Dmitry). Oleg I think that detecting a missing strings is possible with carefully created regexes for responses without missing string, which should be placed before correct regex. However, I think that this is not really the case for using regular expressions, there are better ways to do this. Sorry, but I do not understand what you mean. For me the proof of concept of PREG is tthat it would allow all the features of my REGEXP question type plus new features. Can you please provide examples (based on my French flag colors question, see REGEXP online help) demonstrating this? Looking forward to new developments, (please commit all changes to Moodle contrib server). Joseph
Hide
Permalink
Dmitriy Kolesov added a comment -

Oleg Aleksandrovich,

I will do 1, 3, 4 & 5.
I done 3, see simpletest\testcomplex.php.
Testing on my locale computer with Denwer: 295 passes, 0 fail and 1955 exceptions
(severity: E_NOTICE or E_WARNING).
All is work on my computer.

Show
Dmitriy Kolesov added a comment - Oleg Aleksandrovich, I will do 1, 3, 4 & 5. I done 3, see simpletest\testcomplex.php. Testing on my locale computer with Denwer: 295 passes, 0 fail and 1955 exceptions (severity: E_NOTICE or E_WARNING). All is work on my computer.
Hide
Permalink
Dmitriy Kolesov added a comment -

Done.
1)Lexer get input data from string,
2)LemonPhp files and JlexPhp file move to src folder,
3)Function build_tree move to preg_matcher class from dfa_preg_matcher class (it's parse regex),
4)Function get_result renamed to match,
5)Function form_tree deleted, now use parser.

Show
Dmitriy Kolesov added a comment - Done. 1)Lexer get input data from string, 2)LemonPhp files and JlexPhp file move to src folder, 3)Function build_tree move to preg_matcher class from dfa_preg_matcher class (it's parse regex), 4)Function get_result renamed to match, 5)Function form_tree deleted, now use parser.
Hide
Permalink
Joseph Rézeau added a comment -

Hi Dmitriy and Oleg,
I'm afraid we have a communication problem.
I've just downloaded and tested the latest version of PREG (preg2010-08-10.zip (305 kB)) and I am still getting a lot of error messages when testing. The Hint function does not work (throws fatal errors). There is still no online help available.
Dmitriy does not answer my questions, and writes about files or functions which I do not know about (Lexer, LemonPhp, etc.).

I shall give up testing until your team can provide a fully working version with full instructions,

All the best,

Joseph

Show
Joseph Rézeau added a comment - Hi Dmitriy and Oleg, I'm afraid we have a communication problem. I've just downloaded and tested the latest version of PREG (preg2010-08-10.zip (305 kB)) and I am still getting a lot of error messages when testing. The Hint function does not work (throws fatal errors). There is still no online help available. Dmitriy does not answer my questions, and writes about files or functions which I do not know about (Lexer, LemonPhp, etc.). I shall give up testing until your team can provide a fully working version with full instructions, All the best, Joseph
Hide
Permalink
Dmitriy Kolesov added a comment -

When you do test, which exeptions you get? E_NOTICE and E_WARNING or fatal error? You get fatal error when press hint button?
Are you use Denwer? What are you use? On my computer with Denwer all is work and I can't find error, because I not have error(

Show
Dmitriy Kolesov added a comment - When you do test, which exeptions you get? E_NOTICE and E_WARNING or fatal error? You get fatal error when press hint button? Are you use Denwer? What are you use? On my computer with Denwer all is work and I can't find error, because I not have error(
Hide
Permalink
Joseph Rézeau added a comment -

1- When testing question in preview window I get:
debug mode: ALL: Show all reasonable PHP debug messages
Notice: Trying to get property of non-object in moodle\question\type\preg\questiontype.php on line 137
Notice: Trying to get property of non-object in moodle\question\type\preg\questiontype.php on line 137

2- when clicking Hint button:
Notice: Trying to get property of non-object in moodle\question\type\preg\questiontype.php on line 137
Notice: Trying to get property of non-object in moodle\question\type\preg\questiontype.php on line 137
Fatal error: Call to a member function match() on a non-object in moodle\question\type\preg\questiontype.php on line 54

3.- When saving question, I get a full page of errors, see attached screenshot #1.

4.- I have no idea what Denvwer is and why I should use it. The PREG question should work normally in Moodle.

Joseph

Show
Joseph Rézeau added a comment - 1- When testing question in preview window I get: debug mode: ALL: Show all reasonable PHP debug messages Notice: Trying to get property of non-object in moodle\question\type\preg\questiontype.php on line 137 Notice: Trying to get property of non-object in moodle\question\type\preg\questiontype.php on line 137 2- when clicking Hint button: Notice: Trying to get property of non-object in moodle\question\type\preg\questiontype.php on line 137 Notice: Trying to get property of non-object in moodle\question\type\preg\questiontype.php on line 137 Fatal error: Call to a member function match() on a non-object in moodle\question\type\preg\questiontype.php on line 54 3.- When saving question, I get a full page of errors, see attached screenshot #1. 4.- I have no idea what Denvwer is and why I should use it. The PREG question should work normally in Moodle. Joseph
Hide
Permalink
Oleg Sychev added a comment -

Joseph -
1. I'll load a working version on the CVS once I test it more in 1-2 days, this is preliminary alpha version.
For now the best way is to help us write more unit-tests, look at simpletest/testcomplex.php,
$result->full shows if match is full, $result->index is an index of last correct character, and $result->next is hinted character.
The more unit-tests we have - the better. Unit-testing is much better than manual one. Could you give an example of fails you get during unit-test run now that preg don't use files anymore?

I am not going to write a strict superset of you question just because you wrote it so. I think preg question is about regular expressions and what they could do, not about imitating regex. I didn't like much you solution about missed strings - you repeatedly abuse answers (for entering the correct answer, than for entering missed strings), which makes question editing quite a confusable process (it would be better for you question to add new fiedls to the answer blank for the missing strings and their messages). And I don't know what prevents you from writing a regular expression just omitting missing parts instead of using special syntax for this (knowing they are matched from the first to the last).

For now preg question will be better than regex in hint mode allowing: quantificators *, +, {5,6} (and any other), metacharacter dot (any character) and forward positive asserts.

I beleive in using correct method to do the job. Regular expression isn't a good method to find missing (excess, misplaced) strings - the LCS algorigthm (see http://en.wikipedia.org/wiki/Longest_common_subsequence_problem) used on per-word (instead of per-character) basis is. It will be at core of autofeedback question type. That will require a lot less work from the question author and will result in a better messages (and later when combined with Context-free formal grammar http://en.wikipedia.org/wiki/Context-free_grammar it surpass the regular expressions in power).

Show
Oleg Sychev added a comment - Joseph - 1. I'll load a working version on the CVS once I test it more in 1-2 days, this is preliminary alpha version. For now the best way is to help us write more unit-tests, look at simpletest/testcomplex.php, $result->full shows if match is full, $result->index is an index of last correct character, and $result->next is hinted character. The more unit-tests we have - the better. Unit-testing is much better than manual one. Could you give an example of fails you get during unit-test run now that preg don't use files anymore? I am not going to write a strict superset of you question just because you wrote it so. I think preg question is about regular expressions and what they could do, not about imitating regex. I didn't like much you solution about missed strings - you repeatedly abuse answers (for entering the correct answer, than for entering missed strings), which makes question editing quite a confusable process (it would be better for you question to add new fiedls to the answer blank for the missing strings and their messages). And I don't know what prevents you from writing a regular expression just omitting missing parts instead of using special syntax for this (knowing they are matched from the first to the last). For now preg question will be better than regex in hint mode allowing: quantificators *, +, {5,6} (and any other), metacharacter dot (any character) and forward positive asserts. I beleive in using correct method to do the job. Regular expression isn't a good method to find missing (excess, misplaced) strings - the LCS algorigthm (see http://en.wikipedia.org/wiki/Longest_common_subsequence_problem) used on per-word (instead of per-character) basis is. It will be at core of autofeedback question type. That will require a lot less work from the question author and will result in a better messages (and later when combined with Context-free formal grammar http://en.wikipedia.org/wiki/Context-free_grammar it surpass the regular expressions in power).
Hide
Permalink
Oleg Sychev added a comment -

Dmitriy -

1. You wrote that form_tree deleted, but I did find it in simpletest/testquestiontype.php and many tests there use prefix notation. They should be rewrited.

2. Could you write unit-tests that will test full compliance of scaner and parser with http://www.php.ru/manual/reference.pcre.pattern.syntax.html (and improve a scaner accordingly)?

Show
Oleg Sychev added a comment - Dmitriy - 1. You wrote that form_tree deleted, but I did find it in simpletest/testquestiontype.php and many tests there use prefix notation. They should be rewrited. 2. Could you write unit-tests that will test full compliance of scaner and parser with http://www.php.ru/manual/reference.pcre.pattern.syntax.html (and improve a scaner accordingly)?
Hide
Permalink
Oleg Sychev added a comment -

Dmitriy - look at the lines 46-49 of dfa_preg_matcher.php. Joseph's error is in validation, where you forget to upgrade code for using contructor instead of build_tree call. Please fix it and test question editing too!
Another error is in fclose($file); in preg_matcher class, it is used to call $pseudofile.
Please be careful and don't forget to re-test question after reworking it before uploading.

For tomorrow please:
1. Fix this minor issues
2. Remove all prefix notation from unit-tests (1. in message before)
3. Make all internal functions of dfa_preg_matcher protected.

I'm going to create an Google code repository for our work and rewrite some structural places to give the question more freedom.

Show
Oleg Sychev added a comment - Dmitriy - look at the lines 46-49 of dfa_preg_matcher.php. Joseph's error is in validation, where you forget to upgrade code for using contructor instead of build_tree call. Please fix it and test question editing too! Another error is in fclose($file); in preg_matcher class, it is used to call $pseudofile. Please be careful and don't forget to re-test question after reworking it before uploading. For tomorrow please: 1. Fix this minor issues 2. Remove all prefix notation from unit-tests (1. in message before) 3. Make all internal functions of dfa_preg_matcher protected. I'm going to create an Google code repository for our work and rewrite some structural places to give the question more freedom.
Hide
Permalink
Oleg Sychev added a comment -

Dmitriy - you should look on warnings and notices more thoroughtly. PHP often doesn't give fatal errors where it really should, so many warnings and notices are reporting important problems (like undefined variables or getting property of non-object). Almost all of them should be eliminated.

Show
Oleg Sychev added a comment - Dmitriy - you should look on warnings and notices more thoroughtly. PHP often doesn't give fatal errors where it really should, so many warnings and notices are reporting important problems (like undefined variables or getting property of non-object). Almost all of them should be eliminated.
Hide
Permalink
Dmitriy Kolesov added a comment -

Oleg Aleksandrovich,
I'm sorry. I attach old version by accident. I attached new version now.
I will add tests for parser and lexer tomorrow.

Joseph,
Notices and warnings is not error, it's normal. Trying to get property of non object mean, because I use operator= for property without define variable as object, it's normal. Other notices and warning is not error also.
I have not any fatal error, when I press hint button. I attach old version by accident, may be in new version you will not get fatal error, try it.
Denwer is program for web developing on local computer, I use it and preg work on my computer. If you will use denwer and not get error it's mean than I need other method of search error (without denwer) else I need to find other different between our condition, which is influence to error on pressing of hint button.

Show
Dmitriy Kolesov added a comment - Oleg Aleksandrovich, I'm sorry. I attach old version by accident. I attached new version now. I will add tests for parser and lexer tomorrow. Joseph, Notices and warnings is not error, it's normal. Trying to get property of non object mean, because I use operator= for property without define variable as object, it's normal. Other notices and warning is not error also. I have not any fatal error, when I press hint button. I attach old version by accident, may be in new version you will not get fatal error, try it. Denwer is program for web developing on local computer, I use it and preg work on my computer. If you will use denwer and not get error it's mean than I need other method of search error (without denwer) else I need to find other different between our condition, which is influence to error on pressing of hint button.
Hide
Permalink
Dmitriy Kolesov added a comment -

Oleg Aleksandrovich,

1 & 2 done. I will do protected function tommorrow.
Constructor parse build tree on regex and build dfa on tree, build_tree only build tree without building of dfa, it's different between build_tree and constructor. Dfa not need for validate, therefore I use build_tree.
Validate function is fixed. fclose($pseudofile) now.
Form_tree and prefix form are was deleted.

Show
Dmitriy Kolesov added a comment - Oleg Aleksandrovich, 1 & 2 done. I will do protected function tommorrow. Constructor parse build tree on regex and build dfa on tree, build_tree only build tree without building of dfa, it's different between build_tree and constructor. Dfa not need for validate, therefore I use build_tree. Validate function is fixed. fclose($pseudofile) now. Form_tree and prefix form are was deleted.
Hide
Permalink
Dmitriy Kolesov added a comment -

I will look on warnings and notices, but need some time for it, because there are a lot of warnings and notices.

Show
Dmitriy Kolesov added a comment - I will look on warnings and notices, but need some time for it, because there are a lot of warnings and notices.
Hide
Permalink
Tim Hunt added a comment -

I just want to comment, when you are doing Moodle development, you should have Admin -> Server -> Debugging set to DEVELOPER level, and there should be absolutely no debug output when at that level.

Also, if you are serious about Moodle development, you should find time to read http://docs.moodle.org/en/Development:Coding. (Don't try to read it all at once, just a bit at a time! Start with http://docs.moodle.org/en/Development:Coding_style.)

Show
Tim Hunt added a comment - I just want to comment, when you are doing Moodle development, you should have Admin -> Server -> Debugging set to DEVELOPER level, and there should be absolutely no debug output when at that level. Also, if you are serious about Moodle development, you should find time to read http://docs.moodle.org/en/Development:Coding. (Don't try to read it all at once, just a bit at a time! Start with http://docs.moodle.org/en/Development:Coding_style.)
Hide
Permalink
Oleg Sychev added a comment -

Dmitry, "Trying to get property of non object " usually means you write $xxx->yyy and $xxx is not an object (string or number for example). This is a serious error and should be fixed. You could use print_r($xxx) before the string generating this warning to see what it really is.

To get more insteresting reports on errors you could add following strings to the config.php:
$CFG->debug = 38911;//DEBUG_DEVELOPER
$CRG->debugdisplay = true;

You also may want to add
$CFG->skiplangupgrade = true;
to avoid updating russian languagepack on every update of Moodle code.

Show
Oleg Sychev added a comment - Dmitry, "Trying to get property of non object " usually means you write $xxx->yyy and $xxx is not an object (string or number for example). This is a serious error and should be fixed. You could use print_r($xxx) before the string generating this warning to see what it really is. To get more insteresting reports on errors you could add following strings to the config.php: $CFG->debug = 38911;//DEBUG_DEVELOPER $CRG->debugdisplay = true; You also may want to add $CFG->skiplangupgrade = true; to avoid updating russian languagepack on every update of Moodle code.
Hide
Permalink
Oleg Sychev added a comment -

Dmitry - Please be very careful when writing code, the number of errors in you code is worrying

"Constructor parse build tree on regex and build dfa on tree, build_tree only build tree without building of dfa, it's different between build_tree and constructor. Dfa not need for validate, therefore I use build_tree." You don't have constructor without arguments for dfa_preg_matcher so you coudn't use it that way. I don't recommend adding new constructor without building FA, since it is will need for validation too (both to validate correct answer and to find out if there are too many states in some FA). For testing of parser and scaner you could use preg_matcher class (I'm going to upload new code there today which updates code architecture a bit).

Show
Oleg Sychev added a comment - Dmitry - Please be very careful when writing code, the number of errors in you code is worrying "Constructor parse build tree on regex and build dfa on tree, build_tree only build tree without building of dfa, it's different between build_tree and constructor. Dfa not need for validate, therefore I use build_tree." You don't have constructor without arguments for dfa_preg_matcher so you coudn't use it that way. I don't recommend adding new constructor without building FA, since it is will need for validation too (both to validate correct answer and to find out if there are too many states in some FA). For testing of parser and scaner you could use preg_matcher class (I'm going to upload new code there today which updates code architecture a bit).
Hide
Permalink
Dmitriy Kolesov added a comment -

Oleg Aleksandrovich,

1) A lot of error message about few errors, after fixing constructor disappeared over 1600 error message. There is 636 (0 Trying to get property of non-object) notices and warnings on tests now.
I use "new dfa_preg_matcher;", when dfa_preg_matcher have not constructor without argument in few test. I fix constructor and it not build empty tree now, therefore dissapear "Trying to get property of non-object" and many other errors.

2) You are right, I will use constructor in validate function for validating count of states dfa.

Show
Dmitriy Kolesov added a comment - Oleg Aleksandrovich, 1) A lot of error message about few errors, after fixing constructor disappeared over 1600 error message. There is 636 (0 Trying to get property of non-object) notices and warnings on tests now. I use "new dfa_preg_matcher;", when dfa_preg_matcher have not constructor without argument in few test. I fix constructor and it not build empty tree now, therefore dissapear "Trying to get property of non-object" and many other errors. 2) You are right, I will use constructor in validate function for validating count of states dfa.
Hide
Permalink
Oleg Sychev added a comment -

Dmitriy, please look at new files (commit done,you could merge it with you work using Mercurial). I reworked preg_matcher class to have a good interface and structure suitable to using in the question. It will break many things for now, so you first task is to fix all broken things and move code a bit:

1. Reimplement find_unsupported_operation() as accept_tree (code should be in preg_matcher) and accept_node (code should be in dfa_preg_matcher) functions, they should report errors by adding them to the $this->errors

2. Reimplement validation, using constructor and then check for errors, that was created by accept_tree and other functions - these errors should be showed to the user (so they should actually use get_string, thought my errors don't follow this for now)

3. match function in dfa_preg_matcher should now be match_inner. It should fill $this->full, $this->index and $this->next, but don't return anything; match function is defined in preg_matcher and takes care of caching results

4. match function now returns only full match instead of result - other results are available via special functions. This breaks unit-tests and many other things (i'm sorry), but was necessary because some matchers could support only full matching. Fix unit-tests (i think 'replace' could help you a lot) and validation, don't worry about questiontype class which i'm going to rewrite anyway.

Please fix it tomorrow and hunt down remaining notices, posting result there so I could rework question type.

P.S. I guess it would be good to rename parser_preg.php to preg_parser.php and place both parser and lexer under version control.

Show
Oleg Sychev added a comment - Dmitriy, please look at new files (commit done,you could merge it with you work using Mercurial). I reworked preg_matcher class to have a good interface and structure suitable to using in the question. It will break many things for now, so you first task is to fix all broken things and move code a bit: 1. Reimplement find_unsupported_operation() as accept_tree (code should be in preg_matcher) and accept_node (code should be in dfa_preg_matcher) functions, they should report errors by adding them to the $this->errors 2. Reimplement validation, using constructor and then check for errors, that was created by accept_tree and other functions - these errors should be showed to the user (so they should actually use get_string, thought my errors don't follow this for now) 3. match function in dfa_preg_matcher should now be match_inner. It should fill $this->full, $this->index and $this->next, but don't return anything; match function is defined in preg_matcher and takes care of caching results 4. match function now returns only full match instead of result - other results are available via special functions. This breaks unit-tests and many other things (i'm sorry), but was necessary because some matchers could support only full matching. Fix unit-tests (i think 'replace' could help you a lot) and validation, don't worry about questiontype class which i'm going to rewrite anyway. Please fix it tomorrow and hunt down remaining notices, posting result there so I could rework question type. P.S. I guess it would be good to rename parser_preg.php to preg_parser.php and place both parser and lexer under version control.
Hide
Permalink
Dmitriy Kolesov added a comment -

Ok.

Show
Dmitriy Kolesov added a comment - Ok.
Hide
Permalink
Dmitriy Kolesov added a comment -

Merged and fixed. Notices and warnings fixed. And few minor fixes.

Show
Dmitriy Kolesov added a comment - Merged and fixed. Notices and warnings fixed. And few minor fixes.
Hide
Permalink
Joseph Rézeau added a comment -

Installed 7. Zip Archive preg2010-08-14.zip (321 kB)

Fatal error: Class 'moodle_exception' not found in \moodle\question\type\preg\preg_exception.php on line 15

Show
Joseph Rézeau added a comment - Installed 7. Zip Archive preg2010-08-14.zip (321 kB) Fatal error: Class 'moodle_exception' not found in \moodle\question\type\preg\preg_exception.php on line 15
Hide
Permalink
Dmitriy Kolesov added a comment -

Joseph,

class moodle_exception from new version of moodle, you can paste it in file setuplib.php and it will be work.
File moodle_exception.txt contain code for paste.

Show
Dmitriy Kolesov added a comment - Joseph, class moodle_exception from new version of moodle, you can paste it in file setuplib.php and it will be work. File moodle_exception.txt contain code for paste.
Hide
Permalink
Joseph Rézeau added a comment -

Dmitriy : "class moodle_exception from new version of moodle, you can paste it in file setuplib.php and it will be work."
Does this mean that preg question will not work on a "standard" moodle 1.9.9 version?
I do not want to hack my moodlle 1.9 core files on my test site at the moment.
Why cannot you make a working version of preg question for standard moodle 1.9.9?

Show
Joseph Rézeau added a comment - Dmitriy : "class moodle_exception from new version of moodle, you can paste it in file setuplib.php and it will be work." Does this mean that preg question will not work on a "standard" moodle 1.9.9 version? I do not want to hack my moodlle 1.9 core files on my test site at the moment. Why cannot you make a working version of preg question for standard moodle 1.9.9?
Hide
Permalink
Oleg Sychev added a comment -

Joseph - this is more complex question than you think. Exceptions available from PHP 5, and strictly Moodle 1.9 plugins should work on PHP 4.

Don't use exceptions means work, many dull work that will be not needed since Moodle 2.0, which requires PHP 5. At least I'm not going to do it now, when work required on more serious things. However, I could easy do version for Moodle 1.9 running under PHP 5, if that's suit you.

Technically, you could comment out requiring preg_exception class and all lines with 'throw' statements, and be fine under PHP 4, since exceptions are meant to work under severe programm errors, not in the normal working process.

Show
Oleg Sychev added a comment - Joseph - this is more complex question than you think. Exceptions available from PHP 5, and strictly Moodle 1.9 plugins should work on PHP 4. Don't use exceptions means work, many dull work that will be not needed since Moodle 2.0, which requires PHP 5. At least I'm not going to do it now, when work required on more serious things. However, I could easy do version for Moodle 1.9 running under PHP 5, if that's suit you. Technically, you could comment out requiring preg_exception class and all lines with 'throw' statements, and be fine under PHP 4, since exceptions are meant to work under severe programm errors, not in the normal working process.
Hide
Permalink
Joseph Rézeau added a comment -

Oleg,
Since the preg question will - eventually - have to work in the new moodle 2.0 version - with PHP5 - why not make it work in moodle 2.0 right now.
I have a regularly updated version of moodle 2.0 (at the moment in the preview stage) on my local computer and can easily conduct tests of preg in moodle 2.0.
However, this means that a few changes will have to be made for preg to be fully moodle 2.0 compliant.
I've just tried your latest preg on my latest moodle 2.0 and am getting this (fatal) error message:
stripslashes_safe() not available anymore

I am still willing to test your preg question but I do not want to have to hack either my moodle files or your preg files for that.
All I need is a fully working version of preg for moodle 1.9 or - preferably - for moodle 2.0.

Show
Joseph Rézeau added a comment - Oleg, Since the preg question will - eventually - have to work in the new moodle 2.0 version - with PHP5 - why not make it work in moodle 2.0 right now. I have a regularly updated version of moodle 2.0 (at the moment in the preview stage) on my local computer and can easily conduct tests of preg in moodle 2.0. However, this means that a few changes will have to be made for preg to be fully moodle 2.0 compliant. I've just tried your latest preg on my latest moodle 2.0 and am getting this (fatal) error message: stripslashes_safe() not available anymore I am still willing to test your preg question but I do not want to have to hack either my moodle files or your preg files for that. All I need is a fully working version of preg for moodle 1.9 or - preferably - for moodle 2.0.
Hide
Permalink
Oleg Sychev added a comment -

Dmitriy - this is my second major rework: reworked question type, editing form and database.
I lack time to fully test it, but it would be done (you could do it too) - but I post the code so you could do you work. As before I committed my changes so you could merge them if you do any work in the meantime.

Things for you attention:
1. You probably should delete all traces of question from you db and reinstall it from scratch, since I was forced to edit you fields in install.xml/upgrade/php - some types were choosed wrongly, upgrade procedure won't work for you
2. The question no longer changes response to hint, it should show hinted character on yellow background instead (I wrote code to do so).

You tasks for now:
1. Upgrade DFA matcher to work like any normal regex matcher. I.e. if regexp doesn't start with ^ then match shouldn't necessary start from the start of the string, it could start later (i.e. look for the longest partial or first full match) - you should now set index_first to the index of first character. And if regex doesn't end with $ then even full match could end before end of the string, so $full==true doesn't mean $index == strlen($string)-1. Please add unit tests for this functionality and upgrade lexer/parser as necessary. This should not be very hard.
2. Add case-insensitive mode to matching (modifier 'i' for matcher class). This should be relatively easy, but please add a couple of unit-tests.
3. You shoudn't use preg_match to find syntax errors in regex. Lexer/parser should do that, as they could give more appropriate error messages.
4. dfa_preg_matcher should count the number of states while building automate, so it could stop building and add an error (instead of you code, that check that number after building, which could take a lot of time)
5. I remind you to rename parser_preg.php to preg_parser.php As you are probably going to update parsing, it is good time to do renaming.

Joseph - I hope that after one more iteration from Dmitry, with those 5 things fixed, I could do Moodle 2.0 version for you. I commit it to the CVS.

Show
Oleg Sychev added a comment - Dmitriy - this is my second major rework: reworked question type, editing form and database. I lack time to fully test it, but it would be done (you could do it too) - but I post the code so you could do you work. As before I committed my changes so you could merge them if you do any work in the meantime. Things for you attention: 1. You probably should delete all traces of question from you db and reinstall it from scratch, since I was forced to edit you fields in install.xml/upgrade/php - some types were choosed wrongly, upgrade procedure won't work for you 2. The question no longer changes response to hint, it should show hinted character on yellow background instead (I wrote code to do so). You tasks for now: 1. Upgrade DFA matcher to work like any normal regex matcher. I.e. if regexp doesn't start with ^ then match shouldn't necessary start from the start of the string, it could start later (i.e. look for the longest partial or first full match) - you should now set index_first to the index of first character. And if regex doesn't end with $ then even full match could end before end of the string, so $full==true doesn't mean $index == strlen($string)-1. Please add unit tests for this functionality and upgrade lexer/parser as necessary. This should not be very hard. 2. Add case-insensitive mode to matching (modifier 'i' for matcher class). This should be relatively easy, but please add a couple of unit-tests. 3. You shoudn't use preg_match to find syntax errors in regex. Lexer/parser should do that, as they could give more appropriate error messages. 4. dfa_preg_matcher should count the number of states while building automate, so it could stop building and add an error (instead of you code, that check that number after building, which could take a lot of time) 5. I remind you to rename parser_preg.php to preg_parser.php As you are probably going to update parsing, it is good time to do renaming. Joseph - I hope that after one more iteration from Dmitry, with those 5 things fixed, I could do Moodle 2.0 version for you. I commit it to the CVS.
Hide
Permalink
Oleg Sychev added a comment -

Dmitry - please don't forget to upgrade Moodle 1.9 to latest version too (or just start from the fresh install), because this version use one patch to the Moodle that was added only recently. Question would not work on the old Moodle versions.

Show
Oleg Sychev added a comment - Dmitry - please don't forget to upgrade Moodle 1.9 to latest version too (or just start from the fresh install), because this version use one patch to the Moodle that was added only recently. Question would not work on the old Moodle versions.
Hide
Permalink
Dmitriy Kolesov added a comment -

Oleg Aleksandrovich,

4 will some differently, because count of passages, but not count of state, give a lot of time of building. Compare time of building dfa on a{99,100} (102 states, 103 passages) and a{1,51} (also 102 states, but over 1k passages ), it give very different time, second very great time. Therefore control of passages count, not states count.

Show
Dmitriy Kolesov added a comment - Oleg Aleksandrovich, 4 will some differently, because count of passages, but not count of state, give a lot of time of building. Compare time of building dfa on a{99,100} (102 states, 103 passages) and a{1,51} (also 102 states, but over 1k passages ), it give very different time, second very great time. Therefore control of passages count, not states count.
Hide
Permalink
Oleg Sychev added a comment -

Dmitry,

4. it'sOK to count passages (even better both passages and states). But it should be done during building FA, not by count() function later (a perfect place to use exceptions, but it should be catched and processed to the error message in $this->errors). Otherwise you may not know that you have too many passages because you'll never end building FA.

I hope you understand that 1. could be done easily by trying to match starting from each character in the string, just use loop. But be careful and don't truncate string to start matching from second character - add 'offset' argument to you compaer function instead: there may be difference between these results when using backward asserts etc (that could be supported later). I suggest place loop and finding best match in match_inner() while looking for match with offset in you compare() function.

Show
Oleg Sychev added a comment - Dmitry, 4. it'sOK to count passages (even better both passages and states). But it should be done during building FA, not by count() function later (a perfect place to use exceptions, but it should be catched and processed to the error message in $this->errors). Otherwise you may not know that you have too many passages because you'll never end building FA. I hope you understand that 1. could be done easily by trying to match starting from each character in the string, just use loop. But be careful and don't truncate string to start matching from second character - add 'offset' argument to you compaer function instead: there may be difference between these results when using backward asserts etc (that could be supported later). I suggest place loop and finding best match in match_inner() while looking for match with offset in you compare() function.
Hide
Permalink
Oleg Sychev added a comment -

Dmitry, this is fixed version - use the commit to merge it with you work.

Trying to edit question with DFA engine I found this strange bug: expression
[0-9]+
doesn't match with
123
using DFA engine (match perfectly using preg_php_matcher). Please look what going on.

Show
Oleg Sychev added a comment - Dmitry, this is fixed version - use the commit to merge it with you work. Trying to edit question with DFA engine I found this strange bug: expression [0-9]+ doesn't match with 123 using DFA engine (match perfectly using preg_php_matcher). Please look what going on.
Hide
Permalink
Oleg Sychev added a comment -

I'll probably be unavailable next week.

Dmitriy - please still post there new files after you fixed 1-5 from my previous post.

Then you could work on long-term goals:
1. Implement finding how many characters left to complete the matching (and make sure the next character hinted is from shortest path to the matching).
2. Write unit-tests that will test full compliance of scaner and parser with http://www.php.ru/manual/reference.pcre.pattern.syntax.html (and improve a scaner/parser accordingly).
3. Improve error-reporting from parser to catch common errors (like incorrect brackets), using Lemon error-reporting functions.
4. Make refactoring of dfa_preg_matcher we discussed.
5. Increase operations supported by dfa_preg_matcher, to the fullest extent DFA matcher could do it.

Joseph - please wait for me to commit tested version for 2.0 in the HEAD, don't test Dmitriy files (they would be for 1.9 with exceptions). I created 2.0 Mercurial branch for myself, but have to wait last fix from Dmitriy.

Show
Oleg Sychev added a comment - I'll probably be unavailable next week. Dmitriy - please still post there new files after you fixed 1-5 from my previous post. Then you could work on long-term goals: 1. Implement finding how many characters left to complete the matching (and make sure the next character hinted is from shortest path to the matching). 2. Write unit-tests that will test full compliance of scaner and parser with http://www.php.ru/manual/reference.pcre.pattern.syntax.html (and improve a scaner/parser accordingly). 3. Improve error-reporting from parser to catch common errors (like incorrect brackets), using Lemon error-reporting functions. 4. Make refactoring of dfa_preg_matcher we discussed. 5. Increase operations supported by dfa_preg_matcher, to the fullest extent DFA matcher could do it. Joseph - please wait for me to commit tested version for 2.0 in the HEAD, don't test Dmitriy files (they would be for 1.9 with exceptions). I created 2.0 Mercurial branch for myself, but have to wait last fix from Dmitriy.
Hide
Permalink
Dmitriy Kolesov added a comment -

1. DFA matcher was upgraded for support match part of string with regex.
2.Case insensitive mode was added.
3.Search for syntax error with my parser class, not use preg_match more.
4.Added validation for count of states and passages.
5.parser_preg renamed to preg_parser.

P.s. it's for moodle 1.9.9 with exceptions.

Show
Dmitriy Kolesov added a comment - 1. DFA matcher was upgraded for support match part of string with regex. 2.Case insensitive mode was added. 3.Search for syntax error with my parser class, not use preg_match more. 4.Added validation for count of states and passages. 5.parser_preg renamed to preg_parser. P.s. it's for moodle 1.9.9 with exceptions.
Hide
Permalink
Oleg Sychev added a comment -

Dmitriy

1. I don't like the way you implemented case insensitive matching. Static recursive function is too complex way to do this. In dfa_preg_matcher there should be code where character from the string compared with regex leaf (character or character class). It could be easily modified to ignore case if 'i' present in $this->modifiers. Another way is to do strtolower() on the leafs during parsing in case insensitive mode, it may be good for all matchers. There is no need to walk over tree one more time.
2. What you called 'lock' in regex usually called 'anchor'. It will be better to rename associated fields etc. Also please add a comment to the field you added to the abstract class.
3. There should be unit-tests (both for match and don't match) for situations like:
3.1. ^ab
3.2. ab$
3.3. ^ab$
If some tests are missing, please add them.

Show
Oleg Sychev added a comment - Dmitriy 1. I don't like the way you implemented case insensitive matching. Static recursive function is too complex way to do this. In dfa_preg_matcher there should be code where character from the string compared with regex leaf (character or character class). It could be easily modified to ignore case if 'i' present in $this->modifiers. Another way is to do strtolower() on the leafs during parsing in case insensitive mode, it may be good for all matchers. There is no need to walk over tree one more time. 2. What you called 'lock' in regex usually called 'anchor'. It will be better to rename associated fields etc. Also please add a comment to the field you added to the abstract class. 3. There should be unit-tests (both for match and don't match) for situations like: 3.1. ^ab 3.2. ab$ 3.3. ^ab$ If some tests are missing, please add them.
Hide
Permalink
Dmitriy Kolesov added a comment -

Oleg Aleksandrovich,

I do 1,2 & 3. I convert charclasses in connection field of matcher (it's two dimension array, two nesting foreach loops for exhaustion it).
Also I implement wave for searching count of left characters and add few unit tests for parser and lexer. I'm not add supporting of new operations yet.

P.S. this version of preg for moodle 1.9.9 with exceptions.

Show
Dmitriy Kolesov added a comment - Oleg Aleksandrovich, I do 1,2 & 3. I convert charclasses in connection field of matcher (it's two dimension array, two nesting foreach loops for exhaustion it). Also I implement wave for searching count of left characters and add few unit tests for parser and lexer. I'm not add supporting of new operations yet. P.S. this version of preg for moodle 1.9.9 with exceptions.
Hide
Permalink
Oleg Sychev added a comment -

Dmitriy - Oh, I hoped I could accept the result. But there were two problems:

1. When renaming you wrote 'unchor' instead of 'anchor' How could you miss it?
2. You 'connection' table seems to store characters in each connection, which consumes a lot of memory (how many times the same value are duplicateed?) You should store a reference (or index if they are indexed) to the leaf instead, then no leaf characters will be duplicated.

Show
Oleg Sychev added a comment - Dmitriy - Oh, I hoped I could accept the result. But there were two problems: 1. When renaming you wrote 'unchor' instead of 'anchor' How could you miss it? 2. You 'connection' table seems to store characters in each connection, which consumes a lot of memory (how many times the same value are duplicateed?) You should store a reference (or index if they are indexed) to the leaf instead, then no leaf characters will be duplicated.
Hide
Permalink
Dmitriy Kolesov added a comment -

Oleg Aleksandrovich,

I fix it. Unchor renamed to anchor and dfa_preg_matcher::connection is array of link to string.
I hope you accept it, and it will be beta.

Show
Dmitriy Kolesov added a comment - Oleg Aleksandrovich, I fix it. Unchor renamed to anchor and dfa_preg_matcher::connection is array of link to string. I hope you accept it, and it will be beta.
Hide
Permalink
Oleg Sychev added a comment -

Joseph - I committed BETA-version to the HEAD contrib CVS. It should basically works - for Moodle 2.0. To use hint choose Deterministic finite state automata engine. Please test it on all regular expressions you could and report problems there. Feel free to help us write help strings - you should know English better. You should also know better what should be described thoroughtly to regular teacher about regular expressions.

Dmitry - please write full list of operations and escape-sequences (like \n) supported by you engine for now. Please don't add new operations before refactoring we discussed.

Show
Oleg Sychev added a comment - Joseph - I committed BETA-version to the HEAD contrib CVS. It should basically works - for Moodle 2.0. To use hint choose Deterministic finite state automata engine. Please test it on all regular expressions you could and report problems there. Feel free to help us write help strings - you should know English better. You should also know better what should be described thoroughtly to regular teacher about regular expressions. Dmitry - please write full list of operations and escape-sequences (like \n) supported by you engine for now. Please don't add new operations before refactoring we discussed.
Hide
Permalink
Dmitriy Kolesov added a comment -

My engine support next operations now:
1) character class ( a [a] [abc] [a-c] [a-c3-6] and [^...] ),
2) metasymbol dot,
3) empty (abc|),
4) concatenation,
5) alternative,
6) greed iteration ( * ),
7) greed quantificator plus ( + ),
8) greed quantificators {} ( {3} {,3} {3,} {3,6} ),
9) grouping ( (?:...) ),
10) true forward assert ( (?=...) ),
11) \w\W\s\S\t\d\D ( \w mean [a-zA-Z_0-9] ),
12) hex code of character ( \xhh ),
13) oct code of character (\0dd or \ddd ),
14) greed quantificator ( ? ).
Any other operations is unsupporting now.

Show
Dmitriy Kolesov added a comment - My engine support next operations now: 1) character class ( a [a] [abc] [a-c] [a-c3-6] and [^...] ), 2) metasymbol dot, 3) empty (abc|), 4) concatenation, 5) alternative, 6) greed iteration ( * ), 7) greed quantificator plus ( + ), 8) greed quantificators {} ( {3} {,3} {3,} {3,6} ), 9) grouping ( (?:...) ), 10) true forward assert ( (?=...) ), 11) \w\W\s\S\t\d\D ( \w mean [a-zA-Z_0-9] ), 12) hex code of character ( \xhh ), 13) oct code of character (\0dd or \ddd ), 14) greed quantificator ( ? ). Any other operations is unsupporting now.
Hide
Permalink
Joseph Rézeau added a comment -

Oleg,
Just installed the preg question type on my local installation of Moodle 2.0 Preview 4+ (Build: 20100905).
Notifications send this error message:

Plugin "qtype_preg" is defective or outdated, can not continue, sorry.

Please check that current version of preg is compatible with present Moodle 2.0 Preview 4+ (Build: 20100905) version.
Joseph

Show
Joseph Rézeau added a comment - Oleg, Just installed the preg question type on my local installation of Moodle 2.0 Preview 4+ (Build: 20100905). Notifications send this error message: Plugin "qtype_preg" is defective or outdated, can not continue, sorry. Please check that current version of preg is compatible with present Moodle 2.0 Preview 4+ (Build: 20100905) version. Joseph
Hide
Permalink
Oleg Sychev added a comment -

Joseph - it was just versions number, 20100903 doesn't require any specific number from plugins while 20100905 requires it. You should see similar problems with regex.

Should be fixed now, please try again.

Dmitry - why you dfa matcher tells it isn't supporting subpatterns when simple brackets for grouping are used?
People often use () just instead of (?, since it works the same way when you don't use subpattern matching.
I think you should allow simple brackets for grouping operators, just don't treat them as subpatterns, or many people will be unhappy.

I think it would be better to add new constant SUBPATTERN_MATCHING like PARTIAL_MATCHING etc so engine could tell us if it support subpattern matching. But in abscence of subpattern matching you should just treat ( ) as regular grouping. Please fix it fast and post solution there since it is widely used.

Show
Oleg Sychev added a comment - Joseph - it was just versions number, 20100903 doesn't require any specific number from plugins while 20100905 requires it. You should see similar problems with regex. Should be fixed now, please try again. Dmitry - why you dfa matcher tells it isn't supporting subpatterns when simple brackets for grouping are used? People often use () just instead of (?, since it works the same way when you don't use subpattern matching. I think you should allow simple brackets for grouping operators, just don't treat them as subpatterns, or many people will be unhappy. I think it would be better to add new constant SUBPATTERN_MATCHING like PARTIAL_MATCHING etc so engine could tell us if it support subpattern matching. But in abscence of subpattern matching you should just treat ( ) as regular grouping. Please fix it fast and post solution there since it is widely used.
Hide
Permalink
Joseph Rézeau added a comment -

09:47 15/09/2010
JR testing of Oleg's preg question type
---------------------------------------
What is absolutely needed at the moment is a set of real-world examples. I mean not examples from the world of computing or mathematics, but from everyday language, such as my "French flag colors" example.

I have no idea how to use the Hint feature. It never works. Please provide a step-by-step scenario.
I have no idea how the Hint grade border setting works.

note.- PHP preg extension engine doesn't support hinting
the Use hint button should be disabled if Deterministic finite state automata engine is selected (I see in your script that you want to implement this feature)

The online help is definitely written for computer programers, not for ordinary users (teachers). I hardly understand it myself. For instance I have no idea what the Deterministic finite state automata engine is or what it does. What I need, again is a set of real world examples of how this works.

The question editing interface
The general ergonomy is OK, and works well in Moodle 2.0.

The Exact matching setting is saved when the question is saved. But it is not retrieved from database when editing question again. It is always set at Yes.

The coloring of the student's answer (blue and red) is done from a programer's point of view, not from a pedagogic point of view. You color in blue those elements which "match" a regular expression, whether that expression was set to catch correct or incorrect elements in the answer. This does not mean anything to the student. Please color in blue the "correct" elements and in red the "incorrect" elements (as per my REGEXP question type).

—
If I use this expression: (the French flag|it) is blue, white(,| and) red with the PHP preg extension engine, it works OK.
But if I turn on the Deterministic finite state automata engine, I get an error message Subpattern is unsupported.
—
From a pedagogic point of view, at the moment I see no benefit in the PREG question type over my REGEXP question type. It lacks an important feature of REGEXP: detecting missing strings in the student's answer. Hint facility not working. And I do not see which extra facilities it offers. Please re-state your case, providing real world examples.

Show
Joseph Rézeau added a comment - 09:47 15/09/2010 JR testing of Oleg's preg question type --------------------------------------- What is absolutely needed at the moment is a set of real-world examples. I mean not examples from the world of computing or mathematics, but from everyday language, such as my "French flag colors" example. I have no idea how to use the Hint feature. It never works. Please provide a step-by-step scenario. I have no idea how the Hint grade border setting works. note.- PHP preg extension engine doesn't support hinting the Use hint button should be disabled if Deterministic finite state automata engine is selected (I see in your script that you want to implement this feature) The online help is definitely written for computer programers, not for ordinary users (teachers). I hardly understand it myself. For instance I have no idea what the Deterministic finite state automata engine is or what it does. What I need, again is a set of real world examples of how this works. The question editing interface The general ergonomy is OK, and works well in Moodle 2.0. The Exact matching setting is saved when the question is saved. But it is not retrieved from database when editing question again. It is always set at Yes. The coloring of the student's answer (blue and red) is done from a programer's point of view, not from a pedagogic point of view. You color in blue those elements which "match" a regular expression, whether that expression was set to catch correct or incorrect elements in the answer. This does not mean anything to the student. Please color in blue the "correct" elements and in red the "incorrect" elements (as per my REGEXP question type). — If I use this expression: (the French flag|it) is blue, white(,| and) red with the PHP preg extension engine, it works OK. But if I turn on the Deterministic finite state automata engine, I get an error message Subpattern is unsupported. — From a pedagogic point of view, at the moment I see no benefit in the PREG question type over my REGEXP question type. It lacks an important feature of REGEXP: detecting missing strings in the student's answer. Hint facility not working. And I do not see which extra facilities it offers. Please re-state your case, providing real world examples.
Hide
Permalink
Dmitriy Kolesov added a comment -

I add supporting of subpatterns (...), only as grouping, but not catch substring and link is unsupported.

Joseph, Now your example with colour of French flag must work. I hope.

Show
Dmitriy Kolesov added a comment - I add supporting of subpatterns (...), only as grouping, but not catch substring and link is unsupported. Joseph, Now your example with colour of French flag must work. I hope.
Hide
Permalink
Oleg Sychev added a comment -

Joseph

0. I'm well aware that small docs are a problem for now, however I just couldn't do all thing at once and have to work on preg in these limited free time I have, and code should work first - no one needs docs on the non-working code. I started a Moodle Docs page about it (please look at http://docs.moodle.org/en/Preg_question_type), and just do a basic outline with a very sketchy examples in 3 hours. However, some things about hinting should be clearer for you now.

1. I'm not trying to use regexes for natural language teaching . So it's naturally hard for me to come with natural language examples. I'm done a tool - I may not know it's best use under any circumstances. Could you please help me with them? The situations where examples are needed best:
1.1. + and * quantifiers - they are good at formal languages (invaluable in such regular expressions as a path to the file with any number of folders, or web-address etc), but their use (except obvious whitespacing problem solving) in natural language is still hiding from me
1.2. positive lookahead assertions - see Moodle docs page for description. Tim once mentioned they could be used to easily define expression which will meet you three colors in any possible order, but I don't know how to do this. Negative assertions support would be added later, and they are much more useful than positive ones.

2. Question editing. I'll look for the Exact matching problem (worked fine for me). I would like to use disabling if it would work in Moodle (see the bug mentioned in the comments), for now it does more harm than good. No one from Moodle core developers was assigned to this bug thought and I have no control over them.

3. Coloring, subpatterns and hinting. For now my question doesn't know anything about 'correct' or 'incorrect' answers it catches, it just catches them. I should look for the current version of you question to see how it found out which _element_ it catches, there are only one catch per attempt for now in my question. The subpattern issue was simple overlooking and would be resolved in day or two. For now you could use (?: ) instead of ( ) - it is even more correct from regex point of view (and performance!). About hinting - are you sure it not working? It may not work the way you expected and it has it's reasons - look on the Moodle doc page, there are good examples there. Hint grade border also described there.

4.Engines: for now it just that: DFA our (possibly bugged) code supporting subset of regexp operators (still more than you regex!), but with hinting. PHP preg extension - 100% tested code with full regex operators support and no hinting.

5. You may misunderstand my intentions and current situation. I'm not trying to convince you that PREG is better than REGEXP for you current pedagogical situation. It may be that REGEXP will always be better for you personally (something programmed personally for someone is always better than universal tool - for him) - or it may be that you'll find great use for PREG assertions for example - do creative thinking about new possibilities in you situation, I'm not natural language teacher to advise you there. And you could easily use them both. Development process for PREG is also slightly different than for REGEXP, it is not feature-started. We developed a solid base, quite a complex thing. Only part of it's capabilities are exploited for features right now - but the rest could be done relatively easily and they are great - look at Develpment plans part on the Moodle docs page mentioned above. But first (and while development) we should be sure that this complex base code is working correctly. If you could, please help us testing it using all regular expresions you could and see that it find matches and hints correctly - while we are writing docs and adding new features. We test quite hard (look on the number of unit-tests), but there may be something we omit in our thinking so the totally uninolved tester is desirable.

6. About extra facilities: when PREG for now is better than REGEXP:
6.1. more regex operators supported - quantifiers + and * (does you REGEXP support {2,4} quantifiers?) and assertions - a big win in itself - still you have to discover you uses of them in you situation, you are great at this thought. And I don't see full list of supported (or unsupported) operators in any of you help or web-pages, users must discover this by trial and error;
6.2. more robust - you question used to hang on regexes with many alternatives available (and the number is multiplied with any new alternative set added to the regex), for many regexes it will be faster than you and possible at all (there are small number of regexes where you question will be faster thought, like a{2,200} )
6.3. more solid editing - REGEXP editing interface is messed up - it is all about special cases you should remember (correct me if I wrong somewhere). Enter simple string as a first answer, than regular expressions on the next (which isn't always easy as you must remember about escaping issues, which applies to regexes but no to the string). Don't start you expressions with – or you'll find out they are working totally different (how we are supposed to do "--$a;" matching in you question?) - remember it is special case for missing words, author is too lazy to add a checkbox to the answer blank for that. You could use * (etc) when doing missing words matching, but not when correct answer matching. And all this possibilities aren't clearly covered in the help/docs. There are no such problems with PREG. Each repeated field serves one and only one purpose and have one and only one syntax and will remain to do so.
6.4. hint grade border - would REGEXP hint lead you to a 0% or 10% or 90% grade asnwer? In PREG question authors controls this.
6.5. You examples also have numerous problems, and I'm not sure they all are solvable with REGEXP. For one thing, you French flag example has whitespacing issues - place two spaces (or tab instead of space) between words, or place space before colon and you are incorrect. Missing words examples has problems too: "--.blue." will not catch missing WORDS, only substrings (i.e. if student write "I love blues" it will think that "blue" is there). You should use --.(?<=\b)blue(?=\b). instead - if REGEXP would support \b and assertions.

'Missing words' feature (which I would call 'negative matching') is easy and would be done when there will be time (and Tim check in extra_answer_fields part).

P.S. Regular expressions are a great tool for formal languages. They are too weak to handle natural languages, it is unsolvable - there will be only some features available, only partially. You need at least a context-free grammar (probably even attributive one) instead to handle only natural language syntax (it could do anything regex could and much more). Such question is under development too, but again I couldn't do all things at once - be patient. The fact that you know regular expressions and don't know well context-free grammars doesn't make regular expressions fit for natural language processing, sorry.

Show
Oleg Sychev added a comment - Joseph 0. I'm well aware that small docs are a problem for now, however I just couldn't do all thing at once and have to work on preg in these limited free time I have, and code should work first - no one needs docs on the non-working code. I started a Moodle Docs page about it (please look at http://docs.moodle.org/en/Preg_question_type), and just do a basic outline with a very sketchy examples in 3 hours. However, some things about hinting should be clearer for you now. 1. I'm not trying to use regexes for natural language teaching . So it's naturally hard for me to come with natural language examples. I'm done a tool - I may not know it's best use under any circumstances. Could you please help me with them? The situations where examples are needed best: 1.1. + and * quantifiers - they are good at formal languages (invaluable in such regular expressions as a path to the file with any number of folders, or web-address etc), but their use (except obvious whitespacing problem solving) in natural language is still hiding from me 1.2. positive lookahead assertions - see Moodle docs page for description. Tim once mentioned they could be used to easily define expression which will meet you three colors in any possible order, but I don't know how to do this. Negative assertions support would be added later, and they are much more useful than positive ones. 2. Question editing. I'll look for the Exact matching problem (worked fine for me). I would like to use disabling if it would work in Moodle (see the bug mentioned in the comments), for now it does more harm than good. No one from Moodle core developers was assigned to this bug thought and I have no control over them. 3. Coloring, subpatterns and hinting. For now my question doesn't know anything about 'correct' or 'incorrect' answers it catches, it just catches them. I should look for the current version of you question to see how it found out which _element_ it catches, there are only one catch per attempt for now in my question. The subpattern issue was simple overlooking and would be resolved in day or two. For now you could use (?: ) instead of ( ) - it is even more correct from regex point of view (and performance!). About hinting - are you sure it not working? It may not work the way you expected and it has it's reasons - look on the Moodle doc page, there are good examples there. Hint grade border also described there. 4.Engines: for now it just that: DFA our (possibly bugged) code supporting subset of regexp operators (still more than you regex!), but with hinting. PHP preg extension - 100% tested code with full regex operators support and no hinting. 5. You may misunderstand my intentions and current situation. I'm not trying to convince you that PREG is better than REGEXP for you current pedagogical situation. It may be that REGEXP will always be better for you personally (something programmed personally for someone is always better than universal tool - for him) - or it may be that you'll find great use for PREG assertions for example - do creative thinking about new possibilities in you situation, I'm not natural language teacher to advise you there. And you could easily use them both. Development process for PREG is also slightly different than for REGEXP, it is not feature-started. We developed a solid base, quite a complex thing. Only part of it's capabilities are exploited for features right now - but the rest could be done relatively easily and they are great - look at Develpment plans part on the Moodle docs page mentioned above. But first (and while development) we should be sure that this complex base code is working correctly. If you could, please help us testing it using all regular expresions you could and see that it find matches and hints correctly - while we are writing docs and adding new features. We test quite hard (look on the number of unit-tests), but there may be something we omit in our thinking so the totally uninolved tester is desirable. 6. About extra facilities: when PREG for now is better than REGEXP: 6.1. more regex operators supported - quantifiers + and * (does you REGEXP support {2,4} quantifiers?) and assertions - a big win in itself - still you have to discover you uses of them in you situation, you are great at this thought. And I don't see full list of supported (or unsupported) operators in any of you help or web-pages, users must discover this by trial and error; 6.2. more robust - you question used to hang on regexes with many alternatives available (and the number is multiplied with any new alternative set added to the regex), for many regexes it will be faster than you and possible at all (there are small number of regexes where you question will be faster thought, like a{2,200} ) 6.3. more solid editing - REGEXP editing interface is messed up - it is all about special cases you should remember (correct me if I wrong somewhere). Enter simple string as a first answer, than regular expressions on the next (which isn't always easy as you must remember about escaping issues, which applies to regexes but no to the string). Don't start you expressions with – or you'll find out they are working totally different (how we are supposed to do "--$a;" matching in you question?) - remember it is special case for missing words, author is too lazy to add a checkbox to the answer blank for that. You could use * (etc) when doing missing words matching, but not when correct answer matching. And all this possibilities aren't clearly covered in the help/docs. There are no such problems with PREG. Each repeated field serves one and only one purpose and have one and only one syntax and will remain to do so. 6.4. hint grade border - would REGEXP hint lead you to a 0% or 10% or 90% grade asnwer? In PREG question authors controls this. 6.5. You examples also have numerous problems, and I'm not sure they all are solvable with REGEXP. For one thing, you French flag example has whitespacing issues - place two spaces (or tab instead of space) between words, or place space before colon and you are incorrect. Missing words examples has problems too: "--.blue." will not catch missing WORDS, only substrings (i.e. if student write "I love blues" it will think that "blue" is there). You should use --.(?<=\b)blue(?=\b). instead - if REGEXP would support \b and assertions. 'Missing words' feature (which I would call 'negative matching') is easy and would be done when there will be time (and Tim check in extra_answer_fields part). P.S. Regular expressions are a great tool for formal languages. They are too weak to handle natural languages, it is unsolvable - there will be only some features available, only partially. You need at least a context-free grammar (probably even attributive one) instead to handle only natural language syntax (it could do anything regex could and much more). Such question is under development too, but again I couldn't do all things at once - be patient. The fact that you know regular expressions and don't know well context-free grammars doesn't make regular expressions fit for natural language processing, sorry.
Hide
Permalink
Dmitriy Kolesov added a comment -

Oleg Aleksandrovich,

You are not right in two minor moment:
1) \b is unsupported now, I will add it later,
2) subpatterns (...) is supported for grouping, but not for catching ( (...) is equivalent to (?:...) for dfa_preg_matcher, link is unsupported).

Show
Dmitriy Kolesov added a comment - Oleg Aleksandrovich, You are not right in two minor moment: 1) \b is unsupported now, I will add it later, 2) subpatterns (...) is supported for grouping, but not for catching ( (...) is equivalent to (?:...) for dfa_preg_matcher, link is unsupported).
Hide
Permalink
Joseph Rézeau added a comment -

Hi Oleg,
Many thanks for having taken the time to reply at length. I will follow the numbering of your 16/Sep/10 06:25 AM comment to add my comments here.

0. The Moodle Docs page about Preg question type is a very good start. It is quite clear, well laid-out and answers some of my questions. In some places it is still uses too much of a "computer-language" and there should be more concrete examples of use. Now that it exists it can only be made better and more comprehensive as time goes by and as you pursue your development.

1. I think we agree that our approaches to the use of regular expressions in the analysis of student's answers in the Moodle's Short Answer question are very different. This explains why communication between us is sometimes difficult.
1.1. + and * quantifiers - etc. We agree here that those quantifiers have no use in analysis of the analysis of natural language student's answers. That is why I forbid their use in my REGEXP editing.
1.2. positive / negative lookahead assertions - At one stage in the past I thought those would be useful for my REGEXP question, but I never managed to make them work.

2. Question editing. I'll look for the Exact matching problem (worked fine for me).
Still does not work, it is always set to yes when editing a question, even if you set had it to No in a previous editing session.

3. _Coloring, subpatterns and hinting. _
At the moment I cannot get the coloring and hinting to work as described in the Docs. I think the Docs should explain in more detail all the settings that are needed to make those features (coloring and hinting) work.
------------
Example:
What are the colors of the French flag?
Correct answer: blue, white and red
Settings:
matching engine: Deterministic finite state automata (because PHP preg extension engine doesn't support hinting)
Use Hint: Yes
Hint grade border None
Exact matching: No
Answer1 Answer : blue, white and red
Note: I cannot use the regular expression are blue, white(,| and) red as per your Docs, because I get error message Subpattern is unsupported
Student enters: they are blue, vhite and red (as per your Docs example)
result:
the whole student answer is colored red, except for the letter "a" in "and" which is colored blue. I was expecting blue coloring for letter "v" in "vhite".
Clicking on the Hint button : nothing happens.
----------

4. Engines: I don't really understand.

5. You may misunderstand my intentions and current situation. I'm not trying to convince you that PREG is better than REGEXP for you current pedagogical situation. It may be that REGEXP will always be better for you personally ...
This is not what I am saying. I am saying that REGEXP is better for "ordinary" short answer response analysis, in the field of natural languages, i.e. outside of the realm of computing, logics and mathematics. I would be happy to use PREG if it did all that REGEXP does, plus possibly more things. I am aware that, at the moment, the number of moodlers who use REGEXP throughout the world must be very small indeed. When PREG becomes available, it will be interesting to see how many moodlers use it.
However, I am still quite willing to test PREG and ready to be convinced that it does just that: what can be done with REGEXP, and even more, and in a better way. What hinders my testing right now is that the PREG version in Moodle Contrib is still buggy (see my report above).

6. _About extra facilities: when PREG for now is better than REGEXP:

6.1. more regex operators supported - quantifiers + and * (does you REGEXP support {2,4} quantifiers?) and assertions etc._
Quantifiers are useless for analysis of natural language (but I'll welcome examples showing that they are useful). As for assertions, they might be useful but I still have to find a way to use them!
I agree that my online help is deficient in listing the operators (supported or non-supported). But, as you say yourself, REGEXP is oriented towards the end-user, not to the programmer.

6.2. more robust
That is correct, theoretically. If a very large number of alternatives were made available in the REGEXP regular expressions, the "alternatives engine" might hang. In practise, and again with natural languages in mind, this is not very likely to happen, simply because no teacher would write that many alternative possibilities. But I do agree with you that robustness is not a strong point of REGEXP.

6.3 more solid editing
I agree that the PREG editing window looks quite neat. However, the implications of selecting the various options are not clear per se.
Enter simple string as a first answer, than regular expressions on the next (which isn't always easy as you must remember about escaping issues, which applies to regexes but no to the string).
I agree that your solution is much better: to have a separate "Correct Answer" field (which is checked against the following Correct answers expressions in Answer 1, 2 etc. I can easily make that change to REGEXP and probably will for 2.0.
As for the rest of your remarks, I quite understand that, as a professional programmer, you can be dismayed at the "ad hoc" features of REGEXP. As an amateur programmer but professionnal educator, I have tried to male REGEXP a usable tool for analysis of natural languages answers, within the restricted scope of Moodle quiz questions, nothing more.
I do not understand:
how we are supposed to do "--$a;" matching in you question?
and
author is too lazy to add a checkbox to the answer blank for that

6.4. hint grade border would REGEXP hint lead you to a 0% or 10% or 90% grade asnwer? In PREG question authors controls this.
Since I have not yet managed to make the hint feature to work in PREG, I cannot answer. I expect it's an interesting feature. However, in my everyday use of Moodle quizzes in general and REGEXP in particular, I am far more interested in practise than testing and, for me, the grade is an accessory, not an essential part of the learning process.

6.5. _Your examples also have numerous problems, and I'm not sure they all are solvable with REGEXP.

6.5.1. For one thing, you French flag example has whitespacing issues - place two spaces (or tab instead of space) between words, or place space before colon and you are incorrect._
This is not true. REGEXP has a routine which automatically removes redundant spaces.
Space before colon is correct in French typography and incorrect in English typography. It's up to the teacher to decide how to catch those when writing his regular expressions in the Answers fields.
You can't tab instead of space) between words : tab takes you outside of the input field when you are writing your answer as a student

6.5.2. _Missing words examples has problems too: "--.blue." will not catch missing WORDS, only substrings (i.e. if student write "I love blues" it will think that "blue" is there). You should use --.(?<=\b)blue(?=\b). instead _
I use the following syntax, which works:
--.\bblue\b.
(You're right, my online help does not says this, I'll have to update it).
REGEXP does support \b

6.5.3 _if REGEXP would support \b and assertions.
'Missing words' feature (which I would call 'negative matching') is easy and would be done when there will be time (and Tim check in extra_answer_fields part)._
As you know that "missing words" feature ('negative matching' in programmer's language) is indispensable. I'm looking forward to see how you might implement it in PREG. On the other hand I do not understand "Tim check in extra_answer_fields part". Does it mean that something has to be added to the SHORTANSWER question type first?

P.S. Regular expressions are a great tool for formal languages. They are too weak to handle natural languages, it is unsolvable
I think there is a misunderstanding in the way you and I use the expression "natural languages". I certainly do not want a question type which would incorporate a "context-free grammar" (whatever that is). REGEXP is a very simple tool which is aimed at providing ad hoc analysis of student's SHORTANSWER answers in order to provide meaninful feedback conducive to learning. REGEXP has nothing to do with artificial intelligence, "grammars" etc.
It is precisely because Regular expressions are ... too weak to handle natural languages that in REGEXP I have "tweaked" the standard regular expressions to provide exactly the modest analysis tool that is needed by those teachers who ask questions which can be answered in short answers and need to be tested for correctness, mosty syntax (natural language syntax, of course).

Many thanks for your remarks, Oleg. We do have communication problems, due not to the fact that we are both using a foreign natural language (English) but to the fact that we are each using a different view of the world (the world of programming / the world of didactics). But I am finding this conversation quite enriching as it makes me think again about some of my opinions & choices. I do hope it is as enriching for you.

Joseph

Show
Joseph Rézeau added a comment - Hi Oleg, Many thanks for having taken the time to reply at length. I will follow the numbering of your 16/Sep/10 06:25 AM comment to add my comments here. 0. The Moodle Docs page about Preg question type is a very good start. It is quite clear, well laid-out and answers some of my questions. In some places it is still uses too much of a "computer-language" and there should be more concrete examples of use. Now that it exists it can only be made better and more comprehensive as time goes by and as you pursue your development. 1. I think we agree that our approaches to the use of regular expressions in the analysis of student's answers in the Moodle's Short Answer question are very different. This explains why communication between us is sometimes difficult. 1.1. + and * quantifiers - etc. We agree here that those quantifiers have no use in analysis of the analysis of natural language student's answers. That is why I forbid their use in my REGEXP editing. 1.2. positive / negative lookahead assertions - At one stage in the past I thought those would be useful for my REGEXP question, but I never managed to make them work. 2. Question editing. I'll look for the Exact matching problem (worked fine for me). Still does not work, it is always set to yes when editing a question, even if you set had it to No in a previous editing session. 3. _Coloring, subpatterns and hinting. _ At the moment I cannot get the coloring and hinting to work as described in the Docs. I think the Docs should explain in more detail all the settings that are needed to make those features (coloring and hinting) work. ------------ Example: What are the colors of the French flag? Correct answer: blue, white and red Settings: matching engine: Deterministic finite state automata (because PHP preg extension engine doesn't support hinting) Use Hint: Yes Hint grade border None Exact matching: No Answer1 Answer : blue, white and red Note: I cannot use the regular expression are blue, white(,| and) red as per your Docs, because I get error message Subpattern is unsupported Student enters: they are blue, vhite and red (as per your Docs example) result: the whole student answer is colored red, except for the letter "a" in "and" which is colored blue. I was expecting blue coloring for letter "v" in "vhite". Clicking on the Hint button : nothing happens. ---------- 4. Engines: I don't really understand. 5. You may misunderstand my intentions and current situation. I'm not trying to convince you that PREG is better than REGEXP for you current pedagogical situation. It may be that REGEXP will always be better for you personally ... This is not what I am saying. I am saying that REGEXP is better for "ordinary" short answer response analysis, in the field of natural languages, i.e. outside of the realm of computing, logics and mathematics. I would be happy to use PREG if it did all that REGEXP does, plus possibly more things. I am aware that, at the moment, the number of moodlers who use REGEXP throughout the world must be very small indeed. When PREG becomes available, it will be interesting to see how many moodlers use it. However, I am still quite willing to test PREG and ready to be convinced that it does just that: what can be done with REGEXP, and even more, and in a better way. What hinders my testing right now is that the PREG version in Moodle Contrib is still buggy (see my report above). 6. _About extra facilities: when PREG for now is better than REGEXP: 6.1. more regex operators supported - quantifiers + and * (does you REGEXP support {2,4} quantifiers?) and assertions etc._ Quantifiers are useless for analysis of natural language (but I'll welcome examples showing that they are useful). As for assertions, they might be useful but I still have to find a way to use them! I agree that my online help is deficient in listing the operators (supported or non-supported). But, as you say yourself, REGEXP is oriented towards the end-user, not to the programmer. 6.2. more robust That is correct, theoretically. If a very large number of alternatives were made available in the REGEXP regular expressions, the "alternatives engine" might hang. In practise, and again with natural languages in mind, this is not very likely to happen, simply because no teacher would write that many alternative possibilities. But I do agree with you that robustness is not a strong point of REGEXP. 6.3 more solid editing I agree that the PREG editing window looks quite neat. However, the implications of selecting the various options are not clear per se. Enter simple string as a first answer, than regular expressions on the next (which isn't always easy as you must remember about escaping issues, which applies to regexes but no to the string). I agree that your solution is much better: to have a separate "Correct Answer" field (which is checked against the following Correct answers expressions in Answer 1, 2 etc. I can easily make that change to REGEXP and probably will for 2.0. As for the rest of your remarks, I quite understand that, as a professional programmer, you can be dismayed at the "ad hoc" features of REGEXP. As an amateur programmer but professionnal educator, I have tried to male REGEXP a usable tool for analysis of natural languages answers, within the restricted scope of Moodle quiz questions, nothing more. I do not understand: how we are supposed to do "--$a;" matching in you question? and author is too lazy to add a checkbox to the answer blank for that 6.4. hint grade border would REGEXP hint lead you to a 0% or 10% or 90% grade asnwer? In PREG question authors controls this. Since I have not yet managed to make the hint feature to work in PREG, I cannot answer. I expect it's an interesting feature. However, in my everyday use of Moodle quizzes in general and REGEXP in particular, I am far more interested in practise than testing and, for me, the grade is an accessory, not an essential part of the learning process. 6.5. _Your examples also have numerous problems, and I'm not sure they all are solvable with REGEXP. 6.5.1. For one thing, you French flag example has whitespacing issues - place two spaces (or tab instead of space) between words, or place space before colon and you are incorrect._ This is not true. REGEXP has a routine which automatically removes redundant spaces. Space before colon is correct in French typography and incorrect in English typography. It's up to the teacher to decide how to catch those when writing his regular expressions in the Answers fields. You can't tab instead of space) between words : tab takes you outside of the input field when you are writing your answer as a student 6.5.2. _Missing words examples has problems too: "--.blue." will not catch missing WORDS, only substrings (i.e. if student write "I love blues" it will think that "blue" is there). You should use --.(?<=\b)blue(?=\b). instead _ I use the following syntax, which works: --.\bblue\b. (You're right, my online help does not says this, I'll have to update it). REGEXP does support \b 6.5.3 _if REGEXP would support \b and assertions. 'Missing words' feature (which I would call 'negative matching') is easy and would be done when there will be time (and Tim check in extra_answer_fields part)._ As you know that "missing words" feature ('negative matching' in programmer's language) is indispensable. I'm looking forward to see how you might implement it in PREG. On the other hand I do not understand "Tim check in extra_answer_fields part". Does it mean that something has to be added to the SHORTANSWER question type first? P.S. Regular expressions are a great tool for formal languages. They are too weak to handle natural languages, it is unsolvable I think there is a misunderstanding in the way you and I use the expression "natural languages". I certainly do not want a question type which would incorporate a "context-free grammar" (whatever that is). REGEXP is a very simple tool which is aimed at providing ad hoc analysis of student's SHORTANSWER answers in order to provide meaninful feedback conducive to learning. REGEXP has nothing to do with artificial intelligence, "grammars" etc. It is precisely because Regular expressions are ... too weak to handle natural languages that in REGEXP I have "tweaked" the standard regular expressions to provide exactly the modest analysis tool that is needed by those teachers who ask questions which can be answered in short answers and need to be tested for correctness, mosty syntax (natural language syntax, of course). Many thanks for your remarks, Oleg. We do have communication problems, due not to the fact that we are both using a foreign natural language (English) but to the fact that we are each using a different view of the world (the world of programming / the world of didactics). But I am finding this conversation quite enriching as it makes me think again about some of my opinions & choices. I do hope it is as enriching for you. Joseph
Hide
Permalink
Oleg Sychev added a comment -

Dmitry - will it be hard to you to download and install latest Denwer version and upgrade to the Moodle 2.0? I've setted up Mercurial repository on the Google Code for working on the Moodle plugins, but it working with 2.0 branch of the question. If you could work with it, you'll have no need to download an archive there, you could directly push into Google Code repository. Let me know when you ready and I'm add you as a committer (are you email there valid? I'll need it.).

Our current goals:
1. (easy) Add common errors (like unclosed brackets) support to the parser, using Lemon error rules
2. (easy) Move 'i' (case-insensitive) mode support to the parser - change leaf right on it creation would be faster
3. (medium) Move procedure to check whether a character suits a leaf outside DFA matcher - it doesn't matter which algorithm you are using when it comes to the leafs (maybe we need a meeting to discuss it, please find me next week)
4. (medium) after the 3 we could work on proper Unicode support for some leafs, \w is much more than english letters and numbers, there should be different approach to it as you can't possibly list all Unicode characters for that....
5. (medium) refactor DFA matching code, converting switches to the inheritance
6. (easy after 5) add more assert support, negative asserts are much more useful than positive ones

Joseph
1. Could you track our code using Mercurial repository on the Google Code our you stuck with CVS only?
2. Did you want to add you matching algorithm to PREG question type as one more engine? It could be easily done if you write a matcher class, inheriting preg_matcher, providing some basic functions. You could even benefit from existing regex parser if you want.

Show
Oleg Sychev added a comment - Dmitry - will it be hard to you to download and install latest Denwer version and upgrade to the Moodle 2.0? I've setted up Mercurial repository on the Google Code for working on the Moodle plugins, but it working with 2.0 branch of the question. If you could work with it, you'll have no need to download an archive there, you could directly push into Google Code repository. Let me know when you ready and I'm add you as a committer (are you email there valid? I'll need it.). Our current goals: 1. (easy) Add common errors (like unclosed brackets) support to the parser, using Lemon error rules 2. (easy) Move 'i' (case-insensitive) mode support to the parser - change leaf right on it creation would be faster 3. (medium) Move procedure to check whether a character suits a leaf outside DFA matcher - it doesn't matter which algorithm you are using when it comes to the leafs (maybe we need a meeting to discuss it, please find me next week) 4. (medium) after the 3 we could work on proper Unicode support for some leafs, \w is much more than english letters and numbers, there should be different approach to it as you can't possibly list all Unicode characters for that.... 5. (medium) refactor DFA matching code, converting switches to the inheritance 6. (easy after 5) add more assert support, negative asserts are much more useful than positive ones Joseph 1. Could you track our code using Mercurial repository on the Google Code our you stuck with CVS only? 2. Did you want to add you matching algorithm to PREG question type as one more engine? It could be easily done if you write a matcher class, inheriting preg_matcher, providing some basic functions. You could even benefit from existing regex parser if you want.
Hide
Permalink
Joseph Rézeau added a comment -

Oleg,
1.- I'd rather stick with moodle CVS, with ECLIPSE environement which I use intensively for my own development. Please commit any changes to moodle CVS so I can test.
2.- Thanks for the proposal but, not being a programmer, I'm afraid I would not understand how to go about it.

Show
Joseph Rézeau added a comment - Oleg, 1.- I'd rather stick with moodle CVS, with ECLIPSE environement which I use intensively for my own development. Please commit any changes to moodle CVS so I can test. 2.- Thanks for the proposal but, not being a programmer, I'm afraid I would not understand how to go about it.
Hide
Permalink
Oleg Sychev added a comment -

Joseph - I fixed issue with Exact matching editing, but the code is contained in the Moodle core so I have no immediate control over it - see MDL-24241. Easy thing, but I don't know when Tim will have time to check it in. You could use patch posted there youself (or just look at it and do similar changes in question/type/edit_question_form.php by youself) if you like more speed.

Show
Oleg Sychev added a comment - Joseph - I fixed issue with Exact matching editing, but the code is contained in the Moodle core so I have no immediate control over it - see MDL-24241. Easy thing, but I don't know when Tim will have time to check it in. You could use patch posted there youself (or just look at it and do similar changes in question/type/edit_question_form.php by youself) if you like more speed.
Hide
Permalink
Oleg Sychev added a comment -

Dmitry - you should look better on the Joseph's example. Regex is
are blue, white(,| and) red
Response (student's answer):
they are blue, vhite and red
Exact matching is no.

You engine report one-character partial match 'a' from 'and' instead of much longer 'are blue, '. It is seem to use last match, no the longest (or shortest to complete) one as it should. That is a bug I think. Please fix it fast and add relevant unit-tests. Shortest to complete match should be choosen if several partial matches occur.

P.S. You latest commits shows you spend little effort on unit-testing, the testing cases are very small and not tricky, they are too few. So bugs easily leaks behind them. For the project of such complexity it is not good. When you have time please add more tests for latest functionality.

Show
Oleg Sychev added a comment - Dmitry - you should look better on the Joseph's example. Regex is are blue, white(,| and) red Response (student's answer): they are blue, vhite and red Exact matching is no. You engine report one-character partial match 'a' from 'and' instead of much longer 'are blue, '. It is seem to use last match, no the longest (or shortest to complete) one as it should. That is a bug I think. Please fix it fast and add relevant unit-tests. Shortest to complete match should be choosen if several partial matches occur. P.S. You latest commits shows you spend little effort on unit-testing, the testing cases are very small and not tricky, they are too few. So bugs easily leaks behind them. For the project of such complexity it is not good. When you have time please add more tests for latest functionality.
Hide
Permalink
Joseph Rézeau added a comment -

Oleg Sychev added a comment - 17/Sep/10 03:36 AM

_I fixed issue with Exact matching editing, but the code is contained in the Moodle core so I have no immediate control over it - see MDL-24241. Easy thing, but I don't know when Tim will have time to check it in. You could use patch posted there youself (or just look at it and do similar changes in question/type/edit_question_form.php by youself) if you like more speed. _

Actually, the problem comes from your setting "1" as the default value for the match field, in edit_preg_form, line 41.
This is not necessary, since a Yes/No menu will always default to 1 (=Yes).
$mform->setDefault('exactmatch',1);

If you remove that line, everything works fine - you can use that as a workaround until Tim fixes the problem in MDL-24241.

Show
Joseph Rézeau added a comment - Oleg Sychev added a comment - 17/Sep/10 03:36 AM _I fixed issue with Exact matching editing, but the code is contained in the Moodle core so I have no immediate control over it - see MDL-24241. Easy thing, but I don't know when Tim will have time to check it in. You could use patch posted there youself (or just look at it and do similar changes in question/type/edit_question_form.php by youself) if you like more speed. _ Actually, the problem comes from your setting "1" as the default value for the match field, in edit_preg_form, line 41. This is not necessary, since a Yes/No menu will always default to 1 (=Yes). $mform->setDefault('exactmatch',1); If you remove that line, everything works fine - you can use that as a workaround until Tim fixes the problem in MDL-24241.
Hide
Permalink
Oleg Sychev added a comment -

Dmitry - I fixed you bug with wrong match been choosen described above.

I also improved engine a bit. If any of you unit-tests directly accessed preg_matcher::index_first or preg_matcher::index_last they could be broken, since they are now an arrays (use [0] to find old info). Functions like first_correct_character_index() are working the old way (when called old way, they now could more), so anything using them is safe (that's a good real example how useful getter functions are instead of allowing direct access to the class fields).

You could pull my changes from https://oasychev-moodle-plugins.googlecode.com/hg/ , no password should be required to read access (you could find you code in question/type/preg), so I have no need to post archive there. But it is 2.0 version, so please setup Moodle 2.0 first.

Now please work on moving case-insensitive support to the parser and develop a stronger and trickier unit-tests base.

Joseph - I committed to the CVS new version, where bug you found trying to repeat hinting example from Moodle Docs is fixed - you'll found you thanks in the commit message
Exact matching editing issue should be fixed too - thanks to the Pierre for the fast work.
Please try PREG once more.

I still delaying answering to you long (and very nice indeed) comment because I have some very busy workdays and what little time I have was invested in fixing bugs and developing code.

Show
Oleg Sychev added a comment - Dmitry - I fixed you bug with wrong match been choosen described above. I also improved engine a bit. If any of you unit-tests directly accessed preg_matcher::index_first or preg_matcher::index_last they could be broken, since they are now an arrays (use [0] to find old info). Functions like first_correct_character_index() are working the old way (when called old way, they now could more), so anything using them is safe (that's a good real example how useful getter functions are instead of allowing direct access to the class fields). You could pull my changes from https://oasychev-moodle-plugins.googlecode.com/hg/ , no password should be required to read access (you could find you code in question/type/preg), so I have no need to post archive there. But it is 2.0 version, so please setup Moodle 2.0 first. Now please work on moving case-insensitive support to the parser and develop a stronger and trickier unit-tests base. Joseph - I committed to the CVS new version, where bug you found trying to repeat hinting example from Moodle Docs is fixed - you'll found you thanks in the commit message Exact matching editing issue should be fixed too - thanks to the Pierre for the fast work. Please try PREG once more. I still delaying answering to you long (and very nice indeed) comment because I have some very busy workdays and what little time I have was invested in fixing bugs and developing code.
Hide
Permalink
Oleg Sychev added a comment -

Oops, sorry, Joseph I thanked you in Mercurial commit message but forget in the CVS one...

Show
Oleg Sychev added a comment - Oops, sorry, Joseph I thanked you in Mercurial commit message but forget in the CVS one...
Hide
Permalink
Joseph Rézeau added a comment -

@Oleg: Thanks for the latest bug fixes in PREG. I won't be able to do any testing for approximately the next 10 days. I will report here when I resume testing.
Joseph

Show
Joseph Rézeau added a comment - @Oleg: Thanks for the latest bug fixes in PREG. I won't be able to do any testing for approximately the next 10 days. I will report here when I resume testing. Joseph
Hide
Permalink
Oleg Sychev added a comment -

Dmitriy - please look at the number of issues assigned to you on the http://code.google.com/p/oasychev-moodle-plugins/issues/list (and my one too, you could participate at least by describing why you are do what you do in the first place).

We probably should meet this week to develop a plan for further development of preg - please find me to discuss time of the meeting. There are some serious rewritings ahead but they are necessary. It will be good to have notebook with preg code on the meeting.

Show
Oleg Sychev added a comment - Dmitriy - please look at the number of issues assigned to you on the http://code.google.com/p/oasychev-moodle-plugins/issues/list (and my one too, you could participate at least by describing why you are do what you do in the first place). We probably should meet this week to develop a plan for further development of preg - please find me to discuss time of the meeting. There are some serious rewritings ahead but they are necessary. It will be good to have notebook with preg code on the meeting.
Hide
Permalink
Oleg Sychev added a comment -

Dmitriy - please avoid further work on dfa_preg_matcher::match_inner() and scanner/parser before you could merge my changes. I have done some rework in this place. You could work on leaf matching for now...

Show
Oleg Sychev added a comment - Dmitriy - please avoid further work on dfa_preg_matcher::match_inner() and scanner/parser before you could merge my changes. I have done some rework in this place. You could work on leaf matching for now...
Hide
Permalink
Oleg Sychev added a comment -

I just added a new commit with 2 shiny brand new features:

  • the whole match and subpatterns (PHP preg engine only) captured could now be inserted in feedback using simple placeholders
  • error reporting for custom parser (DFA engine only) now tells meaningful messages about various syntax errors
  • some refactoring of the scaner/parser interaction to make grammar more comprehensible and avoid grammar conflicts
    You could find details at http://docs.moodle.org/en/Preg_question_type
    Please try them and let me know how you like them.

The error reporting part was quite a challenge, LALR(1)-grammar parsers are not very good at this, totally. Took 2 days of hard work, but with some creativity I could get it working (hopefully) even with pesky conditional subpatterns.

P.S. Tim and other professionals, don't worry about rare and big CVS commits there - I keep my code history in Mercurial repository on Google Code with frequent commits and commit only completed work to the Moodle CVS.

Show
Oleg Sychev added a comment - I just added a new commit with 2 shiny brand new features:
  • the whole match and subpatterns (PHP preg engine only) captured could now be inserted in feedback using simple placeholders
  • error reporting for custom parser (DFA engine only) now tells meaningful messages about various syntax errors
  • some refactoring of the scaner/parser interaction to make grammar more comprehensible and avoid grammar conflicts You could find details at http://docs.moodle.org/en/Preg_question_type Please try them and let me know how you like them.
The error reporting part was quite a challenge, LALR(1)-grammar parsers are not very good at this, totally. Took 2 days of hard work, but with some creativity I could get it working (hopefully) even with pesky conditional subpatterns. P.S. Tim and other professionals, don't worry about rare and big CVS commits there - I keep my code history in Mercurial repository on Google Code with frequent commits and commit only completed work to the Moodle CVS.
Hide
Permalink
Oleg Sychev added a comment -

Joseph - I find time at last to answer the questions in you long and very nice reply (I'm sorry if my one was a bit harsh). Following the numbers once again.

0. Have you seen new section about understanding expressions in Moodle Docs page? I wonder does it really make things more clear for the people without programming experience as I intended.

1. first of all I'm not thinking about PREG as a question only to teach natural languages or only formal ones - it should have features for all wanting to use regexes
1.1. +and * - maybe not much use in real languages (unless you want to define syntactical things like ((word1|word2|word3|word4), ?)*((word1|word2|word3|word4)) i.e. comma-separated list of any length), but you seems to use * in you example of missing words feature.

1.2. I think there will be all sorts assertions available in PREG (the most useful ones, probably are negative lookahead and especially lookbehing ones), and maybe even conditional subpatterns which could be quite useful for natural language. But IMHO you could easily implement them in REGEX generating a separate list for any assertion and checking it during matching (thought it may hurt you list of possible matches)

2. I waiting to hear back from you if "exact matching" editing problem disappered now to close relevant issue

3. That was a bug that should be fixed.

4. Engines: PHP preg engine could do subpattern matching, but not hinting and extensive error reporting - while DFA features are contrary. So there is no 'better' choice, it depends on the features you want to use in each question.

5. The number of Moodlers who user PREG is probably much lower than REGEXP for now

6.1. agree that my online help is deficient in listing the operators (supported or non-supported). But, as you say yourself, REGEXP is oriented towards the end-user, not to the programmer. any good help on a language should have descriptive section and a reference section. I'm actually puzzled which things under which circumstances I could use in REGEX, and trial-and-error takes too many time. And do you really think one ought to be a programmer to master regexes? They aren't much harder than arithmetics...

6.2. That is correct, theoretically - well, I encountered it practically two years ago. The number of possible alternatives multiply with every alternative set added, it rise very fast: throw in a bunch of optional spaces and you are there (just imagine a language where space before punctuation mark is optional). Also, PREG contains settings to limit the internal structures and give error on the very large regexes instead of hanging, that is somewhat better than simply hanging REGEXP are used to - you could implement it as well.

6.3. I can easily make that change to REGEXP - I was just wondering why it isn't in REGEXP yet while you agreed this is better about a year ago;
I definitely not like any "ad hoc" features in regex syntax - the regexps are complex enought to write without keeping in mind additional "ad hoc" things.
how we are supposed to do "--$a;" matching in you question?
This is a real-word teaching example of regular expression that starts with – : like "Write a statement to decrement variable with name a" with regex like
– ?$a ?;
then if author forgot about you "ad hoc" syntax he would be stunned. That's the problem with "ad hoc" things. You could have choosen something regular expression couldn't start from instead, like, ironically, "++"
author is too lazy to add a checkbox to the answer blank for that
still better and obvious choice would be to leave "answer" field for regex alone and add "missing" checkbox to the answer blank for this feature.
BTW, am I correct that you "missing words" means answer that works when regex doesn't match?

6.4. Hint grade border. When you enter some characters and repeatedly press hint (adding hinted characters), hint will eventually "lead" you to some answer, matching with one of regexes. The problem is, what answers hint should lead you: any (even 0% ones), only 100% ones or >=60% ones for example. (You are probably won't using partially correct answers at all, but then it doesn't mean no one using REGEXP or PREG not use them). That is what hint grade border for.

6.5. REGEXP has a routine which automatically removes redundant spaces. - I coudn't approve such solution. Cases where the number of spaces is signigicant may be rare (thought I could think about some right now, like Python language teaching for example), but REGEXP is totally useless in them due to this routine. And that is the thing that can be handled by regular expressions themselves, so there is no point in reducing functionality.
_ tab takes you outside of the input field when you are writing your answer as a student_ - that doesn't necessary means it can't be inside answer - it could be copied from some text editor, or inserted in a number of other ways. And there are more spaces in Unicode then just one with 32 number. "\s" is much better for whitespace handling when you dealing with regexes than just typing space in the expression.
--.\bblue\b. ... REGEXP does support \b - it will work if REGEXP support \b as assertion (which it really is), is this the case? Thought I'm suspecting that you use preg_match() for you missing words feature, not you alterative-based engine - or you'll have problems with *.
As you know that "missing words" feature is indispensable. - I always wondering about you attitude: anything you do is indespensable, while anything you coudn't or don't know of aren't needed at all (like you don't give any attention to the misplaced words feature I mentioned for auto-feedback question type, which backs up missed words pretty well). Have patience, it will be done in PREG, but require some thought about how grading should work under any circumstances (correct, partially correct, incorrect answers?) with it etc (am I correct guessing that REGEXP even doesn't return validation error if there is 100% grade regexp with missing words feature for it?).
Does it mean that something has to be added to the SHORTANSWER question type first? no, extra_answer_fields() means that something has to be added to default_questiontype to facilitate adding new fields to the answer.

7. Joseph, you are incorrectly fearing grammars. Context-free grammars (CFGs) are no AI. Actually they were created to understand natural language syntax better (while they excell at formal languages). On the "ad hoc" basis they are probably easier to grasp than even regexes (while their possibilities are greater). They are just a way to describe, how one syntactical units could be grouped to form another ones, i.e. how words and punctuation marks make clauses, word, clauses etc - sentences etc. Here is simple PHP-language example of grammar rule:
while_stmt ::= WHILE '(' expr ')' stmt
that describes that while statement consists of "while" keyword, expression in parenthesis and statement to execute in the loop exactly in this sequence (there will be other rules describing what 'expr' and 'stmt' are). Is it hard? Grammars differs from regexes in 2 ways only:
-1- grammars don't have iteration, only alternative and sequence (iteration can be easily achieved);
-2- there are many rules for the grammar, not one expression, so you could describe grouping in steps, using low-level groups (like 'expr' in the example above) to define high-level ones.
For many years the best ways of any language processing sharing first two steps:
(1) break text down to the smallest meaningful elements (words, punctuation marks etc) using regexes
(2) build more complex syntactical units (clauses, sentences) from them using grammars.

8. I organised my code in two easily separable parts: question type that handle UI, grading etc and matching engine that handle regex matching. Look at public functions of preg_matcher.php, they are easier to grasp. You could easily use our engine in you question type (given appropriate respects) or could add you matching engine to ours covering it with class, that extends preg_matcher (or do both). It's done same way as you create question type classes, only easier since matcher has much simpler interface.

Show
Oleg Sychev added a comment - Joseph - I find time at last to answer the questions in you long and very nice reply (I'm sorry if my one was a bit harsh). Following the numbers once again. 0. Have you seen new section about understanding expressions in Moodle Docs page? I wonder does it really make things more clear for the people without programming experience as I intended. 1. first of all I'm not thinking about PREG as a question only to teach natural languages or only formal ones - it should have features for all wanting to use regexes 1.1. +and * - maybe not much use in real languages (unless you want to define syntactical things like ((word1|word2|word3|word4), ?)*((word1|word2|word3|word4)) i.e. comma-separated list of any length), but you seems to use * in you example of missing words feature. 1.2. I think there will be all sorts assertions available in PREG (the most useful ones, probably are negative lookahead and especially lookbehing ones), and maybe even conditional subpatterns which could be quite useful for natural language. But IMHO you could easily implement them in REGEX generating a separate list for any assertion and checking it during matching (thought it may hurt you list of possible matches) 2. I waiting to hear back from you if "exact matching" editing problem disappered now to close relevant issue 3. That was a bug that should be fixed. 4. Engines: PHP preg engine could do subpattern matching, but not hinting and extensive error reporting - while DFA features are contrary. So there is no 'better' choice, it depends on the features you want to use in each question. 5. The number of Moodlers who user PREG is probably much lower than REGEXP for now 6.1. agree that my online help is deficient in listing the operators (supported or non-supported). But, as you say yourself, REGEXP is oriented towards the end-user, not to the programmer. any good help on a language should have descriptive section and a reference section. I'm actually puzzled which things under which circumstances I could use in REGEX, and trial-and-error takes too many time. And do you really think one ought to be a programmer to master regexes? They aren't much harder than arithmetics... 6.2. That is correct, theoretically - well, I encountered it practically two years ago. The number of possible alternatives multiply with every alternative set added, it rise very fast: throw in a bunch of optional spaces and you are there (just imagine a language where space before punctuation mark is optional). Also, PREG contains settings to limit the internal structures and give error on the very large regexes instead of hanging, that is somewhat better than simply hanging REGEXP are used to - you could implement it as well. 6.3. I can easily make that change to REGEXP - I was just wondering why it isn't in REGEXP yet while you agreed this is better about a year ago; I definitely not like any "ad hoc" features in regex syntax - the regexps are complex enought to write without keeping in mind additional "ad hoc" things. how we are supposed to do "--$a;" matching in you question? This is a real-word teaching example of regular expression that starts with – : like "Write a statement to decrement variable with name a" with regex like – ?$a ?; then if author forgot about you "ad hoc" syntax he would be stunned. That's the problem with "ad hoc" things. You could have choosen something regular expression couldn't start from instead, like, ironically, "++" author is too lazy to add a checkbox to the answer blank for that still better and obvious choice would be to leave "answer" field for regex alone and add "missing" checkbox to the answer blank for this feature. BTW, am I correct that you "missing words" means answer that works when regex doesn't match? 6.4. Hint grade border. When you enter some characters and repeatedly press hint (adding hinted characters), hint will eventually "lead" you to some answer, matching with one of regexes. The problem is, what answers hint should lead you: any (even 0% ones), only 100% ones or >=60% ones for example. (You are probably won't using partially correct answers at all, but then it doesn't mean no one using REGEXP or PREG not use them). That is what hint grade border for. 6.5. REGEXP has a routine which automatically removes redundant spaces. - I coudn't approve such solution. Cases where the number of spaces is signigicant may be rare (thought I could think about some right now, like Python language teaching for example), but REGEXP is totally useless in them due to this routine. And that is the thing that can be handled by regular expressions themselves, so there is no point in reducing functionality. _ tab takes you outside of the input field when you are writing your answer as a student_ - that doesn't necessary means it can't be inside answer - it could be copied from some text editor, or inserted in a number of other ways. And there are more spaces in Unicode then just one with 32 number. "\s" is much better for whitespace handling when you dealing with regexes than just typing space in the expression. --.\bblue\b. ... REGEXP does support \b - it will work if REGEXP support \b as assertion (which it really is), is this the case? Thought I'm suspecting that you use preg_match() for you missing words feature, not you alterative-based engine - or you'll have problems with *. As you know that "missing words" feature is indispensable. - I always wondering about you attitude: anything you do is indespensable, while anything you coudn't or don't know of aren't needed at all (like you don't give any attention to the misplaced words feature I mentioned for auto-feedback question type, which backs up missed words pretty well). Have patience, it will be done in PREG, but require some thought about how grading should work under any circumstances (correct, partially correct, incorrect answers?) with it etc (am I correct guessing that REGEXP even doesn't return validation error if there is 100% grade regexp with missing words feature for it?). Does it mean that something has to be added to the SHORTANSWER question type first? no, extra_answer_fields() means that something has to be added to default_questiontype to facilitate adding new fields to the answer. 7. Joseph, you are incorrectly fearing grammars. Context-free grammars (CFGs) are no AI. Actually they were created to understand natural language syntax better (while they excell at formal languages). On the "ad hoc" basis they are probably easier to grasp than even regexes (while their possibilities are greater). They are just a way to describe, how one syntactical units could be grouped to form another ones, i.e. how words and punctuation marks make clauses, word, clauses etc - sentences etc. Here is simple PHP-language example of grammar rule: while_stmt ::= WHILE '(' expr ')' stmt that describes that while statement consists of "while" keyword, expression in parenthesis and statement to execute in the loop exactly in this sequence (there will be other rules describing what 'expr' and 'stmt' are). Is it hard? Grammars differs from regexes in 2 ways only: -1- grammars don't have iteration, only alternative and sequence (iteration can be easily achieved); -2- there are many rules for the grammar, not one expression, so you could describe grouping in steps, using low-level groups (like 'expr' in the example above) to define high-level ones. For many years the best ways of any language processing sharing first two steps: (1) break text down to the smallest meaningful elements (words, punctuation marks etc) using regexes (2) build more complex syntactical units (clauses, sentences) from them using grammars. 8. I organised my code in two easily separable parts: question type that handle UI, grading etc and matching engine that handle regex matching. Look at public functions of preg_matcher.php, they are easier to grasp. You could easily use our engine in you question type (given appropriate respects) or could add you matching engine to ours covering it with class, that extends preg_matcher (or do both). It's done same way as you create question type classes, only easier since matcher has much simpler interface.
Hide
Permalink
Joseph Rézeau added a comment -

Hi Oleg,
I am desperately trying to get the Hint facility to work in your PREG question type but cannot get it to work at all.
I follow exactly the description you provide in http://docs.moodle.org/en/Preg_question_type#Hinting
1.- Matching engine: Deterministic finite state
2.- Use hint: Yes
3.- Hint grade border: 100%
3b.- Exact matching: No.
4.- Correct answer: are blue, white, red
5a.- Answer 1 : are blue, white(,| and) red / Grade 100% / Feedback message: Correct
5b.- Answer2: .* / Grade 0% / Feedback message: Sorry, try again
6.- Student enters: blue, vhite and red and presses submit button (white is mistakenly spelt vhite)
7.- PREG question replies:
7a.- red wrong ckeck mark
7b.- the student's answer is displayed - in blue - under the input box
7c.- the feedback message "Sorry, try again" is displayed
8.- Student clicks the feedback button: nothing happens
Note: nothing happens at all when feedback button is pressed.
----------------
Please forward step-by-step instructions to make the Hint facility work.

Joseph

Show
Joseph Rézeau added a comment - Hi Oleg, I am desperately trying to get the Hint facility to work in your PREG question type but cannot get it to work at all. I follow exactly the description you provide in http://docs.moodle.org/en/Preg_question_type#Hinting 1.- Matching engine: Deterministic finite state 2.- Use hint: Yes 3.- Hint grade border: 100% 3b.- Exact matching: No. 4.- Correct answer: are blue, white, red 5a.- Answer 1 : are blue, white(,| and) red / Grade 100% / Feedback message: Correct 5b.- Answer2: .* / Grade 0% / Feedback message: Sorry, try again 6.- Student enters: blue, vhite and red and presses submit button (white is mistakenly spelt vhite) 7.- PREG question replies: 7a.- red wrong ckeck mark 7b.- the student's answer is displayed - in blue - under the input box 7c.- the feedback message "Sorry, try again" is displayed 8.- Student clicks the feedback button: nothing happens Note: nothing happens at all when feedback button is pressed. ---------------- Please forward step-by-step instructions to make the Hint facility work. Joseph
Hide
Permalink
Oleg Sychev added a comment -

If you delete Answer 2 (.* 0%) it works?

Show
Oleg Sychev added a comment - If you delete Answer 2 (.* 0%) it works?
Hide
Permalink
Oleg Sychev added a comment -

Joseph - you problem is in you Answer 2. There wasn't anything like that in instructions. Delete it and you get the question working.

PREG question is told to prefer full match over partial one (doing otherwise will result in a very strange effects), and you Answer 2 gives full match for anything - so in this case partial matching won't work at all.

Are you really need this "sorry, try again" message?

Show
Oleg Sychev added a comment - Joseph - you problem is in you Answer 2. There wasn't anything like that in instructions. Delete it and you get the question working. PREG question is told to prefer full match over partial one (doing otherwise will result in a very strange effects), and you Answer 2 gives full match for anything - so in this case partial matching won't work at all. Are you really need this "sorry, try again" message?
Hide
Permalink
Joseph Rézeau added a comment -

Oleg,
1.- The reason I always put a ".*" catchall expression right at the end of my Answers list is in case the previous Answers expressions did not catch anything. Maybe with the PREG Hint feature this is no longer necessary. I will do more testing.
2.- When I remove Answer 2, this is what I find out:
3a.- Student enters: are "blue, vhite and red" and presses submit button (this is the expected behavior)
3b.- student's answer is displayed, "blue, " in blue and "vhite and red" in red crossed out : that is OK
3c.- then, student presses Hint button: nothing happens : this is not what I expect...
4a. Student enters: are "blue, vhite and red" and presses Hint button
4b.- student's answer is displayed, "blue, " in blue and "vhite and red" in red crossed out and, this time the Hint works: the first missing word "w" is displayed on a green background.
OK, now I understand how your Hint facility works. It prevents the student from clicking the Hint button repeatedly, as he has to enter corrections before asking for a new Hint. I think it is pedagogically sound.
However there remains a problem: Once the student has pressed the Submit button, the Hint feature becomes unavailable unless he adds at least one letter to his submitted answer. Something must be done here, to "reset" the Hint facility.
Will continue testing and will report here.
Joseph

Show
Joseph Rézeau added a comment - Oleg, 1.- The reason I always put a ".*" catchall expression right at the end of my Answers list is in case the previous Answers expressions did not catch anything. Maybe with the PREG Hint feature this is no longer necessary. I will do more testing. 2.- When I remove Answer 2, this is what I find out: 3a.- Student enters: are "blue, vhite and red" and presses submit button (this is the expected behavior) 3b.- student's answer is displayed, "blue, " in blue and "vhite and red" in red crossed out : that is OK 3c.- then, student presses Hint button: nothing happens : this is not what I expect... 4a. Student enters: are "blue, vhite and red" and presses Hint button 4b.- student's answer is displayed, "blue, " in blue and "vhite and red" in red crossed out and, this time the Hint works: the first missing word "w" is displayed on a green background. OK, now I understand how your Hint facility works. It prevents the student from clicking the Hint button repeatedly, as he has to enter corrections before asking for a new Hint. I think it is pedagogically sound. However there remains a problem: Once the student has pressed the Submit button, the Hint feature becomes unavailable unless he adds at least one letter to his submitted answer. Something must be done here, to "reset" the Hint facility. Will continue testing and will report here. Joseph
Hide
Permalink
Joseph Rézeau added a comment -

Minor bug when Correct answer contains an apostrope, in Preview question mode

What are the colors of the French flag?
Correct answer = it's blue, white and red
Preview question
Click Fill with correct button
input field is filled with: it\'s blue, white and red
Click Submit:
Answer input field still displays : it\'s blue, white and red
line under displays:
in red:it\'s
in blue: blue, white and red
Correct feedback message is displayed

Show
Joseph Rézeau added a comment - Minor bug when Correct answer contains an apostrope, in Preview question mode What are the colors of the French flag? Correct answer = it's blue, white and red Preview question Click Fill with correct button input field is filled with: it\'s blue, white and red Click Submit: Answer input field still displays : it\'s blue, white and red line under displays: in red:it\'s in blue: blue, white and red Correct feedback message is displayed
Hide
Permalink
Oleg Sychev added a comment -

Thanks, Joseph! I already fixed both bugs (hint after submit and apostrophe one too), but waiting for Tim to commit MDL-24386 for the fix to take full effect (there can be numerous problems with feedback displaying in complex cases unless this is solved), then I'll commit my fixes to Moodle CVS too. (You could easily eliminate aphstrophe problem deleting 'addslashes' call from questiontype.php, I just omit it during 2.0 conversion).

Show
Oleg Sychev added a comment - Thanks, Joseph! I already fixed both bugs (hint after submit and apostrophe one too), but waiting for Tim to commit MDL-24386 for the fix to take full effect (there can be numerous problems with feedback displaying in complex cases unless this is solved), then I'll commit my fixes to Moodle CVS too. (You could easily eliminate aphstrophe problem deleting 'addslashes' call from questiontype.php, I just omit it during 2.0 conversion).
Hide
Permalink
Oleg Sychev added a comment -

Joseph, by the way, did you read about new 'subpattern matching' feature in the docs? I'm sure you can find creative use for it. Sadly, it's incompatible with Hint for now and will be for quite a time (until someone will write NFA or Backtracking engine), thought you could use {$0} with the DFA engine.

Show
Oleg Sychev added a comment - Joseph, by the way, did you read about new 'subpattern matching' feature in the docs? I'm sure you can find creative use for it. Sadly, it's incompatible with Hint for now and will be for quite a time (until someone will write NFA or Backtracking engine), thought you could use {$0} with the DFA engine.
Hide
Permalink
Luis Vázquez de Lara added a comment -

I'm downloaded PREG to moodle 1.994. When tryin to open the page it appears the following error message:
Fatal error: Class 'moodle_exception' not found in /usr/share/moodle/question/type/preg/preg_exception.php on line 15.

Show
Luis Vázquez de Lara added a comment - I'm downloaded PREG to moodle 1.994. When tryin to open the page it appears the following error message: Fatal error: Class 'moodle_exception' not found in /usr/share/moodle/question/type/preg/preg_exception.php on line 15.
Hide
Permalink
Oleg Sychev added a comment -

Luis,

"Latest version" link of PREG with new functionality would work under Moodle 2.0, not 1.9

For 1.9 use "download for Moodle 1.9" link, but there is no hinting there.

Actually, latest version could be very easily tweaked to work with Moodle 1.9 on PHP 5 if you know programming a little (feel free to ask me what should be done if you inclined to do it). I just haven't time to do that job, investing all free time to get new functionality working.

Show
Oleg Sychev added a comment - Luis, "Latest version" link of PREG with new functionality would work under Moodle 2.0, not 1.9 For 1.9 use "download for Moodle 1.9" link, but there is no hinting there. Actually, latest version could be very easily tweaked to work with Moodle 1.9 on PHP 5 if you know programming a little (feel free to ask me what should be done if you inclined to do it). I just haven't time to do that job, investing all free time to get new functionality working.
Hide
Permalink
Luis Vázquez de Lara added a comment -

Hi Oleg:
Thank you very much for your response. I downloaded the 1.9 version and I'm starting to work with it. Yes, I know a little programming and please let me know what should be done to make the latest version work with moodle 1.9. Because I'm not familiar with PHP lenguage, I would need the exact programming syntax.
By the way, Where could I get the documentation to work with PREG? When I click on the help icon it doesn´t lead to anything
Luis

Show
Luis Vázquez de Lara added a comment - Hi Oleg: Thank you very much for your response. I downloaded the 1.9 version and I'm starting to work with it. Yes, I know a little programming and please let me know what should be done to make the latest version work with moodle 1.9. Because I'm not familiar with PHP lenguage, I would need the exact programming syntax. By the way, Where could I get the documentation to work with PREG? When I click on the help icon it doesn´t lead to anything Luis
Hide
Permalink
Luis Vázquez de Lara added a comment -

I created a question with PREG and when trying to preview it, the following message appeared

Failed to load question options from the table question_preg for questionid 1268

Do you think it´s because I first downloaded the version for moodle 2?

Show
Luis Vázquez de Lara added a comment - I created a question with PREG and when trying to preview it, the following message appeared Failed to load question options from the table question_preg for questionid 1268 Do you think it´s because I first downloaded the version for moodle 2?
Hide
Permalink
Oleg Sychev added a comment -

Luis
1. There is "Documentation" link where you downloaded PREG question type. Thought it deals with new 2.0 functionality much, there are good description what regular expressions are too. Are you familiar with them? 1.9 version work very straight, similar to shortanswer, just add regular expressions (without modifiers or enclosing characters) as asnwers, you could also write down correct answer in student-readable form in the relevant field, that's all.
2. Yes, if you installed 2.0 versions with it's DB tables it won't work with 1.9. Unfortunately Moodle 1.9 lacks question type uninstallation, it was included in 2.0 only. You'll need to manually delete all traces of old preg installation from DB - basically question type table + record about it been installed whenever Moodle store it (I don't know, sorry). If you can't do this, you'll be better with fresh Moodle install
3. To "downgrade" 2.0 PREG code to 1.9 you'll need basically two things (don't alter the database then):
3.1. Copy code from the file moodle_exception.txt attached there in the beginning of preg_exception.php (but after "<? php" tag of course)
3.2. Look where 1.9 version used stripslashes_safe and add them there and to all other places where these variables - like responses[''] is used.
Or, alternatively, you could install Moodle 2.0 to play with new version of preg. It is now in beta and may give errors with complex regexes. It will be released approximately with 2.0 version of Moodle.

Show
Oleg Sychev added a comment - Luis 1. There is "Documentation" link where you downloaded PREG question type. Thought it deals with new 2.0 functionality much, there are good description what regular expressions are too. Are you familiar with them? 1.9 version work very straight, similar to shortanswer, just add regular expressions (without modifiers or enclosing characters) as asnwers, you could also write down correct answer in student-readable form in the relevant field, that's all. 2. Yes, if you installed 2.0 versions with it's DB tables it won't work with 1.9. Unfortunately Moodle 1.9 lacks question type uninstallation, it was included in 2.0 only. You'll need to manually delete all traces of old preg installation from DB - basically question type table + record about it been installed whenever Moodle store it (I don't know, sorry). If you can't do this, you'll be better with fresh Moodle install 3. To "downgrade" 2.0 PREG code to 1.9 you'll need basically two things (don't alter the database then): 3.1. Copy code from the file moodle_exception.txt attached there in the beginning of preg_exception.php (but after "<? php" tag of course) 3.2. Look where 1.9 version used stripslashes_safe and add them there and to all other places where these variables - like responses[''] is used. Or, alternatively, you could install Moodle 2.0 to play with new version of preg. It is now in beta and may give errors with complex regexes. It will be released approximately with 2.0 version of Moodle.
Hide
Permalink
Luis Vázquez de Lara added a comment -

Thank you so much. I'll go for the option of upgrading to Moodle 2.0. I can't wait to start playing with the new version
Best regards
Luis

Show
Luis Vázquez de Lara added a comment - Thank you so much. I'll go for the option of upgrading to Moodle 2.0. I can't wait to start playing with the new version Best regards Luis
Hide
Permalink
Luis Vázquez de Lara added a comment -

Hi Oleg:
I upgraded to Moodle 2.0 and after installing PREG an error appears saying that it is not compatible with Moodle 2.0:

Debug info: Plugin is not compatible with Moodle 2.x or later.
Stack trace:

  • line 294 of /lib/upgradelib.php: plugin_defective_exception thrown
  • line 1352 of /lib/upgradelib.php: call to upgrade_plugins()
  • line 290 of /admin/index.php: call to upgrade_noncore()
    Is there something I can do? I downloaded the file preg2010-09-15.zip
    Thanks and happy new year
    Luis
Show
Luis Vázquez de Lara added a comment - Hi Oleg: I upgraded to Moodle 2.0 and after installing PREG an error appears saying that it is not compatible with Moodle 2.0: Debug info: Plugin is not compatible with Moodle 2.x or later. Stack trace:
  • line 294 of /lib/upgradelib.php: plugin_defective_exception thrown
  • line 1352 of /lib/upgradelib.php: call to upgrade_plugins()
  • line 290 of /admin/index.php: call to upgrade_noncore() Is there something I can do? I downloaded the file preg2010-09-15.zip Thanks and happy new year Luis
Hide
Permalink
Oleg Sychev added a comment -

Hi, Luis.
Files placed in this issue were for internal development purpose. Actual version of question stored in Moodle contrib CVS.

To download Preg version for 2.0 you should use standard Moodle Modules and Plugins database - http://moodle.org/mod/data/view.php?d=13&rid=1901 - look for "Download for Moodle 2.0" link (file will be "preg.zip". You'll also find documentation link there which may be helpful. It's would be better to uninstall old version you installed now first if it got installed (Moodle 2.0 could uninstall questions), and install new one clean.

Remember that code for DFA engine (with it's Hint capability) is considered BETA by now - i.e it would work for simple regexes - hey, it's actually passess more than 400 tests - but I already know some complex cases where it would not work. We are just polishing a major rework of DFA engine to commit to the Moodle CVS which would make it more maintainable and fix many complex cases.

If you don't plan to write complex regexes (especially using assertions), than DFA and hinting would work fine, probably even for production. PHP Preg engine working good of course - if there are no bugs in PHP - and could be used for production straight now with regexes of any complexity. You could also enjoy "subpattern matching" feature with it, which was added for 2.0 version.

Show
Oleg Sychev added a comment - Hi, Luis. Files placed in this issue were for internal development purpose. Actual version of question stored in Moodle contrib CVS. To download Preg version for 2.0 you should use standard Moodle Modules and Plugins database - http://moodle.org/mod/data/view.php?d=13&rid=1901 - look for "Download for Moodle 2.0" link (file will be "preg.zip". You'll also find documentation link there which may be helpful. It's would be better to uninstall old version you installed now first if it got installed (Moodle 2.0 could uninstall questions), and install new one clean. Remember that code for DFA engine (with it's Hint capability) is considered BETA by now - i.e it would work for simple regexes - hey, it's actually passess more than 400 tests - but I already know some complex cases where it would not work. We are just polishing a major rework of DFA engine to commit to the Moodle CVS which would make it more maintainable and fix many complex cases. If you don't plan to write complex regexes (especially using assertions), than DFA and hinting would work fine, probably even for production. PHP Preg engine working good of course - if there are no bugs in PHP - and could be used for production straight now with regexes of any complexity. You could also enjoy "subpattern matching" feature with it, which was added for 2.0 version.
Hide
Permalink
Oleg Sychev added a comment - - edited

Oh, and Happy New Year to everyone involved Many thanks for you work and help.

Hoping to have new refactored DFA engine in new year too.

Show
Oleg Sychev added a comment - - edited Oh, and Happy New Year to everyone involved Many thanks for you work and help. Hoping to have new refactored DFA engine in new year too.

People

Vote (0)
Watch (5)

Dates

  • Created:
    Updated:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Moodle. Try JIRA - bug tracking software for your team.