Moodle
  1. Moodle
  2. MDL-26099

Workshop doesn't autmatically switch phases

    Details

    • Testing Instructions:
      Hide

      Testing difficulty: MEDIUM (requires cron job running)

      1. Set up a workshop with submissions deadline set and automatic phase switching enabled
      2. Set up the scheduled allocation method
      3. Switch the workshop to the submission phase
      4. make some submissions on behalf of students
      5. TEST: make sure that once the submissions deadline is reached, the cron job (a) allocates submissions for assessment and (b) the workshop is switched to the assessment phase automatically.

      Show
      Testing difficulty: MEDIUM (requires cron job running) 1. Set up a workshop with submissions deadline set and automatic phase switching enabled 2. Set up the scheduled allocation method 3. Switch the workshop to the submission phase 4. make some submissions on behalf of students 5. TEST: make sure that once the submissions deadline is reached, the cron job (a) allocates submissions for assessment and (b) the workshop is switched to the assessment phase automatically.
    • Difficulty:
      Moderate
    • Affected Branches:
      MOODLE_20_STABLE, MOODLE_21_STABLE, MOODLE_22_STABLE
    • Fixed Branches:
      MOODLE_23_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-26099-workshop-phase-switch
    • Rank:
      15678

      Description

      Workshop doesn't automatically switch phases which is a significant problem. It makes more work for the teacher to have to manually switch phases. There should be the option to automatically switch phases including allocation and release of grades. If there was a release of grade date than even the grades could be calculated and release. Once a workshop has been appropriately set up a teacher should not have to monitor it to do anything.

        Issue Links

          Activity

          Hide
          Michelle Moore added a comment -
          Show
          Michelle Moore added a comment - See related discussion here: http://moodle.org/mod/forum/discuss.php?d=166912#p742311
          Hide
          David Mudrak added a comment -

          Increasing the priority for this and setting 2.3 as the milestone. There is a strong demand for this feature. Many people see this as a regression and they are actually right.

          Show
          David Mudrak added a comment - Increasing the priority for this and setting 2.3 as the milestone. There is a strong demand for this feature. Many people see this as a regression and they are actually right.
          Hide
          Vernellia Randall added a comment -

          I am glad to see this in priority. Missed deadlines seriously affected my evaluations. No matter what time I set to "switch" either something would come up or I would forget. I have been hanging with the idea this would be fixed.

          Show
          Vernellia Randall added a comment - I am glad to see this in priority. Missed deadlines seriously affected my evaluations. No matter what time I set to "switch" either something would come up or I would forget. I have been hanging with the idea this would be fixed.
          Hide
          David Mudrak added a comment -

          Submitting for a peer-review.

          Show
          David Mudrak added a comment - Submitting for a peer-review.
          Hide
          David Mudrak added a comment -

          Attaching some screenshots that illustrate the UI of the new feature.

          Show
          David Mudrak added a comment - Attaching some screenshots that illustrate the UI of the new feature.
          Hide
          David Mudrak added a comment -

          General information about how I implemented this: With regards to automatic phase switching, the only real trouble was to make sure that submissions are allocated for assessment. Till now, all allocation methods were interactive (ie they required teacher's assistance). In the subtask of this issue, a new allocation subplugin workshopallocation_scheduled was developed. It internally uses workshopallocation_random in that way that it lets the teacher to pre-define parameters for the random allocation and executes it later.

          So now, when the teacher defines the submissions deadline and enables the automatic phase switching, two things happen:

          • the first cron job after the submission deadline executes the scheduled allocation
          • right after that, the phase is switched from the submission phase to the assessment phase

          Note there there is no support for automatic switching to other workshop phases for 2.3 planned. That is, the teacher still has to come at the end of the assessment phase and evaluate assessments, let workshop to calculate final grades and close the workshop. Some folks demanded this to be automated, too. But that will not happen in 2.3.

          Show
          David Mudrak added a comment - General information about how I implemented this: With regards to automatic phase switching, the only real trouble was to make sure that submissions are allocated for assessment. Till now, all allocation methods were interactive (ie they required teacher's assistance). In the subtask of this issue, a new allocation subplugin workshopallocation_scheduled was developed. It internally uses workshopallocation_random in that way that it lets the teacher to pre-define parameters for the random allocation and executes it later. So now, when the teacher defines the submissions deadline and enables the automatic phase switching, two things happen: the first cron job after the submission deadline executes the scheduled allocation right after that, the phase is switched from the submission phase to the assessment phase Note there there is no support for automatic switching to other workshop phases for 2.3 planned. That is, the teacher still has to come at the end of the assessment phase and evaluate assessments, let workshop to calculate final grades and close the workshop. Some folks demanded this to be automated, too. But that will not happen in 2.3.
          Hide
          Vernellia Randall added a comment -

          So for Clarification

          (1) Teacher Setups workshop
          (2) Teacher has to manually switch from Setup to Submission
          (3) Workshop can be switched to Submission far in advance of submission date because Workshop won't accept any submission until the date and time.
          (4) At the close of the submission date, Workshop automatically closes the submission phase
          (5) if teacher, during setup phase, selected random allocation, workshop automatically allocates the submitted work and opens the assessment phases.
          (6) if teacher, during setup phase, did not select random allocation, teacher has to manually allocate and manually switches from submission to assessment phase
          (7) once assessment phase is completed teacher has to manually switch from assessment phase to grading phase
          (8) once grading phase is complete, teacher has to manually close the workshop.

          Is this correct? Do you anticipate making the rest of workshop automatic? If so when? If not, Why not?

          Show
          Vernellia Randall added a comment - So for Clarification (1) Teacher Setups workshop (2) Teacher has to manually switch from Setup to Submission (3) Workshop can be switched to Submission far in advance of submission date because Workshop won't accept any submission until the date and time. (4) At the close of the submission date, Workshop automatically closes the submission phase (5) if teacher, during setup phase, selected random allocation, workshop automatically allocates the submitted work and opens the assessment phases. (6) if teacher, during setup phase, did not select random allocation, teacher has to manually allocate and manually switches from submission to assessment phase (7) once assessment phase is completed teacher has to manually switch from assessment phase to grading phase (8) once grading phase is complete, teacher has to manually close the workshop. Is this correct? Do you anticipate making the rest of workshop automatic? If so when? If not, Why not?
          Hide
          Dan Poltawski added a comment -

          Hi David,

          This looks good - some comments nothing particularly major just my own observations.

          1. workshop_scheduled_allocator::delete_instance() is //TODO
          2. (Caveat: I'm not well versed on the limits of workshop subplugins). It feels a bit wrong to me to be hardcoding workshopallocation plugin in workshop_cron(), why not do it 'properly' so all plugins can have cron? Would that also make it possible for someone to make a modified version fo your allocator? (perhaps not, but still seems like we could just avoid this hardcoded plugin choice).
          3. Strict Standards: Declaration of mod_workshop_mod_form::validation() should be compatible with that of moodleform_mod::validation() in /Users/danp/git/moodle/mod/workshop/mod_form.php on line 40

          Some points of intrigue for me from a software design point of view (educate me ):

          1. Why are you changing the commented out log lines like:
            -        // $o[] = 'debug::circle groups = ' . json_encode($circlegroups);
            +        // $result->log('circle groups = ' . json_encode($circlegroups), 'debug');
            

            Are they being used regularly for debugging? Should they use debugging properly?

          2. Why create the workshop_random_allocator_setting::instance_from_object factory method over using the constructor?
          Show
          Dan Poltawski added a comment - Hi David, This looks good - some comments nothing particularly major just my own observations. workshop_scheduled_allocator::delete_instance() is //TODO ( Caveat: I'm not well versed on the limits of workshop subplugins ). It feels a bit wrong to me to be hardcoding workshopallocation plugin in workshop_cron(), why not do it 'properly' so all plugins can have cron? Would that also make it possible for someone to make a modified version fo your allocator? (perhaps not, but still seems like we could just avoid this hardcoded plugin choice). Strict Standards: Declaration of mod_workshop_mod_form::validation() should be compatible with that of moodleform_mod::validation() in /Users/danp/git/moodle/mod/workshop/mod_form.php on line 40 Some points of intrigue for me from a software design point of view (educate me ): Why are you changing the commented out log lines like: - // $o[] = 'debug::circle groups = ' . json_encode($circlegroups); + // $result->log('circle groups = ' . json_encode($circlegroups), 'debug'); Are they being used regularly for debugging? Should they use debugging properly? Why create the workshop_random_allocator_setting::instance_from_object factory method over using the constructor?
          Hide
          David Mudrak added a comment -

          Hi Dan. Thanks for detailed review.

          1. Yes, workshop subplugins do not support uninstall yet. I discussed this issue with Petr yesterday and I suggested an approach of a general uninstall helper method being part of the plugin_manager that would deal with it. Hence to be solved in separate issue.
          2. Note that I am not hardcoding a particular subplugin - what is hardcoded is a particular subplugin type. So if somebody writes another allocation plugin (eg. scheduler that work differently), they can have a cron method there. I just don't give other workshop subplugin types (such as grading strategies or grading evaluators) a chance to have their cron yet. There would be no point of loading all their libraries during the cron just to realize they have no cron method. Once there is a good reason to add cron support for them, we can easily let workshop core to call them. Same approach is used by mod_quiz, for example.
          3. Right. I was just trying to have better code that the parent method has. I did not realize the strict standards here. Will fix.
          4. Change of that comments must be seen in the context of the whole commit. These commented lines were useful during the development but they are just too much verbose even for normal debugging purposes. I just did not want to remove them in case they would be useful one day again.
          5. That's standard factory pattern. There are two factory methods to create instance of the class, depending on the type of given data (text or stdClass). If PHP supported http://en.wikipedia.org/wiki/Method_overloading I would have two __construct() methods in the class, each of them taking different type of parameters.

          Show
          David Mudrak added a comment - Hi Dan. Thanks for detailed review. 1. Yes, workshop subplugins do not support uninstall yet. I discussed this issue with Petr yesterday and I suggested an approach of a general uninstall helper method being part of the plugin_manager that would deal with it. Hence to be solved in separate issue. 2. Note that I am not hardcoding a particular subplugin - what is hardcoded is a particular subplugin type. So if somebody writes another allocation plugin (eg. scheduler that work differently), they can have a cron method there. I just don't give other workshop subplugin types (such as grading strategies or grading evaluators) a chance to have their cron yet. There would be no point of loading all their libraries during the cron just to realize they have no cron method. Once there is a good reason to add cron support for them, we can easily let workshop core to call them. Same approach is used by mod_quiz, for example. 3. Right. I was just trying to have better code that the parent method has. I did not realize the strict standards here. Will fix. 4. Change of that comments must be seen in the context of the whole commit. These commented lines were useful during the development but they are just too much verbose even for normal debugging purposes. I just did not want to remove them in case they would be useful one day again. 5. That's standard factory pattern. There are two factory methods to create instance of the class, depending on the type of given data (text or stdClass). If PHP supported http://en.wikipedia.org/wiki/Method_overloading I would have two __construct() methods in the class, each of them taking different type of parameters.
          Hide
          David Mudrak added a comment -

          Hi Vernellia. Thanks for your consideration of this.

          Yes, all your points are almost correct. Just to clarify some details:

          (re 5) The teacher actually sets up "scheduled allocation" during the setup phase. But yes, at the end the random allocation is automatically executed on submissions deadline, right before the workshop switches into the assessment phase

          (re 6) The workshop would switch into the assessment phase anyway, even if the scheduled allocation was not set up. Those are two independent features.

          Do you anticipate making the rest of workshop automatic? If so when? If not, Why not?

          Well, no. I don't plan to extend these automatic switching to other phases. For the following reasons.

          • The Setup -> Submission switch is useless for the reason you mentioned in (3) - the workshop can be put into the submission phase in advance.
          • The Submission -> Assessment switch was just implemented and will be available since Moodle 2.3
          • The Assessment -> Evaluation switch is semi-useful for the same reason as in the first one. If the assessment deadline is defined, the workshop can stay in the assessment phase safely.
          • The Evaluation -> Closed auto switch phase would be conceptually wrong, as I was trying to explain several times in the forum. I believe that there must always be a real person who comes to the workshop, checks the automatically calculated grades and makes sure they are valid. This person (typically the teacher) is the one who "signs off" the given grades and pushes them to the gradebook (by closing the workshop). This can't be done by the LMS itself. Workshop is not the Quiz. Quiz has pre-defined rules how grades are calculated. In the Workshop, grades are given on the peer-assessment basis and just must be checked.
          Show
          David Mudrak added a comment - Hi Vernellia. Thanks for your consideration of this. Yes, all your points are almost correct. Just to clarify some details: (re 5) The teacher actually sets up "scheduled allocation" during the setup phase. But yes, at the end the random allocation is automatically executed on submissions deadline, right before the workshop switches into the assessment phase (re 6) The workshop would switch into the assessment phase anyway, even if the scheduled allocation was not set up. Those are two independent features. Do you anticipate making the rest of workshop automatic? If so when? If not, Why not? Well, no. I don't plan to extend these automatic switching to other phases. For the following reasons. The Setup -> Submission switch is useless for the reason you mentioned in (3) - the workshop can be put into the submission phase in advance. The Submission -> Assessment switch was just implemented and will be available since Moodle 2.3 The Assessment -> Evaluation switch is semi-useful for the same reason as in the first one. If the assessment deadline is defined, the workshop can stay in the assessment phase safely. The Evaluation -> Closed auto switch phase would be conceptually wrong, as I was trying to explain several times in the forum. I believe that there must always be a real person who comes to the workshop, checks the automatically calculated grades and makes sure they are valid. This person (typically the teacher) is the one who "signs off" the given grades and pushes them to the gradebook (by closing the workshop). This can't be done by the LMS itself. Workshop is not the Quiz. Quiz has pre-defined rules how grades are calculated. In the Workshop, grades are given on the peer-assessment basis and just must be checked.
          Hide
          Dan Poltawski added a comment -

          Thanks David,

          Regarding 5) I understand the use of the factory method for text but don't really understand the advantage of using a factory method in the case of the stdClass over its natural constructor (it seems like passing the stdClass would be 'natural' way to construct this object).

          Show
          Dan Poltawski added a comment - Thanks David, Regarding 5) I understand the use of the factory method for text but don't really understand the advantage of using a factory method in the case of the stdClass over its natural constructor (it seems like passing the stdClass would be 'natural' way to construct this object).
          Hide
          David Mudrak added a comment -

          Well, my understanding of the factory method pattern is that it sort of implies protected constructor. Once a factory pattern is implemented, the factory method(s) should be the only allowed way of instantiating objects. IMHO.

          Show
          David Mudrak added a comment - Well, my understanding of the factory method pattern is that it sort of implies protected constructor. Once a factory pattern is implemented, the factory method(s) should be the only allowed way of instantiating objects. IMHO.
          Hide
          David Mudrak added a comment -

          Rebased against the latest origin/master and fixed the strict standard warning spotted by Dan during the peer-review.

          Submitting for integration.

          Show
          David Mudrak added a comment - Rebased against the latest origin/master and fixed the strict standard warning spotted by Dan during the peer-review. Submitting for integration.
          Hide
          Michelle Moore added a comment - - edited

          In response to Vernellia's comments and your responses . . .

          First, I appreciate the work you've done here and will be very happy to have some of this functionality back. Second, the remainder of my comments are likely similar to those I've made before, but I want to go on record again and hopefully improve my previous arguments.

          In particular, I'd like to see the automatic phase switching extended through the grading evaluation and closing phases of the workshop.

          First, the release of grades and publishing to the gradebook were automatic features in the past, so we're still talking about a feature regression. On the plus side, if we add this automation, we're looking at a big improvement because the grades aren't revealed as they're being calculated so students aren't seeing the average change with each submission--something that caused my students concern in the past.

          Second, while there are plenty of situations where a teacher does need to come into a workshop to "sign off" on the given grades, there are situations where it's not necessary either. For example, if I create a really simple yes/no assessment form or if I have a really good, detailed rubric and/or an experienced set of users, then there's usually little to no intervention required. There have been numerous times where I make no adjustments to workshop grades and it would be great to be able to automate that process so I'm not a roadblock in the process.

          If you do automate that portion of the workshop, it would be really great if the workshop could be set to proceed with a set of either/or options. For instance, I would like to be able to switch to the next phase based on date, but if everyone finishes the assessments early, it would be nice to go ahead and release the scores then. Or, I might want to just set the workshop to release when the assessments are complete without any date condition; this option is less likely, but may meet someone's needs.

          As I think of it, it would actually be nice to have a similar option from the submission to assessment phase too-to have the workshop switch to assessment when all submissions have been received as opposed to switching based just on date. I realize this makes for a new feature request, one I would qualify under "nice to have"-if there's any chance it might be implemented, let me know and I can capture it in a new tracker item.

          Show
          Michelle Moore added a comment - - edited In response to Vernellia's comments and your responses . . . First, I appreciate the work you've done here and will be very happy to have some of this functionality back. Second, the remainder of my comments are likely similar to those I've made before, but I want to go on record again and hopefully improve my previous arguments. In particular, I'd like to see the automatic phase switching extended through the grading evaluation and closing phases of the workshop. First, the release of grades and publishing to the gradebook were automatic features in the past, so we're still talking about a feature regression. On the plus side, if we add this automation, we're looking at a big improvement because the grades aren't revealed as they're being calculated so students aren't seeing the average change with each submission--something that caused my students concern in the past. Second, while there are plenty of situations where a teacher does need to come into a workshop to "sign off" on the given grades, there are situations where it's not necessary either. For example, if I create a really simple yes/no assessment form or if I have a really good, detailed rubric and/or an experienced set of users, then there's usually little to no intervention required. There have been numerous times where I make no adjustments to workshop grades and it would be great to be able to automate that process so I'm not a roadblock in the process. If you do automate that portion of the workshop, it would be really great if the workshop could be set to proceed with a set of either/or options. For instance, I would like to be able to switch to the next phase based on date, but if everyone finishes the assessments early, it would be nice to go ahead and release the scores then. Or, I might want to just set the workshop to release when the assessments are complete without any date condition; this option is less likely, but may meet someone's needs. As I think of it, it would actually be nice to have a similar option from the submission to assessment phase too-to have the workshop switch to assessment when all submissions have been received as opposed to switching based just on date. I realize this makes for a new feature request, one I would qualify under "nice to have"-if there's any chance it might be implemented, let me know and I can capture it in a new tracker item.
          Hide
          Sam Hemelryk added a comment -

          Hi David,

          The changes here look very good, there were only two things that caught my eye.
          1. The changes to the workshop allocation subplugins appear to require any existing allocation subplugin to be revisited, albeit the changes don't appear to be too difficult to wrap ones head around. I see the docs required label has been added so I assume you're on top of documenting the changes and how people can upgrade the custom allocation plugins (do you know if there are any actually?)
          2. There are several todo's in this code, I grep'd the rest of workshop and see that there are already many. Perhaps it would be an idea to try to link those to MDL issues creating them where required. Not worth delaying these changes over by any means.

          Cheers
          Sam

          Show
          Sam Hemelryk added a comment - Hi David, The changes here look very good, there were only two things that caught my eye. 1. The changes to the workshop allocation subplugins appear to require any existing allocation subplugin to be revisited, albeit the changes don't appear to be too difficult to wrap ones head around. I see the docs required label has been added so I assume you're on top of documenting the changes and how people can upgrade the custom allocation plugins (do you know if there are any actually?) 2. There are several todo's in this code, I grep'd the rest of workshop and see that there are already many. Perhaps it would be an idea to try to link those to MDL issues creating them where required. Not worth delaying these changes over by any means. Cheers Sam
          Hide
          David Mudrak added a comment -

          Hi Sam. Thanks for the review. No, I am not aware of any custom allocation subplugins out there. The Plugins registry is empty http://moodle.org/plugins/browse.php?list=category&id=17 and nobody even mentioned any work on their own solution in the forum. It seems to be a good idea to describe the changes in upgrade.txt though, for sure.

          Yes, many todos are left from the development of the workshop rewrite. I am continuously dealing with them as I work on workshop from time to time (like a bulk of improvements for 2.3 that will remove some of them). Do you think it worth creating a new issue to add MDLs to all those todos? If so, I can do it after 2.3 coding freeze.

          Show
          David Mudrak added a comment - Hi Sam. Thanks for the review. No, I am not aware of any custom allocation subplugins out there. The Plugins registry is empty http://moodle.org/plugins/browse.php?list=category&id=17 and nobody even mentioned any work on their own solution in the forum. It seems to be a good idea to describe the changes in upgrade.txt though, for sure. Yes, many todos are left from the development of the workshop rewrite. I am continuously dealing with them as I work on workshop from time to time (like a bulk of improvements for 2.3 that will remove some of them). Do you think it worth creating a new issue to add MDLs to all those todos? If so, I can do it after 2.3 coding freeze.
          Hide
          David Mudrak added a comment -

          Hi Michelle. Thanks for the comments.

          After reading your post, I can see the point of automatic phase switching from Evaluation to Close. Although the same effect can be reached by using the "hidden until" feature of the gradebook, right?

          With regards to automatic calculation of workshop grades, I am not sure I'm convinced still. If such a feature is ever implemented (and of course, the issue can be reported for it), it will definitely require more complex setup than just date-based. But still a lot of unanswered questions are at the table - like what user should be recorded as the one who actually approved (gave) the grade. Imagine there are multiple teachers in the course. At the moment, the one who closes the workshop is the one who is recorded in the gradebook as the responsible grader. I can imagine pretty difficult consequences. Imagine you as the teacher are blamed for grades that some student got, that are signed off by yourself even if you never actually saw them. That even hits some legal apects IMHO. Grades in LMS require special care...

          Show
          David Mudrak added a comment - Hi Michelle. Thanks for the comments. After reading your post, I can see the point of automatic phase switching from Evaluation to Close. Although the same effect can be reached by using the "hidden until" feature of the gradebook, right? With regards to automatic calculation of workshop grades, I am not sure I'm convinced still. If such a feature is ever implemented (and of course, the issue can be reported for it), it will definitely require more complex setup than just date-based. But still a lot of unanswered questions are at the table - like what user should be recorded as the one who actually approved (gave) the grade. Imagine there are multiple teachers in the course. At the moment, the one who closes the workshop is the one who is recorded in the gradebook as the responsible grader. I can imagine pretty difficult consequences. Imagine you as the teacher are blamed for grades that some student got, that are signed off by yourself even if you never actually saw them. That even hits some legal apects IMHO. Grades in LMS require special care...
          Hide
          Michelle Moore added a comment -

          Thank you for your consideration of my ideas--you raise some interesting issues. I'm going to process what you've said and will try to talk to some other workshop users to see what they think too.

          Show
          Michelle Moore added a comment - Thank you for your consideration of my ideas--you raise some interesting issues. I'm going to process what you've said and will try to talk to some other workshop users to see what they think too.
          Hide
          Sam Hemelryk added a comment -

          Hi David,

          Thanks for thinking about the documentation.

          In regards to the todo's I think it is entirely up to you, if you're on top of them and are working your way through them then I am happy
          Having MDLs created and noted for them just helps frame up the work still required, and ensure that none get forgotten and all end up in someone's todo list in tracker.
          Of course workshop is your module and it's a shining example of a module so I do think it should be up to you how you manage it.

          Cheers
          Sam

          Show
          Sam Hemelryk added a comment - Hi David, Thanks for thinking about the documentation. In regards to the todo's I think it is entirely up to you, if you're on top of them and are working your way through them then I am happy Having MDLs created and noted for them just helps frame up the work still required, and ensure that none get forgotten and all end up in someone's todo list in tracker. Of course workshop is your module and it's a shining example of a module so I do think it should be up to you how you manage it. Cheers Sam
          Hide
          Michael de Raadt added a comment -

          During testing, when I enabled scheduled allocation, I saw this notice...

          {"numofreviews":2,"numper":1,"excludesamegroup":false,"removecurrent":false,"assesswosubmission":false,"addselfassessment":false}
          

          It looks like a debug message, but it could be intending to be informative.

          Show
          Michael de Raadt added a comment - During testing, when I enabled scheduled allocation, I saw this notice... {"numofreviews":2,"numper":1,"excludesamegroup":false,"removecurrent":false,"assesswosubmission":false,"addselfassessment":false} It looks like a debug message, but it could be intending to be informative.
          Hide
          Michael de Raadt added a comment -

          Test result: Success

          Tested in master only.

          Phase was automatically shifted and student reviews were automatically allocated to students who had submitted.

          The following message was shown during manually running cron...

          Processing module function workshop_cron ... processing workshop subplugins ...
          Starting workshop allocation methods
          Processing cron function for workshopallocation_scheduled...
          ... executing scheduled allocation in 1 workshop(s) ... done. (30 dbqueries, 0.05 seconds)
          Finished workshop allocation methods
          Processing automatic assessment phase switch in 1 workshop(s) ... done
          ... used 41 dbqueries
          ... used 0.087053060531616 seconds
          done.
          
          Show
          Michael de Raadt added a comment - Test result: Success Tested in master only. Phase was automatically shifted and student reviews were automatically allocated to students who had submitted. The following message was shown during manually running cron... Processing module function workshop_cron ... processing workshop subplugins ... Starting workshop allocation methods Processing cron function for workshopallocation_scheduled... ... executing scheduled allocation in 1 workshop(s) ... done. (30 dbqueries, 0.05 seconds) Finished workshop allocation methods Processing automatic assessment phase switch in 1 workshop(s) ... done ... used 41 dbqueries ... used 0.087053060531616 seconds done.
          Hide
          David Mudrak added a comment -

          Thanks for testing this Michael. Confirmed, that encrypted JSON message is actually a debugging-level message and it was expected to see.

          Show
          David Mudrak added a comment - Thanks for testing this Michael. Confirmed, that encrypted JSON message is actually a debugging-level message and it was expected to see.
          Hide
          Dan Poltawski added a comment -

          Bonza mate!

          Your changes have made it into the Moodle release! Its time to celebrate! Put a shrimp on the barbie and grab a stubby.

          Hooroo

          Show
          Dan Poltawski added a comment - Bonza mate! Your changes have made it into the Moodle release! Its time to celebrate! Put a shrimp on the barbie and grab a stubby. Hooroo
          Hide
          Vernellia Randall added a comment -

          Thank you David!! I am so please to know that is coming in the next release. Thank you for all the work you put into workshop. My university supports saki and I have refused to move because there is nothing like workshop in Saki and this fixes a significant useability issue. Vernellia

          Show
          Vernellia Randall added a comment - Thank you David!! I am so please to know that is coming in the next release. Thank you for all the work you put into workshop. My university supports saki and I have refused to move because there is nothing like workshop in Saki and this fixes a significant useability issue. Vernellia
          Hide
          Jared R. Chapman added a comment -

          David and everyone else working on this project, thank you so much! This is an amazing tool and significantly better than any other out there as far as I could find.

          I would like to throw my emphatic support behind having the option for the review process to close automatically. My guess is that it comes down to use case. Here is mine:

          I have 200+ students. I believe in the power of articulation and written assignments. I also believe in peer review.

          Each week I assign 3 brief writing assignments to each student, that means I have 600+ papers to grade each week. Normally this would not be feasible and the students would have to make due with a less impactful learning experience.

          With moodle 1.9, I set up everything in workshops, the students are assigned papers to review, grades are given, averaged, and automatically assigned to the grade book. Students know that if they are unhappy with the grade they receive, than can appeal to me, and I will grade their paper. Each week I read a fair representation of the papers. (as an aside, automating this appeal would be a nice feature request)

          As far as grading goes, the papers are subjective responses to case studies and self assessments, there is no right answer. Peer reviewers are simply evaluating level of effort using a rubric. 0 for 0 work, 4 for poor work, 8 for acceptable work and 10 for "it is so good it makes you want to cry". As they make this evaluation, they are exposed to the opinions and experiences of 3 of their peers, broadening their own experience. Each semester each student is peer reviewed 105 times, some reviews are too high, some are too low, but in the end I have a pretty solid assessment of each student's performance.

          Now, this process is quite a bit more difficult in the current work shop. Each week I must update 6 workshops for each of my 5 courses, 30 processes to complete. If I am late, or forget, or make a mistake, I receive a maelstrom of email from inconvenienced students.

          While this is better than nothing, it is not better than the old system for me. I hope this use case demonstrates at least one scenario where a beginning to end automated process makes sense.

          Show
          Jared R. Chapman added a comment - David and everyone else working on this project, thank you so much! This is an amazing tool and significantly better than any other out there as far as I could find. I would like to throw my emphatic support behind having the option for the review process to close automatically. My guess is that it comes down to use case. Here is mine: I have 200+ students. I believe in the power of articulation and written assignments. I also believe in peer review. Each week I assign 3 brief writing assignments to each student, that means I have 600+ papers to grade each week. Normally this would not be feasible and the students would have to make due with a less impactful learning experience. With moodle 1.9, I set up everything in workshops, the students are assigned papers to review, grades are given, averaged, and automatically assigned to the grade book. Students know that if they are unhappy with the grade they receive, than can appeal to me, and I will grade their paper. Each week I read a fair representation of the papers. (as an aside, automating this appeal would be a nice feature request) As far as grading goes, the papers are subjective responses to case studies and self assessments, there is no right answer. Peer reviewers are simply evaluating level of effort using a rubric. 0 for 0 work, 4 for poor work, 8 for acceptable work and 10 for "it is so good it makes you want to cry". As they make this evaluation, they are exposed to the opinions and experiences of 3 of their peers, broadening their own experience. Each semester each student is peer reviewed 105 times, some reviews are too high, some are too low, but in the end I have a pretty solid assessment of each student's performance. Now, this process is quite a bit more difficult in the current work shop. Each week I must update 6 workshops for each of my 5 courses, 30 processes to complete. If I am late, or forget, or make a mistake, I receive a maelstrom of email from inconvenienced students. While this is better than nothing, it is not better than the old system for me. I hope this use case demonstrates at least one scenario where a beginning to end automated process makes sense.
          Hide
          Vernellia Randall added a comment -

          Jared, Over a year ago, I made the same case. I continually have problems remember to switch the phases. I just had a major end of the year assignment messed up because I forgot to switch phases. I think that it is a major step to have it switch from submission to assessment. The next step is to allow the total automation if that what a teacher wants.

          Show
          Vernellia Randall added a comment - Jared, Over a year ago, I made the same case. I continually have problems remember to switch the phases. I just had a major end of the year assignment messed up because I forgot to switch phases. I think that it is a major step to have it switch from submission to assessment. The next step is to allow the total automation if that what a teacher wants.
          Hide
          Mary Cooch added a comment - - edited

          The ability to switch phases automatically has been added to 2.3 docs. http://docs.moodle.org/23/en/Workshop_settings#Access_control

          Show
          Mary Cooch added a comment - - edited The ability to switch phases automatically has been added to 2.3 docs. http://docs.moodle.org/23/en/Workshop_settings#Access_control
          Hide
          Helen Foster added a comment -

          Just noting that the new setting is also mentioned in http://docs.moodle.org/23/en/Workshop_FAQ

          Show
          Helen Foster added a comment - Just noting that the new setting is also mentioned in http://docs.moodle.org/23/en/Workshop_FAQ
          Hide
          Jason Bennett added a comment -

          My testing seems to indicate that the phases on switch the next time cron runs after the schedule phase switch time. Is that how it's meant to function?

          Show
          Jason Bennett added a comment - My testing seems to indicate that the phases on switch the next time cron runs after the schedule phase switch time. Is that how it's meant to function?
          Hide
          David Mudrak added a comment -

          Yes, the first cron.php job execution after the scheduled time actually switches the phase. There is no other way in Moodle. That is why the cron.php should be executed every couple of minutes.

          Show
          David Mudrak added a comment - Yes, the first cron.php job execution after the scheduled time actually switches the phase. There is no other way in Moodle. That is why the cron.php should be executed every couple of minutes.
          Hide
          Jason Bennett added a comment -

          Help me understand this. Why do some events in Moodle, a Quiz open and close schedule, for example, happen exactly when scheduled and don't seem to be dependent on the cron schedule? Why can't Workshop behave like that? I see in the Moodle cron docs that the burden to the server isn't so great that scheduling cron to happen often is a problem but not everyone has the ability to change that on their server.

          Thanks.

          Show
          Jason Bennett added a comment - Help me understand this. Why do some events in Moodle, a Quiz open and close schedule, for example, happen exactly when scheduled and don't seem to be dependent on the cron schedule? Why can't Workshop behave like that? I see in the Moodle cron docs that the burden to the server isn't so great that scheduling cron to happen often is a problem but not everyone has the ability to change that on their server. Thanks.
          Hide
          David Mudrak added a comment -

          Quiz open/close schedule is not actually an operation to execute. It is just a condition checked at every time the user comes to the Quiz page. If the current time is within the allowed interval, quiz can be taken. If not, the user is informed. On contrary to that, the scheduled allocation is an operation that must execute something (it must allocate submissions and - in the future - it might send email notifications etc.). It is not just a yes/no check. It's more like sending the subscribed forum posts via email (which happens via cron, too).

          There is a space for improvements, for sure. For example, the Workshop could check for the need of scheduled allocation every time when some user opens the workshop main page. For example, let us say the phase switching (and thence the allocation) is scheduled to 12:40 PM and the cron runs every 30 minutes. So the cron execution at 12:30 did not trigger anything yet, the next execution at 13:00 would do it. We could improve Workshop so that when some user (teacher, student or even a guest) looks at the workshop page at 12:43 PM, it would trigger the phase switch (so the result is what the user expects). Let me know if this would be suitable for your needs and we can open a new issue for that.

          Hope this explains.

          Show
          David Mudrak added a comment - Quiz open/close schedule is not actually an operation to execute. It is just a condition checked at every time the user comes to the Quiz page. If the current time is within the allowed interval, quiz can be taken. If not, the user is informed. On contrary to that, the scheduled allocation is an operation that must execute something (it must allocate submissions and - in the future - it might send email notifications etc.). It is not just a yes/no check. It's more like sending the subscribed forum posts via email (which happens via cron, too). There is a space for improvements, for sure. For example, the Workshop could check for the need of scheduled allocation every time when some user opens the workshop main page. For example, let us say the phase switching (and thence the allocation) is scheduled to 12:40 PM and the cron runs every 30 minutes. So the cron execution at 12:30 did not trigger anything yet, the next execution at 13:00 would do it. We could improve Workshop so that when some user (teacher, student or even a guest) looks at the workshop page at 12:43 PM, it would trigger the phase switch (so the result is what the user expects). Let me know if this would be suitable for your needs and we can open a new issue for that. Hope this explains.
          Hide
          Jason Bennett added a comment -

          Thanks for the explanation. I suspected that Quizzes worked as you describe so thanks for confirming.

          Our cron job runs at 19 minutes after the hour but not after every hour. I could certainly try to convince my web server admin to change that but, as I said, I am certain that other institutions using Moodle, especially some who host externally, don't have any control over the frequency of their cron job. I believe it would be a great improvement if Workshop changed phase based on user views rather than cron. Otherwise, people are going to be surprised when they discover that the phase hasn't switched when it was scheduled to switch. I had that happen with one faculty member who set a workshop activity for 8 sections of a class taught by 4 different professors. It was awkward having to explain to her that the phases would switch at 19 after the hour following her scheduled switch time/date, but only if she scheduled it between 6am and 8pm.

          Show
          Jason Bennett added a comment - Thanks for the explanation. I suspected that Quizzes worked as you describe so thanks for confirming. Our cron job runs at 19 minutes after the hour but not after every hour. I could certainly try to convince my web server admin to change that but, as I said, I am certain that other institutions using Moodle, especially some who host externally, don't have any control over the frequency of their cron job. I believe it would be a great improvement if Workshop changed phase based on user views rather than cron. Otherwise, people are going to be surprised when they discover that the phase hasn't switched when it was scheduled to switch. I had that happen with one faculty member who set a workshop activity for 8 sections of a class taught by 4 different professors. It was awkward having to explain to her that the phases would switch at 19 after the hour following her scheduled switch time/date, but only if she scheduled it between 6am and 8pm.
          Hide
          David Mudrak added a comment -

          OK, makes perfect sense and your use case is serious. I just created MDL-37781 to track this.

          Show
          David Mudrak added a comment - OK, makes perfect sense and your use case is serious. I just created MDL-37781 to track this.

            People

            • Votes:
              26 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: