Details

    • Affected Branches:
      MOODLE_23_STABLE
    • Rank:
      39004

      Description

      add plagiarism api hooks to essay quiz questions

        Activity

        Hide
        Tim Hunt added a comment -

        I am happy to discuss this when the time comes. Some questions to consider:

        1. Do you really mean 'Essay questions', or is some other level of abstraction better. (E.g. what about OU's pmatch question type?)

        2. Is this really a question feature, or a quiz feature? Presumably plagiarism services do better on longer documents, so all responses to a quiz as one submission might be better than doing one question at a time.

        3. When, exactly, do we plagiarism-check the work. Think, for example, about a quiz in adaptive mode.

        I am sure all this is soluble, but there are some design decisions to make.

        Show
        Tim Hunt added a comment - I am happy to discuss this when the time comes. Some questions to consider: 1. Do you really mean 'Essay questions', or is some other level of abstraction better. (E.g. what about OU's pmatch question type?) 2. Is this really a question feature, or a quiz feature? Presumably plagiarism services do better on longer documents, so all responses to a quiz as one submission might be better than doing one question at a time. 3. When, exactly, do we plagiarism-check the work. Think, for example, about a quiz in adaptive mode. I am sure all this is soluble, but there are some design decisions to make.
        Hide
        Kanika Goyal added a comment -

        Hi all,

        I have a problem in deciding when should we check the content for plagiarism? When should
        the event that will sent the data to plagiarism plugin be triggered as essay questions has
        no specific function of it when data is saved/updated.
        Please suggest me the method.

        Thanks,
        Kanika

        Show
        Kanika Goyal added a comment - Hi all, I have a problem in deciding when should we check the content for plagiarism? When should the event that will sent the data to plagiarism plugin be triggered as essay questions has no specific function of it when data is saved/updated. Please suggest me the method. Thanks, Kanika
        Hide
        Tim Hunt added a comment -

        Hmm. Should this be an essay qtype feature, or a quiz feature?

        Show
        Tim Hunt added a comment - Hmm. Should this be an essay qtype feature, or a quiz feature?
        Hide
        Dan Marsden added a comment -

        Thanks Tim - I'm not sure on that myself - keen for some guidance from you on this.

        We need some way of sending content to the plagiarism plugins by use of event triggers when appropriate - we can trigger this event for every edit/save/update and the plagiarism plugin can decide which events to use/ignore.

        We also need some way of displaying a link to a report/similarity score for the content that has been processed - I'd initially thought this would just appear in the screen used by teachers when marking essay style questions but I'm open to feedback.

        are there many other question plugins that allow larger amounts of content to be entered that could be submitted to a plagiarism checker? - for example Turnitin requires more than 100 characters of non-white space.

        Show
        Dan Marsden added a comment - Thanks Tim - I'm not sure on that myself - keen for some guidance from you on this. We need some way of sending content to the plagiarism plugins by use of event triggers when appropriate - we can trigger this event for every edit/save/update and the plagiarism plugin can decide which events to use/ignore. We also need some way of displaying a link to a report/similarity score for the content that has been processed - I'd initially thought this would just appear in the screen used by teachers when marking essay style questions but I'm open to feedback. are there many other question plugins that allow larger amounts of content to be entered that could be submitted to a plagiarism checker? - for example Turnitin requires more than 100 characters of non-white space.
        Hide
        Tim Hunt added a comment -

        Well, you could decide to turn the entire quiz submission into one 'document' to submit to plagiarism checking.

        That gives you a clear event to look out for, the quiz Submit all and finish action, which does trigger a Moodle event.

        You could make a quiz report plug-in to display the results. That would also give you a place to put some UI to configure which questions have their responses submitted for checking.

        Look at mod/quiz/report/responses to see how to display the responses to all the questions in a quiz as plain text (although you will want a more conventional document, not a table row).

        Show
        Tim Hunt added a comment - Well, you could decide to turn the entire quiz submission into one 'document' to submit to plagiarism checking. That gives you a clear event to look out for, the quiz Submit all and finish action, which does trigger a Moodle event. You could make a quiz report plug-in to display the results. That would also give you a place to put some UI to configure which questions have their responses submitted for checking. Look at mod/quiz/report/responses to see how to display the responses to all the questions in a quiz as plain text (although you will want a more conventional document, not a table row).
        Hide
        Dan Marsden added a comment -

        hmmm - I'm not sure that would be all that useful unless we could filter out all the common data from the text doc and I can't see how useful it would be for normal question types - if the doc contained the questions that part would be common across all submissions and appear as a large amount of the doc - I'd imagine it would cause a very high similarity score between docs.

        Is there a way that we can generate a doc that contains only "text" answers for larger questions like the essay q's?

        Show
        Dan Marsden added a comment - hmmm - I'm not sure that would be all that useful unless we could filter out all the common data from the text doc and I can't see how useful it would be for normal question types - if the doc contained the questions that part would be common across all submissions and appear as a large amount of the doc - I'd imagine it would cause a very high similarity score between docs. Is there a way that we can generate a doc that contains only "text" answers for larger questions like the essay q's?
        Hide
        Tim Hunt added a comment -

        When I say "That would also give you a place to put some UI to configure which questions have their responses submitted for checking." I meant to add

        By default I suggest that only essay questions are included, but it might be useful for teachers to be able to include other responses, for example to short-answer or pmatch questions (http://moodle.org/plugins/view.php?plugin=qtype_pmatch).

        While the individual answers to different questions might be quite similar, the collection in total would not be. I am sure people sometimes submit documents for plagiarism checking that are a mixture of a few short-answer questions, and a few longer essays, and that it can give useful results.

        Show
        Tim Hunt added a comment - When I say "That would also give you a place to put some UI to configure which questions have their responses submitted for checking." I meant to add By default I suggest that only essay questions are included, but it might be useful for teachers to be able to include other responses, for example to short-answer or pmatch questions ( http://moodle.org/plugins/view.php?plugin=qtype_pmatch ). While the individual answers to different questions might be quite similar, the collection in total would not be. I am sure people sometimes submit documents for plagiarism checking that are a mixture of a few short-answer questions, and a few longer essays, and that it can give useful results.
        Hide
        Dan Marsden added a comment -

        My initial reaction didn't like this as an initial direction but after some further thought I think it makes sense. Some plagiarism plugins have the abilty to select to exclude certain plagiarised parts from affecting the end similarity score and although I think there's room for the plagiarism services to improve this it makes more sense on the Moodle side.

        Kanika can you please split this into 2 tasks - first write some code in one of the plagiarism plugins (Urkund is a good one to start with) based on the existing submitall and finish action (hard code it to only look at essay qs to start with)
        In future I think we will probably use get_form_elements_module to add a quiz specific setting that allows the user to select which question types to include when sending to the plagiarism service.

        Then write a Quiz report plugin which displays all users that have submitted a quiz and the status of the similarity report - use the standard plagiarism_get_links functions if possible so the code in the quiz report plugin will ideally be quite simple.

        thanks!

        Show
        Dan Marsden added a comment - My initial reaction didn't like this as an initial direction but after some further thought I think it makes sense. Some plagiarism plugins have the abilty to select to exclude certain plagiarised parts from affecting the end similarity score and although I think there's room for the plagiarism services to improve this it makes more sense on the Moodle side. Kanika can you please split this into 2 tasks - first write some code in one of the plagiarism plugins (Urkund is a good one to start with) based on the existing submitall and finish action (hard code it to only look at essay qs to start with) In future I think we will probably use get_form_elements_module to add a quiz specific setting that allows the user to select which question types to include when sending to the plagiarism service. Then write a Quiz report plugin which displays all users that have submitted a quiz and the status of the similarity report - use the standard plagiarism_get_links functions if possible so the code in the quiz report plugin will ideally be quite simple. thanks!
        Hide
        Dan Marsden added a comment -

        Kanika has run out of time to finish this before GSOC ends - but here's the patch she made that started the process - adding it in the comments here so someone else can pick up on this at some point.
        https://github.com/kanikagoyal/moodle/compare/master_MDL-32226_add_plagiarism_api_support_essay_questions_quiz

        Show
        Dan Marsden added a comment - Kanika has run out of time to finish this before GSOC ends - but here's the patch she made that started the process - adding it in the comments here so someone else can pick up on this at some point. https://github.com/kanikagoyal/moodle/compare/master_MDL-32226_add_plagiarism_api_support_essay_questions_quiz
        Hide
        Dan Marsden added a comment -
        Show
        Dan Marsden added a comment - (and here's the corresponding Urkund patch for my reference) https://github.com/kanikagoyal/moodle-plagiarism_urkund/commit/729ca25755f693d7f070441d0f50ba7791eaab4b
        Hide
        Tim Hunt added a comment -
        1. Why add a new event? The quiz was already firing quiz_attempt_submitted.
        2. This is a quiz event, so the name should start quiz_.
        3. All events fired should be documented in db/events.php inside the plugin

        Of course, because of 1., 2. and 3. are irrelevant.

        In the Urkund code, that is not a good way to try to get the responses. For example,

        • What if someone has made a custom qtype a bit like essay, but different? (I suggest looking for manual graded behaviour as a better heuristic.)
        • What if random questions are in use?
        • You are doing DB queries in a loop. You should do one query outside the loop.
          You should use the question engine API. Use $quba = question_engine::load_questions_usage_by_activity($qubaid), and then call methods on that object to get the data.
        Show
        Tim Hunt added a comment - Why add a new event? The quiz was already firing quiz_attempt_submitted. This is a quiz event, so the name should start quiz_. All events fired should be documented in db/events.php inside the plugin Of course, because of 1., 2. and 3. are irrelevant. In the Urkund code, that is not a good way to try to get the responses. For example, What if someone has made a custom qtype a bit like essay, but different? (I suggest looking for manual graded behaviour as a better heuristic.) What if random questions are in use? You are doing DB queries in a loop. You should do one query outside the loop. You should use the question engine API. Use $quba = question_engine::load_questions_usage_by_activity($qubaid), and then call methods on that object to get the data.
        Hide
        Dan Marsden added a comment -

        thanks Tim - I only added the links for future ref as we've run out of time during GSOC - but thanks for the review anyway! - it will help if someone else/me picks this up in future!

        Show
        Dan Marsden added a comment - thanks Tim - I only added the links for future ref as we've run out of time during GSOC - but thanks for the review anyway! - it will help if someone else/me picks this up in future!
        Hide
        tas ras added a comment -

        Hello.
        Is there an estimated time at which the essay questions will support plagiarism?

        Show
        tas ras added a comment - Hello. Is there an estimated time at which the essay questions will support plagiarism?
        Hide
        Dan Marsden added a comment -

        no-one is actively working on this at the moment - it was included as a possible task in last years GSOC project but we didn't manage to finish it.

        I don't plan to include this in a future GSOC project but if someone else writes a patch to include it I'm happy to review it - It will require Tim Hunts approval before landing in core quiz/question code so will need his review as well.

        Show
        Dan Marsden added a comment - no-one is actively working on this at the moment - it was included as a possible task in last years GSOC project but we didn't manage to finish it. I don't plan to include this in a future GSOC project but if someone else writes a patch to include it I'm happy to review it - It will require Tim Hunts approval before landing in core quiz/question code so will need his review as well.

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated: