Moodle
  1. Moodle
  2. MDL-27115

Add Item response theory (IRT) analysis to the available quiz reports

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.9.8, 2.2
    • Fix Version/s: DEV backlog
    • Component/s: Quiz
    • Labels:
    • Database:
      MySQL
    • Difficulty:
      Easy
    • Affected Branches:
      MOODLE_19_STABLE, MOODLE_22_STABLE
    • Rank:
      16758
    1. quiz_irtanalysis.php
      38 kB
      Alexandr Bykov
    2. report.php
      39 kB
      Alexandr Bykov
    3. report.php
      38 kB
      Alexandr Bykov
    4. reportlib.php
      10 kB
      Alexandr Bykov

      Issue Links

        Activity

        Hide
        Alexandr Bykov added a comment -

        Hello. I works with Item Response Theory in analysis of quizes and I want to include this possibility in Moodle.

        Show
        Alexandr Bykov added a comment - Hello. I works with Item Response Theory in analysis of quizes and I want to include this possibility in Moodle.
        Hide
        Alexandr Bykov added a comment -

        File report.php includes possibility of calculation of complication of every task in quiz and student preparedness with application IRT-algorithms.

        Show
        Alexandr Bykov added a comment - File report.php includes possibility of calculation of complication of every task in quiz and student preparedness with application IRT-algorithms.
        Hide
        Tim Hunt added a comment -

        This sounds potentially very interesting. You might like to post about your proposal in the quiz forum: http://moodle.org/mod/forum/view.php?id=737.

        Are you looking at the code in Moodle 1.9 or 2.0? In Moodle 1.9 there was the old Item analysis report. In Moodle 2.0 that was replaced by a new Statistics report. That is still doing old fashioned statistics, not IRT, but the code is probably a bit better.

        Quiz reports are a type of plug-in. Therefore, initially you can develop your code in a separate folder mod/quiz/report/irt (or whatever you want to call it). And you can share your work through Moodle's plugin database: http://docs.moodle.org/en/Development:Guidelines_for_contributed_code

        Show
        Tim Hunt added a comment - This sounds potentially very interesting. You might like to post about your proposal in the quiz forum: http://moodle.org/mod/forum/view.php?id=737 . Are you looking at the code in Moodle 1.9 or 2.0? In Moodle 1.9 there was the old Item analysis report. In Moodle 2.0 that was replaced by a new Statistics report. That is still doing old fashioned statistics, not IRT, but the code is probably a bit better. Quiz reports are a type of plug-in. Therefore, initially you can develop your code in a separate folder mod/quiz/report/irt (or whatever you want to call it). And you can share your work through Moodle's plugin database: http://docs.moodle.org/en/Development:Guidelines_for_contributed_code
        Hide
        Alexandr Bykov added a comment -

        Thank you Tim. I visit links from your message necessarily. So I'm beginner in Moodle and PHP. I written programs on C and Java before, but I'm studying in pedagogical university and I think that Moodle may be very interesting for me and my future job.

        Show
        Alexandr Bykov added a comment - Thank you Tim. I visit links from your message necessarily. So I'm beginner in Moodle and PHP. I written programs on C and Java before, but I'm studying in pedagogical university and I think that Moodle may be very interesting for me and my future job.
        Hide
        Alexandr Bykov added a comment -

        So I deals Moodle 1.9.5, but I not founded it in the list of versions on this site.

        Show
        Alexandr Bykov added a comment - So I deals Moodle 1.9.5, but I not founded it in the list of versions on this site.
        Hide
        Tim Hunt added a comment -

        Moodle 1.9.5 is about 2 years old. You should at least upgrade to 1.9.11, which has a lot of bug and security fixes. However, if you are doing new development like on IRT, it might be more sensible to base your work on Moodle 2.0. However, since your work is mostly about statistical analysis of scores, it probably does not make a big difference which version you use.

        Show
        Tim Hunt added a comment - Moodle 1.9.5 is about 2 years old. You should at least upgrade to 1.9.11, which has a lot of bug and security fixes. However, if you are doing new development like on IRT, it might be more sensible to base your work on Moodle 2.0. However, since your work is mostly about statistical analysis of scores, it probably does not make a big difference which version you use.
        Hide
        Alexandr Bykov added a comment -

        Thank you very much for advices! If I have time, then I will reinstall Moodle today or tomorrow. I'm so happy that you help me with it.

        Show
        Alexandr Bykov added a comment - Thank you very much for advices! If I have time, then I will reinstall Moodle today or tomorrow. I'm so happy that you help me with it.
        Hide
        Tim Hunt added a comment -

        Sounds good. Please can I encourage you to share what you have done with the community. See http://docs.moodle.org/en/Development:Guidelines_for_contributed_code

        Show
        Tim Hunt added a comment - Sounds good. Please can I encourage you to share what you have done with the community. See http://docs.moodle.org/en/Development:Guidelines_for_contributed_code
        Hide
        Alexandr Bykov added a comment -

        reportlib.php ---> /mod/quiz/report/
        report.php ---> /mod/quiz/report/irtanalysis/
        quiz_irtanalysis.php ---> /lang/en_utf8/

        Show
        Alexandr Bykov added a comment - reportlib.php ---> /mod/quiz/report/ report.php ---> /mod/quiz/report/irtanalysis/ quiz_irtanalysis.php ---> /lang/en_utf8/
        Hide
        Alexandr Bykov added a comment -

        I have not much time and I works slowly with idea of IRT in Moodle.

        Show
        Alexandr Bykov added a comment - I have not much time and I works slowly with idea of IRT in Moodle.
        Hide
        Tim Hunt added a comment -

        Note that the language file can go in /mod/quiz/report/irtanalysis/lange/en_utf8, which makes your plugin more self-contained.

        Also, it is better not to change a core Moodle file like /mod/quiz/report/reportlib.php. I have not looked to see what changes you made, but if you need to define extra functions just for your report, it is better to put them in a file like /mod/quiz/report/irtanalysis/locallib.php

        Show
        Tim Hunt added a comment - Note that the language file can go in /mod/quiz/report/irtanalysis/lange/en_utf8, which makes your plugin more self-contained. Also, it is better not to change a core Moodle file like /mod/quiz/report/reportlib.php. I have not looked to see what changes you made, but if you need to define extra functions just for your report, it is better to put them in a file like /mod/quiz/report/irtanalysis/locallib.php
        Hide
        Alexandr Bykov added a comment -

        Thank you very much. I thank about self-contained in this plug-in, today I'll have change location of the files.
        In reportlib.php I have added function "quiz_format_logit_hardness_for_questions".

        Show
        Alexandr Bykov added a comment - Thank you very much. I thank about self-contained in this plug-in, today I'll have change location of the files. In reportlib.php I have added function "quiz_format_logit_hardness_for_questions".
        Hide
        Tim Hunt added a comment -

        I do not think this is a priority in the short time. I think the best thing is to make this functionality available as a quiz report plugin in the plugins database.

        If lots of people install it from there and like it, then we can consider adding it to core Moodle.

        I am leaving this issue open, however, so people have a place to vote for this.

        Show
        Tim Hunt added a comment - I do not think this is a priority in the short time. I think the best thing is to make this functionality available as a quiz report plugin in the plugins database. If lots of people install it from there and like it, then we can consider adding it to core Moodle. I am leaving this issue open, however, so people have a place to vote for this.
        Hide
        Jean-Michel Vedrine added a comment -

        Hello Alexandr,
        I am very interested in your idea of an IRT quiz report.
        I must admit that beside reading a lot of books on IRT I never really used it because I missed a friendly tool (all that I have tried are really unfriendly) to do calculations on my students results.
        As I already made a third party quiz report (responses report) that latter became a core report (see in Moodle 2.0/2.1/2.2) maybe I can help you on the Moodle part of the report code. I see that you started from the overview report, but if you do so you need to delete some parts of the overview report code that aren't really necessary in the irtanalysis report.
        Also I wonder if starting from the analysis (renamed to statistics in Moodle later versions) wouldn't be a better idea as I view IRTanalysis as being question centric rather than attempt centric like overview and responses reports (but maybe I am wrong in thinking that).
        Also one thing that prevented me from starting to code an IRT report was that after reading books I was thinking that to be interesting calculations needed to be done on all attempts in all quizess on a question rather than just the attempts in a given quiz, so I was thinking it was more aq uestion bank report rather than a quiz report; Am I right or wrong ?
        Maybe we can divide the work, you explain what calculations must be done (a mokup of the interface of the report would be ideal) and I try to modify the code accordingly ?
        One other thing I need to say is that I am very interested in a Moodle 2.1/2.2 version as Moodle 1.9/2.0 are at the end of their life and only receive security fixes now. I personnaly use Moodle 2.2.

        Show
        Jean-Michel Vedrine added a comment - Hello Alexandr, I am very interested in your idea of an IRT quiz report. I must admit that beside reading a lot of books on IRT I never really used it because I missed a friendly tool (all that I have tried are really unfriendly) to do calculations on my students results. As I already made a third party quiz report (responses report) that latter became a core report (see in Moodle 2.0/2.1/2.2) maybe I can help you on the Moodle part of the report code. I see that you started from the overview report, but if you do so you need to delete some parts of the overview report code that aren't really necessary in the irtanalysis report. Also I wonder if starting from the analysis (renamed to statistics in Moodle later versions) wouldn't be a better idea as I view IRTanalysis as being question centric rather than attempt centric like overview and responses reports (but maybe I am wrong in thinking that). Also one thing that prevented me from starting to code an IRT report was that after reading books I was thinking that to be interesting calculations needed to be done on all attempts in all quizess on a question rather than just the attempts in a given quiz, so I was thinking it was more aq uestion bank report rather than a quiz report; Am I right or wrong ? Maybe we can divide the work, you explain what calculations must be done (a mokup of the interface of the report would be ideal) and I try to modify the code accordingly ? One other thing I need to say is that I am very interested in a Moodle 2.1/2.2 version as Moodle 1.9/2.0 are at the end of their life and only receive security fixes now. I personnaly use Moodle 2.2.
        Hide
        Alexandr Bykov added a comment -

        Hi! I'm here again after hospital and long-long time-out. I'm sorry for it. I wanna say that plug-in was finished last year. But I don't have time for testing and uploading files. Now I'm installing a latest Moodle.
        Do you know, has anybody created plug-in for irt-analysis yet?

        Show
        Alexandr Bykov added a comment - Hi! I'm here again after hospital and long-long time-out. I'm sorry for it. I wanna say that plug-in was finished last year. But I don't have time for testing and uploading files. Now I'm installing a latest Moodle. Do you know, has anybody created plug-in for irt-analysis yet?
        Hide
        Alexandr Bykov added a comment -

        Hello Jean-Michel Vedrine!
        Good idea! Your offer still valid?

        Show
        Alexandr Bykov added a comment - Hello Jean-Michel Vedrine! Good idea! Your offer still valid?
        Hide
        Jean-Michel Vedrine added a comment -

        Hello Alexandr,
        Sorry to hear you had health problems but happy you are feeling better now.
        To my best knowledge nobody has done an IRT plugin for Moodle.
        My offer is still valid and I am on summer holidays in a few weeks from now, so hopefully I will have more time for Moodle programming.
        If you want to have a look at how the quiz activity has evolved in recent Moodle versions, I suggest you install latest Moodle 2.2.x or even better you wait for Moodle 2.3 that should be out in a few days.
        Or if you can't use anything other than Moodle 1.9 I will developp a 2.2/2.3 version at the same time the 1.9 version of the plugin is done.

        Show
        Jean-Michel Vedrine added a comment - Hello Alexandr, Sorry to hear you had health problems but happy you are feeling better now. To my best knowledge nobody has done an IRT plugin for Moodle. My offer is still valid and I am on summer holidays in a few weeks from now, so hopefully I will have more time for Moodle programming. If you want to have a look at how the quiz activity has evolved in recent Moodle versions, I suggest you install latest Moodle 2.2.x or even better you wait for Moodle 2.3 that should be out in a few days. Or if you can't use anything other than Moodle 1.9 I will developp a 2.2/2.3 version at the same time the 1.9 version of the plugin is done.
        Hide
        Tim Hunt added a comment -

        I am pretty sure that no-one else is working on IRT for Moodle.

        Show
        Tim Hunt added a comment - I am pretty sure that no-one else is working on IRT for Moodle.
        Hide
        Tim Hunt added a comment -

        This issue was assigned to me automatically, however I will not be able to work on this issue in the immediate future. In order to create a truer sense of the state of this issue and to allow other developers to have chance to become involved, I am removing myself as the assignee of this issue.

        For more information, see http://docs.moodle.org/dev/Changes_to_issue_assignment

        Show
        Tim Hunt added a comment - This issue was assigned to me automatically, however I will not be able to work on this issue in the immediate future. In order to create a truer sense of the state of this issue and to allow other developers to have chance to become involved, I am removing myself as the assignee of this issue. For more information, see http://docs.moodle.org/dev/Changes_to_issue_assignment

          People

          • Votes:
            3 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated: