Moodle Community Sites
  1. Moodle Community Sites
  2. MDLSITE-406

Modules and Plugins Database - an improved rating system

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Not a bug
    • Component/s: moodle.org
    • Labels:
      None

      Description

      Just a thought here > I realise we have a rating system for each module & plugin in the database on its respective page (but most ppl don't use it). Moreover there ISN'T a system that actually ranks or compares the credibility of the plugins. We all appreciate the time & effort developers invest to make these plugins available to our community, but lets face it - some are more stable & better maintained than others.

      As a point of comparison, Joomla! CMS gives users the opportunity to favour, rate, recommend, & review extensions. Users can also browse by most favoured, top rated, most popular & reviewed.

      Do we need to devise a more effective system that makes it easier for Moodle community members to make an informed decision about which plugins are(not) reliable?

      Ease of installation, functionality, stability, moodle-like appearance & overall rating are some likely assessment criteria. Administrators, developers & end-users should be able to evaluate the contributions of others. On the same token, this may motivate developers to contribute & further improve their work.

        Issue Links

          Activity

          Hide
          Martin Dougiamas added a comment -

          As ideas surface, can there please be some brief analysis with it of

          • how much is database template configuration and
          • how much is new code in the database module
          Show
          Martin Dougiamas added a comment - As ideas surface, can there please be some brief analysis with it of how much is database template configuration and how much is new code in the database module
          Hide
          Chad Outten added a comment -

          ok, here's my analysis...

          Database template configuration

          ratings: drop menu with values 0-100 exists
          comments: text box for comments exists & is ok

          suggested improvements:

          • add field(s) for reviewers to favour & recommend extensions; values: yes, no
          • add field for user type (drop menu); values: admin, developer, educator etc
          • use a more meaningful rating scale eg. 1-7 stars as per moodle.org forums. is mapping of existing data possible?
          • change 'comment' field title to 'review' & use subtext that will focus reviews eg. "having tested this extension, review it based on: ease of installation, functionality, stability, moodle-like appearance etc"
          • criteria such as ease of installation, functionality, stability, moodle-like appearance & overall rating may be more relevant than 'usefulness'.

          New code in database module

          • count of downloads on a per extension basis will require new code
          • calculations that display most favoured, recommended, reviewed extensions will require new code

          Cheers,
          Chad

          Show
          Chad Outten added a comment - ok, here's my analysis... Database template configuration ratings: drop menu with values 0-100 exists comments: text box for comments exists & is ok suggested improvements: add field(s) for reviewers to favour & recommend extensions; values: yes, no add field for user type (drop menu); values: admin, developer, educator etc use a more meaningful rating scale eg. 1-7 stars as per moodle.org forums. is mapping of existing data possible? change 'comment' field title to 'review' & use subtext that will focus reviews eg. "having tested this extension, review it based on: ease of installation, functionality, stability, moodle-like appearance etc" criteria such as ease of installation, functionality, stability, moodle-like appearance & overall rating may be more relevant than 'usefulness'. New code in database module count of downloads on a per extension basis will require new code calculations that display most favoured, recommended, reviewed extensions will require new code Cheers, Chad
          Hide
          Chad Outten added a comment -

          here's a quick mockup, 3 steps included in attached file...

          1. Developer adds extension to database
          2. Potential users read description, discussion, check out demo &/or downloads, test for self
          3. Users then (select magnifier icon to) review, rate, recommend, favour the extension

          Ideally, step 4 would calculate & display data collected from user reviews > so other potential users can see top rated, most reviewed, recommended, favoured extensions etc

          HTH

          Show
          Chad Outten added a comment - here's a quick mockup, 3 steps included in attached file... 1. Developer adds extension to database 2. Potential users read description, discussion, check out demo &/or downloads, test for self 3. Users then (select magnifier icon to) review, rate, recommend, favour the extension Ideally, step 4 would calculate & display data collected from user reviews > so other potential users can see top rated, most reviewed, recommended, favoured extensions etc HTH
          Hide
          Anthony Borrow added a comment -

          Martin - Here is my summary of what folks are seeking for the Modules and Plugins ratings. First, an ability to specify the role of the person doing the rating (i.e. student, teacher, admin, developer, Moodle Partner?). The characteristics as I summarize/distill them from the discussion at this point are:

          1) Quality of code: 1-10
          2) Active maintainer: Y/N
          3) Overall rating (0-10) [already implemented as 0-100]
          4) Popularity (based on a count of the number of ratings)
          5) Review/comments [already implemented as textbox]
          6) Documentation: 1-10
          7) Usability: 1-10

          One idea I had was to create a feedback for the Modules and Plugins and have the user select the one they are rating. While it would collect the data the feedback module is limited in its ability to filter responses during analysis so that you could see for example how developers rate the quality of code or users rate usability. If we were to do it with feedback, it would be nice if we could limit one response per user per module but feedback does not currently have that capability.

          Another idea I had was the possibility of creating another module type that would allow the rating of other activities (modules) with the ability to create the questions similar to feedback. Essentially I'm thinking along the lines of mdl_ratings with a structure of (id, userid, mod, modinstance), mdl_ratings_question (id, questiontype, question) mdl_ratings_questions (id, ratingsid, ratings_question_id, response). This is simply an idea of taking the application from how to rate this particular database activity and abstracting it any activity (assignment, quiz, etc.).

          I do not have any ideas about how practical that would be so please know that I'm simply brainstorming. My preference is actually toward the feedback module; however, if were to create a way where by any rating could be customized to include multiple rating scales that could be defined that would also work. The next challenge would be how to display those ratings as it would be ideal to have them graphically visible with the ability to filter according to particular fields (which inclines me back toward feedback). Hopefully some of this makes some sense and might be of use.

          Peace - Anthony

          Show
          Anthony Borrow added a comment - Martin - Here is my summary of what folks are seeking for the Modules and Plugins ratings. First, an ability to specify the role of the person doing the rating (i.e. student, teacher, admin, developer, Moodle Partner?). The characteristics as I summarize/distill them from the discussion at this point are: 1) Quality of code: 1-10 2) Active maintainer: Y/N 3) Overall rating (0-10) [already implemented as 0-100] 4) Popularity (based on a count of the number of ratings) 5) Review/comments [already implemented as textbox] 6) Documentation: 1-10 7) Usability: 1-10 One idea I had was to create a feedback for the Modules and Plugins and have the user select the one they are rating. While it would collect the data the feedback module is limited in its ability to filter responses during analysis so that you could see for example how developers rate the quality of code or users rate usability. If we were to do it with feedback, it would be nice if we could limit one response per user per module but feedback does not currently have that capability. Another idea I had was the possibility of creating another module type that would allow the rating of other activities (modules) with the ability to create the questions similar to feedback. Essentially I'm thinking along the lines of mdl_ratings with a structure of (id, userid, mod, modinstance), mdl_ratings_question (id, questiontype, question) mdl_ratings_questions (id, ratingsid, ratings_question_id, response). This is simply an idea of taking the application from how to rate this particular database activity and abstracting it any activity (assignment, quiz, etc.). I do not have any ideas about how practical that would be so please know that I'm simply brainstorming. My preference is actually toward the feedback module; however, if were to create a way where by any rating could be customized to include multiple rating scales that could be defined that would also work. The next challenge would be how to display those ratings as it would be ideal to have them graphically visible with the ability to filter according to particular fields (which inclines me back toward feedback). Hopefully some of this makes some sense and might be of use. Peace - Anthony
          Hide
          Helen Foster added a comment -

          Chad and Anthony, many thanks for your comments

          After giving the issue some thought, and talking to more people about it, rather than spending time customizing the existing modules and plugins database activity, we've come to the conclusion that it would be best to look for an alternative solution. Please see MDLSITE-571. Comments and suggestions are welcome!

          Show
          Helen Foster added a comment - Chad and Anthony, many thanks for your comments After giving the issue some thought, and talking to more people about it, rather than spending time customizing the existing modules and plugins database activity, we've come to the conclusion that it would be best to look for an alternative solution. Please see MDLSITE-571 . Comments and suggestions are welcome!
          Hide
          Anthony Borrow added a comment -

          Just a quick note that I checked with Sun about Netbeans which has a simple but nice looking list of plugins based on popularity and rating. I am beginning to wonder what a good rating algorithm would look like. I tend to think that simplicity is good but we could go anywhere from something like:

          http://www.slideshare.net/spajk/reallife-rating-algorithm

          to something simpler that factors in the number of ratings and then average of ratings. I'm still searching around for other projects that provide code review. I am thinking of asking in the community to see if we might find some volunteers that would be willing to specialize in very particular things in reviewing the code (i.e. sql injection, data validation, coding style, performance, etc.). If they provided some quality ratings in those areas then we could develop a quality algorithm based on code and some reasonably specific metrics.

          I would be interested in ideas from others about what the various rating algorithms could/should look like.

          Peace - Anthony

          Show
          Anthony Borrow added a comment - Just a quick note that I checked with Sun about Netbeans which has a simple but nice looking list of plugins based on popularity and rating. I am beginning to wonder what a good rating algorithm would look like. I tend to think that simplicity is good but we could go anywhere from something like: http://www.slideshare.net/spajk/reallife-rating-algorithm to something simpler that factors in the number of ratings and then average of ratings. I'm still searching around for other projects that provide code review. I am thinking of asking in the community to see if we might find some volunteers that would be willing to specialize in very particular things in reviewing the code (i.e. sql injection, data validation, coding style, performance, etc.). If they provided some quality ratings in those areas then we could develop a quality algorithm based on code and some reasonably specific metrics. I would be interested in ideas from others about what the various rating algorithms could/should look like. Peace - Anthony
          Hide
          gavin henrick added a comment -

          I think there is a range of issues to be assessed, i currently assess modules based on a few issues some of these are below and I am sure others will have even more they use:

          Downloading/Installing

          • Availability on trusted download site
          • Ease of install
          • Documentation for install
          • Type of addon (patching core files, or a block etc..)

          Project
          Will it be there in 1 year, is it available for all versions, or some

          • Project Maintainability (Is it one guy who made something once and isnt around anymore)
          • Project Version support (only latest moodle? or all moodles since 1.6 ?)
          • Project life (new project or old)
          • Project Backer (Paid for ongoing updates..)
          • Code Design (extra Dependencies/php versions/etc table indexing)
          • Popularity (recommended/lots of downloads)

          Impact on Existing Moodle

          • Security (XSS, etc..)
          • Performance (heavy queries, complex code... slow down pages?)
          • Resource usage (take a lot of disk space, or db or none ...)
          • backup/restore works?
          • unit tests?

          Actual Usage

          • Documentation for usage
          • Interface usability - teacher/admin
          • Interface usability - student
          • Is it what it says it is ?

          These are some of the things people need to think about before installing an addon, and maybe others?

          Show
          gavin henrick added a comment - I think there is a range of issues to be assessed, i currently assess modules based on a few issues some of these are below and I am sure others will have even more they use: Downloading/Installing Availability on trusted download site Ease of install Documentation for install Type of addon (patching core files, or a block etc..) Project Will it be there in 1 year, is it available for all versions, or some Project Maintainability (Is it one guy who made something once and isnt around anymore) Project Version support (only latest moodle? or all moodles since 1.6 ?) Project life (new project or old) Project Backer (Paid for ongoing updates..) Code Design (extra Dependencies/php versions/etc table indexing) Popularity (recommended/lots of downloads) Impact on Existing Moodle Security (XSS, etc..) Performance (heavy queries, complex code... slow down pages?) Resource usage (take a lot of disk space, or db or none ...) backup/restore works? unit tests? Actual Usage Documentation for usage Interface usability - teacher/admin Interface usability - student Is it what it says it is ? These are some of the things people need to think about before installing an addon, and maybe others?
          Hide
          Chad Outten added a comment -

          @Gavin: thanks for your comment - I agree with your ideas. You provide some key considerations relating to the evaluation of third party plugins. Do you have any thoughts in terms of how these consideration might best be represented on a web form, page - so users can provide evaluative feedback of benefit to other moodlers who are investigating the merit of third party plugins?

          Show
          Chad Outten added a comment - @Gavin: thanks for your comment - I agree with your ideas. You provide some key considerations relating to the evaluation of third party plugins. Do you have any thoughts in terms of how these consideration might best be represented on a web form, page - so users can provide evaluative feedback of benefit to other moodlers who are investigating the merit of third party plugins?
          Hide
          Daniele Cordella added a comment -

          my 2 cents of quick contribution (sorry, my spare time is really nothing).
          Some automatic indicators of plugin quality shoud be:
          -> the fame of the author/s (age of his/her first moodle login, number of posts, number of bug fixed, number of posts rated as useful, if he/she is a developer, (some ideas could be fetched from ohloh)...)
          -> number of download per time unit of the plugin
          -> date of last update
          -> number of updates per week/months/semester
          -> rate from users using the plugin

          Ciao.

          Show
          Daniele Cordella added a comment - my 2 cents of quick contribution (sorry, my spare time is really nothing). Some automatic indicators of plugin quality shoud be: -> the fame of the author/s (age of his/her first moodle login, number of posts, number of bug fixed, number of posts rated as useful, if he/she is a developer, (some ideas could be fetched from ohloh)...) -> number of download per time unit of the plugin -> date of last update -> number of updates per week/months/semester -> rate from users using the plugin Ciao.
          Hide
          Davo Smith added a comment -

          Related to this, I have at least one plugin that is rated '0/100' on the back of only 1 person rating it (with no feedback given as to exactly why they gave it such a rating).

          I'd be disappointed if such a rating immediately buried the module, never to be seen again (particularly as it's a plugin that I find very useful, as do quite a lot of other people who have emailed me or posted on forums about it).

          Show
          Davo Smith added a comment - Related to this, I have at least one plugin that is rated '0/100' on the back of only 1 person rating it (with no feedback given as to exactly why they gave it such a rating). I'd be disappointed if such a rating immediately buried the module, never to be seen again (particularly as it's a plugin that I find very useful, as do quite a lot of other people who have emailed me or posted on forums about it).
          Hide
          Helen Foster added a comment -

          Assigning to Super Sam

          Show
          Helen Foster added a comment - Assigning to Super Sam
          Hide
          Helen Foster added a comment -

          Closing as the Moodle plugins directory has now replaced the modules and plugins database.

          Show
          Helen Foster added a comment - Closing as the Moodle plugins directory has now replaced the modules and plugins database.

            People

            • Votes:
              5 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development