Moodle
  1. Moodle
  2. MDL-10383

Make modules to declare and use Groupings

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.9
    • Fix Version/s: 1.9
    • Component/s: Groups
    • Labels:
      None
    • Rank:
      34584

      Description

      Groupings were introduced in version 1.8 as a way to organize groups within a course. However, groupings do not have any practical use in version 1.8 or current 1.9 (HEAD). As in 1.8/1.9 Groupings are more a useless complication than a useful feature. To be productive Groupings must be USED by activity modules. I have proposed a whole new range of groups/groupings features combining the design goals (but not yet implemented) of OU and the implemented version of Groupings working at ULPGC. The spec proposal may be found at http://docs.moodle.org/en/Development:Groupings_and_Groups

      At this stage I will concentrate in two new features:

      a) Making possible to use different groupings by different activities.
      Activities can declare a grouping to use. This means that when that particular activity is set as visible/separate groups, the particular groups used are those belonging to the declared grouping. Hence, we can have simultaneous activities where users are distributed according to different criteria, activities & resources may be targeted to different audiences.

      The needed changes are adding a field "groupingid" to the courses and course_modules tables to hold the grouping that the course itself or each module instance will use when set in visible/separate groupmode. By "using grouping" I mean that grouping-aware functions should operate nor on all groups associated with a course but only those associated with a grouping. So, you may have a forum declaring a grouping and thus using a particular set of groups. At the same time other activity in the course, let say an assignment may declare other grouping and thus treat the same users as distributed in a completely different set of groups.

      "Use grouping" must be added to the configuration form for each module instance, parallel to "groupmode" attribute. Best way is to add to te common module settings

      Module code must be adapted to ensure that $cm->groupingid property now existing is passed to groups-API functions within the module code to return/manage only those groups belonging to the declared grouping.

      b) Group-based content delivery
      Once each activity, and resources, may declare a grouping to use, it is possible to make that those users that are not members of any group of that grouping cannot access to the module content (being it an interactive activity or a Resource module instance)
      A simple checking for group membership at the beggining of the view.php file for each module would do the trick
      Group-membership may be checked at course page (function print_section() ) to make links to forbidden modules invisible to users without suitable access.

      At ULPGC we have a test site with these features ENABLED, that is, already coded. You may find it at http://moodle.ulpgc.es/moodle19ulpgc/course/view.php?id=3
      Try login as user "prof01", "prof02" or "estudiante01", "estudiante02" ... with password "moodle" for all

      The file attached is a unified diff against HEAD that can be used to explore proposed code changes.

      1. diff-ulpgc_groupings-08-07-07
        69 kB
        Enrique Castro
      2. groups_cleanup2.patch
        114 kB
        Petr Škoda
      3. groups.dia
        2 kB
        Petr Škoda
      4. groups-db-libchanges.diff
        55 kB
        Matt Clarkson
      5. patch-Bugz-2617-group_cm.diff
        44 kB
        Nick Freear
      6. patch-MDL-10383-groups-db-v6.diff
        57 kB
        Nick Freear
      7. patch-MDL-10383-groups-lib-clean-p1.diff
        30 kB
        Nick Freear

        Issue Links

          Activity

          Hide
          Petr Škoda added a comment -

          Hi! I do not think we should commit any groupings changes before fixing the db structure first. It is IMO needed because we must solve backup/restore (including restore on other site), access control and global groups.

          Could we discuss the problems via Skype please? (my id is petr.skoda)

          Show
          Petr Škoda added a comment - Hi! I do not think we should commit any groupings changes before fixing the db structure first. It is IMO needed because we must solve backup/restore (including restore on other site), access control and global groups. Could we discuss the problems via Skype please? (my id is petr.skoda)
          Hide
          Nick Freear added a comment -

          Hi Enrique, Petr and all!

          Thank you for putting work in to a specification Enrique. I have also started a spec, which I am putting on the Wiki here,
          http://docs.moodle.org/en/Development:Groupings_OU

          Please don't think that I am trying to compete with you - I think it's useful for us all to lay out our "stalls" (to use an English expression), and I hope we can draw together your's, Petr's and my ideas to develop a comprehensive set of requirements.

          I see adding grouping support in modules as essential, but also as a later step following a process of re-working the group code, and modifying the database schema. We can also work towards site-wide groups, but note that the work Juliette White did originally showed that many users (teachers etc.) don't need this, but they do need better support for groups in courses and activities.

          I've had a look at the patch Enrique - a lot of useful code. I noticed that your approach to the user-interface to set the grouping for a course module was different to ours - we have a class 'MoodleQuickForm_modgrouping', similar to the class 'MoodleQuickForm_modgroupmode', and an extra parameter to the function 'standard_coursemodule_elements(... $supportsgroupings=false)'. See,
          http://docs.moodle.org/en/Development:Groupings_OU#Groupings_for_course_modules

          I look forward to exchanging views on Skype and/or the forums! Yours

          Nick

          Show
          Nick Freear added a comment - Hi Enrique, Petr and all! Thank you for putting work in to a specification Enrique. I have also started a spec, which I am putting on the Wiki here, http://docs.moodle.org/en/Development:Groupings_OU Please don't think that I am trying to compete with you - I think it's useful for us all to lay out our "stalls" (to use an English expression), and I hope we can draw together your's, Petr's and my ideas to develop a comprehensive set of requirements. I see adding grouping support in modules as essential, but also as a later step following a process of re-working the group code, and modifying the database schema. We can also work towards site-wide groups, but note that the work Juliette White did originally showed that many users (teachers etc.) don't need this, but they do need better support for groups in courses and activities. I've had a look at the patch Enrique - a lot of useful code. I noticed that your approach to the user-interface to set the grouping for a course module was different to ours - we have a class 'MoodleQuickForm_modgrouping', similar to the class 'MoodleQuickForm_modgroupmode', and an extra parameter to the function 'standard_coursemodule_elements(... $supportsgroupings=false)'. See, http://docs.moodle.org/en/Development:Groupings_OU#Groupings_for_course_modules I look forward to exchanging views on Skype and/or the forums! Yours Nick
          Hide
          Enrique Castro added a comment - - edited

          Hi Nick,
          Nice to read you. I think we are basically in agreement.

          Your spec and mine cover different aspects of Groups/groupings, one form the point of view of the user teacher and other from de code programmer. I think we would need to put together and consolidate the three views (with Petr) in a single document, or at least coherent parts of a whole. That's what Martin D and Moodle community expects, I think.

          Agreement points:
          a) Grouping support at modules is essential and first task to accomplish. I think Moodle 1.9 cannot lack this feature
          b) Global groups are quite problematic, mainly because backup/restore: Delay adoption
          c) 6-table database structure is OK: most flexible, keeps all possibilities open for future additions. I do not see a power argument to change it.

          Disagreement:
          I strongly oppose making mandatory that any group must belong to a grouping. This implies to create a "default grouping" to hold groups, makes arise the problem of "not in a grouping"/orphaned groups. This is also a problem when restoring old courses: they do not have a notion of "grouping". But this Groupings-first is not built in into DB or API, is a UI-imposed restriction.

          UI criticism in 1.8/1.9
          I think the main point to take into account is that the proposed UI insisted in creating Grouping first and then adding groups to a grouping. In pre-1.8 Moodle GROUPS are the main actors and I think that they should remain so: allow to create and manage groups independently of groupings. ONLY if needed, the teacher may use groupings as a way to organize groups and to separate sets of groups for use in different activities.

          So I defend a groups-UI separate in tabs. One tab to manage groups and adding users to groups (the "classic") and another, separate, tab to manage groupings and populate groupings with groups. In this way the old-style logic and behavior is preserved, and new features may be introduced as new tabs as required.

          API Code polishing
          I basically agree with the changes indicated (MV,D). I do not see the rationale to maintain a separate lib/modulelib.php. The only "extra" point at module level is that groupingid is fixed. All functions in modulelib.php perform the same task as functions on basicgroups or grouping libs. For instance:

          groups_m_get_group == groups_get_group_settings() or groups_groupids_to_groups()
          groups_m_get_groups_for_user() == groups_get_groups_for_user_in_grouping() provided groupingid is known and passed

          My point is that, in Python words, "better explicit than implicit". All m functions perform tasks that are covered by basic functions, provided groupingid parameter is passed. Why to complicate future maintaining tasks duplicating functions?

          Summary: We all agree groupings at modules is THE feature to add in 1.9. I think the main point now is in groups UI.
          Should we work to re-make the classic three pane UI with formslib? I think so

          Show
          Enrique Castro added a comment - - edited Hi Nick, Nice to read you. I think we are basically in agreement. Your spec and mine cover different aspects of Groups/groupings, one form the point of view of the user teacher and other from de code programmer. I think we would need to put together and consolidate the three views (with Petr) in a single document, or at least coherent parts of a whole. That's what Martin D and Moodle community expects, I think. Agreement points: a) Grouping support at modules is essential and first task to accomplish. I think Moodle 1.9 cannot lack this feature b) Global groups are quite problematic, mainly because backup/restore: Delay adoption c) 6-table database structure is OK: most flexible, keeps all possibilities open for future additions. I do not see a power argument to change it. Disagreement: I strongly oppose making mandatory that any group must belong to a grouping. This implies to create a "default grouping" to hold groups, makes arise the problem of "not in a grouping"/orphaned groups. This is also a problem when restoring old courses: they do not have a notion of "grouping". But this Groupings-first is not built in into DB or API, is a UI-imposed restriction. UI criticism in 1.8/1.9 I think the main point to take into account is that the proposed UI insisted in creating Grouping first and then adding groups to a grouping. In pre-1.8 Moodle GROUPS are the main actors and I think that they should remain so: allow to create and manage groups independently of groupings. ONLY if needed, the teacher may use groupings as a way to organize groups and to separate sets of groups for use in different activities. So I defend a groups-UI separate in tabs. One tab to manage groups and adding users to groups (the "classic") and another, separate, tab to manage groupings and populate groupings with groups. In this way the old-style logic and behavior is preserved, and new features may be introduced as new tabs as required. API Code polishing I basically agree with the changes indicated (MV,D). I do not see the rationale to maintain a separate lib/modulelib.php. The only "extra" point at module level is that groupingid is fixed. All functions in modulelib.php perform the same task as functions on basicgroups or grouping libs. For instance: groups_m_get_group == groups_get_group_settings() or groups_groupids_to_groups() groups_m_get_groups_for_user() == groups_get_groups_for_user_in_grouping() provided groupingid is known and passed My point is that, in Python words, "better explicit than implicit". All m functions perform tasks that are covered by basic functions, provided groupingid parameter is passed. Why to complicate future maintaining tasks duplicating functions? Summary: We all agree groupings at modules is THE feature to add in 1.9. I think the main point now is in groups UI. Should we work to re-make the classic three pane UI with formslib? I think so
          Hide
          Petr Škoda added a comment -

          Hi!

          my priorities seem to be different from both of you. The global groupings is the most difficult problem we are facing, that is why we should solve it first. I am not saying we should implement it first, but we should IMO find some solution first and then talk about the rest.

          Originally everything was contained in courses - nothing was shared between them. It was nice and easy and everything worked fine. In 1.7 was a major breakthrough - global roles. As expected anything shared between course was difficult to implement, there are still some problems left and the resulting code is much slower. Course enrolments are one of those problems, I would really like to use global groupings to work around some problems there too.

          Groups shared in groupings might seem fine when thinking about course groups only, but it will IMO cause trouble when we introduce global groups. If we do not share groups we need only minimal access control in the groupings - capabilities in system context for global groupings and in course context for course groupings. If we share groups we need to deal with it at the group level which would be much more complicated.

          Shared groups in groupings - do we really need it? What are the use cases? How often do you need to reuse one group in several groupings inside one course?

          • Sharing groups in groupings from one course? Not really sure we need it - more complicated UI, more complicated code.
          • Sharing of groups between groupings from different courses? No. Use global groupings instead.

          I agree the UI in 1.8.0 is not good, I do not think we need to discuss it much. I always wanted separate grouping and group forms because not everybody needs groupings. The orphaned groups were really bad idea, internally there should IMO always exist default course grouping which would be hidden in UI if no other groupings used in course.

          I am working on my proposal too, adding some more info and explanation...

          Thanks everybody for working on this issue!!!

          Show
          Petr Škoda added a comment - Hi! my priorities seem to be different from both of you. The global groupings is the most difficult problem we are facing, that is why we should solve it first. I am not saying we should implement it first, but we should IMO find some solution first and then talk about the rest. Originally everything was contained in courses - nothing was shared between them. It was nice and easy and everything worked fine. In 1.7 was a major breakthrough - global roles. As expected anything shared between course was difficult to implement, there are still some problems left and the resulting code is much slower. Course enrolments are one of those problems, I would really like to use global groupings to work around some problems there too. Groups shared in groupings might seem fine when thinking about course groups only, but it will IMO cause trouble when we introduce global groups. If we do not share groups we need only minimal access control in the groupings - capabilities in system context for global groupings and in course context for course groupings. If we share groups we need to deal with it at the group level which would be much more complicated. Shared groups in groupings - do we really need it? What are the use cases? How often do you need to reuse one group in several groupings inside one course? Sharing groups in groupings from one course? Not really sure we need it - more complicated UI, more complicated code. Sharing of groups between groupings from different courses? No. Use global groupings instead. I agree the UI in 1.8.0 is not good, I do not think we need to discuss it much. I always wanted separate grouping and group forms because not everybody needs groupings. The orphaned groups were really bad idea, internally there should IMO always exist default course grouping which would be hidden in UI if no other groupings used in course. I am working on my proposal too, adding some more info and explanation... Thanks everybody for working on this issue!!!
          Hide
          Enrique Castro added a comment -

          Hi Petr,
          My main problem is that I do not see at least some of the the problems you anticipate with global groups.

          I agree that backup/restore is a problem, but if global groups are flagged and given a unique ID in DB this problem may be overcome, Am I right?
          You mention the sharing of groups and access capabilities. Well, my view on that is quiet simple: Global groups must be locked (read-only items) at course level. The only thing a user may do with a global group at course level is to select it for use. Cannot add, edit, delete users change it in any way. The only access check available and meaningful at course level is - "is user X member of group G?" Nothing more.
          My understanding of roles& permissions system is minimal so, probably, I am too blunt to see the complication,. Petr, please, could you explain what do you see as showstopper in the global groups-access capabilities interaction?

          You demand use cases for sharing a group in groupings (I understand have a group belonging to several groupings).

          Example A (from N Hansen http://moodle.org/mod/forum/discuss.php?d=52134)
          She has a course with three units and three levels of payment
          Pay1 give acces to unit1
          Pay2 give acces to both unit 1 and 2
          Pay3 give access to al units form 1 to 3.
          Let say she has three groups Pay1-3 according to the amount payed (a convenient way to classify users).
          Unit 1 activities would use GroupingA=(Pay1, Pay2, Pay3) ie: any student can see and participate in these activities
          Unit 2 activities would use GroupingB=(Pay2, Pay3)
          Unit3 activities would use GroupingC=(Pay3) only users in Pay3 group may see the content

          I agree that all cases of group sharing between groupings can me managed duplicating the group or adding those users to another group in the other grouping. This flexibility, sharing within the course, is a matter on convenience.

          My criticism is the other way around. Not about having a group in several groupings, what I see essential is to have groups without groupings. Your scheme imposes that every group must belong to a grouping (and afterwards, to only one). But pre-1.8 groups did not belong to any grouping. Yes, your scheme demands a hidden "default grouping", special and different from the rest of groupings. I see this as a complication. Once groupings and groups are separated having a group in 0, 1 or n groupings are natural cases.

          I wholeheartedly agree that sharing course-based groups is an aberration. Read-only global groups are for that.

          Show
          Enrique Castro added a comment - Hi Petr, My main problem is that I do not see at least some of the the problems you anticipate with global groups. I agree that backup/restore is a problem, but if global groups are flagged and given a unique ID in DB this problem may be overcome, Am I right? You mention the sharing of groups and access capabilities. Well, my view on that is quiet simple: Global groups must be locked (read-only items) at course level. The only thing a user may do with a global group at course level is to select it for use. Cannot add, edit, delete users change it in any way. The only access check available and meaningful at course level is - "is user X member of group G?" Nothing more. My understanding of roles& permissions system is minimal so, probably, I am too blunt to see the complication,. Petr, please, could you explain what do you see as showstopper in the global groups-access capabilities interaction? You demand use cases for sharing a group in groupings (I understand have a group belonging to several groupings). Example A (from N Hansen http://moodle.org/mod/forum/discuss.php?d=52134 ) She has a course with three units and three levels of payment Pay1 give acces to unit1 Pay2 give acces to both unit 1 and 2 Pay3 give access to al units form 1 to 3. Let say she has three groups Pay1-3 according to the amount payed (a convenient way to classify users). Unit 1 activities would use GroupingA=(Pay1, Pay2, Pay3) ie: any student can see and participate in these activities Unit 2 activities would use GroupingB=(Pay2, Pay3) Unit3 activities would use GroupingC=(Pay3) only users in Pay3 group may see the content I agree that all cases of group sharing between groupings can me managed duplicating the group or adding those users to another group in the other grouping. This flexibility, sharing within the course, is a matter on convenience. My criticism is the other way around. Not about having a group in several groupings, what I see essential is to have groups without groupings. Your scheme imposes that every group must belong to a grouping (and afterwards, to only one). But pre-1.8 groups did not belong to any grouping. Yes, your scheme demands a hidden "default grouping", special and different from the rest of groupings. I see this as a complication. Once groupings and groups are separated having a group in 0, 1 or n groupings are natural cases. I wholeheartedly agree that sharing course-based groups is an aberration. Read-only global groups are for that.
          Hide
          Petr Škoda added a comment -

          Hi Enrique,

          hmm, that example with different levels of payments does not seem right to me, I would expect this to be handled with roles and new type of enrolment plugins (multiple instances of paypal plugin each with different enrolment role and money). This is a special case and maybe using different courses + one shared would be possible - again improved enrolment plugins doing enrolments into several courses with one payment would be useful in this case. Are there any other known uses for shared groups in groupings?

          Groups without a grouping were bad from the programmers view - the problem is the speed of SQL queries, you have to combine results of two different queries. Also to get better performance we have to integrate groups more with the roles framework - you often need to know if user is in some group and has capability to do something, or get all users with capability belonging to some group. In fact the hidden grouping is a major simplification from the coding point of view and speed improvement too.

          Seems we are getting to some conclusions, we do not agree on:

          1/ sharing of groups in course groupings

          • minuses: more complex code + needs UI
          • pluses: please describe, I could not find any yet - one good use case might change my mind

          2/ orphan groups

          • minuses: more complex code
          • pluses: please describe - me, Martin and OU think there are none

          -------

          The relation of roles and groups is another problematic question. Some ppl proposed to assign roles in groups - it is not reasonable IMO. There should not be any group contexts for roles.

          Your proposal to use groups to restrict access to activities seems to be OK for separate groups. However we can not use it for visible groups mode because it would break backwards compatibility (Am I right?). We might add a new "restricted" mode that would allow access into that activity if member of at least one group in selected grouping or maybe a new attribute in course_module to restrict access based on group membership, specifying group mode for resources sounds strange to me. The implementation should not be hard IMO, users should understand easily too. My +1 for that.

          --------------

          I did not speak much about the implementation yet, I would prefer to have one library file lib/grouplib.php with public functions used from modules + private ones used from UI, the lib/accesslib.php would contain some groups related code too. The ui for global and course groupings + groups should be in /group/ directory. I do not think there should be so many functions that are calling each other, sooner or later we will have to optimize db access with SQL queries instead of PHP loops, we might do that during implementation - why not? If groups are useful ppl will be using them more - I would not like to see performance problems, roles and enrolments are causing enough trouble already. Using global groupings for course enrolment purposes might help. Special global group enrolment plugins might be another improvement. (I will have to write some use cases, that should explain it easily.)

          The session code for active grouping+group will have to be rewritten too, the current code has problems when having two browser windows each with different activity+grouping. Nothing serious, but there are some other hidden legacy problems and strange coding style in older code. We should IMHO trow out all older code and some new too and do a clean+faster implementation with new API that is based on needs found during module fixing - not as before having API first and then starting module conversions . I have already removed some unused/unfinished files and functions during my last groupings cleanup.
          I do not think we need to keep the legacy API anymore, there were so many changes in 1.7/1.8/1.9 that no 3rd party code works anymore without modifications anyway - though there should be detailed docs describing the conversions and how to use new API.

          The upgrade code should use sql if possible only, no calls to group functions. It is very tricky, we have to watch for changed functions used during upgrade which might be using tables before they are upgraded. We already have these headaches with roles code

          Show
          Petr Škoda added a comment - Hi Enrique, hmm, that example with different levels of payments does not seem right to me, I would expect this to be handled with roles and new type of enrolment plugins (multiple instances of paypal plugin each with different enrolment role and money). This is a special case and maybe using different courses + one shared would be possible - again improved enrolment plugins doing enrolments into several courses with one payment would be useful in this case. Are there any other known uses for shared groups in groupings? Groups without a grouping were bad from the programmers view - the problem is the speed of SQL queries, you have to combine results of two different queries. Also to get better performance we have to integrate groups more with the roles framework - you often need to know if user is in some group and has capability to do something, or get all users with capability belonging to some group. In fact the hidden grouping is a major simplification from the coding point of view and speed improvement too. Seems we are getting to some conclusions, we do not agree on: 1/ sharing of groups in course groupings minuses: more complex code + needs UI pluses: please describe, I could not find any yet - one good use case might change my mind 2/ orphan groups minuses: more complex code pluses: please describe - me, Martin and OU think there are none ------- The relation of roles and groups is another problematic question. Some ppl proposed to assign roles in groups - it is not reasonable IMO. There should not be any group contexts for roles. Your proposal to use groups to restrict access to activities seems to be OK for separate groups. However we can not use it for visible groups mode because it would break backwards compatibility (Am I right?). We might add a new "restricted" mode that would allow access into that activity if member of at least one group in selected grouping or maybe a new attribute in course_module to restrict access based on group membership, specifying group mode for resources sounds strange to me. The implementation should not be hard IMO, users should understand easily too. My +1 for that. -------------- I did not speak much about the implementation yet, I would prefer to have one library file lib/grouplib.php with public functions used from modules + private ones used from UI, the lib/accesslib.php would contain some groups related code too. The ui for global and course groupings + groups should be in /group/ directory. I do not think there should be so many functions that are calling each other, sooner or later we will have to optimize db access with SQL queries instead of PHP loops, we might do that during implementation - why not? If groups are useful ppl will be using them more - I would not like to see performance problems, roles and enrolments are causing enough trouble already. Using global groupings for course enrolment purposes might help. Special global group enrolment plugins might be another improvement. (I will have to write some use cases, that should explain it easily.) The session code for active grouping+group will have to be rewritten too, the current code has problems when having two browser windows each with different activity+grouping. Nothing serious, but there are some other hidden legacy problems and strange coding style in older code. We should IMHO trow out all older code and some new too and do a clean+faster implementation with new API that is based on needs found during module fixing - not as before having API first and then starting module conversions . I have already removed some unused/unfinished files and functions during my last groupings cleanup. I do not think we need to keep the legacy API anymore, there were so many changes in 1.7/1.8/1.9 that no 3rd party code works anymore without modifications anyway - though there should be detailed docs describing the conversions and how to use new API. The upgrade code should use sql if possible only, no calls to group functions. It is very tricky, we have to watch for changed functions used during upgrade which might be using tables before they are upgraded. We already have these headaches with roles code
          Hide
          Enrique Castro added a comment -

          OK, lets add some use cases. These come from teacher demands I have experienced at ULPG moodle site. Those demands triggered our development of groupings for 1.6.

          Example B
          Grouping Project A: ProjectA01, ProjectA02, ... ProjectA25 (25 groups of 3 people)
          Each 3-member team work in a different topic These groups are used in a Wiki and a forum.
          Grouping Writing: Writing01, Writing02, Writing03 .. Writing12 (12 groups of 5 people)
          Each 5-member team work in a different topic These groups are used in a Wiki and a forum.

          The teacher detects that Project03, Project17, Writing9 and Writing12 are working on related topics and have specific needs. He decided to add some resources for them and only them to see.
          Set up a Resource page as separate groups using Grouping Special =(Project03, Project17, Writing9, Writing12)

          I know, this may be done with other tools:
          a) the additional material could be added to the Forum, but that means the teacher needs to replicate 4 times the same message (six, eight times?) Each time he wants to update information, the replication issue arises again
          b) Creating a new Special group and putting it into Special Grouping. Ok ¿But what about people changing group? Teacher need to keep track that those on Project03, Project17, Writing9 and Writing12 were added to Special and review Special each time the other groups are changed. A task on teacher shoulders that supposedly CMS tool was making easier, automatic, not more complex.

          Example C
          Several teachers share a course. Each teaches a "Theory" group,
          Grouping Sections= Theory01, Theory02, Theory03 each with one teacher in it
          Forum News uses grouping Sections as separate groups: allows announcements for all students and for each sections independently.

          Grouping Labs= Lab01, Lab02, ... Lab06. Each teacher is responsible for 2 Lab groups.
          Wiki Lab results uses grouping Labs for students to write lab reports

          Students are distributed at Lab and Theory groups at will: there is no rule like Theory01 students go to Lab01+Lab02 groups.

          Each one of three teachers wants a "Doubts" forum accessible only for the students they teach, and
          In addition, they want to add content (resources) private for their students
          Solution: Each teacher have a Grouping. For instance, TeacherA grouping=(Theory1+Lab01+Lab03). In this way each teacher can add activities or resources that only their set of students can access.

          Alternative: creating additional groups and adding users manually, again. I anticipate that if we go this route then in a near future we will discussing ways to "clone" groups and synchronize membership from one to other: Teachers demanding that members of GroupA become members of groupB automagically. Sharing groups makes this unnecessary.

          Example D: overcoming a known problem in forum
          When you have a forum working in "separate groups" mode the teacher can post a message to "all participants" but those messages are "read-only": students cannot reply to that post due to different groupid.
          Course with two sections
          Grouping Sections= (English101, English102). Each with a teacher
          Grouping Project=(Project01, Project02, Project03 ... Project12, + English101, English102)
          Now students are provided with a forum for private discussions about the projects. The teachers can use English101/102 to start a discussion related to these projects that concerns to all their students.

          This may be done alternatively with a separate forum for those "project but common" discussions. But most teachers (and students) want to keep topics organized. Not having a "general forum" for any type of discussions. Questions about projects in the projects forum, questions about theory in the class forum.

          Example F. Using global groups
          Global grouping "Foreign Students"=(Resident, Erasmus, Non-UE)
          Global Grouping "Disabilities"=(Disabled) (Disabled people are entitled by law to certain benefits: specially composed texts, exams ... )
          Now go to course level. A course with two sections.
          Grouping Sections= (Chemistry101, Chemistry102)
          The teachers have a News forum in separate groups mode for general announcements, exam warnings etc. Usually Erasmus students are in an "special regime" and are a separate target for many messages and instructions.
          If group sharing across groupings were allowed it would be possible to just add global group Erasmus to course grouping Sections and have News forums to manage (Section01, Section02, Erasmus) as separate groups.
          If not, teachers have to add a whole new forum "News for foreigners" using "Foreign Students" grouping. And students have to watch for news at two different forums.

          You have some resources or activities specially developed for students with disabilities. Let say, special reading texts. They are configure ti use Grouping=Disabilities. You have some immigrant students with a low level of communication language that could also benefit from accessing those resources. You may duplicate those, now using a local Grouping Special. If you could create a local Grouping Special=(global Disabled + local special) they all would access to the same resources or activities.

          I think this is a significant limitation, with your scheme global Groupings are shared, rather than individual global Groups. And you cannot create a grouping at course level using course-local groups and global groups together. I do not see many use cases to utilize strictly global groupings at course activities. Such rigid global groupings/groups are useful mainly for enrolement purposes, not actual use in activities.

          -----------------------------------------
          So, let see the disagreements

          >1/ sharing of groups in course groupings
          > * minuses: more complex code + needs UI
          Needs a different UI, yes, actually a one much more like the classic one, Please see images at http://docs.moodle.org/en/Development:Groupings_and_Groups and working demo at http://moodle.ulpgc.es/moodle19ulpgc/course/view.php?id=3 (login as "prof01" password "moodle" or "estudiante01" password "moodle")

          Code complexity? Only that you cannot deduce uniquely the groupingid from groupid. However, that's not usually the task but the other way around. Groupingid must come from $course or $cm objects

          > * pluses: please describe, the above use case

          I do make a point on group sharing at course and site level. I would see the potential uses of groups severely handicapped if not allowed to share groups.

          2/ orphan groups
          > * minuses: more complex code
          I have perform the hack to get groupings twice at ULPGC site. Fisrt with 1.5 I did use a "default grouping" to hold course groups if no grouping was created explicitly. When we upgraded to 1.6 I refactored the system and I got a "simper" framework without that "default grouping". Of perhaps, is simply that the second time I was more familiar with groups functions.

          > * pluses: please describe - me, Martin and OU think there are none
          Mainly teacher confort. Groups and groupings are demanded and a required feature, but not all teachers need groupings. If groups are used only for separate course sections then old-style moodle is enough. Presenting a UI where first groupings are to be created and then groups added within is a complication there.

          On the other hand, restoring old backup copies from 1.6, 1.5 versions into >1.8 versions would be more natural if "orphan groups" (groups with a courseid but no groupingid associated) are preserved: better backward compatibility. If not , restore functions must ensure that "default grouping" is created and associated with orphan groups coming from the backup.

          I do not see this point of orphan groups as crucial. This is a minor implementation detail: choosing if modifying a piece of the code or the other bu the important features for the teacher will work either.

          -----------------------------------------------

          With respect to the other concerns,

          SPEED
          I am by no mean a DBA.
          I have been inspecting the code for the most used complex functions in groups API, those that allow you to retrieve groups for a user and compare how the task woul be done with your 4-table fixed-grouping scheme or the six-table loose OU scheme. In all cases the complexity of the queries is similar. In OU scheme there are more tables, but not all are used simultaneously.

          groups_get_groups($courseid): 2-table join vs 1-table query if only groupids are need, a 2-table
          groups_get_groups_in_grouping($groupingid): 1-table query vs 1 if only groupids or a 2-table join for group objects
          groups_get_groups_for_user($userid, $courseid): both schemes use a 3-table join
          groups_get_groups_for_user_in_grouping($userid, $groupingid): 2-table join vs a 2-table join for groupids or a 3-table join for group objects
          groups_is_member_of_some_group_in_grouping($userid, $groupingid): both 2-table joins
          groups_get_groups_not_in_grouping($groupingid, $courseid): both 2-table joins

          So, it seems that some functions may be slower for the 6-table UO scheme, but other are actually simpler. I do not think that to avoid the extra complexity of groups_get_groups_for_user_in_grouping($userid, $groupingid) using a 3-table join we should refuse to use the extra flexibility of the scheme. How much slower would be this function with one and the other scheme? Is it worth that extra speed the lack of group sharing between groupings.

          -------------------
          Restricting access to resources and activities
          Yes, the implementation makes the restriction for "separate groups" mode only: if groups are separated and you are not in any group you are off-limits. For instance in a separate groups forum ungrouped students cannot participate in the discussions. I think that adding a new attribute "restricted" distinct from "groupmode" would break that intuitive understanding.

          As for Resources, I see it as a natural extension. Think of a "group mode" Resource as a "group wiki": you have in fact several wikis, one for each group. I am not thinking in adding such multipage capability to resources in any time in a near or distant future.
          For the moment: you setup the resource as "separate groups" and those "not grouped" cannot access. That's the only "use of groups" of the resource module (Resources are modules, after all). But in future other "uses of groups" may arise.

          -------------------
          About implementation
          I agree with the general view, simplification into one library file, with a reduction of overall API functions. Indeed may are not used nor needed for modules to use groups powerfully . I would use, as in other parts of Moodle, default parameters to combine some of them.

          Throwing away the legacy functions is drastic but feasible: those functions are not called that much. I can mention that I hacked modules in 1.5 and 1.6 for groupings in less than a week once I had the groups functions clear in library.

          Show
          Enrique Castro added a comment - OK, lets add some use cases. These come from teacher demands I have experienced at ULPG moodle site. Those demands triggered our development of groupings for 1.6. Example B Grouping Project A: ProjectA01, ProjectA02, ... ProjectA25 (25 groups of 3 people) Each 3-member team work in a different topic These groups are used in a Wiki and a forum. Grouping Writing: Writing01, Writing02, Writing03 .. Writing12 (12 groups of 5 people) Each 5-member team work in a different topic These groups are used in a Wiki and a forum. The teacher detects that Project03, Project17, Writing9 and Writing12 are working on related topics and have specific needs. He decided to add some resources for them and only them to see. Set up a Resource page as separate groups using Grouping Special =(Project03, Project17, Writing9, Writing12) I know, this may be done with other tools: a) the additional material could be added to the Forum, but that means the teacher needs to replicate 4 times the same message (six, eight times?) Each time he wants to update information, the replication issue arises again b) Creating a new Special group and putting it into Special Grouping. Ok ¿But what about people changing group? Teacher need to keep track that those on Project03, Project17, Writing9 and Writing12 were added to Special and review Special each time the other groups are changed. A task on teacher shoulders that supposedly CMS tool was making easier, automatic, not more complex. Example C Several teachers share a course. Each teaches a "Theory" group, Grouping Sections= Theory01, Theory02, Theory03 each with one teacher in it Forum News uses grouping Sections as separate groups: allows announcements for all students and for each sections independently. Grouping Labs= Lab01, Lab02, ... Lab06. Each teacher is responsible for 2 Lab groups. Wiki Lab results uses grouping Labs for students to write lab reports Students are distributed at Lab and Theory groups at will: there is no rule like Theory01 students go to Lab01+Lab02 groups. Each one of three teachers wants a "Doubts" forum accessible only for the students they teach, and In addition, they want to add content (resources) private for their students Solution: Each teacher have a Grouping. For instance, TeacherA grouping=(Theory1+Lab01+Lab03). In this way each teacher can add activities or resources that only their set of students can access. Alternative: creating additional groups and adding users manually, again. I anticipate that if we go this route then in a near future we will discussing ways to "clone" groups and synchronize membership from one to other: Teachers demanding that members of GroupA become members of groupB automagically. Sharing groups makes this unnecessary. Example D: overcoming a known problem in forum When you have a forum working in "separate groups" mode the teacher can post a message to "all participants" but those messages are "read-only": students cannot reply to that post due to different groupid. Course with two sections Grouping Sections= (English101, English102). Each with a teacher Grouping Project=(Project01, Project02, Project03 ... Project12, + English101, English102) Now students are provided with a forum for private discussions about the projects. The teachers can use English101/102 to start a discussion related to these projects that concerns to all their students. This may be done alternatively with a separate forum for those "project but common" discussions. But most teachers (and students) want to keep topics organized. Not having a "general forum" for any type of discussions. Questions about projects in the projects forum, questions about theory in the class forum. Example F. Using global groups Global grouping "Foreign Students"=(Resident, Erasmus, Non-UE) Global Grouping "Disabilities"=(Disabled) (Disabled people are entitled by law to certain benefits: specially composed texts, exams ... ) Now go to course level. A course with two sections. Grouping Sections= (Chemistry101, Chemistry102) The teachers have a News forum in separate groups mode for general announcements, exam warnings etc. Usually Erasmus students are in an "special regime" and are a separate target for many messages and instructions. If group sharing across groupings were allowed it would be possible to just add global group Erasmus to course grouping Sections and have News forums to manage (Section01, Section02, Erasmus) as separate groups. If not, teachers have to add a whole new forum "News for foreigners" using "Foreign Students" grouping. And students have to watch for news at two different forums. You have some resources or activities specially developed for students with disabilities. Let say, special reading texts. They are configure ti use Grouping=Disabilities. You have some immigrant students with a low level of communication language that could also benefit from accessing those resources. You may duplicate those, now using a local Grouping Special. If you could create a local Grouping Special=(global Disabled + local special) they all would access to the same resources or activities. I think this is a significant limitation, with your scheme global Groupings are shared, rather than individual global Groups. And you cannot create a grouping at course level using course-local groups and global groups together. I do not see many use cases to utilize strictly global groupings at course activities. Such rigid global groupings/groups are useful mainly for enrolement purposes, not actual use in activities. ----------------------------------------- So, let see the disagreements >1/ sharing of groups in course groupings > * minuses: more complex code + needs UI Needs a different UI, yes, actually a one much more like the classic one, Please see images at http://docs.moodle.org/en/Development:Groupings_and_Groups and working demo at http://moodle.ulpgc.es/moodle19ulpgc/course/view.php?id=3 (login as "prof01" password "moodle" or "estudiante01" password "moodle") Code complexity? Only that you cannot deduce uniquely the groupingid from groupid. However, that's not usually the task but the other way around. Groupingid must come from $course or $cm objects > * pluses: please describe, the above use case I do make a point on group sharing at course and site level. I would see the potential uses of groups severely handicapped if not allowed to share groups. 2/ orphan groups > * minuses: more complex code I have perform the hack to get groupings twice at ULPGC site. Fisrt with 1.5 I did use a "default grouping" to hold course groups if no grouping was created explicitly. When we upgraded to 1.6 I refactored the system and I got a "simper" framework without that "default grouping". Of perhaps, is simply that the second time I was more familiar with groups functions. > * pluses: please describe - me, Martin and OU think there are none Mainly teacher confort. Groups and groupings are demanded and a required feature, but not all teachers need groupings. If groups are used only for separate course sections then old-style moodle is enough. Presenting a UI where first groupings are to be created and then groups added within is a complication there. On the other hand, restoring old backup copies from 1.6, 1.5 versions into >1.8 versions would be more natural if "orphan groups" (groups with a courseid but no groupingid associated) are preserved: better backward compatibility. If not , restore functions must ensure that "default grouping" is created and associated with orphan groups coming from the backup. I do not see this point of orphan groups as crucial. This is a minor implementation detail: choosing if modifying a piece of the code or the other bu the important features for the teacher will work either. ----------------------------------------------- With respect to the other concerns, SPEED I am by no mean a DBA. I have been inspecting the code for the most used complex functions in groups API, those that allow you to retrieve groups for a user and compare how the task woul be done with your 4-table fixed-grouping scheme or the six-table loose OU scheme. In all cases the complexity of the queries is similar. In OU scheme there are more tables, but not all are used simultaneously. groups_get_groups($courseid): 2-table join vs 1-table query if only groupids are need, a 2-table groups_get_groups_in_grouping($groupingid): 1-table query vs 1 if only groupids or a 2-table join for group objects groups_get_groups_for_user($userid, $courseid): both schemes use a 3-table join groups_get_groups_for_user_in_grouping($userid, $groupingid): 2-table join vs a 2-table join for groupids or a 3-table join for group objects groups_is_member_of_some_group_in_grouping($userid, $groupingid): both 2-table joins groups_get_groups_not_in_grouping($groupingid, $courseid): both 2-table joins So, it seems that some functions may be slower for the 6-table UO scheme, but other are actually simpler. I do not think that to avoid the extra complexity of groups_get_groups_for_user_in_grouping($userid, $groupingid) using a 3-table join we should refuse to use the extra flexibility of the scheme. How much slower would be this function with one and the other scheme? Is it worth that extra speed the lack of group sharing between groupings. ------------------- Restricting access to resources and activities Yes, the implementation makes the restriction for "separate groups" mode only: if groups are separated and you are not in any group you are off-limits. For instance in a separate groups forum ungrouped students cannot participate in the discussions. I think that adding a new attribute "restricted" distinct from "groupmode" would break that intuitive understanding. As for Resources, I see it as a natural extension. Think of a "group mode" Resource as a "group wiki": you have in fact several wikis, one for each group. I am not thinking in adding such multipage capability to resources in any time in a near or distant future. For the moment: you setup the resource as "separate groups" and those "not grouped" cannot access. That's the only "use of groups" of the resource module (Resources are modules, after all). But in future other "uses of groups" may arise. ------------------- About implementation I agree with the general view, simplification into one library file, with a reduction of overall API functions. Indeed may are not used nor needed for modules to use groups powerfully . I would use, as in other parts of Moodle, default parameters to combine some of them. Throwing away the legacy functions is drastic but feasible: those functions are not called that much. I can mention that I hacked modules in 1.5 and 1.6 for groupings in less than a week once I had the groups functions clear in library.
          Hide
          Petr Škoda added a comment -

          Thanks for the valuable input!

          I would like to sum it a bit for Martin:

          • shared groups in global groupings - NO (or maybe yes if sharing allowed on the course level, can be changed later)
          • shared groups between groupings from different courses - NO (backup nightmare)
          • shared groups in groupings within one course - MAYBE, MartinD decides (db structure depends on this)
          • orphan groups - internally NO, automatic creation of first course grouping which is set as default and hidden in UI if only one grouping in course

          Timeframe:

          • local groupings fully working in 1.9
          • global groupings in 2.0 (patch for 1.9.x?)
          • group enrolment in 2.0

          People:

          • coding: Enrique?, Nick?
          • code review: Petr, Sam?, Nick?
          • groups code maintainer in 1.9: Enrique? (== group bugs assigned in the tracker)
          • documentation: ?

          Nicolas might help with the UI, if I remember it correctly he was working on the original one in 1.8.0.

          Show
          Petr Škoda added a comment - Thanks for the valuable input! I would like to sum it a bit for Martin: shared groups in global groupings - NO (or maybe yes if sharing allowed on the course level, can be changed later) shared groups between groupings from different courses - NO (backup nightmare) shared groups in groupings within one course - MAYBE, MartinD decides (db structure depends on this) orphan groups - internally NO, automatic creation of first course grouping which is set as default and hidden in UI if only one grouping in course Timeframe: local groupings fully working in 1.9 global groupings in 2.0 (patch for 1.9.x?) group enrolment in 2.0 People: coding: Enrique?, Nick? code review: Petr, Sam?, Nick? groups code maintainer in 1.9: Enrique? (== group bugs assigned in the tracker) documentation: ? Nicolas might help with the UI, if I remember it correctly he was working on the original one in 1.8.0.
          Hide
          Nick Freear added a comment - - edited

          Petr, Enrique, Sam and I have had further discussions on Skype. Thank you all for the input. Some points,

          • I am only able work on the groups re-factor for a limited time, so after discussion with Petr we have found a way to divide the work into tasks that can be assigned to different people.
          • Task 1, implement new DB schema, upgrade and backup/restore, test, with minimal changes to the libraries to support this. So, libraries will remain unfactored here. Est time, 3.5-4 weeks, assigned to Nick, review by Petr, sam? I'm on holiday 28 July - 12 August (honeymoon!), so estimated delivery 24 August: unfortunately it won't make 1.9 Beta unless that is delayed.
          • Task 2, library re-factor, assigned to Enrique or other (To Be Confirmed), review by Petr, Nick, sam?
          • Task 3, grouping support in modules - initially one module, eg. Database, Wiki, Forum. Can be before or after task 2; Enrique or other (TBC), with assistance from Nick, review by Petr, Nick...
          • Task 4, 5... user-interface, documentation, global groupings, group enrollment ...
          • As indicated by Petr in his comment above, do we want shared groups in groupings within one course? Petr and OU think not, Enrique thinks yes.

          Martin, please can you consider this as soon as possible. I'd like to get started on the work

          Thank you. Yours

          Nick

          Show
          Nick Freear added a comment - - edited Petr, Enrique, Sam and I have had further discussions on Skype. Thank you all for the input. Some points, I am only able work on the groups re-factor for a limited time, so after discussion with Petr we have found a way to divide the work into tasks that can be assigned to different people. Task 1, implement new DB schema, upgrade and backup/restore, test, with minimal changes to the libraries to support this. So, libraries will remain unfactored here. Est time, 3.5-4 weeks, assigned to Nick, review by Petr, sam? I'm on holiday 28 July - 12 August (honeymoon!), so estimated delivery 24 August: unfortunately it won't make 1.9 Beta unless that is delayed. Task 2, library re-factor, assigned to Enrique or other (To Be Confirmed), review by Petr, Nick, sam? Task 3, grouping support in modules - initially one module, eg. Database, Wiki, Forum. Can be before or after task 2; Enrique or other (TBC), with assistance from Nick, review by Petr, Nick... Task 4, 5... user-interface, documentation, global groupings, group enrollment ... We are agreed on using Petr's DB schema, except for 1 outstanding issue, http://docs.moodle.org/en/Image:groupsdb.png http://docs.moodle.org/en/Development:Groupings_OU#Petr.27s_proposal As indicated by Petr in his comment above, do we want shared groups in groupings within one course? Petr and OU think not, Enrique thinks yes. Martin, please can you consider this as soon as possible. I'd like to get started on the work Thank you. Yours Nick
          Hide
          Enrique Castro added a comment -

          Hi, nice chat this morning.
          The key point to decide is to keep "group sharing between groupings" or not.. At the implementation level that means to decide if groups table should have a field holding a unique groupingid for each group. That is: each group must belong to one and only one grouping in the whole site.

          I wholeheartly advice no: teachers do want to use a group of students at more than one activity (but not in the same grouping). Groupings are sets of groups, as groups are sets of users. At first Moodle had single-group membership, each users could be in a group and only in a group, but use demanded to extend that. It's the same thing with groups and groupings. Once you have several groups in a course you can put together those groups differently for some activities.

          GroupA is (John, Peter and Mary). You, as teacher, do not want to have groupA in Grouping01 used in Activity 01, and groupA'=(John, Peter and Mary) again, in grouping02 for activity02 that engages those and groups B and C. And even groupA''=(John, Peter and Mary) again, in grouping02 for activity02 that engages those and groups G, H, R. The teacher wants one group name identifiying that people, and then use that group in different activities =different groupings.

          We have been discussing the alternative approach of "cloning" groups from one grouping to other. Making a GroupA in grouping 01 and a GroupB in grouping02 that happens to have the same users at members. But this solution must imply code to keep group membership synchronized: if added a user to GroupA, GroupB automatically changes. If not doing so we are creating a mess to work with by the teachers. The teacher would need to remember, magically, that GroupB is supposed to be a copy of GroupA and manually edit B when changing A.

          Groups for activities are not rigid and invariable. They may be created as course flows, and members changing as needed (even self-assigned). A different beast from groups viewed as class cohorts or enrolment-assemblage that are not usually teacher controlled and invariable during course length.

          The problem of not sharing groups is even worse, from the teacher point of view, when considering global groups. It is unlikely that a teacher needs to use a globally-set criterion to distribute users into activity-based groups, at least in large universities. I do not see how grouping students by course, by year in the studies, can be used at course level in course activities (may be used for enrolment, that's another story)
          In the use case above, the teacher can take advantage on including a global group in a local grouping. That

          The new BD schema proposed by Nick Freear http://docs.moodle.org/en/Image:groupsdb.png WITH the grouping_groups table and without the groupingid filed in groups table would meet the requirement to allow a group to be part of several groupings. However, I do not see that scheme much more simple, or facilitating faster queries that current scheme.
          --------------------

          Both "simplified" DB schemas make global groupings rather than groups (assuming implicitly that groups in a global grouping are global groups) However, at site level what are really needed groupings or groups? Are global groups always organized into sets that can be put separated into different groupings? Do all courses in a site always use global groups in exactly the same groupings(if use one group in teh grouping always using the other together, too). Above I have presented use cases not in this direction.

          Sharing global groups would make mandatory to label those groups as unique in the DB for proper management, mostly during backup/restore. So uniqueid, synchroleid, manualsych shoul exist also at group table, not only groupings table.

          --------------------
          Summarizing

          • shared groups between groupings from different courses - NO (backup nightmare)
          • orphan groups - internally NO, automatic creation of first course grouping which is set as default and hidden in UI if only one grouping in course
          • shared groups in groupings within one course - MAYBE, MartinD decides (db structure depends on this).
          • shared groups in global groupings - maybe yes if sharing allowed on the course level

          ------------

          Dedication: I have six weeks clear, four with full-day dedication. Afterwards in part-time
          I can take responsibility for maintaining code, chasing bugs etc. of the tasks assigned.

          We all agree that groupings at module level are a must for 1.9. Adding group-based restriction access is trivial once module-level groupings are working.

          Show
          Enrique Castro added a comment - Hi, nice chat this morning. The key point to decide is to keep "group sharing between groupings" or not.. At the implementation level that means to decide if groups table should have a field holding a unique groupingid for each group. That is: each group must belong to one and only one grouping in the whole site. I wholeheartly advice no: teachers do want to use a group of students at more than one activity (but not in the same grouping). Groupings are sets of groups, as groups are sets of users. At first Moodle had single-group membership, each users could be in a group and only in a group, but use demanded to extend that. It's the same thing with groups and groupings. Once you have several groups in a course you can put together those groups differently for some activities. GroupA is (John, Peter and Mary). You, as teacher, do not want to have groupA in Grouping01 used in Activity 01, and groupA'=(John, Peter and Mary) again, in grouping02 for activity02 that engages those and groups B and C. And even groupA''=(John, Peter and Mary) again, in grouping02 for activity02 that engages those and groups G, H, R. The teacher wants one group name identifiying that people, and then use that group in different activities =different groupings. We have been discussing the alternative approach of "cloning" groups from one grouping to other. Making a GroupA in grouping 01 and a GroupB in grouping02 that happens to have the same users at members. But this solution must imply code to keep group membership synchronized: if added a user to GroupA, GroupB automatically changes. If not doing so we are creating a mess to work with by the teachers. The teacher would need to remember, magically, that GroupB is supposed to be a copy of GroupA and manually edit B when changing A. Groups for activities are not rigid and invariable. They may be created as course flows, and members changing as needed (even self-assigned). A different beast from groups viewed as class cohorts or enrolment-assemblage that are not usually teacher controlled and invariable during course length. The problem of not sharing groups is even worse, from the teacher point of view, when considering global groups. It is unlikely that a teacher needs to use a globally-set criterion to distribute users into activity-based groups, at least in large universities. I do not see how grouping students by course, by year in the studies, can be used at course level in course activities (may be used for enrolment, that's another story) In the use case above, the teacher can take advantage on including a global group in a local grouping. That The new BD schema proposed by Nick Freear http://docs.moodle.org/en/Image:groupsdb.png WITH the grouping_groups table and without the groupingid filed in groups table would meet the requirement to allow a group to be part of several groupings. However, I do not see that scheme much more simple, or facilitating faster queries that current scheme. -------------------- Both "simplified" DB schemas make global groupings rather than groups (assuming implicitly that groups in a global grouping are global groups) However, at site level what are really needed groupings or groups? Are global groups always organized into sets that can be put separated into different groupings? Do all courses in a site always use global groups in exactly the same groupings(if use one group in teh grouping always using the other together, too). Above I have presented use cases not in this direction. Sharing global groups would make mandatory to label those groups as unique in the DB for proper management, mostly during backup/restore. So uniqueid, synchroleid, manualsych shoul exist also at group table, not only groupings table. -------------------- Summarizing shared groups between groupings from different courses - NO (backup nightmare) orphan groups - internally NO, automatic creation of first course grouping which is set as default and hidden in UI if only one grouping in course shared groups in groupings within one course - MAYBE, MartinD decides (db structure depends on this). shared groups in global groupings - maybe yes if sharing allowed on the course level ------------ Dedication: I have six weeks clear, four with full-day dedication. Afterwards in part-time I can take responsibility for maintaining code, chasing bugs etc. of the tasks assigned. We all agree that groupings at module level are a must for 1.9. Adding group-based restriction access is trivial once module-level groupings are working.
          Hide
          Petr Škoda added a comment -

          The sharing of groups in different groupings is a potential problem - if we allow it at the database level ppl will sooner or later modify it to share groups between groupings from different courses. This is a real problem that would be very, very hard to solve and we would be forced to deal with it in future.

          I object that cloning (== making a copy without any link to original group/grouping) would confuse teachers, they can not use groupings now. In majority of cases they would either use the same grouping or a new one - either created by hand or cloning or generated by some group distribution tool or student group selfassignment. I do not agree that shared groupings are easier to understand than cloned groupings and that the UI is easier to make/understand for shared groups. Shared groups require more complex UI IMO.

          Anyway we could implement both shared and cloning of groups at the same time. The clone UI is just one field in new group/grouping form - select group/grouping to clone.

          People that really want synchronised groups from different groupings might alse create automatic group synchroniser instead of db sharing - it would not have the same backup/restore or permissions problems because there would still be two separate groups each in its own grouping. This could be done with just one field in group table - groups with the same random id in this field would be synchronised when adding/removing users into these groups. There would not be needed any core changes, just added synchroniser into group UI. You could also link groups from different courses without any internal problems if needed.

          Show
          Petr Škoda added a comment - The sharing of groups in different groupings is a potential problem - if we allow it at the database level ppl will sooner or later modify it to share groups between groupings from different courses. This is a real problem that would be very, very hard to solve and we would be forced to deal with it in future. I object that cloning (== making a copy without any link to original group/grouping) would confuse teachers, they can not use groupings now. In majority of cases they would either use the same grouping or a new one - either created by hand or cloning or generated by some group distribution tool or student group selfassignment. I do not agree that shared groupings are easier to understand than cloned groupings and that the UI is easier to make/understand for shared groups. Shared groups require more complex UI IMO. Anyway we could implement both shared and cloning of groups at the same time. The clone UI is just one field in new group/grouping form - select group/grouping to clone. People that really want synchronised groups from different groupings might alse create automatic group synchroniser instead of db sharing - it would not have the same backup/restore or permissions problems because there would still be two separate groups each in its own grouping. This could be done with just one field in group table - groups with the same random id in this field would be synchronised when adding/removing users into these groups. There would not be needed any core changes, just added synchroniser into group UI. You could also link groups from different courses without any internal problems if needed.
          Hide
          Enrique Castro added a comment - - edited

          Hi Petr
          If a teacher need to use GroupA in grouping01 and grouping02 I see as more simple and natural
          a) Create GroupA
          b) Create Groupings Grouping01 and Grouping02
          c) Add GroupA to grouping01 and add groupA to grouping02

          than the other way
          a) Create grouping01 and GroupA within (with other groups)
          b) Create Grouping02 and GroupA (the same name, empty group)
          c) Make a link (new function) between GroupA (with students) and groupA (without students)

          Once GroupA in groupin02 is linked to groupA in grouping01 you no longer should be able to add/remove users from it. That would break the linking logic in the first place. So you have to make checkings for that case in code and UI, synchronizing groups has side-effects too, as sharing. The best synchronizing is using the very same group, not a "clone".

          You are concerned by 3rd parties, other people making their own modifications to Moodle. Well, either those modifications do work AND are valuable for an ample audience, the why not incorporate them?; Or they are too specialized and only useful locally: let that people live with their code.

          I concede that I have a biased view. I do see lot of uses to shared groups within courses because I am using them. I do not see any use for sharing groups across courses because I see course level as a logical barrier. Sharing groups betwen courses has no meaning if courses has different user enrolled (groups are not names, but the collection of people). Even if two courses have the same students I doubt I would use the same groupings or groups: if they are different courses they must have different topics: need different activities and different groups because not everybody is equally able at all topics. So, I minimize risk of sharing across course boundaries. Please, Petr, can you provide some use case for sharing groups between courses? (local groups, not global).

          A comparison: sharing/publishing quiz questions categories. At ULPGC teachers see that as a mess. If you have 1500 courses and some of then "publish" categories rapidly that list is a complete mess where it is very difficult to locate the category you want to use. Even more, if the question text links to course-based files (e. g. images) that are lost in the new destination course. Result, at ULPGC teachers are sharing questions by sharing exported files (XML).

          Petr, apart from the sharing problem, do you see any other showstopper for the 6-table scheme?
          I would propose to add groupings at module level to 1.9 HEAD as it is, this putting groupings to work at last. Then let people use and opine and make any radical changes for 2.0.

          Show
          Enrique Castro added a comment - - edited Hi Petr If a teacher need to use GroupA in grouping01 and grouping02 I see as more simple and natural a) Create GroupA b) Create Groupings Grouping01 and Grouping02 c) Add GroupA to grouping01 and add groupA to grouping02 than the other way a) Create grouping01 and GroupA within (with other groups) b) Create Grouping02 and GroupA (the same name, empty group) c) Make a link (new function) between GroupA (with students) and groupA (without students) Once GroupA in groupin02 is linked to groupA in grouping01 you no longer should be able to add/remove users from it. That would break the linking logic in the first place. So you have to make checkings for that case in code and UI, synchronizing groups has side-effects too, as sharing. The best synchronizing is using the very same group, not a "clone". You are concerned by 3rd parties, other people making their own modifications to Moodle. Well, either those modifications do work AND are valuable for an ample audience, the why not incorporate them?; Or they are too specialized and only useful locally: let that people live with their code. I concede that I have a biased view. I do see lot of uses to shared groups within courses because I am using them. I do not see any use for sharing groups across courses because I see course level as a logical barrier. Sharing groups betwen courses has no meaning if courses has different user enrolled (groups are not names, but the collection of people). Even if two courses have the same students I doubt I would use the same groupings or groups: if they are different courses they must have different topics: need different activities and different groups because not everybody is equally able at all topics. So, I minimize risk of sharing across course boundaries. Please, Petr, can you provide some use case for sharing groups between courses? (local groups, not global). A comparison: sharing/publishing quiz questions categories. At ULPGC teachers see that as a mess. If you have 1500 courses and some of then "publish" categories rapidly that list is a complete mess where it is very difficult to locate the category you want to use. Even more, if the question text links to course-based files (e. g. images) that are lost in the new destination course. Result, at ULPGC teachers are sharing questions by sharing exported files (XML). Petr, apart from the sharing problem, do you see any other showstopper for the 6-table scheme? I would propose to add groupings at module level to 1.9 HEAD as it is, this putting groupings to work at last. Then let people use and opine and make any radical changes for 2.0.
          Hide
          Petr Škoda added a comment - - edited

          Simple and natural? That seems to be a bit relative

          a) create a grouping1
          b) define groups groupA, groupB

          c) create new grouping2
          d) add new groupC - select "Clone groupA" in new group form
          e) modify groupC members (does not affect groupA)
          f) add new groupB - select "Link to groupB" in new group form

          g) create new grouping3 - select "Clone+link groups from grouping1"

          h) you can change/review the linking later in any grouping

          I can not see any problems you describe above - if you link two groupings, the result is sum of all members in both groups; if you add one user into grouping the internal function for adding group members could sync the rest of linked groups; if you remove user from linked group, removing function syncs all linked groups. Only several places deal with group membership - group UI, user import, restore, enrolment (any more?)- they should use library functions for any adding/removing of members anyway - ok, it might be better to move it into core
          As I said before, linking has IMO fewer side effects than sharing because you still have groups attached to groupings - we could deal with the linking easily in backup/restore code IMHO.

          The UI could hide the difference between sharing and linking IMO - but I think that linking would be in fact more flexible; for example you could link based on group name; you could break the link and recreate it; you could decide what to do when linking groups sum members, use one group or the other, we would require unique group names in groupings only (you could reuse group1,group15 in several groupings in one course). The nice thing is that these extensions can be all done in 3rd party code without major breaking of future upgrades. We could also implement the linking later, not in 1.9, without major db changes (one db field only in groups table). There would be no need for global groups/groupings except the enrolment purposes - but it might be solved by different means.

          The more I think about it the more I like the linking - flexible, few side effects, easier db schema, easier modifications...

          Global groups or linking across courses? Simple use case is big course with old style tutor groups that is being split into two courses. Or when I was at grammar school we had the same 2 groups for several classes - chemistry lab, physics lab and biology lab. They are using moodle at that school now, I bet they still use the same fixed groupings for several classes Or at the university here in Liberec they have groups of students that are the same for many laboratory practices (I do not know how to call it in English properly, sorry). This is very common it the first years, later when the students specialize the groups are not fixed anymore.

          Show
          Petr Škoda added a comment - - edited Simple and natural? That seems to be a bit relative a) create a grouping1 b) define groups groupA, groupB c) create new grouping2 d) add new groupC - select "Clone groupA" in new group form e) modify groupC members (does not affect groupA) f) add new groupB - select "Link to groupB" in new group form g) create new grouping3 - select "Clone+link groups from grouping1" h) you can change/review the linking later in any grouping I can not see any problems you describe above - if you link two groupings, the result is sum of all members in both groups; if you add one user into grouping the internal function for adding group members could sync the rest of linked groups; if you remove user from linked group, removing function syncs all linked groups. Only several places deal with group membership - group UI, user import, restore, enrolment (any more?)- they should use library functions for any adding/removing of members anyway - ok, it might be better to move it into core As I said before, linking has IMO fewer side effects than sharing because you still have groups attached to groupings - we could deal with the linking easily in backup/restore code IMHO. The UI could hide the difference between sharing and linking IMO - but I think that linking would be in fact more flexible; for example you could link based on group name; you could break the link and recreate it; you could decide what to do when linking groups sum members, use one group or the other, we would require unique group names in groupings only (you could reuse group1,group15 in several groupings in one course). The nice thing is that these extensions can be all done in 3rd party code without major breaking of future upgrades. We could also implement the linking later, not in 1.9, without major db changes (one db field only in groups table). There would be no need for global groups/groupings except the enrolment purposes - but it might be solved by different means. The more I think about it the more I like the linking - flexible, few side effects, easier db schema, easier modifications... Global groups or linking across courses? Simple use case is big course with old style tutor groups that is being split into two courses. Or when I was at grammar school we had the same 2 groups for several classes - chemistry lab, physics lab and biology lab. They are using moodle at that school now, I bet they still use the same fixed groupings for several classes Or at the university here in Liberec they have groups of students that are the same for many laboratory practices (I do not know how to call it in English properly, sorry). This is very common it the first years, later when the students specialize the groups are not fixed anymore.
          Hide
          Enrique Castro added a comment -

          Hi Petr,
          Linking two groups is a NEW concept, not existing previously in Moodle.
          If I a teacher have Groups A, B, C, D and Groupings 01, 02 and he wants that GroupA be a member of grouping 01 AND 02 the simple, conceptually, thing to do is just that: Add GroupA to grouping 01 and to grouping 02. Just like it sounds, no extra concept.

          Within your scheme if a teacher wants groupA=(John, Paul & Mary) in two groupings he first create a new empty group in the second grouping, and then he links it to the former. Even both linked groups may have different name, but collect together the same students. This is a radically different concept than simply "use groupA in two groupings". Linking is an additional concept or property of groups. What you are doing is crening a NEW groups, which happens to have the same members (that's a secondary condition). Linking may have its merits and uses, but it's something completely diffrent to the groups concempt in the first place.

          I think that our opposing positions arise from different views of groups. It seems that you are thinking mainly in "rigid" class-level groups. At grammar school you have third grade, and probably Third Grade A, Third grade B and third grade C. They have each a teacher assigned, even e different room. The students in Third Grade A are in A for all the course, rarely change. Those gropups are not decided by the class-teacher but come done, imposed by School management.

          The main point of this issue is to get ACTIVITY-SPECIFIC GROUPS. Groupings are not and end by themselves. They are useful because they allow to use different groups at different activities. I am the teacher of Third Grade A. I want my pupils to discuss about a topic in groups of 5. I set up those groups and a forum or wike where eah group can discuss its topic without interference from the others. But net week I have other excercise, now to be done by other groups (I want 5-member groups, but not the same 5 members for all activities). That's the use of groupings. So you have many many groups. It is not unusual to have 20-30 groups in each grouping (activity).

          Now the next step. GroupC of activity 01 and group H of activity 03 are working in a similar topic. I want to add extra resources for them to see (and only them). A simple solution is adding those resources for a GroupingX=(GroupA+GroupH)

          There is another point of view that separate our positions: the relative weight of groups and groupings. I have GROUPS as first actors, users are distributed into Groups, existing by themselves. Only in a second step groups are organized in sets, in groupings. I can reorganize the same bunch of groups into different sets, different groupings for specialized usage (grouping may impose extra conditions to how groups behave). My impression is that you have GROUPINGS as main actors, first create grouping, then groups associated to it rigidly.

          The fact is that in mixed environments, and at activity level, groups, are created at teacher will, not rigidly and top-down imposed by organization enrolment structure. I as teacher create and mix as desired.

          In your example about lab practices. I do have several Laboratory practice groups in my course, and they are fixed for all lab practices (but not other activities). I have LabA and LabB, with 24 users each (all 24 are in the lab room together at the same hour). Then I have LabA01, LabA02 ... LabA12 are two-student groups (practice experiments are performed in pairs). For announcements I have a grouping (Theory, LabA, LabB). That allows me to post messages in a forum (using that grouping) taylored for each of those groups (for instance, next lab session for LabA is Wed 16.00 hours, LabB is Friday 10.00 AM). For lab reports I have a Lab Grouping=(LabA, LabB, LabA01, LabA02 ....., LabB01, LabB02

          In that way each students has access to two wiki pages, one "private" for the working pair (where they write their own particular report) and a common "lab" page where all pairs can write and put in common results of experiments

          I could use two wikis, one for pair-group reports and another for lab-level reports. The second would use a grouping (LabA, LabB), note I am reusing the LabA, LabB groups with the Announcement (Theory, LabA, LabB). "Theory" groups is not all students (there are several theory groups, for other teachers).

          I do not know if you have Lab practices in common for students of different courses. For instance, having Lab01 for students of Chem102 and Biochem101 and Biology203. That would be best managed by a metacourse, with students going into groups within metacourse based in the child course they come from. Managing this with regular separate courses and "linking" some group of Chem102 to other group of Lab0, that will open, fully, the concept of trans-course linking or coupling, the whole issue you try to keep well apart from temptation.

          Offering a UI for sharing groups WITHIN course is less dangerous in the long run. It keeps things really within course boundaries.

          Show
          Enrique Castro added a comment - Hi Petr, Linking two groups is a NEW concept, not existing previously in Moodle. If I a teacher have Groups A, B, C, D and Groupings 01, 02 and he wants that GroupA be a member of grouping 01 AND 02 the simple, conceptually, thing to do is just that: Add GroupA to grouping 01 and to grouping 02. Just like it sounds, no extra concept. Within your scheme if a teacher wants groupA=(John, Paul & Mary) in two groupings he first create a new empty group in the second grouping, and then he links it to the former. Even both linked groups may have different name, but collect together the same students. This is a radically different concept than simply "use groupA in two groupings". Linking is an additional concept or property of groups. What you are doing is crening a NEW groups, which happens to have the same members (that's a secondary condition). Linking may have its merits and uses, but it's something completely diffrent to the groups concempt in the first place. I think that our opposing positions arise from different views of groups. It seems that you are thinking mainly in "rigid" class-level groups. At grammar school you have third grade, and probably Third Grade A, Third grade B and third grade C. They have each a teacher assigned, even e different room. The students in Third Grade A are in A for all the course, rarely change. Those gropups are not decided by the class-teacher but come done, imposed by School management. The main point of this issue is to get ACTIVITY-SPECIFIC GROUPS. Groupings are not and end by themselves. They are useful because they allow to use different groups at different activities. I am the teacher of Third Grade A. I want my pupils to discuss about a topic in groups of 5. I set up those groups and a forum or wike where eah group can discuss its topic without interference from the others. But net week I have other excercise, now to be done by other groups (I want 5-member groups, but not the same 5 members for all activities). That's the use of groupings. So you have many many groups. It is not unusual to have 20-30 groups in each grouping (activity). Now the next step. GroupC of activity 01 and group H of activity 03 are working in a similar topic. I want to add extra resources for them to see (and only them). A simple solution is adding those resources for a GroupingX=(GroupA+GroupH) There is another point of view that separate our positions: the relative weight of groups and groupings. I have GROUPS as first actors, users are distributed into Groups, existing by themselves. Only in a second step groups are organized in sets, in groupings. I can reorganize the same bunch of groups into different sets, different groupings for specialized usage (grouping may impose extra conditions to how groups behave). My impression is that you have GROUPINGS as main actors, first create grouping, then groups associated to it rigidly. The fact is that in mixed environments, and at activity level, groups, are created at teacher will, not rigidly and top-down imposed by organization enrolment structure. I as teacher create and mix as desired. In your example about lab practices. I do have several Laboratory practice groups in my course, and they are fixed for all lab practices (but not other activities). I have LabA and LabB, with 24 users each (all 24 are in the lab room together at the same hour). Then I have LabA01, LabA02 ... LabA12 are two-student groups (practice experiments are performed in pairs). For announcements I have a grouping (Theory, LabA, LabB). That allows me to post messages in a forum (using that grouping) taylored for each of those groups (for instance, next lab session for LabA is Wed 16.00 hours, LabB is Friday 10.00 AM). For lab reports I have a Lab Grouping=(LabA, LabB, LabA01, LabA02 ....., LabB01, LabB02 In that way each students has access to two wiki pages, one "private" for the working pair (where they write their own particular report) and a common "lab" page where all pairs can write and put in common results of experiments I could use two wikis, one for pair-group reports and another for lab-level reports. The second would use a grouping (LabA, LabB), note I am reusing the LabA, LabB groups with the Announcement (Theory, LabA, LabB). "Theory" groups is not all students (there are several theory groups, for other teachers). I do not know if you have Lab practices in common for students of different courses. For instance, having Lab01 for students of Chem102 and Biochem101 and Biology203. That would be best managed by a metacourse, with students going into groups within metacourse based in the child course they come from. Managing this with regular separate courses and "linking" some group of Chem102 to other group of Lab0, that will open, fully, the concept of trans-course linking or coupling, the whole issue you try to keep well apart from temptation. Offering a UI for sharing groups WITHIN course is less dangerous in the long run. It keeps things really within course boundaries.
          Hide
          Nick Freear added a comment -

          Petr,
          In the latest incarnation of your DB schema (9:13 17 July) the table names have become plural again - groups, groupings and so on. Is this deliberate?
          http://docs.moodle.org/en/Image:groupsdb.png#file

          I was happy with 'group', grouping, but I'm not concerned either way. Just checking!
          Thanks
          Nick

          Show
          Nick Freear added a comment - Petr, In the latest incarnation of your DB schema (9:13 17 July) the table names have become plural again - groups, groupings and so on. Is this deliberate? http://docs.moodle.org/en/Image:groupsdb.png#file I was happy with 'group', grouping, but I'm not concerned either way. Just checking! Thanks Nick
          Hide
          Petr Škoda added a comment -

          Enrique, we are both talking about different things - I am talking about internal implementation now. You are talking about interface and usage. There is no reason why we could not hide the "linking" in UI and pretend we are sharing the groups. But I am not sure I like the sharing in UI, I would prefer either only linking or switch to emulate sharing (the implementation would not be difficult).

          Please note that groupings do not exists in Moodle 1.8.1 officially yet, we can still change them in any way now and it will not be unnatural or confusing.
          Yes, I am thinking about groupings as something above groups that separates them.

          I have changed the table names to make it more compatible with 1.7 or earlier.

          Show
          Petr Škoda added a comment - Enrique, we are both talking about different things - I am talking about internal implementation now. You are talking about interface and usage. There is no reason why we could not hide the "linking" in UI and pretend we are sharing the groups. But I am not sure I like the sharing in UI, I would prefer either only linking or switch to emulate sharing (the implementation would not be difficult). Please note that groupings do not exists in Moodle 1.8.1 officially yet, we can still change them in any way now and it will not be unnatural or confusing. Yes, I am thinking about groupings as something above groups that separates them. I have changed the table names to make it more compatible with 1.7 or earlier.
          Hide
          Enrique Castro added a comment - - edited

          Hi Petr, the aim of my former comment was to try to reconcile our distant points of view by making clear that we were pointing to the same object at different angles.

          Talking about implementation, I do have proposed one. There is a diff attached since I opened the issue. That implementation covers all discussed features but global groups (taht can be easily incorporated) and has several others. Your proposal not only reduces functionality for end user but requires writing and testing new code for "linking".

          You see linking simple at implementation but i see that now add/remove functions need to check first if the
          In UI, when "adding a group to a grouping" we need to check first if the group belong to another grouping. If not, go ahead, if already used, proceed to transparently make a new group and link to the former. I see those checking as more complex code, not simpler.

          I see "linking" a problem for understand and use, in the long term. If I add/remove users to GroupA (that happens to be used at groupings 01, 05 and 07): But if I have GroupA in grouping 01 and GroupHH in grouping04 becomes linked to GroupA; how do the teacher realize that fact afterwards, when actually using the groups? In you previous post you comment in various ways on linking: Second group linked to former (one cannot add/remove users to linked group), or bidirectional, or linked plus add/remove individual users in the linked groups: too confusing for use.

          I argue that once the "linking" mechanism is on place, it can be abused to create a nightmare of cross-linked groups just as the sharing of groups.

          Reviewing you last DB scheme
          http://docs.moodle.org/en/Image:groupsdb.png
          I see you have further reduced it by making groupings course-associated: introducing a courseid field in groupings table.
          That modification works for grouping-based activities, but that implementation cannot manage Global Groupings (groupings not associated with a courseid). So that scheme is only a transient one. When Global Groups were addressed that scheme must be dropped for other more general.

          If your mandatory goal is to keep groups and groupings course-based to avoid inter-course dependencies then a tested solution is to make groups and groupings course-associated by introducing explicitly a courseid field. I have the schema
          http://docs.moodle.org/en/images_en/c/ce/Groups-tables2.png
          working on moodle1.6 in current ULPGC site.
          This DB structure ignores global groups/groupings. It would be possible to incorporate them in future by making courseid=0 to mean Global group/grouping

          With this structure in is impossible to make any 3rd party extension that introduces course-crosslinking: groups and groupings are strictly course-based. But it is possible to have a group in more than one grouping.

          Petr, sincerely, I do not understand your reasons to insist in "a group can belong to just one unique grouping" when both groups and groupings are course-associated.

          --------------------
          Another approach to global groups: use "linking" to create local-global groups

          In all discussion about global groups I have understand, always, that a global group/grouping used at a course level really meant "those users or the global group that happens to be enrolled in this course". This behavior can be emulated the other way around.
          a)create a groupX
          b) instead of populating it with users using the UI tool selecting users from a list, the UI provides other separate tool: Add users from Global Group. No list of users but a list of global groups and the users that actually become members of groupX are those in global groups selected that actually are enrolled at the course.
          In this approach synchronization of global-linked groups is maintained at global-groups editing level: each time a global group is changed (user Added/removed) all linked local groups are tested for user add/removal.

          Show
          Enrique Castro added a comment - - edited Hi Petr, the aim of my former comment was to try to reconcile our distant points of view by making clear that we were pointing to the same object at different angles. Talking about implementation, I do have proposed one. There is a diff attached since I opened the issue. That implementation covers all discussed features but global groups (taht can be easily incorporated) and has several others. Your proposal not only reduces functionality for end user but requires writing and testing new code for "linking". You see linking simple at implementation but i see that now add/remove functions need to check first if the In UI, when "adding a group to a grouping" we need to check first if the group belong to another grouping. If not, go ahead, if already used, proceed to transparently make a new group and link to the former. I see those checking as more complex code, not simpler. I see "linking" a problem for understand and use, in the long term. If I add/remove users to GroupA (that happens to be used at groupings 01, 05 and 07): But if I have GroupA in grouping 01 and GroupHH in grouping04 becomes linked to GroupA; how do the teacher realize that fact afterwards, when actually using the groups? In you previous post you comment in various ways on linking: Second group linked to former (one cannot add/remove users to linked group), or bidirectional, or linked plus add/remove individual users in the linked groups: too confusing for use. I argue that once the "linking" mechanism is on place, it can be abused to create a nightmare of cross-linked groups just as the sharing of groups. Reviewing you last DB scheme http://docs.moodle.org/en/Image:groupsdb.png I see you have further reduced it by making groupings course-associated: introducing a courseid field in groupings table. That modification works for grouping-based activities, but that implementation cannot manage Global Groupings (groupings not associated with a courseid). So that scheme is only a transient one. When Global Groups were addressed that scheme must be dropped for other more general. If your mandatory goal is to keep groups and groupings course-based to avoid inter-course dependencies then a tested solution is to make groups and groupings course-associated by introducing explicitly a courseid field. I have the schema http://docs.moodle.org/en/images_en/c/ce/Groups-tables2.png working on moodle1.6 in current ULPGC site. This DB structure ignores global groups/groupings. It would be possible to incorporate them in future by making courseid=0 to mean Global group/grouping With this structure in is impossible to make any 3rd party extension that introduces course-crosslinking: groups and groupings are strictly course-based. But it is possible to have a group in more than one grouping. Petr, sincerely, I do not understand your reasons to insist in "a group can belong to just one unique grouping" when both groups and groupings are course-associated. -------------------- Another approach to global groups: use "linking" to create local-global groups In all discussion about global groups I have understand, always, that a global group/grouping used at a course level really meant "those users or the global group that happens to be enrolled in this course". This behavior can be emulated the other way around. a)create a groupX b) instead of populating it with users using the UI tool selecting users from a list, the UI provides other separate tool: Add users from Global Group. No list of users but a list of global groups and the users that actually become members of groupX are those in global groups selected that actually are enrolled at the course. In this approach synchronization of global-linked groups is maintained at global-groups editing level: each time a global group is changed (user Added/removed) all linked local groups are tested for user add/removal.
          Hide
          Petr Škoda added a comment -

          I was against the shared groups since the very beginning - this discussion itself is the proof that once the db structure got into core people started using it for sharing of groups, which will inevitably evolve into sharing of groups anywhere. We can not prevent it and it would be hard to solve in the future IMHO. Now is the last possibility to reverse it.

          The linking is more flexible than sharing and safer for us IMO. There are more possibilities how to implement it in UI - either by fully emulating sharing or creating new not yet imagined possibilities (which I tried to describe above). The worst thing that might happen with linking during backup/restore/import/export/upgrade is that the groups are unlinked, in case of sharing the problems could be worse. I do not understand your examples of problems with linking UI, I am sure there are ways to make it intuitive, easy to use and more flexible than sharing. Remember there are no groupings or sharing of groups yet in Moodle 1.8.1.

          The implementation complexity is another topic - we could implement first only groupings + cloning of groups in 1.9. The internal code and UI would be much simpler than today. later we could implement the linking and new UI features for group distribution, student self group assignment, etc. The side effect of my last proposed scheme is much better backwards compatibility with 1.7

          Global groupings are not my priority, linking+cloning would partially work around it, the enrolment trouble could be solved in a different way (not sure if the groups are the optimal way).

          My +1 to work on less features for 1.9 and more polished code and better APIs. We changed API+UI+db in 1.8, then again UI in 1.8.1 because groups did not work much/groupings did not work at all yet - we can not change UI+API+db in 1.9 and in 2.0 again

          There is not much time to do the coding and testing, the 1.9 beta is closer each day.

          Show
          Petr Škoda added a comment - I was against the shared groups since the very beginning - this discussion itself is the proof that once the db structure got into core people started using it for sharing of groups, which will inevitably evolve into sharing of groups anywhere. We can not prevent it and it would be hard to solve in the future IMHO. Now is the last possibility to reverse it. The linking is more flexible than sharing and safer for us IMO. There are more possibilities how to implement it in UI - either by fully emulating sharing or creating new not yet imagined possibilities (which I tried to describe above). The worst thing that might happen with linking during backup/restore/import/export/upgrade is that the groups are unlinked, in case of sharing the problems could be worse. I do not understand your examples of problems with linking UI, I am sure there are ways to make it intuitive, easy to use and more flexible than sharing. Remember there are no groupings or sharing of groups yet in Moodle 1.8.1. The implementation complexity is another topic - we could implement first only groupings + cloning of groups in 1.9. The internal code and UI would be much simpler than today. later we could implement the linking and new UI features for group distribution, student self group assignment, etc. The side effect of my last proposed scheme is much better backwards compatibility with 1.7 Global groupings are not my priority, linking+cloning would partially work around it, the enrolment trouble could be solved in a different way (not sure if the groups are the optimal way). My +1 to work on less features for 1.9 and more polished code and better APIs. We changed API+UI+db in 1.8, then again UI in 1.8.1 because groups did not work much/groupings did not work at all yet - we can not change UI+API+db in 1.9 and in 2.0 again There is not much time to do the coding and testing, the 1.9 beta is closer each day.
          Hide
          Enrique Castro added a comment - - edited

          Hi Petr,
          I understand that you are against any sharing across course boundaries. Sharing groups of one course with other course.

          Sharing within the same course is not abomination, it's natural. We do "share" users between activities (module instances), we do not "clone" a user to have access to another module. W do "share" users between groups. If a users is in three groups the userid is added to all three groups, not "cloned" or something special. There is no exception in code to check before adding a user to a group if teh user already belongs to other groups. If I have a group in a course, why should be imposed to "clone" it to access other activities?, why to "clone" it?, an exceptional procedure to add to a second grouping?.

          I think that your understandable prevention to the backup nightmare of shared groups across courses is extending beyond reasonable limits if precluding group sharing within a course.

          I have module-level groupings workin on 1.6 Moodle in our production site. Both groups and groupings and ONLY local, without any trace of globally or cross-course shared groups. But with groups belonging to several groupings within a course.

          I have a las proposal to try to reconcile both views. From your last scheme
          http://docs.moodle.org/en/Image:groupsdb.png
          You are introducing courseid into groupings table, thus tightly associating each grouping with a single course. You have groupingid in groups table thus, associating a group with a single grouping and hence with a single course.

          I propose to remove groupingid from groups table and substitute with courseid. (that's the way I have both tables with 1.6). So each group would be explicitly associated with a single course. No ambiguity, no confusion for backup restore. We would need groupings_members table relating groupingid and groupid to hold the membership relation between groups and groupings. The schema is

          http://docs.moodle.org/en/Image:Groups-tables2.png

          In this way I think that is possible to get both goals
          a) Each group and grouping is explicitly associated to a unique courseid.
          b) It is possible to have a single group in a course to belong to several groupings in that course.

          It is always possible to write code that behaves contrary to moodle philosophy. For instance, it is possible to add people to a group that are not enrolled in the course. Just happens that the UI only present a list of course-users. Nothing in groups_add_member() function neither in the DB structure enforce a rule not to add a userid of a non-enroled user to a group. I think that the above proposal keeps the same compromise. Courseid is clearly stated and groups can be added to several groupings.

          We must provide a solution for Global Groups, since they are demanded by other people (not me neither ULPGC).
          For this particular purpose I think that "linking" or cloning would be a very good idea indeed.

          Global groups are managed (created, edited user add/removal) by admins or alike in a separate UI, available at site level, not course context. That means a UI (and API functions) distinct and separate from regular course-groups.

          We wouldn't have "global groups" at course level. All course groups would be "local" (courseid-associated)
          First we create a local Group, then instead of adding users, by use of a separate UI tool we link that local group to a "global group"
          linking means that only those users that happens to be be members of the global group and, simultaneously, are enrolled in teh current curse are added to the local group.

          The advantage of this approach over "generalized linking" is parsimony. Current group API would work untouched. The synchronization of membership when users were added(removed would be confined and controlled by global groups functions. E.g. each time a user is added to a global groups local groups are queried to identify and update linked local groups.

          But we all agree that global groups are something for the future. We must consider them now just to ensure that we are not forcing ourselves to go back and forth about them like the grouping issue in 1.8/1.8.1, 1.9.

          I think that we all agree that the only feature that is feasible to add to 1.9 are activity-level groupings (and grouping-based access that it's trivial once the former is done). And this task is waiting only for decision on DB structure. I think that
          http://docs.moodle.org/en/Image:Groups-tables2.png
          solves your concerns and preserves my usability demands. If we agree on that structure I can implement it for 1.9 in a couple of weeks, with UI etc. (have you seen the proposed UI?)

          Even, "linking" isn' strictly needed for activity-level groupings. Linking/sharing is a usability issue. Even if using your scheme
          http://docs.moodle.org/en/Image:groupsdb.png
          The provisions may be done in DB tables but not used yet (I mean keep linkid and the add the linking code to add to groups functions later) but activity-level groupings must be added. That's a real teaching gain, unavailable before. The issue of if groups are shared or not is secondary.

          Show
          Enrique Castro added a comment - - edited Hi Petr, I understand that you are against any sharing across course boundaries. Sharing groups of one course with other course. Sharing within the same course is not abomination, it's natural. We do "share" users between activities (module instances), we do not "clone" a user to have access to another module. W do "share" users between groups. If a users is in three groups the userid is added to all three groups, not "cloned" or something special. There is no exception in code to check before adding a user to a group if teh user already belongs to other groups. If I have a group in a course, why should be imposed to "clone" it to access other activities?, why to "clone" it?, an exceptional procedure to add to a second grouping?. I think that your understandable prevention to the backup nightmare of shared groups across courses is extending beyond reasonable limits if precluding group sharing within a course. I have module-level groupings workin on 1.6 Moodle in our production site. Both groups and groupings and ONLY local, without any trace of globally or cross-course shared groups. But with groups belonging to several groupings within a course. I have a las proposal to try to reconcile both views. From your last scheme http://docs.moodle.org/en/Image:groupsdb.png You are introducing courseid into groupings table, thus tightly associating each grouping with a single course. You have groupingid in groups table thus, associating a group with a single grouping and hence with a single course. I propose to remove groupingid from groups table and substitute with courseid. (that's the way I have both tables with 1.6). So each group would be explicitly associated with a single course. No ambiguity, no confusion for backup restore. We would need groupings_members table relating groupingid and groupid to hold the membership relation between groups and groupings. The schema is http://docs.moodle.org/en/Image:Groups-tables2.png In this way I think that is possible to get both goals a) Each group and grouping is explicitly associated to a unique courseid. b) It is possible to have a single group in a course to belong to several groupings in that course. It is always possible to write code that behaves contrary to moodle philosophy. For instance, it is possible to add people to a group that are not enrolled in the course. Just happens that the UI only present a list of course-users. Nothing in groups_add_member() function neither in the DB structure enforce a rule not to add a userid of a non-enroled user to a group. I think that the above proposal keeps the same compromise. Courseid is clearly stated and groups can be added to several groupings. We must provide a solution for Global Groups, since they are demanded by other people (not me neither ULPGC). For this particular purpose I think that "linking" or cloning would be a very good idea indeed. Global groups are managed (created, edited user add/removal) by admins or alike in a separate UI, available at site level, not course context. That means a UI (and API functions) distinct and separate from regular course-groups. We wouldn't have "global groups" at course level. All course groups would be "local" (courseid-associated) First we create a local Group, then instead of adding users, by use of a separate UI tool we link that local group to a "global group" linking means that only those users that happens to be be members of the global group and, simultaneously, are enrolled in teh current curse are added to the local group. The advantage of this approach over "generalized linking" is parsimony. Current group API would work untouched. The synchronization of membership when users were added(removed would be confined and controlled by global groups functions. E.g. each time a user is added to a global groups local groups are queried to identify and update linked local groups. But we all agree that global groups are something for the future. We must consider them now just to ensure that we are not forcing ourselves to go back and forth about them like the grouping issue in 1.8/1.8.1, 1.9. I think that we all agree that the only feature that is feasible to add to 1.9 are activity-level groupings (and grouping-based access that it's trivial once the former is done). And this task is waiting only for decision on DB structure. I think that http://docs.moodle.org/en/Image:Groups-tables2.png solves your concerns and preserves my usability demands. If we agree on that structure I can implement it for 1.9 in a couple of weeks, with UI etc. (have you seen the proposed UI?) Even, "linking" isn' strictly needed for activity-level groupings. Linking/sharing is a usability issue. Even if using your scheme http://docs.moodle.org/en/Image:groupsdb.png The provisions may be done in DB tables but not used yet (I mean keep linkid and the add the linking code to add to groups functions later) but activity-level groupings must be added. That's a real teaching gain, unavailable before. The issue of if groups are shared or not is secondary.
          Hide
          Petr Škoda added a comment -

          groups db schema - proposed by Enrique

          Show
          Petr Škoda added a comment - groups db schema - proposed by Enrique
          Hide
          Enrique Castro added a comment -

          OK, it seems that we have agreed on a DB schema that is optimal for all. Discussions are fertile!!!
          Petr has attached a Dia drawing of the final structure.

          The keypoints are:
          a) Both groups and groupings tables include a "courseid" filed and thus are univocally associated with a unique course
          b) Groups are related to groupings loosely, through the groups_groupings table. Then a group may belong to none, one or many groupings but remain associated to a single course.

          This structure is the most backward-compatible of those discussed, it's quite similar to groups in moodle versions 1.5 and 1.6. That eases upgrading. With this structure, all kind of groups/groupings relationships can be supported at single-course level.

          The advantages of this structure are:
          a) There is no room for cross-course interactions at DB level
          b) It's simpler and smaller than 1.8 structure
          c) Most used queries for user- or course-groups are simple
          d) Allows re-use and sharing of groups in multiple groupings within a course

          Global groups do not fit into the scheme but can be emulated. Global groups are managed at site level separately from course-level groups. In DB global groups and groupings may be identified by courseid=0 (or added to separate tables)
          All groups used in a course must be local groups, the linking mechanism guarantees that user-members are added as

          Another agreement.
          Grouping-based access restriction to resources and activities should not overload the "separate groups" mode due to backward compatibility issues. A new switch "grouping-restricted" (or similar name) is added to course_modules table and to common module settings along with groupmode and grouping.

          Nick, Sam, what do yo think in OU about this?

          Pending topics:

          Nick, if you are busy with other things I can take this (and take also bugfixing responsibility and workload)

          Show
          Enrique Castro added a comment - OK, it seems that we have agreed on a DB schema that is optimal for all. Discussions are fertile!!! Petr has attached a Dia drawing of the final structure. The keypoints are: a) Both groups and groupings tables include a "courseid" filed and thus are univocally associated with a unique course b) Groups are related to groupings loosely, through the groups_groupings table. Then a group may belong to none, one or many groupings but remain associated to a single course. This structure is the most backward-compatible of those discussed, it's quite similar to groups in moodle versions 1.5 and 1.6. That eases upgrading. With this structure, all kind of groups/groupings relationships can be supported at single-course level. The advantages of this structure are: a) There is no room for cross-course interactions at DB level b) It's simpler and smaller than 1.8 structure c) Most used queries for user- or course-groups are simple d) Allows re-use and sharing of groups in multiple groupings within a course Global groups do not fit into the scheme but can be emulated. Global groups are managed at site level separately from course-level groups. In DB global groups and groupings may be identified by courseid=0 (or added to separate tables) All groups used in a course must be local groups, the linking mechanism guarantees that user-members are added as Another agreement. Grouping-based access restriction to resources and activities should not overload the "separate groups" mode due to backward compatibility issues. A new switch "grouping-restricted" (or similar name) is added to course_modules table and to common module settings along with groupmode and grouping. Nick, Sam, what do yo think in OU about this? Pending topics: library cleanup, how much pruning? how far? For 1.9 or later? global groups: clearly later UI : should be discuss the UI? There are some schreenshots at http://docs.moodle.org/en/Development:Groupings_and_Groups Nick, if you are busy with other things I can take this (and take also bugfixing responsibility and workload)
          Hide
          Petr Škoda added a comment -

          attaching fresh db schema - the image version is in docs wiki, there is some more explanation

          Show
          Petr Škoda added a comment - attaching fresh db schema - the image version is in docs wiki, there is some more explanation
          Hide
          Nick Freear added a comment -

          Hi,
          I'm trying to keep up with reading comments on this bug!

          Enrique - thanks for the latest DB schema - I think we have concensus! I have started implementing the schema changes and upgrade. Will let petr and you know when I have a patch...

          A minor point, can we have the 'courseid' field immediately after 'id' in the groupings table, so it matches the 'groups' table? So, id, +courseid, name, description, +exclusivegroups, +maxgroupsize ...

          Regarding library cleanup "how much pruning?" - quite a bit, some of it specified in these sections - I'll flesh this out,
          http://docs.moodle.org/en/Development:Groupings_OU#Folder:_group

          I think we're envisioning a single 'grouplib' ((or groupinglib)) file from a merge of: basicgrouplib.php, groupinglib.php, utillib.php. All group/db/*lib.php should go, lib/courselib, automaticlib etc. to go.

          Legacylib function calls should be fazed out, but I think this file is best left separate, as is modulelib.php - we at the OU are using some of these functions and they are quite distinct. Get rid of unused functions (check against ou-moodle), and reduce number of functions generally.

          Hope that's a start. Must dash - talk tomorrow. Yours
          Nick

          Show
          Nick Freear added a comment - Hi, I'm trying to keep up with reading comments on this bug! Enrique - thanks for the latest DB schema - I think we have concensus! I have started implementing the schema changes and upgrade. Will let petr and you know when I have a patch... A minor point, can we have the 'courseid' field immediately after 'id' in the groupings table, so it matches the 'groups' table? So, id, +courseid, name, description, +exclusivegroups, +maxgroupsize ... Regarding library cleanup "how much pruning?" - quite a bit, some of it specified in these sections - I'll flesh this out, http://docs.moodle.org/en/Development:Groupings_OU#Folder:_group I think we're envisioning a single 'grouplib' ((or groupinglib)) file from a merge of: basicgrouplib.php, groupinglib.php, utillib.php. All group/db/*lib.php should go, lib/courselib, automaticlib etc. to go. Legacylib function calls should be fazed out, but I think this file is best left separate, as is modulelib.php - we at the OU are using some of these functions and they are quite distinct. Get rid of unused functions (check against ou-moodle), and reduce number of functions generally. Hope that's a start. Must dash - talk tomorrow. Yours Nick
          Hide
          Sam Marshall added a comment -

          I don't think it's appropriate for groups to belong to 0 groupings. This should not be permitted as it complicates the interface - when you select a grouping for an activity, you want to be choosing either: (a) a grouping, or (b) 'all groupings'.

          You don't want a third 'groups that are not in any grouping' option (let alone a fourth 'all groupings but not including groups that aren't in any grouping') - and this turns out to be necessary if you have groups in no grouping. Far better to ensure that all groups are in a default grouping for the course, unless specified otherwise.

          Overall I preferred the schema when it required every group to be in exactly one grouping But I think I don't have any practical problems with this provided we require groups to be in at least one grouping.

          Show
          Sam Marshall added a comment - I don't think it's appropriate for groups to belong to 0 groupings. This should not be permitted as it complicates the interface - when you select a grouping for an activity, you want to be choosing either: (a) a grouping, or (b) 'all groupings'. You don't want a third 'groups that are not in any grouping' option (let alone a fourth 'all groupings but not including groups that aren't in any grouping') - and this turns out to be necessary if you have groups in no grouping. Far better to ensure that all groups are in a default grouping for the course, unless specified otherwise. Overall I preferred the schema when it required every group to be in exactly one grouping But I think I don't have any practical problems with this provided we require groups to be in at least one grouping.
          Hide
          Petr Škoda added a comment - - edited

          I do not think that "orphan" groups are a problem with this schema. It is UI problem only IMHO.

          I like the proposed tabbed interface (http://docs.moodle.org/en/Development:Groupings_and_Groups) with separate groupings and groups forms. It solves the sharing of groups in an easy and understandable way.

          For the activities and course I think that "Legacy == no grouping == all groups in course" + all groupings should be fine.

          Show
          Petr Škoda added a comment - - edited I do not think that "orphan" groups are a problem with this schema. It is UI problem only IMHO. I like the proposed tabbed interface ( http://docs.moodle.org/en/Development:Groupings_and_Groups ) with separate groupings and groups forms. It solves the sharing of groups in an easy and understandable way. For the activities and course I think that "Legacy == no grouping == all groups in course" + all groupings should be fine.
          Hide
          Enrique Castro added a comment -

          Hi Sam,
          I would have no strong objection to having a default grouping to put groups at first instance (and then moving when other groupings are created)
          Provided that:
          a) upgrade code must ensure that "default" grouping is created for all courses
          b) "default" grouping cannot be deleted: an exception must be introduce into code to avoid this
          c) When deleting groupings, Where their groups go? Code must ensure that they are added to "default" grouping

          I have no strong objection, but I do not like "default" grouping because that creates an "exception" a grouping that is different from others in behavior without clear cause (from code view)

          This is a matter of business logic, usability (from users point of view) and backwards compatibility:

          • Backwards compatibility: before 1.8 all groups were in flat list, without groupings. Restoring old backups will work untouched if no default grouping is allowed; otherwise restore code must create the "default" grouping
          • Not all teachers need groupings. Allowing to use groups without groupings keep those teachers using Moodle just as before: upgrade do not force them to change the way they work.
          • The proposed UI emphasizes Groups first. You can create a bunch of groups and use them in a grouping or not. Groupings are a secondary thing used to organize groups into sets fed to activities: A way to specify what of all groups should be used in a specific activity. But groups exists per se (talking not about implementation code but teacher mind). A teacher may have some groups created but not yet used in any grouping or any activity, waiting for their moment on course time-flow. He may slowly add users (as they make their mind about a topic, as they enroll etc) into groups (unusable at activities because not in grouping) and afterwards put those groups to use in a grouping.
          • There could be a confusion about "default" grouping. We as programmers are talking abut code-default. However, the course has a groupingid meaning the grouping to use for separate/visible groups when no other is declared at module level, aka "default" grouping for users. Since this is selectable may be that "default course grouping" is not the same as "default groups grouping". This more a matter of carefully selection of names, and language-specific. But allowing "no grouping=all groups" avoids the problem.
          • I agree that for activities only "all groups" or "groups of grouping X" should be allowed. That menu, at modules, should show a list of groupings plus "All groups" option. Other combinations, 'groups that are not in any grouping' option and the like simple are not considered explicitly.

          In the code at the attached diff you may find examples. Unfortunately my UI demo uses pre-formslib code. I may concentrate on creating the UI with formslib while you refactor the groups libraries, if that may help.

          Show
          Enrique Castro added a comment - Hi Sam, I would have no strong objection to having a default grouping to put groups at first instance (and then moving when other groupings are created) Provided that: a) upgrade code must ensure that "default" grouping is created for all courses b) "default" grouping cannot be deleted: an exception must be introduce into code to avoid this c) When deleting groupings, Where their groups go? Code must ensure that they are added to "default" grouping I have no strong objection, but I do not like "default" grouping because that creates an "exception" a grouping that is different from others in behavior without clear cause (from code view) This is a matter of business logic, usability (from users point of view) and backwards compatibility: Backwards compatibility: before 1.8 all groups were in flat list, without groupings. Restoring old backups will work untouched if no default grouping is allowed; otherwise restore code must create the "default" grouping Not all teachers need groupings. Allowing to use groups without groupings keep those teachers using Moodle just as before: upgrade do not force them to change the way they work. The proposed UI emphasizes Groups first. You can create a bunch of groups and use them in a grouping or not. Groupings are a secondary thing used to organize groups into sets fed to activities: A way to specify what of all groups should be used in a specific activity. But groups exists per se (talking not about implementation code but teacher mind). A teacher may have some groups created but not yet used in any grouping or any activity, waiting for their moment on course time-flow. He may slowly add users (as they make their mind about a topic, as they enroll etc) into groups (unusable at activities because not in grouping) and afterwards put those groups to use in a grouping. There could be a confusion about "default" grouping. We as programmers are talking abut code-default. However, the course has a groupingid meaning the grouping to use for separate/visible groups when no other is declared at module level, aka "default" grouping for users. Since this is selectable may be that "default course grouping" is not the same as "default groups grouping". This more a matter of carefully selection of names, and language-specific. But allowing "no grouping=all groups" avoids the problem. I agree that for activities only "all groups" or "groups of grouping X" should be allowed. That menu, at modules, should show a list of groupings plus "All groups" option. Other combinations, 'groups that are not in any grouping' option and the like simple are not considered explicitly. In the code at the attached diff you may find examples. Unfortunately my UI demo uses pre-formslib code. I may concentrate on creating the UI with formslib while you refactor the groups libraries, if that may help.
          Hide
          Nick Freear added a comment -

          This is version 6 of a the patch to upgrade to the the new DB schema, from 1.7./1.6. etc, and 1.8.. For the 1.8. upgrade I've chosen the route of adding/removing fields and renaming tables - I did try creating new tables and transferring data, this would be slower.

          It's based on the 18 July 'groupsdb' image, http://docs.moodle.org/en/Development:Groups - with these modifications:

          • groupings table: courseid after id column, not after description.
          • groupings_groups: added 'timeadded' field.

          We (Tim Hunt and myself) still have big problems about orphan groups - in my/sam's absence Tim is going to discuss, I think!

          Other patches to follow.

          Show
          Nick Freear added a comment - This is version 6 of a the patch to upgrade to the the new DB schema, from 1.7. /1.6. etc, and 1.8. . For the 1.8. upgrade I've chosen the route of adding/removing fields and renaming tables - I did try creating new tables and transferring data, this would be slower. It's based on the 18 July 'groupsdb' image, http://docs.moodle.org/en/Development:Groups - with these modifications: groupings table: courseid after id column, not after description. groupings_groups: added 'timeadded' field. We (Tim Hunt and myself) still have big problems about orphan groups - in my/sam's absence Tim is going to discuss, I think! Other patches to follow.
          Hide
          Nick Freear added a comment -

          For future reference:

          • This big patch from 23 February was used internally for ou-moodle to add groupings support centrally to 'course_modules' and to the Moodle WIKI. It would need alterations, but gives an indication of what we have done.
          • In OU-moodle, we've since added groupings to: forum, quiz reports, and data activities.
          Show
          Nick Freear added a comment - For future reference: This big patch from 23 February was used internally for ou-moodle to add groupings support centrally to 'course_modules' and to the Moodle WIKI. It would need alterations, but gives an indication of what we have done. In OU-moodle, we've since added groupings to: forum, quiz reports, and data activities.
          Hide
          Nick Freear added a comment -

          I attach a patch that can form part 1 of the library cleanup - it just removes unused files. (Part '0' was Petr removing 'groupui' directory .)

          Please review these 3 patches. Test the first one if possible. Many thanks

          Nick

          Show
          Nick Freear added a comment - I attach a patch that can form part 1 of the library cleanup - it just removes unused files. (Part '0' was Petr removing 'groupui' directory .) Please review these 3 patches. Test the first one if possible. Many thanks Nick
          Hide
          Nick Freear added a comment -

          Note, the first patch 'groups-db-v6' upgrades the database, but does not include the minimal library changes to use the new DB schema. That is the next step. I'm off on leave for 2 weeks, so I'll discuss and continue then. Yours

          Nick

          Show
          Nick Freear added a comment - Note, the first patch 'groups-db-v6' upgrades the database, but does not include the minimal library changes to use the new DB schema. That is the next step. I'm off on leave for 2 weeks, so I'll discuss and continue then. Yours Nick
          Hide
          Martin Dougiamas added a comment -

          I'm happy if you guys are agreed on the schema ... can we get the tables done and into HEAD ASAP (before we branch for 1.9) so that groups at least work as they did before (but on the new tables) so that groupings can be added after branching possibly?

          Is everyone agreed on Nick's patches above?

          Show
          Martin Dougiamas added a comment - I'm happy if you guys are agreed on the schema ... can we get the tables done and into HEAD ASAP (before we branch for 1.9) so that groups at least work as they did before (but on the new tables) so that groupings can be added after branching possibly? Is everyone agreed on Nick's patches above?
          Hide
          Matt Clarkson added a comment -

          I have a series of commits on my local git repo that update moodle to the new groups schema and update the code to maintain the existing groups functionality using the new database schema.

          Schema and upgrade:

          http://git.catalyst.net.nz/gitweb?p=moodle-r2.git;a=commit;h=b68d2070f280fd87945b4fe96783a68fd391f25a
          http://git.catalyst.net.nz/gitweb?p=moodle-r2.git;a=commit;h=c6f949250d9fb2c67aec9ebcd771a8c9f05565de

          Upgrade code to use new schema:

          http://git.catalyst.net.nz/gitweb?p=moodle-r2.git;a=commit;h=9c11a1d709739859dab46e5e9605f11a4bb8dd90
          http://git.catalyst.net.nz/gitweb?p=moodle-r2.git;a=commit;h=7ce3afb45182127e45df4563f9a6211b5560f535
          http://git.catalyst.net.nz/gitweb?p=moodle-r2.git;a=commit;h=265eb0379f7f39e4faa58e6aabbc083e063534c5

          Please review these patches and let me know if you are happy for them to go into CVS head.

          -Matt

          Show
          Matt Clarkson added a comment - I have a series of commits on my local git repo that update moodle to the new groups schema and update the code to maintain the existing groups functionality using the new database schema. Schema and upgrade: http://git.catalyst.net.nz/gitweb?p=moodle-r2.git;a=commit;h=b68d2070f280fd87945b4fe96783a68fd391f25a http://git.catalyst.net.nz/gitweb?p=moodle-r2.git;a=commit;h=c6f949250d9fb2c67aec9ebcd771a8c9f05565de Upgrade code to use new schema: http://git.catalyst.net.nz/gitweb?p=moodle-r2.git;a=commit;h=9c11a1d709739859dab46e5e9605f11a4bb8dd90 http://git.catalyst.net.nz/gitweb?p=moodle-r2.git;a=commit;h=7ce3afb45182127e45df4563f9a6211b5560f535 http://git.catalyst.net.nz/gitweb?p=moodle-r2.git;a=commit;h=265eb0379f7f39e4faa58e6aabbc083e063534c5 Please review these patches and let me know if you are happy for them to go into CVS head. -Matt
          Hide
          Martin Dougiamas added a comment -

          Petr, could you please take care of reviewing and adding Matt's patches to HEAD today?

          Show
          Martin Dougiamas added a comment - Petr, could you please take care of reviewing and adding Matt's patches to HEAD today?
          Hide
          Martin Dougiamas added a comment -

          (and thanks again, Matt!)

          Show
          Martin Dougiamas added a comment - (and thanks again, Matt!)
          Hide
          Matt Clarkson added a comment -

          I have attached a patch against the current cvs head.

          Show
          Matt Clarkson added a comment - I have attached a patch against the current cvs head.
          Hide
          Petr Škoda added a comment -

          Sorry I did not notice the linked patches and was having some skype problems (caused by the latest linux beta) last week.
          So far the patch seems to be ok, going to review/test it more tomorrow - I hope we can get it into HEAD asap tomorrow...

          Show
          Petr Škoda added a comment - Sorry I did not notice the linked patches and was having some skype problems (caused by the latest linux beta) last week. So far the patch seems to be ok, going to review/test it more tomorrow - I hope we can get it into HEAD asap tomorrow...
          Hide
          Martin Dougiamas added a comment -

          I really really really really want to release Beta today ... Hopefully with these tables, so can you make this a #0 priority today please?

          Show
          Martin Dougiamas added a comment - I really really really really want to release Beta today ... Hopefully with these tables, so can you make this a #0 priority today please?
          Hide
          Petr Škoda added a comment -

          should be in cvs

          Show
          Petr Škoda added a comment - should be in cvs
          Hide
          Martin Dougiamas added a comment -

          (dance) Thanks, Petr!

          Show
          Martin Dougiamas added a comment - (dance) Thanks, Petr!
          Hide
          Petr Škoda added a comment -

          adding latest cleanup patch - to be committed today

          Show
          Petr Škoda added a comment - adding latest cleanup patch - to be committed today
          Hide
          Martin Dougiamas added a comment -

          Closing this because I think it's OK. Please file new bugs for anything missing.

          Show
          Martin Dougiamas added a comment - Closing this because I think it's OK. Please file new bugs for anything missing.
          Hide
          Petr Škoda added a comment -

          reopening, there is still one problem with group selector if use has accessallgroups in one module only
          solution should be to add subarray into SESSION->activegroup for ppl with accessallgroups

          testing patch....

          Show
          Petr Škoda added a comment - reopening, there is still one problem with group selector if use has accessallgroups in one module only solution should be to add subarray into SESSION->activegroup for ppl with accessallgroups testing patch....
          Hide
          Petr Škoda added a comment -

          committed into cvs, reclosing

          Show
          Petr Škoda added a comment - committed into cvs, reclosing
          Hide
          Martin Dougiamas added a comment -

          I've made a small fix to that last patch, Petr, because $cm doesn't exist, but I made an assumption that you mean the course context, not module context. Can you please check on this?

          http://moodle.cvs.sourceforge.net/moodle/moodle/lib/grouplib.php?r1=1.21&r2=1.22

          Show
          Martin Dougiamas added a comment - I've made a small fix to that last patch, Petr, because $cm doesn't exist, but I made an assumption that you mean the course context, not module context. Can you please check on this? http://moodle.cvs.sourceforge.net/moodle/moodle/lib/grouplib.php?r1=1.21&r2=1.22
          Hide
          Petr Škoda added a comment -

          oops, my sloppy copy/paste without proper testing again - sorry
          reclosing

          Show
          Petr Škoda added a comment - oops, my sloppy copy/paste without proper testing again - sorry reclosing

            People

            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: