Moodle Community Sites
  1. Moodle Community Sites
  2. MDLSITE-2892

Improve the integration list to move delayed issues to top of pile

    Details

    • Type: Bug Bug
    • Status: Development in progress
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Component/s: Integration
    • Labels:
      None

      Description

      Unfortunately we don't have a way to sort by when submitted for integration and our existing searches are easily fooled by comments on issues so it usually depends on integrators remembering to search for delayed issues first.

      We should find a way to force the delayed issues to the top of the list for integration when they have been delayed one week.

        Gliffy Diagrams

          Issue Links

            Activity

            Hide
            Eloy Lafuente (stronk7) added a comment - - edited

            Yeah,

            I've looked for custom orders like this in the past. And basically the conclusion has been that we need a custom field for this, able to calculate an ordering rank by itself or, at least an ordereable value for certain conditions.

            The main problem is that the "ORDER BY" clause does not allow any expression (basically) so you must provide it with something already ordered, aka the custom field.

            Note there are some plugins like:

            that enables easy creation of custom fields. In fact I think I asked for the later some time ago to be installed. So we wouldn't need to create it with the Java SDK, in theory. I don't know if such fields are searchable or no.

            Show
            Eloy Lafuente (stronk7) added a comment - - edited Yeah, I've looked for custom orders like this in the past. And basically the conclusion has been that we need a custom field for this, able to calculate an ordering rank by itself or, at least an ordereable value for certain conditions. The main problem is that the "ORDER BY" clause does not allow any expression (basically) so you must provide it with something already ordered, aka the custom field. Note there are some plugins like: https://innovalog.atlassian.net/wiki/display/JMCF/JIRA+Misc+Custom+Fields https://marketplace.atlassian.com/plugins/com.onresolve.jira.groovy.groovyrunner that enables easy creation of custom fields. In fact I think I asked for the later some time ago to be installed. So we wouldn't need to create it with the Java SDK, in theory. I don't know if such fields are searchable or no.
            Hide
            Damyon Wiese added a comment -

            Just noting that I used groovyrunner in the past and it was pretty good.

            Show
            Damyon Wiese added a comment - Just noting that I used groovyrunner in the past and it was pretty good.
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Pasting thoughts from the integration chat:

            think we can automate the "delayed" thing, thanks to CiBoT automations. Sort of:

            • Create new, hidden everywhere, "integration prioritizer" custom field.
            • Make all our filters to always sort by that field DESC
            • Create a job, to be executed every 24h against issues awaiting integration, calculating the value for the field based on whatever we want, for example, for now: existence of "delayed" comment in the issue.

            It's not perfect coz requires CiBoT, but hey, we are already dependent on it for many other tasks.

            https://tracker.moodle.org/issues/?filter=14000&jql=project%20%3D%20MDL%20AND%20status%20%3D%20%22Waiting%20for%20integration%20review%22%20AND%20text%20~%20%22The%20integration%20of%20this%20issue%20has%20been%20delayed%20to%20next%20week%22

            It has the advantage of not requiring any new Jira plugin and should continue working without problem always. Plus, we can easily add new "priorities" to the calculations and all them will be the first sorting criteria (getting precedence over priority, votes...).

            Ciao

            Show
            Eloy Lafuente (stronk7) added a comment - Pasting thoughts from the integration chat: think we can automate the "delayed" thing, thanks to CiBoT automations. Sort of: Create new, hidden everywhere, "integration prioritizer" custom field. Make all our filters to always sort by that field DESC Create a job, to be executed every 24h against issues awaiting integration, calculating the value for the field based on whatever we want, for example, for now: existence of "delayed" comment in the issue. It's not perfect coz requires CiBoT, but hey, we are already dependent on it for many other tasks. https://tracker.moodle.org/issues/?filter=14000&jql=project%20%3D%20MDL%20AND%20status%20%3D%20%22Waiting%20for%20integration%20review%22%20AND%20text%20~%20%22The%20integration%20of%20this%20issue%20has%20been%20delayed%20to%20next%20week%22 It has the advantage of not requiring any new Jira plugin and should continue working without problem always. Plus, we can easily add new "priorities" to the calculations and all them will be the first sorting criteria (getting precedence over priority, votes...). Ciao
            Hide
            Dan Poltawski added a comment -

            Makes sense - all issues have integration priority of 0, when something like delay added, we update the issue to +1 to it.

            Only thing I disagree with is the criteria for finding them:

            • Using 'text' as metadata always makes me sick.. which I would ignore, but..
            • If an issue is delayed once, it'll have the magic text for increasing integration priority for future.
            Show
            Dan Poltawski added a comment - Makes sense - all issues have integration priority of 0, when something like delay added, we update the issue to +1 to it. Only thing I disagree with is the criteria for finding them: Using 'text' as metadata always makes me sick.. which I would ignore, but.. If an issue is delayed once, it'll have the magic text for increasing integration priority for future.
            Hide
            Eloy Lafuente (stronk7) added a comment -

            1) we can look exclusively for comments in the last week (and only those made by integrators). Of course we can also create a new label for that, but don't think it's really needed and, as far as the label is not sortable either, it does not add much value.

            2) we can reset the "priority" field back to 0 when CiBoT reopens an issue. So if it goes back to integration 1 week later, then the prioritization won't apply because of 1 (comment too old).

            Show
            Eloy Lafuente (stronk7) added a comment - 1) we can look exclusively for comments in the last week (and only those made by integrators). Of course we can also create a new label for that, but don't think it's really needed and, as far as the label is not sortable either, it does not add much value. 2) we can reset the "priority" field back to 0 when CiBoT reopens an issue. So if it goes back to integration 1 week later, then the prioritization won't apply because of 1 (comment too old).
            Hide
            Dan Poltawski added a comment -

            Makes sense.

            Show
            Dan Poltawski added a comment - Makes sense.
            Hide
            Eloy Lafuente (stronk7) added a comment - - edited

            I've created the "Integration priority" custom field (numeric, defaults to 0). And it has been added to both the "start review" and "stop review" screens. Mainly for testing purposes, once working ok surely we'll move them away from those screens and it will be 100% under CiBoT control.

            Show
            Eloy Lafuente (stronk7) added a comment - - edited I've created the "Integration priority" custom field (numeric, defaults to 0). And it has been added to both the "start review" and "stop review" screens. Mainly for testing purposes, once working ok surely we'll move them away from those screens and it will be 100% under CiBoT control.
            Hide
            Eloy Lafuente (stronk7) added a comment - - edited

            Crap, have to pre-fill all existing issues with 0 because ordering ("Integration priority" DESC) is not working as expected, putting all nulls before the 2 issues having a number there (see the end of the list):

            https://tracker.moodle.org/issues/?jql=project%20%3D%20MDL%20AND%20(%22Currently%20in%20integration%22%20is%20not%20EMPTY%20OR%20status%20in%20(%22Integration%20review%20in%20progress%22%2C%20%22Waiting%20for%20testing%22%2C%20%22Testing%20in%20progress%22%2C%20%22Problem%20during%20testing%22%2C%20Tested))%20ORDER%20BY%20%22Integration%20priority%22%20DESC%2C%20priority%20DESC%2C%20votes%20DESC%2C%20%22Last%20comment%20date%22%20ASC

            Edited: New issues should not be a problem as far as the custom field has been defined with a 0 default that should be applied on creation. Will check, anyway.

            Show
            Eloy Lafuente (stronk7) added a comment - - edited Crap, have to pre-fill all existing issues with 0 because ordering ("Integration priority" DESC) is not working as expected, putting all nulls before the 2 issues having a number there (see the end of the list): https://tracker.moodle.org/issues/?jql=project%20%3D%20MDL%20AND%20(%22Currently%20in%20integration%22%20is%20not%20EMPTY%20OR%20status%20in%20(%22Integration%20review%20in%20progress%22%2C%20%22Waiting%20for%20testing%22%2C%20%22Testing%20in%20progress%22%2C%20%22Problem%20during%20testing%22%2C%20Tested))%20ORDER%20BY%20%22Integration%20priority%22%20DESC%2C%20priority%20DESC%2C%20votes%20DESC%2C%20%22Last%20comment%20date%22%20ASC Edited: New issues should not be a problem as far as the custom field has been defined with a 0 default that should be applied on creation. Will check, anyway.
            Show
            Eloy Lafuente (stronk7) added a comment - Ok, have updated current under integration issues to "Integration priority" to 0 and now the ordering is working as expected, putting the prioritized higher: https://tracker.moodle.org/issues/?jql=project%20%3D%20MDL%20AND%20(%22Currently%20in%20integration%22%20is%20not%20EMPTY%20OR%20status%20in%20(%22Integration%20review%20in%20progress%22%2C%20%22Waiting%20for%20testing%22%2C%20%22Testing%20in%20progress%22%2C%20%22Problem%20during%20testing%22%2C%20Tested))%20ORDER%20BY%20%22Integration%20priority%22%20DESC%2C%20priority%20DESC%2C%20votes%20DESC%2C%20%22Last%20comment%20date%22%20ASC Will check what happens with new issues right now... ciao
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Perfect, seems that all new issues being created since the new field was created are getting the correct 0 value:

            https://tracker.moodle.org/issues/?jql=project%20%3D%20MDL%20AND%20%22Integration%20priority%22%20is%20not%20EMPTY%20and%20%22Currently%20in%20integration%22%20is%20empty%20ORDER%20BY%20key%20ASC

            So, for all the currently unresolved issues... I'm going to perform some bulk/script execution to fill all them, or perhaps will look if it's possible to set them to 0 when they are moved to current integration and the field is empty for them. Surely that will make the trick is a progressive and safe way.

            Show
            Eloy Lafuente (stronk7) added a comment - Perfect, seems that all new issues being created since the new field was created are getting the correct 0 value: https://tracker.moodle.org/issues/?jql=project%20%3D%20MDL%20AND%20%22Integration%20priority%22%20is%20not%20EMPTY%20and%20%22Currently%20in%20integration%22%20is%20empty%20ORDER%20BY%20key%20ASC So, for all the currently unresolved issues... I'm going to perform some bulk/script execution to fill all them, or perhaps will look if it's possible to set them to 0 when they are moved to current integration and the field is empty for them. Surely that will make the trick is a progressive and safe way.
            Hide
            Eloy Lafuente (stronk7) added a comment -

            All Integration Dashboard filters are now using that new field to better control the integration priorities.

            TODO: Build the CiBoT automatisms, namely:

            1) Set to 1 all the issues awaiting integration that have been delayed (looking for comment contents by date = last week).
            2) Set to 1 all the security issues awaiting integration.
            3) Set back to 0 all the reopened issues.
            4) Set to 0 all the issues awaiting integration and being empty.

            Ciao

            Show
            Eloy Lafuente (stronk7) added a comment - All Integration Dashboard filters are now using that new field to better control the integration priorities. TODO: Build the CiBoT automatisms, namely: 1) Set to 1 all the issues awaiting integration that have been delayed (looking for comment contents by date = last week). 2) Set to 1 all the security issues awaiting integration. 3) Set back to 0 all the reopened issues. 4) Set to 0 all the issues awaiting integration and being empty. Ciao
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Ref: While thinking about this, I realized MDLSITE-3769 may be really possible once we all start using the standard delayed message.

            Show
            Eloy Lafuente (stronk7) added a comment - Ref: While thinking about this, I realized MDLSITE-3769 may be really possible once we all start using the standard delayed message.
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Note: If MDLSITE-3769 is verified to be working ok... we'll use the very same search to detect the delayed issues that we must set their priority to 1 (point 1) of the above TODO.

            Show
            Eloy Lafuente (stronk7) added a comment - Note: If MDLSITE-3769 is verified to be working ok... we'll use the very same search to detect the delayed issues that we must set their priority to 1 (point 1) of the above TODO.

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Development