Uploaded image for project: 'Moodle Community Sites'
  1. Moodle Community Sites
  2. MDLSITE-6324

Run the release process towards the creation of all the 311_STABLE stuff

    XMLWordPrintable

    Details

      Description

      Important note

      This issue includes the information to both start (like for 3.10) and continue (like for 3.11) a new parallel development cycle. Please keep it updated with any extra information. And use it (clone) for future cases.

      This is a clone of MDLSITE-6220, where the same (or similar) actions were performed to the creation of the MOODLE_310_STABLE branch.

      In this case, here we'll be creating the MOODLE_311_STABLE branch (generically named MOODLE_XYZ along the document for easier future cloning).

      Goal

      The goal of this issue is to end with all the systems (git, tracker, jenkins, codebase...) ready for working with the new MOODLE_XYZ_STABLE development branch (despite its name).

      Requirements:

      • Wait for the on-sync period of the previous release to be finished. Then, ASAP, proceed to create the new branch.
      • Base branch to split from: When starting the parallel development period (with MOODLE_XY(Z-1)_STABLE) we branched from master. That's the branch from which, now, to continue with the parallel development period, we'll be creating the new MOODLE_XYZ_STABLE branch.
      • To be able to perform the whole process, some permissions are needed:
        • Write access to various moodlehq repositories (gitlab and github) are needed.
        • Integration permissions to be able to handle versions in the tracker.
        • Filter creation (filter_manager credentials required).
        • Admin permissions in the tracker to be able to create new fields and assign them to various screens and, finally, reindex.

      Process:

      Note that some of the tasks listed below are not always needed, some are only required when the initial parallel development period was started). The list below aims to be complete (as a reference for future parallel dev periods), and will include comments about when each task was applied for current 311 or no.

      A) Run a good part of the release process (always).

      This step is the one effectively creating the git branches, track fields, CI new jobs, adjusting everything for the branch being created (MOODLE_XYZ_STABLE). Note that it excludes any step related with releasing any version, because we aren't doing that now, we are just creating a new branch.

      Here there is a complete list of all the actions to perform (when an action corresponds to some exact step in the release process doc, it's referenced):

      Initial steps
      1. integration.git
        • Ensure that weeklies have been already released (their windows packages generated... @ 09:00 AU aprox - all integrators receive notifications about that).
        • Fetch and merge / pull all changes to local integration.git clone.
        • In local integration.git clone, create the new MOODLE_XYZ_STABLE branch, branching from the "base branch to split from" decided in the requirements.
        • Push the the new branch to origin integration.git
      2. security git:
        • Fetch and merge / pull all changes to local security.git clone.
        • Follow exactly the same steps than the integration.git ones, but with these 2 branches: MOODLE_XYZ_STABLE and lastbased-MOODLE_XYZ_STABLE (each one branching from its own - and different - "base branch to split from".
        • Verify that any security issue that may exist in the previous base branch is also in the new MOODLE_XYZ_STABLE one.
        • Push the the new branch to origin security
      Steps from "a week prior"
      1. Logged in as filter_manager, in the Tracker, review all the filters and duplicate as many as needed to point to the new created branch. (step #1 in the release process).
      2. Create the needed PRs for CI repositories (nightlyjobs and moodle-ci-runner), as stated in the process. (step #13)
      Steps from "Packaging"
      1. Configure mdlrelease locally to know about the new development branch). Push changes upstream. Ask all the team to update their mdlrelease clones. (step #4).
      2. Update the old CI server jobs as stated in the process. (step #4).
        • Configure the "05. Check version files" jobs to define the interval of allowed versions for each branch (310 - now fixed to release date, 311 - interval since branch day till release day and master - fixed to 2w after last parallel release date ). (step #4).
        • Configure the "25. Compare databases ..." master job to verify upgrade from the just created XYZ branch. (step #4).
      3. Merge the needed PRs for CI repos, as stated in the process. Note that the 4th PR (enable langupgrade tests) cannot be merged until the new language packs are available, see section E) below. (step #4).
      4. In new CI infrastructure, clone all jobs from master to new view as stated in the process.(step #4).
      5. Share with workplace and mobile about the new branch, in case they want jobs running against it.
      6. Create the corresponding windows packager (@ downloads) and modify stats.php as stated in the process. (step #8).
      7. Create the new "Pull branch" and "Pull diff" fields in the tracker and add them to all screens and CI jobs (step #10).
      8. Edit all the jobs commented in that step (#10) to meet the new fields and new branch just created.
      9. Create an MDL issue to integrate the minimal changes needed for the new branch to work. Note that it's different if we are continuing an existing parallel cycle (like this issue case is) or if we are starting a new parallel development cycle:
        • For an existing cycle, see MDL-70229 (really simple, just a few changes in version, backup and travis are needed).
        • For a new cycle, see MDL-69475 (it needs all the changes applied for an existing PLUS plan a mega-bump of all master versions to avoid any overlapping of versions).
      10. Once the issue in the previous point has been integrated and all tests are passing... run the mdlrelease process (prerelease and release) only for that new branch (--branch MOODLE_XYZ_STABLE), that will create the branch @ moodle.git and other upstream clones.
      11. Protect the new branch in github interface. (step #14)
      12. Enable the new development branch @ downloads, see the changes to cfg.php performed by c1450db & 246ed79 @ serverscripts.git
      13. (optional) To force the generation of the first packages for the branch to be built, run, as vh-downloadmoodleorg the /var/www/vhosts/download.moodle.org/bin/moodle-package-update-local script at downloads, check output.
        1. Note1: If not executed, packages will be generated when rolling next weeklies, together with the rest of branches.
        2. Note2: You can check that they have been created @ https://download.moodle.org/stableXYZ/ (zip, tgz and checksums)
        3. Note3: Important! The packages (normal and windows) won't show @ downloads.moodle.org yet, that's normal, they are made visible - aka, released - by the release automatic execution script (see next point).
      Steps from "Release day"
      1. The execution @ 09:00 AU (any day, because they correspond to a dev branch) will process the new branch and downloads for it should be prepared automatically. (#1)
      2. Verify all looks in place (as a development version). (#2)
      Steps from "2 weeks after release"
      1. moodle-behat-extension (#1)
        • Create the new branch (MOODLE_XYZ_STABLE) @ moodle-behat-extension.
        • Tag its HEAD with 3.XYZ, verify it's auto-deployed @ packagist.
        • Create an issue (like MDL-70192 for 311_STABLE), to make core to point to it.
      Final steps
      1. Announce the creation of the new branch @ HQ and general dev chats.
      2. Announce the creation the new branch, explaining versions and some basic rules in the next integration exposed post (311 example)

      B) Some details to confirm, so everything is ready (always).

      Some of them need to be run always, others only sometimes (or partially). Create clones and link them here when they need to be run for the new MOODLE_XYZ_STABLE branch just created. Here there is a list (still, please check all them):

      • MDLSITE-6227 : Ensure that mdk is ready for the new branch. Always needs to be checked. For 3.11, changes were already done, so no new issue. The linked issue corresponds to 3.10.
      • MDLSITE-6230 : Ensure that external testers have their environment ready. Always needs to be checked. Nothing especial for 3.11, the work was done, for 3.10, in the linked issue.

      C) Versions handling in the tracker (not always, only when new parallel cycle begins).

      These steps don't need to be run when a parallel development period continues (like now, for 311), but only when such a period begins (like it was for 310). It's important to perform all the following actions together, so nothing is created/edited by anyone else while proceeding with them. Following list details them with links to the issue where they were performed:

      1. In the tracker, rename current version in master to parallel development created (example 4.0 => 3.10). Both MDL and CONTRIB. That way, everything scheduled for 4.0 was reassigned automatically to 3.10.
      2. Together with the above, create the new real master version (scheduled for when the parallel period ends (example new 4.0 created, for Nov. 2021). That way we get a new 4.0 version where issues really targeting it can be created.
      3. Immediately after previous point, modify all the issues that are commenting about the original master version just rename to comment about its replacement (and also next versions, especially deprecation epics rearrange). For example, for 3.10 we did modify 4.0 => 3.10, 4.1 => 3.11, 4.2 => 4.0, 4.3 => 4.1... all future cases. Here you can find more details about the searches performed . Verify that the affect and fix version match the new comments.

      D) Other changes to core (not always, only when new parallel cycle begins).

      These steps don't need to be run when a parallel development period continues (like now, for 311), but only when such a period begins (like it was for 310). They are changes in core, corresponding/following all the changes performed in the Tracker around versions. Following list details them with links to the issue where they were performed:

      1. MDL-69479 : Change all the existing upgrade notes to the new branches (example 4.0 => 3.10). That way everything that originally was changed for 4.0 now will point to 3.10, matching the changes in the tracker (C1 above).
      2. MDL-69521 : Change all the comments in core to the new branches (4.0 => 3.10, 4.1 => 3.11, 4.2 => 4.0, 4.3 => 4.1... all future cases). That way everything will match the moved versions in the tracker (C3 above).

      E) Other tasks (always)

      1. Ask for the XYZ language packs to be created ASAP.
      2. Once the language packs are available the "lang upgrade" tests can be enabled back in the CI servers by merging the 4th pull request created in section A).

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              stronk7 Eloy Lafuente (stronk7)
              Reporter:
              stronk7 Eloy Lafuente (stronk7)
              Participants:
              Component watchers:
              Eloy Lafuente (stronk7), Marina Glancy, Sander Bangma, Eloy Lafuente (stronk7)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: