Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-77215

Allow db migrate tool to be parallelized

    XMLWordPrintable

Details

    Description

      If you are doing a large migration and the bottle neck isn't the databases themselves we should be able to improve the wall clock time of the total migration by doing different sets of tables in parallel. 

      Proposing to add cli arguments to the migrate script so that individual tables or sets of tables can be done and then you can spawn N migrate scripts.

      admin/tool/dbtransfer/cli/migrate.php

      I think a simple table name match would be simple and flexible ie:

      php migrate.php --tables="a*"

      would do all tables starting with a. It would also match at the start of the table name. You could just shard on each letter of the alphabet easily this way, or you could be more specific and target some tables.

      This might need to be tied to the component rather than the raw table name, because when we clean up the logs the logs are tied to component.

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            brendanheywood Brendan Heywood
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Clockify

                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.