With the new AI subsystem introduced in MDL-80889, we need to collect usage data from registered sites for this new functionality. This will be valuable in understanding the usage of the subsystem in LMS and be a necessary source of data for further development as we continue to improve and extend the AI functionality in LMS.

The `ai_action_register` table that is introduced by MDL-80889 contains the raw data we want. Processing will be required to aggregate and extract only the required data. Primarily we are after information about:

• How many actions were processed in a given time period (time between registration data update periods ~ 1 week),
• What provider was used for the action
• What was the success rate of the calls
• What was the majority error code for failures
• The time range used to collect the data

In JSON the extracted data will look something like:

{

    "OpenAI API Provider": {

        "generate_text": {

            "success_count": 3,

            "fail_count": 1,

            "predominant_error": 403,

            "average_time": 2

        },

        "generate_image": {

            "success_count": 1,

            "fail_count": 1,

            "predominant_error": 403,

            "average_time": 20

        }

    },

    "time_range": {

        "timefrom": 1723001586,

        "timeto": 1723606386

    }

},

{

    "Azure": {

        "generate_text": {

            "success_count": 3,

            "fail_count": 1,

            "predominant_error": 403,

            "average_time": 2

        },

        "generate_image": {

            "success_count": 1,

            "fail_count": 1,

            "predominant_error": 403,

            "average_time": 20

        }

    },

    "time_range": {

        "timefrom": 1723001586,

        "timeto": 1723606386

    }

}

In the above example the data is grouped by provider, with the actions underneath. It could also be grouped by action and then provider. There are going to be more actions than providers in a standard install.

The specified metrics we want to collect above don't include any, but to be extra clear this work doesn't extract or transmit any personal data (PII).