Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-82977

AI: Provider instances

XMLWordPrintable

    • MOODLE_500_STABLE
    • MOODLE_500_STABLE
    • main_MDL-82977
    • Hide

      Test managing instances

      (These tests are largely covered by behat)

      Create, delete  and configure provider instances:

      1. Log into the site as an admin
      2. Navigate to Site administration > AI >  AI providers
      3. Confirm there are no providers present and "Nothing to display" notifaction is visible
      4. Click the "Create a new provider instance button".
      5. Confirm "OpenAI API Provider" is the default value for the "Choose AI Provider plugin " form value
      6. Enter a name for the provider instance
      7. Enter a valid api key into the "OpenAI API key" form field
      8. Enter a valid organisation ID into the "OpenAI organization ID" form field
      9. Click the "Create Instance button".
      10. Confirm that a "Provider instance created" success notification is displayed
      11. Confirm that there is now a table displaying the details for the configured provider instance
      12. Confirm the provider instance is disabled (row is greyed out in table).
      13. Click the "enable" switch for the provider instance.
      14. Confirm that a "enabled" success notification is displayed
      15. Confirm the provider instance is enabled (row is NOT greyed out in table).
      16. Click the "Create a new provider instance button".
      17. Choose "Azure AI API Provider" for the "Choose AI Provider plugin " form value.
      18. Enter a name for the provider instance
      19. Enter a valid api key into the "Azure AI API key" form field
      20. Enter a valid endpoint URL into the "Azure AI API key endpoint" form field
      21. Click the "Create Instance button".
      22. Confirm that a "Provider instance created" success notification is displayed
      23. Click the "settings" link for the Azure AI provider instance.
      24. From the actions settings table click the "settings" link for the "Generate text" action from the "Action settings" table.
      25. Enter a valid deployment ID into the "Deployment ID" form field
      26. Click the "Save changes" button
      27. Confirm a "action settings updated" success notification is displayed

      Test Provider instance operation

      1. Log into the site as an admin
      2. Navigate to Site administration > AI providers
      3. Configure and configure and enable an AI Provider instance using the setup procedure from the previous test
      4. Navigate to Site administration > AI >  AI placements
      5. Enable the "Text editor placement" placement
      6. Navigate to the user profile of the logged in admin user
      7. Click "Edit profile" for the admin user
      8. Confirm that both "AI generate image" and "AI generate text" are available from the "Generate AI content" Text editor toolbar, in the description section of the profile
      9. Confirm AI text can be generated successfully
      10. Confirm AI images can be generated successfully

      Upgrade testing

      When an LMS instance is upgraded from 4.5 to this patch, existing configured providers are converted to provider instances.

      To test the upgrade:

      1. Install an LMS 4.5 instance.
      2. Log into the site as an admin
      3. Enable and configure the Azure AI provider plugin.
      4. Configure and don't enable the OpenAI provider plugin.
      5. Checkout and apply this code branch
      6. Run the LMS upgrade process to upgrade the code
      7. Confirm the site upgrades without any errors or warnings
      8. Log into the site as an admin
      9. Confirm that there are two configured provider instances. One Azure AI instance and one OpenAI instance
      10. Confirm the Azure AI instance is enabled and the OpenAI instance is disabled
      11. Confirm both instances have the same configuration as they did before the upgrade
      12. Test the providers are working (use the testing instructions from MDL-82627 as a guide)

      For further testing you can also test the upgrade process for providers that are not configured. In the case where a provider is disabled and not configured a provider instance will not be created on upgrade. What determines if a provider is "configured" is if there is an API key set.

      Show
      Test managing instances (These tests are largely covered by behat) Create, delete  and configure provider instances: Log into the site as an admin Navigate to Site administration > AI >  AI providers Confirm there are no providers present and "Nothing to display" notifaction is visible Click the "Create a new provider instance button". Confirm "OpenAI API Provider" is the default value for the "Choose AI Provider plugin " form value Enter a name for the provider instance Enter a valid api key into the "OpenAI API key" form field Enter a valid organisation ID into the "OpenAI organization ID" form field Click the "Create Instance button". Confirm that a "Provider instance created" success notification is displayed Confirm that there is now a table displaying the details for the configured provider instance Confirm the provider instance is disabled (row is greyed out in table). Click the "enable" switch for the provider instance. Confirm that a "enabled" success notification is displayed Confirm the provider instance is enabled (row is NOT greyed out in table). Click the "Create a new provider instance button". Choose "Azure AI API Provider" for the "Choose AI Provider plugin " form value. Enter a name for the provider instance Enter a valid api key into the "Azure AI API key" form field Enter a valid endpoint URL into the "Azure AI API key endpoint" form field Click the "Create Instance button". Confirm that a "Provider instance created" success notification is displayed Click the "settings" link for the Azure AI provider instance. From the actions settings table click the "settings" link for the "Generate text" action from the "Action settings" table. Enter a valid deployment ID into the "Deployment ID" form field Click the "Save changes" button Confirm a "action settings updated" success notification is displayed Test Provider instance operation Log into the site as an admin Navigate to Site administration > AI providers Configure and configure and enable an AI Provider instance using the setup procedure from the previous test Navigate to Site administration > AI >  AI placements Enable the "Text editor placement" placement Navigate to the user profile of the logged in admin user Click "Edit profile" for the admin user Confirm that both "AI generate image" and "AI generate text" are available from the "Generate AI content" Text editor toolbar, in the description section of the profile Confirm AI text can be generated successfully Confirm AI images can be generated successfully Upgrade testing When an LMS instance is upgraded from 4.5 to this patch, existing configured providers are converted to provider instances. To test the upgrade: Install an LMS 4.5 instance. Log into the site as an admin Enable and configure the Azure AI provider plugin. Configure and don't enable the OpenAI provider plugin. Checkout and apply this code branch Run the LMS upgrade process to upgrade the code Confirm the site upgrades without any errors or warnings Log into the site as an admin Confirm that there are two configured provider instances. One Azure AI instance and one OpenAI instance Confirm the Azure AI instance is enabled and the OpenAI instance is disabled Confirm both instances have the same configuration as they did before the upgrade Test the providers are working (use the testing instructions from MDL-82627 as a guide) For further testing you can also test the upgrade process for providers that are not configured. In the case where a provider is disabled and not configured a provider instance will not be created on upgrade. What determines if a provider is "configured" is if there is an API key set.
    • Hide

      Code verified against automated checks with warnings.

      Checked MDL-82977 using repository: https://github.com/mattporritt/moodle

      Should these errors be fixed?

      Built on: Tue Jan 21 09:53:33 PM UTC 2025

      Show
      Code verified against automated checks with warnings. Checked MDL-82977 using repository: https://github.com/mattporritt/moodle main (0 errors / 2 warnings) [branch: main_MDL-82977 | CI Job ] overview (0/0) , phplint (0/0) , phpcs (0/0) , js (0/0) , css (0/0) , phpdoc (0/0) , commit (0/0) , savepoint (0/0) , thirdparty (0/0) , externalbackup (0/2) , grunt (0/0) , shifter (0/0) , mustache (0/0) , gherkin (0/0) , Should these errors be fixed? Built on: Tue Jan 21 09:53:33 PM UTC 2025
    • Hide

      Launching automatic jobs for branch main_MDL-82977

      Built on: Tue Jan 21 05:44:56 AM UTC 2025

      Show
      Launching automatic jobs for branch main_ MDL-82977 https://ci.moodle.org/view/Testing/job/DEV.02%20-%20Developer-requested%20PHPUnit/17996/ PHPUnit (sqlsrv / complete) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/61477/ Behat (NonJS - boost and classic / complete) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/61478/ Behat (Firefox - boost / complete) --> known random. https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/61479/ Behat (Firefox - classic / complete) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/61480/ App tests (stable app version) / complete) --> unrelated app failures Built on: Tue Jan 21 05:44:56 AM UTC 2025
    • 6

      Add the ability to have more than one instance of a provider plugin.

      This will allow having providers for specific tasks. So, you can use a more efficient model for lightweight tasks like summarisation, and a more fully featured model for text generation. Another example is defaulting to using a cheaper model with a lower token limit, and then falling back to a more expensive model if a request is too large for the default model.

      Functionally, this will be similar to the way that SMS gateways can be added, see: MDL-81732

      As this will land after the initial implementation, for existing sites that have configured providers; they should be converted to Provider instances as part of the upgrade process

        1. MDL-82977-1-1.png
          MDL-82977-1-1.png
          784 kB
        2. MDL-82977-1-2.png
          MDL-82977-1-2.png
          1.21 MB
        3. MDL-82977-2.png
          MDL-82977-2.png
          586 kB
        4. MDL-82977 no breadcrumb.png
          MDL-82977 no breadcrumb.png
          93 kB

            matt.porritt@moodle.com Matt Porritt
            matt.porritt@moodle.com Matt Porritt
            Meirza Meirza
            Safat Shahin Safat Shahin
            Ron Carl Alfon Yu Ron Carl Alfon Yu
            Votes:
            3 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 4 days, 6 hours, 45 minutes
                4d 6h 45m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.