Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-68138

Setting smtpmaxbulk > 1 doesn't keep SMTP connection alive

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not a bug
    • Icon: Minor Minor
    • None
    • 3.8, 4.0.5, 4.1
    • Email
    • MOODLE_38_STABLE, MOODLE_400_STABLE, MOODLE_401_STABLE
    • MDL-68138-401
    • MDL-68138-master
    • Hide

      Prerequisites:

      1. Email delivery service + SMTP, e.g. Mailjet
      2. Disposable email service, e.g. Mailinator
      3. Fill in Moodle No-reply address (noreplyaddress) with noreply_mdl68138@mailinator.com. Please use a valid email address because we need to validate it later.

      Setup Mailjet SMTP:

      1. Create an account at Mailjet.com
      2. Get an SMTP account by clicking the "Retrieve your API credentials" button on the Configuration page (https://.../account/setup)
      3. To get Password (Secret Key). Click the gear icon and select Reset Secret Key
      4. At this step, you already have API Key as the SMTP user and Secret Key as SMTP Password. Below is the complete SMTP configuration:
        SMTP Host: in-v3.mailjet.com:587
        SMTP Security: TLS
        SMTP Auth Type: Login
        SMTP Username: <Mailjet API Key>
        SMTP Password: <Mailjet Secret Key>
        Read more at: https://dev.mailjet.com/smtp-relay/configuration/
      1. We need to add a sender address to validate. Set the email address on the Sender domains & addresses page (https://.../account/sender)
      2. At the bottom of the page, click Add a sender address and fill in the form with the below data:
        Label: No Reply
        Email: noreply_mdl68138@mailinator.com
        Select the Both/I don't know radio button
      1. Click Add to save the new sender address.
      2. Open noreply_mdl68138@mailinator.com by going to mailinator.com and filling in noreply_mdl68138 in the Enter Public Mailinator Inbox. Or click the URL https://www.mailinator.com/v4/public/inboxes.jsp?to=noreply_mdl68138.
      3. Select email from The Mailjet Team. Click "Activate now" in the email body.
      4. Back to the Mailjet Sender domains & addresses page. If the email address validation success you will see that the label status will turn from Pending to Active.

      Please take a break and have a bite of a snack before continuing.

      Instructions:

      1. Apply the patch.
      2. Edit config.php and put the followings config:
        $CFG->debugsmtp = true;
        $CFG->debugdeveloper = true;
      3. Setup SMTP Outgoing mail configuration on Site Administration (admin/settings.php?section=outgoingmailconfig):
        SMTP Host in-v3.mailjet.com:587
        SMTP Security: TLS
        SMTP Auth Type: Login
        SMTP Username: <Mailjet API Key>
        SMTP Password: <Mailjet Secret Key>
      4. Create 10 test users
      5. Download mailtest2.php (minor fixing from the original) in the Attachments.
      6. Copy the file into moodle/admin/cli directory
      7. Open your favourite terminal to run CLI. Go to Moodles' root directory.
      8. Run the script: 

        php admin/cli/mailtest2.php --emails=10 --smtpmaxbulk=2 | tee emails_10_2

      9. Run the script:

        grep -o "Connection: opened" emails_10_2 | wc -l

      10. Confirm. The opened connection is 5.
      11. Run the script: 

        grep -o "Connection: closed" emails_10_2 | wc -l

      12. Confirm. The closed connection is 5.
      13. Go to https://app.mailjet.com/stats. Confirm. All emails were in delivered status.
      14. Go to Mailinator.com. Confirm. All emails have been received successfully.

      Please repeat step 4 with different values:

      20 emails and smtpmaxbulk = 5

      php admin/cli/mailtest2.php --emails=20 --smtpmaxbulk=5 | tee emails_20_5

      Confirm. The open & closed connection is 4.

      10 emails and smtpmaxbulk is not set (default)

      php admin/cli/mailtest2.php --emails=10 | tee emails_10_1

      Confirm. The open & closed connection is 10. If we don't specify smtpmaxbulk, it will use the default value, 1.

       

      The result of opened and closed connection amount is the emails divided by the smtpmaxbulk. e.g. emails = 20, smtpmaxbulk = 5 then the result is 20/5 = 4 opened/closed connections.

      Please remember. Mailjet free plan limits a user to send a maximum of 200 emails per day. If the email was not delivered, there is a possibility that the quota has run out.

      Show
      Prerequisites: Email delivery service + SMTP, e.g. Mailjet Disposable email service, e.g. Mailinator Fill in Moodle No-reply address (noreplyaddress) with noreply_mdl68138@mailinator.com . Please use a valid email address because we need to validate it later. Setup Mailjet SMTP: Create an account at Mailjet.com Get an SMTP account by clicking the "Retrieve your API credentials" button on the Configuration page ( https://.../account/setup) To get Password (Secret Key). Click the gear icon and select Reset Secret Key At this step, you already have API Key as the SMTP user and Secret Key as SMTP Password. Below is the complete SMTP configuration: SMTP Host: in-v3.mailjet.com:587 SMTP Security: TLS SMTP Auth Type: Login SMTP Username: <Mailjet API Key> SMTP Password: <Mailjet Secret Key> Read more at: https://dev.mailjet.com/smtp-relay/configuration/ We need to add a sender address to validate. Set the email address on the Sender domains & addresses page ( https://.../account/sender ) At the bottom of the page, click Add a sender address and fill in the form with the below data: Label: No Reply Email: noreply_mdl68138@mailinator.com Select the Both/I don't know radio button Click Add to save the new sender address. Open noreply_mdl68138@mailinator.com by going to mailinator.com and filling in noreply_mdl68138 in the Enter Public Mailinator Inbox. Or click the URL https://www.mailinator.com/v4/public/inboxes.jsp?to=noreply_mdl68138 . Select email from The Mailjet Team. Click "Activate now" in the email body. Back to the Mailjet Sender domains & addresses page. If the email address validation success you will see that the label status will turn from Pending to Active. Please take a break and have a bite of a snack before continuing. Instructions: Apply the patch. Edit config.php and put the followings config: $CFG->debugsmtp = true; $CFG->debugdeveloper = true; Setup SMTP Outgoing mail configuration on Site Administration (admin/settings.php?section=outgoingmailconfig): SMTP Host in-v3.mailjet.com:587 SMTP Security: TLS SMTP Auth Type: Login SMTP Username: <Mailjet API Key> SMTP Password: <Mailjet Secret Key> Create 10 test users Download mailtest2.php (minor fixing from the original) in the Attachments. Copy the file into moodle/admin/cli directory Open your favourite terminal to run CLI. Go to Moodles' root directory. Run the script:  php admin /cli/mailtest2 .php --emails=10 --smtpmaxbulk=2 | tee emails_10_2 Run the script: grep -o "Connection: opened" emails_10_2 | wc -l Confirm . The opened connection is 5. Run the script:  grep -o "Connection: closed" emails_10_2 | wc -l Confirm . The closed connection is 5. Go to https://app.mailjet.com/stats . Confirm . All emails were in delivered status. Go to Mailinator.com. Confirm . All emails have been received successfully. Please repeat step 4 with different values: 20 emails and smtpmaxbulk = 5 php admin /cli/mailtest2 .php --emails=20 --smtpmaxbulk=5 | tee emails_20_5 Confirm . The open & closed connection is 4 . 10 emails and smtpmaxbulk is not set (default) php admin /cli/mailtest2 .php --emails=10 | tee emails_10_1 Confirm . The open & closed connection is 10 . If we don't specify smtpmaxbulk, it will use the default value, 1.   The result of opened and closed connection amount is the emails divided by the smtpmaxbulk . e.g. emails = 20, smtpmaxbulk = 5 then the result is 20/5 = 4 opened/closed connections. Please remember. Mailjet free plan limits a user to send a maximum of 200 emails per day. If the email was not delivered, there is a possibility that the quota has run out.
    • 6
    • Team Hedgehog Sprint 1.2, Team Hedgehog Sprint 1.3

      When using email_to_user() function and setting smtpmaxbulk > 1 the connection is always closed after sending each email.

      I attached mailtest script which will have to alter path to config.php

      Create a moodle test instance

      Make a xs test course from developer setting page

      Run script 

      sudo php admin/toolbox/cli/mailtest.php --smtpmaxbulk=100 | tee somefile

      Sending 100 emails with 'smtpmaxbulk' set at 1 against Moodle version 3.8
      Took (350.826794) seconds 

      Sending 100 emails with 'smtpmaxbulk' set at 100 against Moodle version 3.8
      Took (352.816759) seconds

      When hacking getmailer function to force SMTP to keepalive performance is better $prevkeepalive = true;

      https://github.com/moodle/moodle/blob/MOODLE_38_STABLE/lib/moodlelib.php#L5782

      Sending 100 emails with 'smtpmaxbulk' set at 100 against Moodle version 3.8
      Took (130.510947) seconds

       

        1. currentcondition.txt
          21 kB
        2. mailtest_buffer.php
          2 kB
        3. mailtest.php
          2 kB
        4. mailtest2.php
          2 kB
        5. smtpmaxbulk2_email4.txt
          20 kB
        6. smtpmaxbulk5_email20.txt
          90 kB

            meirza.arson@moodle.com Meirza
            twilliams Troy Williams
            Safat Shahin Safat Shahin
            Huong Nguyen Huong Nguyen
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 days, 34 minutes
                2d 34m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.