Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-51816

Clustered MUC memcached hangs after exhausting max connections

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Waiting for peer review
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.9.2, 3.5.6, 3.6, 3.7
    • Fix Version/s: None
    • Component/s: Caching
    • Testing Instructions:
      Hide

      Configure two moodle instances cachestore_memcached

      Instance A:

      • name: localmemcached
      • prefix: prefixa
      • server: 127.0.0.1

      Instance B:

      • name: localmemcached
      • prefix: prefixb
      • server: 127.0.0.1

      Define in both instances  "localmemcached" as the default application cache store.

      Without this patch both instances will user the same cache and, for example, if it's theme is configured with different colours on of them will loose it's configuration.

      Show
      Configure two moodle instances cachestore_memcached Instance A: name: localmemcached prefix: prefixa server: 127.0.0.1 Instance B: name: localmemcached prefix: prefixb server: 127.0.0.1 Define in both instances  "localmemcached" as the default application cache store. Without this patch both instances will user the same cache and, for example, if it's theme is configured with different colours on of them will loose it's configuration.
    • Affected Branches:
      MOODLE_29_STABLE, MOODLE_35_STABLE, MOODLE_36_STABLE, MOODLE_37_STABLE
    • Pull Master Branch:

      Description

      We have a Moodle cluster with 3 web nodes that holds 10 different Moodle instances.
      We are using clustered memcached MUC configuration.
      Combining the 10 servers users count, we get around 80K users.
      So I get roughly just a few hundreds of concurrent users at a time.

      We started a local memcached on each web with 6K max connection, and we keep getting the Moodle web servers hang after exhausting max connections. every time it hangs with raise it. Now, we are at 22K. and with no comfortable feeling that it is the end.

      Looking at the code, I did not see any connection pool that reuses open connection and also no timeout configuration for unused connection.

      I am not sure if it is a Moodle memcached MUC issue or a php-memcached or a memcached issue. Please advise.

        Attachments

          Activity

            People

            • Assignee:
              tusefomal Ferran Recio
              Reporter:
              nadavkav Nadav Kavalerchik
              Participants:
              Component watchers:
              Matteo Scaramuccia, Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Sara Arjona (@sarjona), Víctor Déniz Falcón
            • Votes:
              7 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated: