Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-56675

memcached getAllKeys broken from memcached version 1.4.23

XMLWordPrintable

    • MOODLE_31_STABLE, MOODLE_32_STABLE
    • MOODLE_31_STABLE, MOODLE_32_STABLE
    • MDL-56675-master
    • Hide

      This issue needs testing on PHP 5.6 and PHP 7+.
      Only PHP56 is affected by this issue.

      Essentially this change disables the memcached MUC store if it is configured to be shared, and the incompatability is detected.

      To test you need to install:

      1. a recent version of memcached and an old version (older than 1.4.23)
      2. libmemcached 1.0.18 (the only release in the past 3 years)
      3. The latest versions of php-memcached (2.2.0 for PHP <7; 3.0.X for PHP >= 7)
      Testing
      1. Start memcached four times on different ports:
        • 2 on the old version
        • 2 on the new version
      2. Configure the memcached test instances (you'll need to change which servers are in use for each test) 

        define('TEST_CACHESTORE_MEMCACHED_TESTSERVERS', "localhost:11211\nlocalhost:11212");

      3. Run the cachestore_memcached_testsuite phpunit testsuite
      4. You should expect to see the following:
      PHP Version 1 x Memcached <= 1.4.22 1 x Memcached > 1.4.23 1 x Memcached <= 1.4.22 + 1 x Memcached > 1.4.23 2 x Memcached <= 1.4.22 2 x Memcached > 1.4.23
      PHP 56 With clustered test skipped With 2 tests (clustered + multiuse) skipped With multi-use test skipped With multi-use test skipped
      PHP 70 With clustered test skipped With clustered test skipped
      Show
      This issue needs testing on PHP 5.6 and PHP 7+. Only PHP56 is affected by this issue. Essentially this change disables the memcached MUC store if it is configured to be shared, and the incompatability is detected. To test you need to install: a recent version of memcached and an old version (older than 1.4.23) libmemcached 1.0.18 (the only release in the past 3 years) The latest versions of php-memcached (2.2.0 for PHP <7; 3.0.X for PHP >= 7) Testing Start memcached four times on different ports: 2 on the old version 2 on the new version Configure the memcached test instances (you'll need to change which servers are in use for each test) define('TEST_CACHESTORE_MEMCACHED_TESTSERVERS', "localhost:11211\nlocalhost:11212"); Run the cachestore_memcached_testsuite phpunit testsuite You should expect to see the following: PHP Version 1 x Memcached <= 1.4.22 1 x Memcached > 1.4.23 1 x Memcached <= 1.4.22 + 1 x Memcached > 1.4.23 2 x Memcached <= 1.4.22 2 x Memcached > 1.4.23 PHP 56 With clustered test skipped With 2 tests (clustered + multiuse) skipped With multi-use test skipped With multi-use test skipped PHP 70 With clustered test skipped With clustered test skipped

      It's become apparent during testing of MDL-56273 that the memcached server breaks silently for us and has done since memcached version 1.4.23 when they rewrote their slab behaviour.

      What does this mean: Well, because we use getAllKeys to purge caches on shared instances, and getAllKeys returns false in this error situation, which evaluates to empty, and which means there are no keys, we never purge the memcached cache.

      Since we don't have any error checking, we silently accept all of this.

      This is potentially a major problem as the cache is constantly left stale if it has been configured to be shared.

      This functionality was introduced by MDL-48506, and MDL-53967 has further hidden the issue.

      I would propose that we remove the memcached plugin from core ASAP as there has been no movement on the issue in https://github.com/php-memcached-dev/php-memcached/issues/203

      It looks like this is down to the way in which the php memcached driver fetchs all keys - it blindly interrogates all of the slabs from 0 to 63. However, 63 is an invalid slab and with the rewrite of slab support in memcached 1.4.23, this errors in memcached instead of exiting.

      Really this is a bug in php-memcached, as a result of a change of protocol response in memcached as a whole.

      This has been broken since April 2015.

      The real problems for us are:

      1. we do not check the result of hasAllKeys properly
      2. we continue to use the cache without purging it
      3. we have no way to inform admins that it's broken

            dobedobedoh Andrew Lyons
            dobedobedoh Andrew Lyons
            John Okely John Okely
            David Monllaó David Monllaó
            Ryan Wyllie Ryan Wyllie
            Votes:
            1 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved:

                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.