Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-70687

Redis session lock expiration should default shorter than session timeout

    XMLWordPrintable

Details

    • MOODLE_39_STABLE
    • MDL-70687_default-redis-session-lock-expiration-to-php-execution-time
    • Hide

      Setup

      1. Install the PHP Redis extension (replace "X" as your server's PHP version, e.g. 8.0)

        sudo apt update && sudo apt install -y phpX-redis

      2. Restart your web server.
      3. Confirm the redis PHP extension is enabled: "php -i | grep redis" - You should see a bunch of lines related to redis.
      4. Install redis. e.g. using Docker

        docker run --name redis -p 6379:6379 -d redis

      5. Setup redis as the session handler, e.g. copy the following to your config.php:

        $CFG->session_handler_class = '\core\session\redis';
        $CFG->session_redis_host = '127.0.0.1';
        $CFG->session_redis_port = 6379;
        $CFG->session_redis_acquire_lock_timeout = 2;

      6. Configure session timeout in config.php

        $CFG->sessiontimeout = 30;

      7. Configure the max_execution_time in your php.ini

        max_execution_time = 60

      1. Install the session breaker plugin to help with testing:

        git clone https://github.com/catalyst/moodle-tool_sessionbreaker/ admin/tool/sessionbreaker

      Before applying the patch

      Test 1
      1. Visit this page, which will lock the session for 2 minutes (which is way longer than the configured session timeout): /admin/tool/sessionbreaker/bad.php?wait=120
      2. In a second tab immediately load this page: /admin/tool/sessionbreaker/good.php
      3. Confirm after roughly 2, 15 and 25 seconds it always throws an exception.

        Unable to obtain lock for session id [...]

      4. Confirm after 35 seconds it no longer throws an exception. It shows that the sessiontimeout is respected.
      Test 2
      1. Change the session timeout in config.php

        $CFG->sessiontimeout = 60;

      2. Change the max_execution_time in your php.ini

        max_execution_time = 30

      3. Restart your web server.
      4. Visit this page, which will lock the session for 2 minutes (which is way longer than the configured session timeout): /admin/tool/sessionbreaker/bad.php?wait=120
      5. In a second tab immediately load this page: /admin/tool/sessionbreaker/good.php
      6. Confirm after roughly 2, 15, 25 and 35 seconds it always throws an exception.

        Unable to obtain lock for session id [...]

      7. Confirm after 60 seconds it no longer throws an exception. It shows that the sessiontimeout is still respected, and the max_execution_time has no effect.

      After applying the patch

      Test 3
      1. Change the session timeout in config.php

        $CFG->sessiontimeout = 30;

      2. Change the max_execution_time in your php.ini

        max_execution_time = 60

      3. Visit this page, which will lock the session for 2 minutes (which is way longer than the configured session timeout): /admin/tool/sessionbreaker/bad.php?wait=120
      4. In a second tab immediately load this page: /admin/tool/sessionbreaker/good.php
      5. Confirm after roughly 2, 15 and 25 seconds it always throws an exception.

        Unable to obtain lock for session id [...]

      6. Confirm after 35 seconds it no longer throws an exception. It shows that the sessiontimeout is respected, which is lower than the max_execution_time.
      Test 4
      1. Change the session timeout in config.php

        $CFG->sessiontimeout = 60;

      2. Change the max_execution_time in your php.ini

        max_execution_time = 30

      3. Restart your web server.
      4. Visit this page, which will lock the session for 2 minutes (which is way longer than the configured session timeout): /admin/tool/sessionbreaker/bad.php?wait=120
      5. In a second tab immediately load this page: /admin/tool/sessionbreaker/good.php
      6. Confirm after roughly 2, 15 and 25 seconds it always throws an exception.

        Unable to obtain lock for session id [...]

      7. Confirm after 35 seconds it no longer throws an exception. It shows that the max_execution_time is now respected, which is lower than the sessiontimeout.
      Show
      Setup Install the PHP Redis extension (replace " X " as your server's PHP version, e.g. 8.0) sudo apt update && sudo apt install -y phpX-redis Restart your web server. Confirm the redis PHP extension is enabled: " php -i | grep redis " - You should see a bunch of lines related to redis. Install redis. e.g. using Docker docker run --name redis -p 6379:6379 -d redis Setup redis as the session handler, e.g. copy the following to your config.php : $CFG ->session_handler_class = '\core\session\redis' ; $CFG ->session_redis_host = '127.0.0.1' ; $CFG ->session_redis_port = 6379; $CFG ->session_redis_acquire_lock_timeout = 2; Configure session timeout in config.php $CFG ->sessiontimeout = 30; Configure the max_execution_time in your php.ini max_execution_time = 60 Install the session breaker plugin to help with testing: git clone https://github.com/catalyst/moodle-tool_sessionbreaker/ admin/tool/sessionbreaker Before applying the patch Test 1 Visit this page, which will lock the session for 2 minutes (which is way longer than the configured session timeout): /admin/tool/sessionbreaker/bad.php?wait=120 In a second tab immediately load this page: /admin/tool/sessionbreaker/good.php Confirm after roughly 2, 15 and 25 seconds it always throws an exception. Unable to obtain lock for session id [...] Confirm after 35 seconds it no longer throws an exception. It shows that the sessiontimeout is respected. Test 2 Change the session timeout in config.php $CFG ->sessiontimeout = 60; Change the max_execution_time in your php.ini max_execution_time = 30 Restart your web server. Visit this page, which will lock the session for 2 minutes (which is way longer than the configured session timeout): /admin/tool/sessionbreaker/bad.php?wait=120 In a second tab immediately load this page: /admin/tool/sessionbreaker/good.php Confirm after roughly 2, 15, 25 and 35 seconds it always throws an exception. Unable to obtain lock for session id [...] Confirm after 60 seconds it no longer throws an exception. It shows that the sessiontimeout is still respected, and the max_execution_time has no effect. After applying the patch Test 3 Change the session timeout in config.php $CFG ->sessiontimeout = 30; Change the max_execution_time in your php.ini max_execution_time = 60 Visit this page, which will lock the session for 2 minutes (which is way longer than the configured session timeout): /admin/tool/sessionbreaker/bad.php?wait=120 In a second tab immediately load this page: /admin/tool/sessionbreaker/good.php Confirm after roughly 2, 15 and 25 seconds it always throws an exception. Unable to obtain lock for session id [...] Confirm after 35 seconds it no longer throws an exception. It shows that the sessiontimeout is respected, which is lower than the max_execution_time . Test 4 Change the session timeout in config.php $CFG ->sessiontimeout = 60; Change the max_execution_time in your php.ini max_execution_time = 30 Restart your web server. Visit this page, which will lock the session for 2 minutes (which is way longer than the configured session timeout): /admin/tool/sessionbreaker/bad.php?wait=120 In a second tab immediately load this page: /admin/tool/sessionbreaker/good.php Confirm after roughly 2, 15 and 25 seconds it always throws an exception. Unable to obtain lock for session id [...] Confirm after 35 seconds it no longer throws an exception. It shows that the max_execution_time is now respected, which is lower than the sessiontimeout .

    Description

      Currently the redis session lock expiration defaults to the session timeout. The session timeout is typically multiple hours.

      If a problem leads to a redis session lock not getting released properly, any subsequent request by a user in that session will wait to acquire the lock and eventually fail. The only way for the user to recover is to either delete their cookies or to wait several hours until the session times out.

      A reasonable session lock expiration is the maximum time a PHP request can be processed for. After this time expires without the lock being released it's clear that the php process must have ended without releasing it (there is potential for some system time not being counted, so to be on the safe side one could increase the default a bit over this minimum). Waiting any longer just causes more errors and problems for the users, so is not advisable.

      So this patch changes the default for session lock expiration in redis to the PHP max execution time. It can still be overwritten through $CFG->session_redis_lock_expire 

      I found the problem on 3.9.4 but it's still an issue on master.

      Attachments

        Activity

          People

            Daniel Ziegenberg Daniel Ziegenberg
            naderman Nils Adermann
            Jordi Pujol-Ahulló Jordi Pujol-Ahulló
            Jake Dallimore, Mathew May, Mihail Geshoski, Matteo Scaramuccia, Andrew Lyons, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze, Stevani Andolo
            Votes:
            16 Vote for this issue
            Watchers:
            19 Start watching this issue

            Dates

              Created:
              Updated: