Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-69121

Allow redis session store to use zip or zStd for compression like redis MUC

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide
      Prerequisite
      1. Install Redis service (daemon) on your Moodle server (see https://docs.moodle.org/37/en/Redis_cache_store#Installing_Redis_server) if you do not already have it.
      2. Install the PHP extension Redis (see https://docs.moodle.org/37/en/Redis_cache_store#Installing_Redis_php_driver) if you do not already have it.
      3. Start the Redis daemon (eg. redis-server /usr/local/etc/redis.conf).
      4. Install the PHP extension zstd see https://github.com/kjdev/php-ext-zstd for information on ways to install this by reading the README.
      5. Create a couple test courses with activities. This will just be needed to navigate around and simulate a session. The Moodle tool for creating courses is useful to quickly make test courses (/admin/tool/generator/maketestcourse.php)
       Configuration

      Add the following to the config.php file

      $CFG->session_handler_class = '\core\session\redis';
      $CFG->session_redis_host = '127.0.0.1';

      Test 1 (no compression)
      1. Clear the local Redis data
        1. redis-cli

        2. 127.0.0.1:6379> FLUSHALL

        3. 127.0.0.1:6379> config resetstat

      2. Login to Moodle
      3. Navigate around the test courses and some activities. (Note each course and activity visited, as it's important to visit the same courses for each compression type.)
      4. Check the session data in redis-cli tool
        1. redis-cli

        2. 127.0.0.1:6379> keys *

        3. Use that key to view data

          127.0.0.1:6379> get {keyfromlaststep}

        4. Note the data is a string representation of an object with human readable keys (i.e. lastname, email)
        5. Get the performance stats from redis-cli:

          127.0.0.1:6379> info stats
          

        6. Take a note of the total_net_input_bytes value and total_net_output_bytes value
      Test 2 (gzip compression)

      Add this line to the config.php

      $CFG->session_redis_compressor = 'gzip';

      Repeat test 1 making sure to visit the same courses and activities as before. On the last step note that the data is no longer human readable and compressed.

      Verify the output from the redis-cli get command is no longer human readable.

      Verify that the total_net_input_bytes and total_net_output_bytes values have decreased.

      Test 3 (zstd compression)

      Modify the $CFG->session_redis_compressor value set in config.php from 1 to 2

      $CFG->session_redis_compressor = 'zstd';

      Repeat test 1 making sure to visit the same courses and activities as before.

      Verify the output from the redis-cli get command is no longer human readable.

      Verify that the total_net_input_bytes and total_net_output_bytes values have decreased.

      Test 4 (unit tests)

      Add the following to your config.php file

      define('TEST_CACHESTORE_REDIS_TESTSERVERS', '127.0.0.1');
      

      Run the unit tests

      vendor/bin/phpunit lib/tests/session_redis_test.php
      

      Show
      Prerequisite Install Redis service (daemon) on your Moodle server (see  https://docs.moodle.org/37/en/Redis_cache_store#Installing_Redis_server ) if you do not already have it. Install the PHP extension Redis (see  https://docs.moodle.org/37/en/Redis_cache_store#Installing_Redis_php_driver ) if you do not already have it. Start the Redis daemon (eg.  redis-server /usr/local/etc/redis.conf ). Install the PHP extension zstd see  https://github.com/kjdev/php-ext-zstd  for information on ways to install this by reading the README. Create a couple test courses with activities. This will just be needed to navigate around and simulate a session. The Moodle tool for creating courses is useful to quickly make test courses (/admin/tool/generator/maketestcourse.php)  Configuration Add the following to the config.php file $CFG->session_handler_class = '\core\session\redis'; $CFG->session_redis_host = '127.0.0.1'; Test 1 (no compression) Clear the local Redis data redis-cli 127.0.0.1:6379> FLUSHALL 127.0.0.1:6379> config resetstat Login to Moodle Navigate around the test courses and some activities. ( Note each course and activity visited, as it's important to visit the same courses   for each compression type. ) Check the session data in redis-cli tool redis-cli 127.0.0.1:6379> keys * Use that key to view data 127.0.0.1:6379> get {keyfromlaststep} Note the data is a string representation of an object with human readable keys (i.e. lastname, email) Get the performance stats from redis-cli: 127.0.0.1:6379> info stats Take a note of the total_net_input_bytes value and total_net_output_bytes value Test 2 (gzip compression) Add this line to the config.php $CFG->session_redis_compressor = 'gzip' ; Repeat test 1 making sure to visit the same courses and activities as before. On the last step note that the data is no longer human readable and compressed. Verify the output from the redis-cli get command is no longer human readable. Verify that the total_net_input_bytes and total_net_output_bytes values have decreased. Test 3 (zstd compression) Modify the $CFG->session_redis_compressor value set in config.php from 1 to 2 $CFG->session_redis_compressor = 'zstd' ; Repeat test 1 making sure to visit the same courses and activities as before. Verify the output from the redis-cli get command is no longer human readable. Verify that the total_net_input_bytes and total_net_output_bytes values have decreased. Test 4 (unit tests) Add the following to your config.php file define('TEST_CACHESTORE_REDIS_TESTSERVERS', '127.0.0.1'); Run the unit tests vendor/bin/phpunit lib/tests/session_redis_test.php
    • Affected Branches:
      MOODLE_311_STABLE, MOODLE_400_STABLE
    • Fixed Branches:
      MOODLE_311_STABLE
    • Pull Master Branch:
      MDL-69121-redis-session-store-compression

      Description

      To make the session storage and IO smaller

      More or less cut and paste job from:

      https://tracker.moodle.org/browse/MDL-66428

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jamie.stamp Jamie Stamp
              Reporter:
              brendanheywood Brendan Heywood
              Peer reviewer:
              Brendan Heywood
              Integrator:
              Eloy Lafuente (stronk7)
              Tester:
              Eloy Lafuente (stronk7)
              Participants:
              Component watchers:
              Adrian Greeve, Jake Dallimore, Mathew May, Mihail Geshoski, Peter Dias, Sujith Haridasan, Matteo Scaramuccia, Andrew Nicols, Dongsheng Cai, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Fix Release Date:
                10/May/21

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 hour, 5 minutes
                  1h 5m