Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-69121

Allow redis session store to use zip or zStd for compression like redis MUC

    XMLWordPrintable

Details

    • MOODLE_311_STABLE, MOODLE_400_STABLE
    • MOODLE_311_STABLE
    • MDL-69121-redis-session-store-compression
    • Hide
      Prerequisite
      1. Install Redis service (daemon) on your Moodle server (see https://docs.moodle.org/37/en/Redis_cache_store#Installing_Redis_server) if you do not already have it.
      2. Install the PHP extension Redis (see https://docs.moodle.org/37/en/Redis_cache_store#Installing_Redis_php_driver) if you do not already have it.
      3. Start the Redis daemon (eg. redis-server /usr/local/etc/redis.conf).
      4. Install the PHP extension zstd see https://github.com/kjdev/php-ext-zstd for information on ways to install this by reading the README.
      5. Create a couple test courses with activities. This will just be needed to navigate around and simulate a session. The Moodle tool for creating courses is useful to quickly make test courses (/admin/tool/generator/maketestcourse.php)
       Configuration

      Add the following to the config.php file

      $CFG->session_handler_class = '\core\session\redis';
      $CFG->session_redis_host = '127.0.0.1';

      Test 1 (no compression)
      1. Clear the local Redis data
        1. redis-cli

        2. 127.0.0.1:6379> FLUSHALL

        3. 127.0.0.1:6379> config resetstat

      2. Login to Moodle
      3. Navigate around the test courses and some activities. (Note each course and activity visited, as it's important to visit the same courses for each compression type.)
      4. Check the session data in redis-cli tool
        1. redis-cli

        2. 127.0.0.1:6379> keys *

        3. Use that key to view data

          127.0.0.1:6379> get {keyfromlaststep}

        4. Note the data is a string representation of an object with human readable keys (i.e. lastname, email)
        5. Get the performance stats from redis-cli:

          127.0.0.1:6379> info stats
          

        6. Take a note of the total_net_input_bytes value and total_net_output_bytes value
      Test 2 (gzip compression)

      Add this line to the config.php

      $CFG->session_redis_compressor = 'gzip';

      Repeat test 1 making sure to visit the same courses and activities as before. On the last step note that the data is no longer human readable and compressed.

      Verify the output from the redis-cli get command is no longer human readable.

      Verify that the total_net_input_bytes and total_net_output_bytes values have decreased.

      Test 3 (zstd compression)

      Modify the $CFG->session_redis_compressor value set in config.php from 1 to 2

      $CFG->session_redis_compressor = 'zstd';

      Repeat test 1 making sure to visit the same courses and activities as before.

      Verify the output from the redis-cli get command is no longer human readable.

      Verify that the total_net_input_bytes and total_net_output_bytes values have decreased.

      Test 4 (unit tests)

      Add the following to your config.php file

      define('TEST_CACHESTORE_REDIS_TESTSERVERS', '127.0.0.1');
      

      Run the unit tests

      vendor/bin/phpunit lib/tests/session_redis_test.php
      

      Show
      Prerequisite Install Redis service (daemon) on your Moodle server (see  https://docs.moodle.org/37/en/Redis_cache_store#Installing_Redis_server ) if you do not already have it. Install the PHP extension Redis (see  https://docs.moodle.org/37/en/Redis_cache_store#Installing_Redis_php_driver ) if you do not already have it. Start the Redis daemon (eg.  redis-server /usr/local/etc/redis.conf ). Install the PHP extension zstd see  https://github.com/kjdev/php-ext-zstd  for information on ways to install this by reading the README. Create a couple test courses with activities. This will just be needed to navigate around and simulate a session. The Moodle tool for creating courses is useful to quickly make test courses (/admin/tool/generator/maketestcourse.php)  Configuration Add the following to the config.php file $CFG->session_handler_class = '\core\session\redis'; $CFG->session_redis_host = '127.0.0.1'; Test 1 (no compression) Clear the local Redis data redis-cli 127.0.0.1:6379> FLUSHALL 127.0.0.1:6379> config resetstat Login to Moodle Navigate around the test courses and some activities. ( Note each course and activity visited, as it's important to visit the same courses   for each compression type. ) Check the session data in redis-cli tool redis-cli 127.0.0.1:6379> keys * Use that key to view data 127.0.0.1:6379> get {keyfromlaststep} Note the data is a string representation of an object with human readable keys (i.e. lastname, email) Get the performance stats from redis-cli: 127.0.0.1:6379> info stats Take a note of the total_net_input_bytes value and total_net_output_bytes value Test 2 (gzip compression) Add this line to the config.php $CFG->session_redis_compressor = 'gzip' ; Repeat test 1 making sure to visit the same courses and activities as before. On the last step note that the data is no longer human readable and compressed. Verify the output from the redis-cli get command is no longer human readable. Verify that the total_net_input_bytes and total_net_output_bytes values have decreased. Test 3 (zstd compression) Modify the $CFG->session_redis_compressor value set in config.php from 1 to 2 $CFG->session_redis_compressor = 'zstd' ; Repeat test 1 making sure to visit the same courses and activities as before. Verify the output from the redis-cli get command is no longer human readable. Verify that the total_net_input_bytes and total_net_output_bytes values have decreased. Test 4 (unit tests) Add the following to your config.php file define('TEST_CACHESTORE_REDIS_TESTSERVERS', '127.0.0.1'); Run the unit tests vendor/bin/phpunit lib/tests/session_redis_test.php

    Description

      To make the session storage and IO smaller

      More or less cut and paste job from:

      https://tracker.moodle.org/browse/MDL-66428

      Attachments

        Issue Links

          Activity

            People

              jamie.stamp Jamie Stamp
              brendanheywood Brendan Heywood
              Brendan Heywood Brendan Heywood
              Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
              Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
              Adrian Greeve, Jake Dallimore, Mathew May, Mihail Geshoski, Sujith Haridasan, Matteo Scaramuccia, Andrew Lyons, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze, Stevani Andolo
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                17/May/21

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 hour, 5 minutes
                  1h 5m