Moodle
  1. Moodle
  2. MDL-30026

Prevent session lock waits from stalling server processes forever

    Details

    • Testing Instructions:
      Hide

      For developers:
      1/ run the functional DB tests under sqlsrv. Verify that no error related with "test_session_locks" happens.

      2/ no need to perform the tests under other DBs, nothing has changed there (last week we introduced and tested that).

      Show
      For developers: 1/ run the functional DB tests under sqlsrv. Verify that no error related with "test_session_locks" happens. 2/ no need to perform the tests under other DBs, nothing has changed there (last week we introduced and tested that).
    • Affected Branches:
      MOODLE_20_STABLE, MOODLE_21_STABLE, MOODLE_22_STABLE
    • Fixed Branches:
      MOODLE_22_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-30026_sqlsrv
    • Rank:
      19568

      Description

      On many of our hosted sites we sometimes get periods of unavailability or out of memory conditions which trace back to a single user.

      Most often in these cases, the user will be performing some long-running process (e.g backup, users upload) and attempts to continue opening things from another browser tab. The new requests get backed up behind the long request waiting to grab a session lock. This can also be triggered by bug, especially infinite loops in code sections with increased time limit.

      This results in stuck server processes which may not be freed for hours, and can cause out-of-memory issues if more processes are started to handle other requests.

      Over time, it should be a goal to migrate these long processes out of web requests and into background processing but it would be nice to have some kind of catchall to prevent these and bugs from causing site unavailability.

      The suggestion is to, where possible, place a time limit on how long Moodle will wait to acquire a session lock before giving up. While this does cause an error to the user, they do not get hung browser windows forever, and the site remains available.

      Please see the linked patch for an attempt at doing this for Postgres.

        Issue Links

          Activity

          Tony Levi created issue -
          Tony Levi made changes -
          Field Original Value New Value
          Priority Minor [ 4 ] Major [ 3 ]
          Labels netspot partner patch
          Petr Škoda made changes -
          Assignee moodle.com [ moodle.com ] Petr Škoda (skodak) [ skodak ]
          Petr Škoda made changes -
          Status Open [ 1 ] Development in progress [ 3 ]
          Petr Škoda made changes -
          Status Development in progress [ 3 ] Waiting for integration review [ 10010 ]
          Pull Master Diff URL https://github.com/skodak/moodle/compare/master...w44_MDL-30026_m22_sessionconnect
          Pull Master Branch w44_MDL-30026_m22_sessionconnect
          Pull from Repository git://github.com/skodak/moodle.git
          Fix Version/s 2.2 [ 10656 ]
          Testing Instructions For developers:
          1/ create simple moodle script with sleep(1000) in it;
          2/ open in one window, then open other window with normal page
          3/ observer timeout behaviour and verify timeout errors
          Eloy Lafuente (stronk7) made changes -
          Currently in integration Yes [ 10041 ]
          Eloy Lafuente (stronk7) made changes -
          Status Waiting for integration review [ 10010 ] Integration review in progress [ 10004 ]
          Integrator stronk7
          Eloy Lafuente (stronk7) made changes -
          Link This issue has been marked as being related by MDL-30242 [ MDL-30242 ]
          Eloy Lafuente (stronk7) made changes -
          Testing Instructions For developers:
          1/ create simple moodle script with sleep(1000) in it;
          2/ open in one window, then open other window with normal page
          3/ observer timeout behaviour and verify timeout errors
          For developers:
          1/ create simple moodle script with sleep(1000) in it;
          2/ open in one window, then open other window with normal page
          3/ observer timeout behaviour and verify timeout errors
          4/ run the functional DB tests under the 5DBs (mysql, pgsql, mssql, sqlsrv and oci). Verify that no error related with "test_session_locks" happens.

          Warn: it's very important to test it under sqlsrv because the fix there has been implemented in "blind-mode" and the 2 developer involved have no, right now, access to run under that DB.
          Eloy Lafuente (stronk7) made changes -
          Status Integration review in progress [ 10004 ] Waiting for testing [ 10005 ]
          Sam Hemelryk made changes -
          Status Waiting for testing [ 10005 ] Testing in progress [ 10011 ]
          Tester samhemelryk
          Sam Hemelryk made changes -
          Status Testing in progress [ 10011 ] Problem during testing [ 10007 ]
          Eloy Lafuente (stronk7) made changes -
          Assignee Petr Škoda (skodak) [ skodak ] Eloy Lafuente (stronk7) [ stronk7 ]
          Eloy Lafuente (stronk7) made changes -
          Status Problem during testing [ 10007 ] Integration review in progress [ 10004 ]
          Eloy Lafuente (stronk7) made changes -
          Status Integration review in progress [ 10004 ] Waiting for testing [ 10005 ]
          Eloy Lafuente (stronk7) made changes -
          Status Waiting for testing [ 10005 ] Testing in progress [ 10011 ]
          Eloy Lafuente (stronk7) made changes -
          Status Testing in progress [ 10011 ] Tested [ 10006 ]
          Eloy Lafuente (stronk7) made changes -
          Status Tested [ 10006 ] Closed [ 6 ]
          Resolution Fixed [ 1 ]
          Currently in integration Yes [ 10041 ]
          Integration date 15/Nov/11
          Eloy Lafuente (stronk7) made changes -
          Resolution Fixed [ 1 ]
          Status Closed [ 6 ] Reopened [ 4 ]
          Eloy Lafuente (stronk7) made changes -
          Status Reopened [ 4 ] Development in progress [ 3 ]
          Eloy Lafuente (stronk7) made changes -
          Status Development in progress [ 3 ] Waiting for integration review [ 10010 ]
          Pull Master Diff URL https://github.com/skodak/moodle/compare/master...w44_MDL-30026_m22_sessionconnect https://github.com/stronk7/moodle/compare/master...MDL-30026_sqlsrv
          Pull Master Branch w44_MDL-30026_m22_sessionconnect MDL-30026_sqlsrv
          Pull from Repository git://github.com/skodak/moodle.git git://github.com/stronk7/moodle.git
          Testing Instructions For developers:
          1/ create simple moodle script with sleep(1000) in it;
          2/ open in one window, then open other window with normal page
          3/ observer timeout behaviour and verify timeout errors
          4/ run the functional DB tests under the 5DBs (mysql, pgsql, mssql, sqlsrv and oci). Verify that no error related with "test_session_locks" happens.

          Warn: it's very important to test it under sqlsrv because the fix there has been implemented in "blind-mode" and the 2 developer involved have no, right now, access to run under that DB.
          For developers:
          1/ run the functional DB tests under sqlsrv. Verify that no error related with "test_session_locks" happens.

          2/ no need to perform the tests under other DBs, nothing has changed there (last week we introduced and tested that).
          Eloy Lafuente (stronk7) made changes -
          Integrator stronk7
          Tester samhemelryk
          Michael de Raadt made changes -
          Labels netspot partner patch netspot partner patch triaged
          Aparup Banerjee made changes -
          Integrator nebgor
          Aparup Banerjee made changes -
          Status Waiting for integration review [ 10010 ] Integration review in progress [ 10004 ]
          Currently in integration Yes [ 10041 ]
          Aparup Banerjee made changes -
          Status Integration review in progress [ 10004 ] Waiting for testing [ 10005 ]
          Affects Version/s 2.1.2 [ 10851 ]
          Affects Version/s 2.0.5 [ 10950 ]
          Affects Version/s 2.0 [ 10122 ]
          Affects Version/s 2.1 [ 10370 ]
          Affects Version/s 2.2 [ 10656 ]
          Eloy Lafuente (stronk7) made changes -
          Affects Version/s 2.2 [ 10656 ]
          Aparup Banerjee made changes -
          Status Waiting for testing [ 10005 ] Testing in progress [ 10011 ]
          Tester nebgor
          Aparup Banerjee made changes -
          Status Testing in progress [ 10011 ] Tested [ 10006 ]
          Eloy Lafuente (stronk7) made changes -
          Status Tested [ 10006 ] Closed [ 6 ]
          Resolution Fixed [ 1 ]
          Currently in integration Yes [ 10041 ]
          Integration date 15/Nov/11 25/Nov/11
          Petr Škoda made changes -
          Link This issue is duplicated by MDL-31524 [ MDL-31524 ]
          Aparup Banerjee made changes -
          Link This issue has a non-specific relationship to MDL-32236 [ MDL-32236 ]
          Martin Dougiamas made changes -
          Link This issue caused a regression MDL-31870 [ MDL-31870 ]

            People

            • Votes:
              3 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: