Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-63401

User expiry does not work as it should

XMLWordPrintable

    • MOODLE_33_STABLE, MOODLE_34_STABLE, MOODLE_35_STABLE, MOODLE_36_STABLE
    • MOODLE_33_STABLE, MOODLE_34_STABLE, MOODLE_35_STABLE
    • MDL-63401-master
    • Hide

      Setup

      1. Create a new site
      2. Create test data by running the attached script (63401.php)
      3. Login as Admin/DPO
      4. Navigate to Site administration -> Users -> Privacy and Policies
      5. Set the requireallenddatesforuserdeletion setting to be checked
      6. Navigate to Site administration -> Users -> Privacy and Policies -> Data registry
      7. Create two new purposes:
        1. User:
          1. Fill in some Lawful bases (e.g. Consent)
          2. Set a retention period of 7 days
        2. Everything else:
          1. Fill in some Lawful bases (e.g. Consent)
          2. Set a retention period of 3 months
      8. Navigate Back to the registry
      9. Choose "User" in the tree
        1. Set the Purpose to the first purpose you created
      10. Choose "Site" in the tree
        1. Set the Purpose to the second (everything else) purpose you created

      Test 1: Users have all logged in recently

      1. Run the expired retention period task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period"

      2. Login as Admin/DPO
      3. Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion
      4. Select "User" from the dropdown
        1. Confirm that there were no contexts listed

      Test 2: Users have all logged in recently

      1. Run the attached "63401-lastaccess.php" script with the following arguments: 

        php 63401-lastaccess.php --usernamefilter='d%' --days=-21

      2. Run the expired retention period task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period"

      3. Login as Admin/DPO
      4. Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion
      5. Select "User" from the dropdown
        1. Confirm that you see the following users listed:
          1. d1
          2. d2
      6. Select "Blocks" from th dropdown
      7. Confirm that there are no blocks listed

      Test 3: Users have all logged in again

      1. Run the attached "63401-lastaccess.php" script with the following arguments: 

        php 63401-lastaccess.php --usernamefilter='d%' --days=0

      2. Run the expired retention period task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period"

      3. Login as Admin/DPO
      4. Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion
      5. Select "User" from the dropdown
        1. Confirm that there were no contexts listed
      6. Select "Blocks" from th dropdown
      7. Confirm that there are no blocks listed

      Test 4: Users haven't logged in again

      1. Run the attached "63401-lastaccess.php" script with the following arguments: 

        php 63401-lastaccess.php --usernamefilter='d%' --days=-21

      2. Run the expired retention period task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period"

      3. Login as Admin/DPO
      4. Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion
      5. Select "User" from the dropdown
        1. Confirm that you see the following users listed:
          1. d1
          2. d2
      6. Select all and then delete them
      7. Select "Blocks" from th dropdown
      8. Confirm that there are no blocks listed
      9. Run the attached "63401-showall.php" script: 

        php 63401-showall.php

        1. Confirm that the only users listed are d1 and d2
        2. Confirm that they are both listed as 'Approved'

      Test 5: Users have all logged in again (they're lazy slacker students)

      1. Run the attached "63401-lastaccess.php" script with the following arguments: 

        php 63401-lastaccess.php --usernamefilter='d%' --days=0

      2. Run the expired retention period task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period"

      3. Login as Admin/DPO
      4. Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion
      5. Select "User" from the dropdown
        1. Confirm that there were no contexts listed
      6. Select "Blocks" from th dropdown
      7. Confirm that there are no blocks listed
      8. Run the attached "63401-showall.php" script: 

        php 63401-showall.php

        1. Confirm that no users are listed

      Test 6: Users haven't logged in again

      1. Run the attached "63401-lastaccess.php" script with the following arguments: 

        php 63401-lastaccess.php --usernamefilter='d%' --days=-21

      2. Run the expired retention period task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period"

      3. Login as Admin/DPO
      4. Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion
      5. Select "User" from the dropdown
        1. Confirm that you see the following users listed:
          1. d1
          2. d2
      6. Select all and then delete them
      7. Select "Blocks" from th dropdown
      8. Confirm that there are no blocks listed
      9. Run the attached "63401-showall.php" script: 

        php 63401-showall.php

        1. Confirm that the only users listed are d1 and d2
        2. Confirm that they are both listed as 'Approved'
      10. Oh no... the users are logging in again!!!
      11. Run the attached "63401-lastaccess.php" script with the following arguments: 

        php 63401-lastaccess.php --usernamefilter='d%' --days=0

      12. And now run the "Delete expired contexts" task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\delete_expired_contexts"

        1. Confirm that both users were processed but skipped
      13. Run the attached "63401-showall.php" script: 

        php 63401-showall.php

        1. Confirm that no users are listed

      Test 7: They're finally going

      1. Run the attached "63401-lastaccess.php" script with the following arguments: 

        php 63401-lastaccess.php --usernamefilter='d%' --days=-21

      2. Run the expired retention period task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period"

      3. Login as Admin/DPO
      4. Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion
      5. Select "User" from the dropdown
        1. Confirm that you see the following users listed:
          1. d1
          2. d2
      6. Select all and then delete them
      7. Select "Blocks" from th dropdown
      8. Confirm that there are no blocks listed
      9. Run the attached "63401-showall.php" script: 

        php 63401-showall.php

        1. Confirm that the only users listed are d1 and d2
        2. Confirm that they are both listed as 'Approved'
      10. And now run the "Delete expired contexts" task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\delete_expired_contexts"

        1. Confirm that both users were processed
        2. Confirm that 2 user contexts were reported as expired
      11. Run the attached "63401-showall.php" script: 

        php 63401-showall.php

        1. Confirm that two empty entries show as cleaned

      Test 8: Courses without end dates are meaningless

      1. Navigate to Site administration -> Users -> Privacy and Policies
      2. Set the requireallenddatesforuserdeletion setting to be unchecked
      3. Run the attached "63401-lastaccess.php" script with the following arguments: 

        php 63401-lastaccess.php --usernamefilter='d%' --days=-21

      4. Run the expired retention period task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period"

      5. Login as Admin/DPO
      6. Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion
      7. Select "User" from the dropdown
        1. Confirm that you see the following users listed:
          1. d5
      8. Select all and then delete them
      9. Select "Blocks" from th dropdown
      10. Confirm that there are no blocks listed
      11. Run the attached "63401-showall.php" script: 

        php 63401-showall.php

        1. Confirm that the d5 is in the list
        2. Confirm that d5 is listed as 'Approved'
      12. And now run the "Delete expired contexts" task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\delete_expired_contexts"

        1. Confirm that the user was processed
        2. Confirm that 1 user contexts were reported as expired
      13. Run the attached "63401-showall.php" script: 

        php 63401-showall.php

        1. Confirm that an additinoal empty entry shows as cleaned

      Test 9: Update a course to have expired

      1. Open the course which expired yesterday
      2. Edit settings
      3. Change the expiry date to 1st May 2018
      4. Run the expired retention period task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period"

      5. Login as Admin/DPO
      6. Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion
      7. Select "Course" from the dropdown
        1. Confirm that you see the following courses listed:
          1. c_enddate_past
      8. Select "User" from th dropdown
        1. Confirm that you see the following users listed:
          1. d4
      9. Select "Blocks" from th dropdown
      10. Confirm that there are no blocks listed
      11. Navigate back to Course
      12. Delete the course
      13. Run the attached "63401-showall.php" script: 

        php 63401-showall.php

        1. Confirm that the "Ended yesterday" course is listed as 'Approved'
      14. And now run the "Delete expired contexts" task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\delete_expired_contexts"

      15. Run the attached "63401-showall.php" script: 

        php 63401-showall.php

        1. Confirm that the "Ended yesterday" course is listed as 'Cleaned'

      The Resurrection

      1. Open the course which expired yesterday
      2. Edit settings
        1. Change the expiry date to 1st September 2018
      3. Add an assignment to the course
      4. Run the expired retention period task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period"

      5. Run the attached "63401-showall.php" script: 

        php 63401-showall.php

        1. Confirm that the "Ended yesterday" course is no longer listed at all
      6. Change the end date back to 1st May 2018
      7. Run the expired retention period task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period"

      8. Run the attached "63401-showall.php" script: 

        php 63401-showall.php

        1. Confirm that the "Ended yesterday" course is now listed again as Expired
        2. Confirm that the assignment you created is also listed
      9. Find the course in the UI
      10. Select just the course for deletion 

        php 63401-showall.php

        1. Confirm that both are listed as approved
      11. And now run the "Delete expired contexts" task: 

        php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\delete_expired_contexts"

      12. Run the attached "63401-showall.php" script: 

        php 63401-showall.php

        1. Confirm that both are listed as cleaned
      Show
      Setup Create a new site Create test data by running the attached script ( 63401.php ) Login as Admin/DPO Navigate to Site administration -> Users -> Privacy and Policies Set the requireallenddatesforuserdeletion setting to be checked Navigate to Site administration -> Users -> Privacy and Policies -> Data registry Create two new purposes: User: Fill in some Lawful bases (e.g. Consent) Set a retention period of 7 days Everything else: Fill in some Lawful bases (e.g. Consent) Set a retention period of 3 months Navigate Back to the registry Choose " User " in the tree Set the Purpose to the first purpose you created Choose " Site " in the tree Set the Purpose to the second (everything else) purpose you created Test 1: Users have all logged in recently Run the expired retention period task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period" Login as Admin/DPO Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion Select " User " from the dropdown Confirm that there were no contexts listed Test 2: Users have all logged in recently Run the attached " 63401-lastaccess.php " script with the following arguments: php 63401-lastaccess.php --usernamefilter='d%' --days=-21 Run the expired retention period task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period" Login as Admin/DPO Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion Select " User " from the dropdown Confirm that you see the following users listed : d1 d2 Select " Blocks " from th dropdown Confirm that there are no blocks listed Test 3: Users have all logged in again Run the attached " 63401-lastaccess.php " script with the following arguments: php 63401-lastaccess.php --usernamefilter='d%' --days=0 Run the expired retention period task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period" Login as Admin/DPO Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion Select " User " from the dropdown Confirm that there were no contexts listed Select " Blocks " from th dropdown Confirm that there are no blocks listed Test 4: Users haven't logged in again Run the attached " 63401-lastaccess.php " script with the following arguments: php 63401-lastaccess.php --usernamefilter='d%' --days=-21 Run the expired retention period task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period" Login as Admin/DPO Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion Select " User " from the dropdown Confirm that you see the following users listed : d1 d2 Select all and then delete them Select " Blocks " from th dropdown Confirm that there are no blocks listed Run the attached " 63401-showall.php " script: php 63401-showall.php Confirm that the only users listed are d1 and d2 Confirm that they are both listed as 'Approved' Test 5: Users have all logged in again (they're lazy slacker students) Run the attached " 63401-lastaccess.php " script with the following arguments: php 63401-lastaccess.php --usernamefilter='d%' --days=0 Run the expired retention period task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period" Login as Admin/DPO Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion Select " User " from the dropdown Confirm that there were no contexts listed Select " Blocks " from th dropdown Confirm that there are no blocks listed Run the attached " 63401-showall.php " script: php 63401-showall.php Confirm that no users are listed Test 6: Users haven't logged in again Run the attached " 63401-lastaccess.php " script with the following arguments: php 63401-lastaccess.php --usernamefilter='d%' --days=-21 Run the expired retention period task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period" Login as Admin/DPO Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion Select " User " from the dropdown Confirm that you see the following users listed : d1 d2 Select all and then delete them Select " Blocks " from th dropdown Confirm that there are no blocks listed Run the attached " 63401-showall.php " script: php 63401-showall.php Confirm that the only users listed are d1 and d2 Confirm that they are both listed as 'Approved' Oh no... the users are logging in again !!! Run the attached " 63401-lastaccess.php " script with the following arguments: php 63401-lastaccess.php --usernamefilter='d%' --days=0 And now run the "Delete expired contexts" task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\delete_expired_contexts" Confirm that both users were processed but skipped Run the attached " 63401-showall.php " script: php 63401-showall.php Confirm that no users are listed Test 7: They're finally going Run the attached " 63401-lastaccess.php " script with the following arguments: php 63401-lastaccess.php --usernamefilter='d%' --days=-21 Run the expired retention period task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period" Login as Admin/DPO Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion Select " User " from the dropdown Confirm that you see the following users listed : d1 d2 Select all and then delete them Select " Blocks " from th dropdown Confirm that there are no blocks listed Run the attached " 63401-showall.php " script: php 63401-showall.php Confirm that the only users listed are d1 and d2 Confirm that they are both listed as 'Approved' And now run the "Delete expired contexts" task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\delete_expired_contexts" Confirm that both users were processed Confirm that 2 user contexts were reported as expired Run the attached " 63401-showall.php " script: php 63401-showall.php Confirm that two empty entries show as cleaned Test 8: Courses without end dates are meaningless Navigate to Site administration -> Users -> Privacy and Policies Set the requireallenddatesforuserdeletion setting to be unchecked Run the attached " 63401-lastaccess.php " script with the following arguments: php 63401-lastaccess.php --usernamefilter='d%' --days=-21 Run the expired retention period task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period" Login as Admin/DPO Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion Select " User " from the dropdown Confirm that you see the following users listed : d5 Select all and then delete them Select " Blocks " from th dropdown Confirm that there are no blocks listed Run the attached " 63401-showall.php " script: php 63401-showall.php Confirm that the d5 is in the list Confirm that d5 is listed as 'Approved' And now run the "Delete expired contexts" task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\delete_expired_contexts" Confirm that the user was processed Confirm that 1 user contexts were reported as expired Run the attached " 63401-showall.php " script: php 63401-showall.php Confirm that an additinoal empty entry shows as cleaned Test 9: Update a course to have expired Open the course which expired yesterday Edit settings Change the expiry date to 1st May 2018 Run the expired retention period task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period" Login as Admin/DPO Navigate to Site administration -> Users -> Privacy and Policies -> Data deletion Select " Course " from the dropdown Confirm that you see the following courses listed : c_enddate_past Select " User " from th dropdown Confirm that you see the following users listed : d4 Select " Blocks " from th dropdown Confirm that there are no blocks listed Navigate back to Course Delete the course Run the attached " 63401-showall.php " script: php 63401-showall.php Confirm that the "Ended yesterday" course is listed as ' Approved ' And now run the "Delete expired contexts" task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\delete_expired_contexts" Run the attached " 63401-showall.php " script: php 63401-showall.php Confirm that the "Ended yesterday" course is listed as ' Cleaned ' The Resurrection Open the course which expired yesterday Edit settings Change the expiry date to 1st September 2018 Add an assignment to the course Run the expired retention period task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period" Run the attached " 63401-showall.php " script: php 63401-showall.php Confirm that the "Ended yesterday" course is no longer listed at all Change the end date back to 1st May 2018 Run the expired retention period task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\expired_retention_period" Run the attached " 63401-showall.php " script: php 63401-showall.php Confirm that the "Ended yesterday" course is now listed again as Expired Confirm that the assignment you created is also listed Find the course in the UI Select just the course for deletion php 63401-showall.php Confirm that both are listed as approved And now run the "Delete expired contexts" task: php admin/tool/task/cli/schedule_task.php --execute="\tool_dataprivacy\task\delete_expired_contexts" Run the attached " 63401-showall.php " script: php 63401-showall.php Confirm that both are listed as cleaned
    • GDPR Followup Sprint 1

      This issue covers three main issues:

      1. users are currently expired when they finish a course, not when the courses that they are in expire. This currently means that some data is removed before it should be
      2. it is not possible to expire a user if they are enrolled in any open-ended courses
      3. any user with a user block is not deletable due to a mis-understanding in the API as to how to handle blocks – it treats them all as belonging to a course

      This issue solves these by:

      1. switch to respecting the course expiry as a dependency of user expiry.
      2. provide a system setting to ignore open-ended courses when calculating the user expiry (rely purely on last login time)
      3. rewrite the dataprivacy expiry manager to not separate out course contexts from user contexts using context level, but instead do so by actual context.

      The biggest change in this patchset is for the third item.
      The original code assumed that any block was always a course block, and ignored the potential for user blocks. Because of this user expiry was essentially not possible if the user had any blocks on their Dashboard.
      I felt that the best way to handle this was:

      1. to merge the user and course context expiry managers. The existing code separated them out based purely on their context level, which is not an accurate gauge of their type
      2. to change the behaviour such that any child context of a user is bundled in with the user context. It does not make sense not to do so as we do not allow any data registry configuration (purpose/category) for a specific user or for children of these anyway.

      In order to complete this work I also moved the location of capability checks from the API class to the endpoints which call them. These tests do not belong in the API, and we have found this in other related issues (user bulk deletion, is_site_dpo changes, etc).

      This work also necessitated that we finally stop deleting the user context, and instead just delete the related content.
      As far as I'm aware, the only time we delete other contexts is when the record itself is removed. We do not remove the user record at any time, therefore we should not remove the context. The context has many things stored against it and, although some items are removed during the context deletion, this only applies to removal of core items (and not even all of them). It does not support deletion of user context data in plugins at all. Again, this same change has been identified as a necessity in other related issues.

        1. 1.PNG
          1.PNG
          126 kB
        2. 2.PNG
          2.PNG
          169 kB
        3. 3.PNG
          3.PNG
          185 kB
        4. 4.PNG
          4.PNG
          258 kB
        5. 5.PNG
          5.PNG
          223 kB
        6. 6.PNG
          6.PNG
          315 kB
        7. 63401.php
          10 kB
        8. 63401-lastaccess.php
          1 kB
        9. 63401-showall.php
          0.6 kB

            dobedobedoh Andrew Lyons
            dobedobedoh Andrew Lyons
            Adrian Greeve Adrian Greeve
            Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
            Janelle Barcega Janelle Barcega
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 3 hours, 10 minutes
                3h 10m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.