Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-62660

Data export download links don't expire

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide
      Prerequisites
      1. You need to have (or be able to set up) the ability to receive emails from your test Moodle site, such as installing and enabling Mailcatcher.
      2. You also need database access to be able to confirm the files are being deleted after expiry.
      Initial setup - before upgrading to include this patch
      1. Open your terminal and go to your Moodle instance's root directory and checkout the latest weekly release:
        1. master

          git checkout a9428fab113c47b8468d4b736254db118683c527

        2. 35

          git checkout 72337efdf7191ce7fffb9991580ee245547d195c

        3. 34

          git checkout 94409c69b1d7f7cf7f7133ac6c09b5697ed2469e

        4. 33

          git checkout 8987e225da68a4fa3870008f49c15f9ac3ab73b2

      2. Log in as the admin user (who should automatically be assigned as the Privacy Officer).
      3. If your admin user is brand new, so does not already have content that would be exported with a data privacy requests, add some (eg create some user calendar entries, add a forum post to a course etc).
      4. Go to Site administration > users > Privacy settings and mark the checkbox for setting "tool_dataprivacy | contactdataprotectionofficer"
      5. Navigate to Site administration > users > add a new user, and create a new user with username 'before62660'.
      6. Create a second user, with username 'completed62660'.
      7. Open Site administration > Users, then click 'Data requests' (within the Privacy and policies section).
      8. Click 'New request', then in the 'Requesting for' search, type 'before62660' and select that user.
      9. Populate 'Type' with 'Delete all of my personal data', and comment 'Testing deletion 1', then click 'Save changes'.
      10. Click 'New request', then in the 'Requesting for' search, type 'completed62660' and select that user.
      11. Populate 'Type' with 'Export all of my personal data', and comment 'Testing completed', then click 'Save changes'.
      12. Run your test site's cron (either from the command line, or in a new tab through the Moodle site as admin).
      13. Refresh the 'Data requests' page (or return to it if you navigated away to run the cron).
      14. Next to the 'Testing deletion 1' request, click 'Actions', then click 'Approve request', then in the modal popup, click 'Approve request'.
      15. Repeat for the 'Testing completed' request, so that is also approved.
      16. Re-run your site's cron.
      17. Refresh or navigate back to the 'Data requests' page
        1. CONFIRM the status of both 'Testing deletion 1' and 'Testing completed' are 'Complete'.
      Further setup
      1. Reset the codebase with the patch now applied.
        1. master

          git checkout master

        2. 35

          git checkout MOODLE_35_STABLE

        3. 34

          git checkout MOODLE_34_STABLE

        4. 33

          git checkout MOODLE_33_STABLE

      2. Navigate to Site administration
        1. CONFIRM you are presented with upgrade prompt. Complete those to ensure the latest data privacy request changes are applied.
      3. Navigate to Site administration > Users, under the Privacy and Policies heading, click Privacy settings.
        1. CONFIRM there is now a "Data request expiry" setting, with the default values set to "1 weeks".
      4. Update the Data request expiry period to be "5 minutes", and click Save changes. This will allow you to test completed data privacy requests that are both active (existing functionality), as well as expired (new functionality).
        1. CONFIRM You see the Changes saved dialogue, so the expiry period has been updated.
      5. Navigate to Site administration > users > add a new user, and create a new user with username 'test62660'. This will be used for a deletion request test later.
      Testing upgrade updates 'completed' statuses to their newer replacements
      1. Open Site administration > Users, then click 'Data requests' (within the Privacy and policies section).
        1. CONFIRM the status of 'Testing deletion 1' is now 'Deleted'.
        2. CONFIRM the status of 'Testing completed' is now 'Download ready'.
      Testing prior to request expiring
      1. (Still logged in as the admin user) navigate to the admin user's profile page (via the dropdown at the top right of the site).
      2. Under Privacy and policies, click 'Data requests', then click the 'New request' button.
      3. In the form that loads, set Type to 'Export all of my personal data', and Comments as 'Testing expiry', then click 'Save changes'.
        1. CONFIRM the data request now appears in the list of Data requests with Status 'pending'
        2. CONFIRM the 'pending' status label appears in blue (label colour was updated from grey).
      4. Run your test site's cron (either from the command line, or in a new tab through the Moodle site as admin).
      5. Refresh the Data requests page
        1. CONFIRM the status of the 'Testing expiry' request is now 'Awaiting approval'.
      6. Open Site administration > Users, then click 'Data requests' (within the Privacy and policies section).
      7. Next to your 'Testing expiry' request, click 'Actions', then click 'Approve request', then in the modal popup, click 'Approve request'.
        1. CONFIRM the status of the request now appears as 'Approved'.
      8. Re-run your site's cron.
      9. Return to your user's Data requests (through user profile as detailed earlier).
        1. CONFIRM the 'Testing expiry' request now has the status 'Download ready' (this label should be green).
      10. On the 'Testing expiry' request row, click Actions
        1. CONFIRM the 'Download' option is visible.
      11. Click 'Download', and save the export.zip file. Keep this tab open for use later.
      12. Open The latest email received with the subject 'Data request: Export all of my personal data'.
      13. Click the 'Download' link within the email.
        1. CONFIRM you are prompted to download the data export.
      Testing request expiry prevents access and removes files
      1. Wait until the expiry period has elapsed (it should be 5 minutes after the received time of the download email).
      2. Make sure you don't refresh or access again the list of requests, because it would expiry the requests.
      3. Click the 'Download' link again from the email.
        1. CONFIRM you are redirected to your test Moodle site, and receive the error 'Sorry, the requested file could not be found'.
      4. Return to your admin user's data requests page and refresh the page (you should still have it open from earlier, but if not, reopen it).
        1. CONFIRM the 'Testing expiry' request now displays the status 'Expired' (this label should be grey).
        2. CONFIRM if you mouse-over the 'Expired' label, you now see an explanation of the status as a tooltip/popover.
        3. CONFIRM the 'Actions' dropdown is no longer available for this request.
      5. Navigate to Site administration > Users, then click 'Data requests' (within the Privacy and policies section).
      6. Click the Actions option on the 'Testing expiry' row
        1. CONFIRM the only option is 'view request'
      Testing scheduled task removes files
      1. Create a new request
      2. Run cron
      3. Approve the new request
      4. Wait until the expiry period has elapsed (it should be 5 minutes after the received time of the download email).
      5. Open your database tool, and view the mdl_files table for your test Moodle site. To make it easier to view the relevant content, you can filter/select by component = 'tool_dataprivacy' and filearea = 'export'
      6. Find the 2 rows related to your 'Testing expiry' request (it should be the most recent component = tool_dataprivacy rows with filename '.' (the directory) and 'export.zip' (the file)). Note down the values in the 'id' column, so you can identify the rows are removed during later steps.
      7. As the site admin, navigate to Site administration > Server, then within the 'Server' section, click 'Scheduled tasks'.
      8. CONFIRM you can find the scheduled task 'Delete files from completed data requests that have expired'.
        1. On that scheduled task's row, click 'Run now'.
      9. In the Confirmation modal popup, click 'Run now'.
        1. CONFIRM the scheduled task output includes '1 expired completed data requests have been deleted'.
        2. CONFIRM the scheduled task output includes 'Scheduled task complete: Delete files from completed data requests that have expired'.
      10. Return to the database
        1. CONFIRM the two rows whose IDs you noted down earlier are now removed from the mdl_files table (you may need to refresh if you still have this open from earlier steps).
      Testing expired status is handled correctly
      1. Repeat steps 2-9 from "Testing request expiry prevents access and removes files" above
        1. CONFIRM you see the same results.
      Testing new deletion request status, label and filter
      1. Log in as user 'test62660'.
      2. Navigate to the user's profile page (via the dropdown at the top right of the site).
      3. Under Privacy and policies, click 'Data requests', then click the 'New request' button.
      4. In the form that loads, set Type to 'Delete all of my personal data', and Comments as 'Deletion expiry 1', then click 'Save changes'.
        1. CONFIRM the data request now appears in the list of Data requests with Status 'pending'
        2. CONFIRM the 'pending' status label appears in blue (label colour was updated from grey).
      5. Run your test site's cron (either from the command line, or by logging in as a site admin, and running them through the Moodle site).
      6. Log in as the same admin as earlier tests, then open Site administration > Users, then click 'Data requests' (within the Privacy and policies section).
      7. Next to the 'Deletion expiry 2' request, click 'Actions', then click 'Approve request', then in the modal popup, click 'Approve request'.
        1. CONFIRM the status of the request now appears as 'Approved'.
      8. Re-run your site's cron.
      9. Refresh the Data requests page
        1. CONFIRM the 'Deletion expiry 1' request now have the status 'Deleted' (this label should be green).
      10. Click the 'Filter' dropdown on the Data requests page
        1. CONFIRM there is a 'Status: Deleted' option.
      11. Click the 'Status: Deleted' option
        1. CONFIRM clicking on that option filters the list of data requests to show 'Deletion expiry 1' requests.
      12. Still logged as user 'test62660', refresh your requests page and CONFIRM you session has been time out.
      13. Attempt to log in as user 'test62660'
        1. CONFIRM you receive an error "Invalid login, please try again".
      Show
      Prerequisites You need to have (or be able to set up) the ability to receive emails from your test Moodle site, such as installing and enabling Mailcatcher. You also need database access to be able to confirm the files are being deleted after expiry. Initial setup - before upgrading to include this patch Open your terminal and go to your Moodle instance's root directory and checkout the latest weekly release: master git checkout a9428fab113c47b8468d4b736254db118683c527 35 git checkout 72337efdf7191ce7fffb9991580ee245547d195c 34 git checkout 94409c69b1d7f7cf7f7133ac6c09b5697ed2469e 33 git checkout 8987e225da68a4fa3870008f49c15f9ac3ab73b2 Log in as the admin user (who should automatically be assigned as the Privacy Officer). If your admin user is brand new, so does not already have content that would be exported with a data privacy requests, add some (eg create some user calendar entries, add a forum post to a course etc). Go to Site administration > users > Privacy settings and mark the checkbox for setting "tool_dataprivacy | contactdataprotectionofficer" Navigate to Site administration > users > add a new user, and create a new user with username 'before62660'. Create a second user, with username 'completed62660'. Open Site administration > Users, then click 'Data requests' (within the Privacy and policies section). Click 'New request', then in the 'Requesting for' search, type 'before62660' and select that user. Populate 'Type' with 'Delete all of my personal data', and comment 'Testing deletion 1', then click 'Save changes'. Click 'New request', then in the 'Requesting for' search, type 'completed62660' and select that user. Populate 'Type' with 'Export all of my personal data', and comment 'Testing completed', then click 'Save changes'. Run your test site's cron (either from the command line, or in a new tab through the Moodle site as admin). Refresh the 'Data requests' page (or return to it if you navigated away to run the cron). Next to the 'Testing deletion 1' request, click 'Actions', then click 'Approve request', then in the modal popup, click 'Approve request'. Repeat for the 'Testing completed' request, so that is also approved. Re-run your site's cron. Refresh or navigate back to the 'Data requests' page CONFIRM the status of both 'Testing deletion 1' and 'Testing completed' are 'Complete'. Further setup Reset the codebase with the patch now applied. master git checkout master 35 git checkout MOODLE_35_STABLE 34 git checkout MOODLE_34_STABLE 33 git checkout MOODLE_33_STABLE Navigate to Site administration CONFIRM you are presented with upgrade prompt. Complete those to ensure the latest data privacy request changes are applied. Navigate to Site administration > Users, under the Privacy and Policies heading, click Privacy settings. CONFIRM there is now a "Data request expiry" setting, with the default values set to "1 weeks". Update the Data request expiry period to be "5 minutes", and click Save changes. This will allow you to test completed data privacy requests that are both active (existing functionality), as well as expired (new functionality). CONFIRM You see the Changes saved dialogue, so the expiry period has been updated. Navigate to Site administration > users > add a new user, and create a new user with username 'test62660'. This will be used for a deletion request test later. Testing upgrade updates 'completed' statuses to their newer replacements Open Site administration > Users, then click 'Data requests' (within the Privacy and policies section). CONFIRM the status of 'Testing deletion 1' is now 'Deleted'. CONFIRM the status of 'Testing completed' is now 'Download ready'. Testing prior to request expiring (Still logged in as the admin user) navigate to the admin user's profile page (via the dropdown at the top right of the site). Under Privacy and policies, click 'Data requests', then click the 'New request' button. In the form that loads, set Type to 'Export all of my personal data', and Comments as 'Testing expiry', then click 'Save changes'. CONFIRM the data request now appears in the list of Data requests with Status 'pending' CONFIRM the 'pending' status label appears in blue (label colour was updated from grey). Run your test site's cron (either from the command line, or in a new tab through the Moodle site as admin). Refresh the Data requests page CONFIRM the status of the 'Testing expiry' request is now 'Awaiting approval'. Open Site administration > Users, then click 'Data requests' (within the Privacy and policies section). Next to your 'Testing expiry' request, click 'Actions', then click 'Approve request', then in the modal popup, click 'Approve request'. CONFIRM the status of the request now appears as 'Approved'. Re-run your site's cron. Return to your user's Data requests (through user profile as detailed earlier). CONFIRM the 'Testing expiry' request now has the status 'Download ready' (this label should be green). On the 'Testing expiry' request row, click Actions CONFIRM the 'Download' option is visible. Click 'Download', and save the export.zip file. Keep this tab open for use later. Open The latest email received with the subject 'Data request: Export all of my personal data'. Click the 'Download' link within the email. CONFIRM you are prompted to download the data export. Testing request expiry prevents access and removes files Wait until the expiry period has elapsed (it should be 5 minutes after the received time of the download email). Make sure you don't refresh or access again the list of requests, because it would expiry the requests. Click the 'Download' link again from the email. CONFIRM you are redirected to your test Moodle site, and receive the error 'Sorry, the requested file could not be found'. Return to your admin user's data requests page and refresh the page (you should still have it open from earlier, but if not, reopen it). CONFIRM the 'Testing expiry' request now displays the status 'Expired' (this label should be grey). CONFIRM if you mouse-over the 'Expired' label, you now see an explanation of the status as a tooltip/popover. CONFIRM the 'Actions' dropdown is no longer available for this request. Navigate to Site administration > Users, then click 'Data requests' (within the Privacy and policies section). Click the Actions option on the 'Testing expiry' row CONFIRM the only option is 'view request' Testing scheduled task removes files Create a new request Run cron Approve the new request Wait until the expiry period has elapsed (it should be 5 minutes after the received time of the download email). Open your database tool, and view the mdl_files table for your test Moodle site. To make it easier to view the relevant content, you can filter/select by component = 'tool_dataprivacy' and filearea = 'export' Find the 2 rows related to your 'Testing expiry' request (it should be the most recent component = tool_dataprivacy rows with filename '.' (the directory) and 'export.zip' (the file)). Note down the values in the 'id' column, so you can identify the rows are removed during later steps. As the site admin, navigate to Site administration > Server, then within the 'Server' section, click 'Scheduled tasks'. CONFIRM you can find the scheduled task 'Delete files from completed data requests that have expired'. On that scheduled task's row, click 'Run now'. In the Confirmation modal popup, click 'Run now'. CONFIRM the scheduled task output includes '1 expired completed data requests have been deleted'. CONFIRM the scheduled task output includes 'Scheduled task complete: Delete files from completed data requests that have expired'. Return to the database CONFIRM the two rows whose IDs you noted down earlier are now removed from the mdl_files table (you may need to refresh if you still have this open from earlier steps). Testing expired status is handled correctly Repeat steps 2-9 from "Testing request expiry prevents access and removes files" above CONFIRM you see the same results. Testing new deletion request status, label and filter Log in as user 'test62660'. Navigate to the user's profile page (via the dropdown at the top right of the site). Under Privacy and policies, click 'Data requests', then click the 'New request' button. In the form that loads, set Type to 'Delete all of my personal data', and Comments as 'Deletion expiry 1', then click 'Save changes'. CONFIRM the data request now appears in the list of Data requests with Status 'pending' CONFIRM the 'pending' status label appears in blue (label colour was updated from grey). Run your test site's cron (either from the command line, or by logging in as a site admin, and running them through the Moodle site). Log in as the same admin as earlier tests, then open Site administration > Users, then click 'Data requests' (within the Privacy and policies section). Next to the 'Deletion expiry 2' request, click 'Actions', then click 'Approve request', then in the modal popup, click 'Approve request'. CONFIRM the status of the request now appears as 'Approved'. Re-run your site's cron. Refresh the Data requests page CONFIRM the 'Deletion expiry 1' request now have the status 'Deleted' (this label should be green). Click the 'Filter' dropdown on the Data requests page CONFIRM there is a 'Status: Deleted' option. Click the 'Status: Deleted' option CONFIRM clicking on that option filters the list of data requests to show 'Deletion expiry 1' requests. Still logged as user 'test62660', refresh your requests page and CONFIRM you session has been time out. Attempt to log in as user 'test62660' CONFIRM you receive an error "Invalid login, please try again".
    • Affected Branches:
      MOODLE_35_STABLE
    • Fixed Branches:
      MOODLE_33_STABLE, MOODLE_34_STABLE, MOODLE_35_STABLE
    • Epic Link:
    • Pull 3.5 Branch:
    • Pull Master Branch:
      MDL-62660-master
    • Sprint:
      GDPR Followup Sprint 1

      Description

      From the initial discussions during the planning phase of the GDPR project, it was agreed that download links for data export should have expiry dates. Somehow, this has been missed.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  10/Sep/18