Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-64652

Personal data export dies after exhausting all the available memory

XMLWordPrintable

      We are experiencing this on moodle.org running on latest 3.6.2 version. I've requested my personal data to be exported. Once the request is approved, the adhoc task is created and executed as expected. The output of that processing looks like this

      [PID 5857] Execute adhoc task: tool_dataprivacy\task\process_data_request_task
      [PID 5857] ... started 04:21:05. Current memory use 5.7MB.
      [PID 5857] Pre-processing request...
      [PID 5857]   Fetching data from 486 components (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing antivirus_clamav (1/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing availability_completion (2/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing availability_date (3/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing availability_grade (4/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing availability_group (5/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing availability_grouping (6/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing availability_profile (7/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_calculated (8/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_calculatedmulti (9/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_calculatedsimple (10/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_ddimageortext (11/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_ddmarker (12/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_ddwtos (13/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_description (14/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_essay (15/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_gapselect (16/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_match (17/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_missingtype (18/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_multianswer (19/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_multichoice (20/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_numerical (21/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_random (22/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_randomsamatch (23/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_shortanswer (24/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing qtype_truefalse (25/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing mod_assign (26/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing mod_assignment (27/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing mod_book (28/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing mod_chat (29/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing mod_choice (30/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing mod_data (31/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing mod_feedback (32/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing mod_folder (33/486) (Monday, 21 January 2019, 9:21 PM)
      [PID 5857]     Processing mod_forum (34/486) (Monday, 21 January 2019, 9:21 PM)
      

      where is gets stuck for a while (seems like 3 hours), and then eventually the process dies with

      PHP Fatal error:  Allowed memory size of 8589934592 bytes exhausted (tried to allocate 4096 bytes) in .../lib/dml/mysqli_native_moodle_recordset.php on line 55
      

      There are 4309 records in the forum_posts with userid = 1601 (my user id), to give an estimation about the amount of exported data.

      Those 8GB are roughly a half of the whole RAM available on that machine.

      We will appreciate an advise from HQ experts in the area, pinging dobedobedoh and stronk7. Also letting know carlopolizzi as this has impact on our legal obligations, and prg3, the site's sysadmin.

        1. adhoc_task.log
          27 kB
          David Mudrák (@mudrd8mz)
        2. adhoc_task2.log.gz
          2.06 MB
          David Mudrák (@mudrd8mz)
        3. adhoc_task3.log.gz
          2.01 MB
          David Mudrák (@mudrd8mz)

            dobedobedoh Andrew Lyons
            mudrd8mz David Mudrák (@mudrd8mz)
            David Mudrák (@mudrd8mz) David Mudrák (@mudrd8mz)
            Jun Pataleta Jun Pataleta
            CiBoT CiBoT
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 hours
                2h

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.