Moodle
  1. Moodle
  2. MDL-31145

Resource Files (Sections) not being deleted

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.1, 2.2
    • Fix Version/s: DEV backlog
    • Component/s: Files API
    • Labels:
    • Environment:
      LAMP Stack -- Let me know if you need more information
    • Database:
      MySQL
    • Affected Branches:
      MOODLE_21_STABLE, MOODLE_22_STABLE
    • Rank:
      37577

      Description

      When a teacher loads an image/video/file into the section of a course, and then later deletes that image/video/file, the image remains on the backside (within the system files).

      The image does not leave the system files at any point and remains. The image/video/file is backed up when the course is backed up therefore creating a larger backup when the resource is actually not in the course.

      Can you provide some insight on how to solve this issue, as it appears that even if a file is deleted, it never is really deleted, and this causes some problems with growth and file management in the future.

      Replication steps:

      1. Login to Course as Teacher
      2. Click Edit Section
      3. Insert Image into Section
      4. Save Section and Image
      5. Delete Image from Section
      6. File remains in the system files

        Issue Links

          Activity

          Hide
          Julia Hengstler added a comment - - edited

          This issue first appeared when I created a course template for my faculty in mid-2011. In the development phases, I played with embedding a video that was downloaded into the course shell. I subsequently deleted it. From the "instructor" perspective it appeared 'gone'. I did this in a couple of course summary sections in the same development shell. I embedded a downloaded video then deleted it. When I thought the shell had been edited and cleaned of unnecessary content since it appeared to be on the "front" topic page, I later tried to clone the shell as the "master" version of the faculty's course template. The cloning failed. I got an error message that the resulting backup file was way too large. I knew the backup shouldn't have been that large, as all that 'showed' were labels, a few dummy pages, and some 80x80px icons. Thus under the assumption that the shell was edited and those videos effectively deleted. I waited for a few days for cron jobs to run.

          I also have limited admin status on my Moodle instance. After a few days, I went through the System Files to the course category & specific course folder only to find that days after I had deleted the content from the course shell--the content remained. That was in the summer/fall of 2011. I reported it to our IT and at that time there was online discussion re. Moodle 2.0 cron jobs not deleting content from the back end of courses. I had to have the IT department manually remove the 'deleted' files that were ballooning the course.

          Show
          Julia Hengstler added a comment - - edited This issue first appeared when I created a course template for my faculty in mid-2011. In the development phases, I played with embedding a video that was downloaded into the course shell. I subsequently deleted it. From the "instructor" perspective it appeared 'gone'. I did this in a couple of course summary sections in the same development shell. I embedded a downloaded video then deleted it. When I thought the shell had been edited and cleaned of unnecessary content since it appeared to be on the "front" topic page, I later tried to clone the shell as the "master" version of the faculty's course template. The cloning failed. I got an error message that the resulting backup file was way too large. I knew the backup shouldn't have been that large, as all that 'showed' were labels, a few dummy pages, and some 80x80px icons. Thus under the assumption that the shell was edited and those videos effectively deleted. I waited for a few days for cron jobs to run. I also have limited admin status on my Moodle instance. After a few days, I went through the System Files to the course category & specific course folder only to find that days after I had deleted the content from the course shell--the content remained. That was in the summer/fall of 2011. I reported it to our IT and at that time there was online discussion re. Moodle 2.0 cron jobs not deleting content from the back end of courses. I had to have the IT department manually remove the 'deleted' files that were ballooning the course.
          Hide
          Julia Hengstler added a comment -

          Now it's Jan 2012. I decided to remove icons as embedded images in my master course template. I moved all common icons to a static folder on a server external to the development shell. I edited that shell by deleting all instances of icons and images, then re-inserting them as a URL-linked image from that static file on the external server. I did that on Jan 3/2012. As admin, I went in to look, and the embedded version of those images were still in the System's course files.
          I asked our tech how often cron jobs run: I was told every 45 min and 1 main run per day.

          I returned through admin rights to look at the system files for that development shell today-- Jan 12, only to find that none of the files had been deleted by cron jobs. In a rough calculation, between when I deleted the files from the "topics" section, and when I looked on Jan 12th, approximately 224 cron jobs should have run & 7-8 main cron jobs. Despite this schedule, the image files remained in the back end of the course.

          Now I realize that for small files, this is not a huge issue: they don't take up much space. However, if my faculty uploads large files like videos, etc. and the same thing happens where the content is never really deleted it balloons the course. In fact, they will never even be truly aware of it until the day they get notice that they can't make a backup of their course, or they are running out of space.

          I thought one of the perks of Moodle 2.+ was to be the automated pruning & deleting of unused content from the back end through these cron jobs?

          Show
          Julia Hengstler added a comment - Now it's Jan 2012. I decided to remove icons as embedded images in my master course template. I moved all common icons to a static folder on a server external to the development shell. I edited that shell by deleting all instances of icons and images, then re-inserting them as a URL-linked image from that static file on the external server. I did that on Jan 3/2012. As admin, I went in to look, and the embedded version of those images were still in the System's course files. I asked our tech how often cron jobs run: I was told every 45 min and 1 main run per day. I returned through admin rights to look at the system files for that development shell today-- Jan 12, only to find that none of the files had been deleted by cron jobs. In a rough calculation, between when I deleted the files from the "topics" section, and when I looked on Jan 12th, approximately 224 cron jobs should have run & 7-8 main cron jobs. Despite this schedule, the image files remained in the back end of the course. Now I realize that for small files, this is not a huge issue: they don't take up much space. However, if my faculty uploads large files like videos, etc. and the same thing happens where the content is never really deleted it balloons the course. In fact, they will never even be truly aware of it until the day they get notice that they can't make a backup of their course, or they are running out of space. I thought one of the perks of Moodle 2.+ was to be the automated pruning & deleting of unused content from the back end through these cron jobs?
          Hide
          Michael de Raadt added a comment -

          It can take some time for clean-up processes to take place. "Trashed" files are deleted after approximately 24 hours (depending on cron regularity) and draft files after four days. It might be worth checking if these files remain after a period of time. If they do, then we have a problem.

          Show
          Michael de Raadt added a comment - It can take some time for clean-up processes to take place. "Trashed" files are deleted after approximately 24 hours (depending on cron regularity) and draft files after four days. It might be worth checking if these files remain after a period of time. If they do, then we have a problem.
          Hide
          Brent Lee added a comment -

          Hi Michael,

          We have gone approx 5 weeks and no crons have deleted. I did wait some time.

          Note, that this behaviour happens on all of my instances of Moodle, I am running 8 instances, and all have the same behaviour.

          Cheers,

          Brent.

          Show
          Brent Lee added a comment - Hi Michael, We have gone approx 5 weeks and no crons have deleted. I did wait some time. Note, that this behaviour happens on all of my instances of Moodle, I am running 8 instances, and all have the same behaviour. Cheers, Brent.
          Hide
          Petr Škoda added a comment -

          Hello,

          I suppose by "delete file" you mean edit HTML text and remove tags linking to the file - if yes there is no code that would look for these "orphaned" files that are not used in html any more. The automatic cleanup in filepool (the area where file contents are stored) is done for files that are manually deleted - but surprisingly there is no way to delete existing files that were uploaded into HTML editor (well there is a hack to do that, disable Javascript in your browser and use the fallback UI). If that is your case please vote in MDL-28019.

          Petr

          Show
          Petr Škoda added a comment - Hello, I suppose by "delete file" you mean edit HTML text and remove tags linking to the file - if yes there is no code that would look for these "orphaned" files that are not used in html any more. The automatic cleanup in filepool (the area where file contents are stored) is done for files that are manually deleted - but surprisingly there is no way to delete existing files that were uploaded into HTML editor (well there is a hack to do that, disable Javascript in your browser and use the fallback UI). If that is your case please vote in MDL-28019 . Petr
          Hide
          Debbie Mowry added a comment - - edited

          Yes, this is a huge issue because it effectively means you cannot update a course if you have any sort of completion tracking involved. I came into a situation where a course backup is 1.4GB in size because of videos. Those videos have been moved to an external source and linked to but the course remains 1.4GB in size even though the uploaded videos are gone. Going through threads I've found two things: 1) files are supposed to be fixed after 4 days 2) delete the actual original activity/resource and create new. The first doesn't work after waiting more than a week the backup is still 1.4GB. The second works but we can't implement (fix) on production because there doesn't seem to be a way to manually update completion history. If we delete the original activity/resource all the students will have to re-view everything just so we can have a manageable backup. I'm willing to manually update completion history like you would grades in a grade book to fix this issue but I can't find a way of doing that—I can export but I can't find anything that tells me how to import or manually mark something as complete that a student has already done.

          Most things in Moodle 2 have been a step up but file management from Moodle 1.9 to Moodle 2 has been a HUGE step backwards.

          Show
          Debbie Mowry added a comment - - edited Yes, this is a huge issue because it effectively means you cannot update a course if you have any sort of completion tracking involved. I came into a situation where a course backup is 1.4GB in size because of videos. Those videos have been moved to an external source and linked to but the course remains 1.4GB in size even though the uploaded videos are gone. Going through threads I've found two things: 1) files are supposed to be fixed after 4 days 2) delete the actual original activity/resource and create new. The first doesn't work after waiting more than a week the backup is still 1.4GB. The second works but we can't implement (fix) on production because there doesn't seem to be a way to manually update completion history. If we delete the original activity/resource all the students will have to re-view everything just so we can have a manageable backup. I'm willing to manually update completion history like you would grades in a grade book to fix this issue but I can't find a way of doing that—I can export but I can't find anything that tells me how to import or manually mark something as complete that a student has already done. Most things in Moodle 2 have been a step up but file management from Moodle 1.9 to Moodle 2 has been a HUGE step backwards.
          Hide
          Gianluca Amato added a comment -

          I noticed that while the Moodle 2.2 Repositories FAQ, at the question "How can I delete an embedded file previously added using the file picker?", has the answer "There is no easy way to delete files embedded in text editor areas, see MDL-28019 for more information", from Moodle 2.3 onwards the answer is "There is no need to do anything, as any unused files are detected and then deleted automatically after a period of time." However, I am using Moodle 2.4, and files embedded in HTML are not deleted. Is it a bug in the documentation, or a problem in my installation?

          Show
          Gianluca Amato added a comment - I noticed that while the Moodle 2.2 Repositories FAQ, at the question "How can I delete an embedded file previously added using the file picker?", has the answer "There is no easy way to delete files embedded in text editor areas, see MDL-28019 for more information", from Moodle 2.3 onwards the answer is "There is no need to do anything, as any unused files are detected and then deleted automatically after a period of time." However, I am using Moodle 2.4, and files embedded in HTML are not deleted. Is it a bug in the documentation, or a problem in my installation?

            People

            • Votes:
              3 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated: