Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-71990

File API seems to be capable of silently losing files (very rarely)

    XMLWordPrintable

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 3.10.4
    • None
    • Files API
    • None
    • MOODLE_310_STABLE

    Description

      I just detected a pretty worrying problem with a file:

      All the data in the database is consistent. There is a row in mdl_files saying this file exists, which relates to a particular quiz attempt which is also present and correct.

      However, when you look up the contenthash in filedir, the file is not there, and (and this leads to errors being logged when people try to access the file, which is what lead me to this).

      I konw this is a tricky distributed commit situation between the file system and the database, but I would have expected the code to make some effort to ensure consistency.

      However, brendanheywood commented:

      its not uncommon, most migrations we do from other hosting has some base error rate of missing files, and I think there are very small race conditions that lead to this. Error rate of around 2 per million

      we've internally talked about swapping the write order to disk and then db, but then you might end up with the reverse situation, it probably needs to be something like a file journal, eg write a temp record, write the file, then update the record.

      the error rate is high enough for us to notice but low enough for us to have not chased it much yet

      The other odd thing is that I don't see any errors logged from the time when the file was written. That means the student had no idea that their file was not saved (and so no hint that they should re-try.) At the very least, we need to try to ensure that the file API throws an exception if the file is not writted to disc.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              timhunt Tim Hunt
              Matteo Scaramuccia, David Woloszyn, Huong Nguyen, Jake Dallimore, Meirza, Michael Hawkins, Raquel Ortega, Safat Shahin, Stevani Andolo
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: