Skip to content

Dead-lock and large dangling files #294

@TheQuantumPhysicist

Description

@TheQuantumPhysicist

Describe the bug
I allowed large file uploads because I use MicroBin only in my local network. But then after starting a large upload (700 MB), I refreshed the page while it was at ~50% by mistake, and that lead to many problems:

  1. The instance was completely unreachable and I had to force-restart the container (dead-lock, apparently)
  2. The partial upload artifact remained in the attachments directory as a dangling file with no reference, bloating the storage required for MicroBin.
  3. While the upload was happening, the instance wasn't reachable from other machines. Now I understand this may not be an issue and might be desired behavior (seems like the implementation is that does the work is not concurrent async), but it's worth mentioning because it might be an issue with a bad lock algorithm of the database. I can imagine this could happen if the database is locked before the upload and waits for the upload to finish, and all other instances wait for the lock to expire before being able to query the website.

To Reproduce
Steps to reproduce the behavior:

  1. Upload a very large file
  2. When it's in the middle, refresh the page
  3. Now the instance is dead. The attachment is a dangling file that can only be deleted manually by the admin controlling the server.

Expected behavior

  1. A marker should be registered for new files that gets them deleted if they're incomplete.
  2. You might wanna consider allowing async operation where concurrent calls to the website is allowed (this is a wish, not a demand, please don't misunderstand me, thank you for the great work). I understand this might be an sqlite issue due to its unique locks to the file, and if this is the case, this has to be done in multiple steps, where first we lock the file, write a "started" marker entry, then unlock, and then decide what to do when the upload it finished. If a restart is done with the marker of "started" but not "finished upload", then the file is deleted automatically at some point as part of some regular cleanups or something.

Thank you for the great work. Your tool is awesome and very useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions