RFC: Storage providers

[Daniel15]

This is something I'd like to implement in Bigcapital, but I'd like to get feedback on the API design first.

Problem

Bigcapital requires the use of S3. However, when self-hosting, it's beneficial to be able to just store the files locally. In my case, Bigcapital will be running on a VPS with 150GB disk space, so it makes sense to just store the files locally rather than paying for an S3-compatible service like S3, Backblaze B2, Cloudflare R2, etc.

The requirement to use S3 can also cause confusion during setup (e.g. #846 and #785, plus I've seen it discussed on Discord).

Solution

Remove the tight coupling with S3 and abstract the handling of files out into interfaces:

interface StorageProvider {
  /**
   * Validate that the storage provider is configured correctly.
   */
  async validateConfig(): Promise<void>;

  /**
   * Store a file. Returns an object representing the uploaded file.
   */
  async upload(file): Promise<File>;

  /**
   * Gets a `File` instance given a fileKey from the database.
   */
  getFileFromKey(fileKey: string): File;
}

interface File {
  /**
  * File path, relative to the root of the storage provider.
  */
  path: string;

  /**
   * Builds a URL to this file.
   */
  buildURL(): string;

  /**
   * Gets the contents of this file.
   */
  async getContents(): Promise<string>;
}

Initially, implement S3StorageProvider and LocalStorageProvider.

[linear]

BIG-319

[abouolia]

Yes, that sounds like a great idea. Since S3 uploading is being done with Multer S3, properly can follow the same pattern but with local provider. and switching between the two providers would just by a new env variable. I’d be happy to review the PR. @Daniel15