Deprecate and Remove LevelDB Subsystem

[janl]

Introduction

Running npm i creates a large number of warning about out of date dependencies. Many of these come from the LevelDB ecosystem, that CouchDB uses for Node.js persistence.

I have tried to update our dependencies to latest LevelDB releases, but I found two major issues:

1. We are relying on an older version of the ecosystem. It has undergone a major rewrite and we’d have to adapt to the new system in many many places.

2. The new ecosystem is also barely maintained. We have open issues about npm i failing on ARM systems because of a Python build incompatibility.

I do not think going forward, the LevelDB is a good choice for Node.js persistence for PouchDB.

The Future

Instead, Node.js and other server-side JS runtimes have adopted SQLite as a standard part of their distributions.

And PouchDB used to have a WebSQL adapter (that is compatible with SQLite).

I have done a proof of concept to have PouchDB run on Node.js with its built-in SQLite module and it works fine, thanks to all the existing code we have from the WebSQL days.

This would also open us to support the other runtimes if we wanted to, and it makes it easier to bundle PouchDB with native-wrapped mobile web apps that want to use SQLite for better persistence (WebKit/Safari evacuate all local data after 7 days).

I am happy to provide my proof of concept as a starting point for this transition to the project.

The Path

To make sure we do not let folks that still rely on LevelDB, we should support a forward migration path. This is what this could look like:

1. Release a PouchDB version that adds SQLite support to node builds and keeps LevelDB support as is.

• People using LevelDB can use this version to do a one-time migration via replication to SQLite

2. The next major version of PouchDB after that will remove the LevelDB ecosystem and make PouchDB more lean and lighter of deprecated or abandoned dependencies.

Thanks to @alxndrsn for helping refine this plan.

Consequences

The LevelDB abstraction is very useful and supports multiple backends, not just LevelDB. The project would lose this additional flexibility, but given where the JS ecosystem is going, I think this is the right call to make today.

One of these abstractions is an in-memory adapter for PouchDB which is used in testing of PouchDB and its plugins a lot. The replacement SQLite adapter should allow for SQLite in-memory databases to achieve the same.

Discussion

What do you think?

[pjfanning]

I don't have a strong opinion but https://github.com/facebook/rocksdb is a better maintained fork of LevelDB.

[janl]

@pjfanning that does not help with the out-of-dateness of the Level-ecosystem that we would have to put in between PouchDB and RocksDB. I assume LevelDB itself is also still supported, the problem is the Node.js goop that PouchDB uses as an abstraction layer that is out of date.

[gladson]

#9100, #9071