Use Node.js native sqlite instead of better-sqlite3 #1357

i-am-the-slime · 2026-01-04T18:14:13Z

Replace better-sqlite3 with Node.js's built-in node:sqlite module to avoid native compilation dependencies.

Changes

Replaced @libsql/client (turso) with Node.js native node:sqlite module
Removed external dependency from package.json
Implementation closely follows the original better-sqlite3 code style
Added explicit directory creation and Promise wrapping for FFI compatibility
Suppressed experimental feature warnings in the entry point

Benefits

Zero external dependencies for SQLite functionality
No native compilation required (no node-gyp)
Built-in Node.js module - will stabilize over time
Minimal code changes from the original better-sqlite3 implementation
Same performance - local file-based SQLite database

Testing

All tests pass: 235/239 (same as baseline - 4 pre-existing failures unrelated to SQLite changes)

Note

The node:sqlite module is marked as experimental in Node.js, but it's stable enough for our use case. The warning is suppressed in the CLI entry point.

wclr · 2026-01-05T06:56:17Z

Maybe it is better to move to nodejs build-in sql support https://nodejs.org/api/sqlite.html (available from Node 22)

Spago next is a new tool and still not widely used so I don't think that there need to worry about old node version and ignore node platform development.

i-am-the-slime · 2026-01-05T10:01:42Z

@wclr sure, let's get some buy-in first from the maintainers and then we can also go that route.

Main motivation is to avoid having to rebuild spago for different node versions.

Replaces @libsql/client with Node.js's built-in node:sqlite module. Changes: - Replace libsql/turso client with DatabaseSync from node:sqlite - Remove @libsql/client dependency from package.json - Remove --external:@libsql/client from bundler config - Add NODE_NO_WARNINGS=1 to suppress experimental sqlite warnings - Add explicit directory creation (was built-in with better-sqlite3) - Use db.exec() for PRAGMA statements (db.pragma() doesn't exist) - Wrap all operations in Promise.resolve() for FFI compatibility Benefits: - Zero external dependencies for SQLite - Simpler implementation (no remote database support needed) - Nearly identical API to the original better-sqlite3 version All tests pass (235/239, same as baseline - 4 pre-existing failures unrelated to SQLite changes).

- Add missing getMetadataImpl export (was declared in FFI but not exported) - Use --no-warnings flag to suppress experimental SQLite warnings - In bin/index.dev.js shebang for direct execution - In test/Prelude.purs when spawning spago in tests - Add trailing newline to Db.js

f-f · 2026-01-18T21:54:12Z

bin/index.dev.js

@@ -1,4 +1,4 @@
-#!/usr/bin/env node
+#!/usr/bin/env -S node --no-warnings


Removing all the warnings is a terrible idea for a set of reasons 😄

Let's just exclude the warnings for experimental modules: --no-warnings=ExperimentalWarning

f-f · 2026-01-18T22:00:53Z

bin/src/Main.purs

  FS.mkdirp Paths.globalCachePath
  logDebug $ "Global cache: " <> Path.quote Paths.globalCachePath

  -- Make sure we have git and purs


Node will fail with a horrible error if it doesn't find the module, so I think we should catch it at the startup and error out then.

Implementation sketch:

import Data.Int as Int import Data.String as String import Effect.Console as Console import Node.Process as Process -- first thing inside main: main = do -- Check that we're running on Node.js >= 22.5.0 (required for node:sqlite) let -- version is like "v22.5.0" or "22.5.0" versionStr = String.stripPrefix (String.Pattern "v") Process.version # fromMaybe Process.version parts = String.split (String.Pattern ".") versionStr case Array.take 2 parts >>= traverse Int.fromString of Just [major, minor] | major > 22 -> pure unit | major == 22 && minor >= 5 -> pure unit | otherwise -> do Console.error $ "Error: spago requires Node.js v22.5.0 or later (found " <> Process.version <> ")" Process.exit' 1 _ -> do -- couldn't parse, issue a warning and let it crash later ..

together with this we should also add the engines field to the package.json. Node won't enforce it on install, but will complain about it:

{ "engines": { "node": ">=22.5.0" } }

f-f · 2026-01-18T22:01:33Z

src/Spago/Db.js

+        fs.mkdirSync(dir, { recursive: true });
+      }
+    } catch (err) {
+      // ignore - will fail later if needed


why are we ignoring the error here?

f-f · 2026-01-18T22:05:55Z

src/Spago/Db.js

+      .prepare("SELECT * FROM package_sets WHERE compiler = ? ORDER BY date DESC LIMIT 1")
+      .get(compiler);


we use positional arguments in a few places, we should use them everywhere, e.g.:

Suggested change

.prepare("SELECT * FROM package_sets WHERE compiler = ? ORDER BY date DESC LIMIT 1")

.get(compiler);

.prepare("SELECT * FROM package_sets WHERE compiler = @compiler ORDER BY date DESC LIMIT 1")

.get({ compiler });

f-f · 2026-01-18T22:06:55Z

src/Spago/Db.js

-    .all();
-  return row;
-}
+  return Promise.resolve().then(() => {


The existing code is in Effect, and this database code is sync as well, why turn everything into Promises?

@f-f

- Use Node.js native node:sqlite module (requires Node >= 22.5.0). - Refactor DB layer to use synchronous API and named parameters. - Add Node.js version check at startup. - Suppress ExperimentalWarning in dev and bundle entry points. - Address review comments from @f-f.

@f-f

- Use Node.js native node:sqlite module (requires Node >= 22.5.0). - Refactor DB layer to use synchronous API and named parameters. - Add Node.js version check at startup with helpful warning. - Scope ExperimentalWarning suppression to the database connection logic. - Update package.json engines field. - Update bin/index.dev.js shebang to suppress ExperimentalWarning. - Address review comments from @f-f.

Use turso/libsql instead of better-sqlite3

156ef65

i-am-the-slime changed the title ~~Use turso instead of better-sqlite to avoid gyp dependency~~ Use Node.js native sqlite instead of better-sqlite3 Jan 12, 2026

f-f reviewed Jan 18, 2026

View reviewed changes

i-am-the-slime marked this pull request as draft January 23, 2026 09:28

i-am-the-slime force-pushed the turso-libsql branch 2 times, most recently from af35edb to ee76757 Compare January 23, 2026 22:34

i-am-the-slime force-pushed the turso-libsql branch from ee76757 to d394531 Compare January 23, 2026 23:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use Node.js native sqlite instead of better-sqlite3 #1357

Use Node.js native sqlite instead of better-sqlite3 #1357

i-am-the-slime commented Jan 4, 2026 •

edited

Loading

Uh oh!

wclr commented Jan 5, 2026

Uh oh!

i-am-the-slime commented Jan 5, 2026

Uh oh!

f-f Jan 18, 2026

Uh oh!

f-f Jan 18, 2026

Uh oh!

f-f Jan 18, 2026

Uh oh!

f-f Jan 18, 2026

Uh oh!

f-f Jan 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -1,4 +1,4 @@
		#!/usr/bin/env node
		#!/usr/bin/env -S node --no-warnings

		.prepare("SELECT * FROM package_sets WHERE compiler = ? ORDER BY date DESC LIMIT 1")
		.get(compiler);

Use Node.js native sqlite instead of better-sqlite3 #1357

Are you sure you want to change the base?

Use Node.js native sqlite instead of better-sqlite3 #1357

Conversation

i-am-the-slime commented Jan 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Benefits

Testing

Note

Uh oh!

wclr commented Jan 5, 2026

Uh oh!

i-am-the-slime commented Jan 5, 2026

Uh oh!

f-f Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

f-f Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

f-f Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

f-f Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

f-f Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

i-am-the-slime commented Jan 4, 2026 •

edited

Loading