ES Module Migration Plan for rclnodejs

[mahmoud-ghalayini]

Goal

Migrate rclnodejs from CommonJS to ES Modules, enabling modern import { Node } from 'rclnodejs' syntax while maintaining backward compatibility for existing CJS consumers.

File Inventory

Category	Count	Notes
lib/*.js	50 files	Core library (all use module.exports)
rosidl_gen/*.js	13 files	9 generators + 4 templates
rosidl_parser/*.js	2 files	IDL parser utilities
rostsd_gen/*.js	1 file	TypeScript declaration generator
types/*.d.ts	36 files	TypeScript declarations
index.js	1 file	Main entry point

Dependencies ESM Status

Package	Status	Strategy
rxjs	✅ ESM native	Direct import
debug	✅ ESM native	Direct import
bindings	⚠️ CJS only	createRequire()
walk	⚠️ CJS only	createRequire()
json-bigint	⚠️ CJS only	createRequire()
@rclnodejs/ref-struct-di	⚠️ CJS only	createRequire()
@rclnodejs/ref-array-di	⚠️ CJS only	createRequire()
third_party/ref-napi	⚠️ CJS (bundled)	Keep as CJS
node-addon-api	Build-time only	No change needed

Critical Issues Identified

1. Circular Dependencies

• lib/rate.js → index.js: Creates Rate instances using rclnodejs.init() and rclnodejs.createNode()
• index.js → lib/node.js → lib/lifecycle.js: Deferred imports at bottom of index.js
• Solution: Convert to dynamic import() or refactor dependency injection

2. Dynamic Requires in Generated Code

• rosidl_gen/templates/message-template.js generates CJS code with:

const ref = require('../../third_party/ref-napi');
const StructType = require('@rclnodejs/ref-struct-di')(ref);

• Decision: Keep generated code as CJS (files go to generated/ directory)

3. Native Module Loading

• lib/native_loader.js uses bindings('rclnodejs')
• Solution: Wrap with createRequire()

---

Migration Phases

Phase 1: Preparation

Files: package.json, scripts/*.js, binding.gyp, rosidl_gen/*.js, rosidl_parser/*.js, rostsd_gen/*.js

Rename all files that must remain CJS to .cjs extension:

1. Rename scripts/*.js → scripts/*.cjs ✅ (Complete)
2. Rename rosidl_gen/*.js → rosidl_gen/*.cjs
3. Rename rosidl_parser/*.js → rosidl_parser/*.cjs
4. Rename rostsd_gen/*.js → rostsd_gen/*.cjs
5. Update all references in package.json, binding.gyp, and internal requires
6. Ensure tests still pass before starting conversion

---

Phase 2: Core Library + Entry Point

Files: 50 JS files in lib/, index.js

⚠️ Atomic Change: All steps below must be done together to maintain testability
We split files into fewer PRs but pipelines won't work until then

• For each file:
  • Remove 'use strict';
  • const X = require('./x.js') → import X from './x.js'
  • const { A, B } = require('./x.js') → import { A, B } from './x.js'
  • module.exports = X → export default X
  • module.exports = { A, B } → export { A, B }
• Special handling:
  • native_loader.js - Use createRequire() for CJS-only bindings package
  • rate.js - Dynamic import for circular dependency
• Add "type": "module" to package.json
• Test: npm test should pass

---

Phase 3: Package Configuration & Bundler

Files: package.json, tsup.config.js

After source conversion is complete, enable dual package exports:

{
  "type": "module",
  "engines": { "node": ">= 18.0.0" },
  "main": "./dist/index.cjs",
  "module": "./dist/index.js",
  "exports": {
    ".": {
      "import": {
        "types": "./types/index.d.ts",
        "default": "./dist/index.js"
      },
      "require": {
        "types": "./types/index.d.ts",
        "default": "./dist/index.cjs"
      }
    },
    "./generated/*": "./generated/*"
  }
}

Use tsup to bundle ESM source → CJS output for backward compatibility:

// tsup.config.js
export default {
  entry: ['index.js'],
  format: ['esm', 'cjs'],
  outDir: 'dist',
  external: [/generated\//, /third_party\//],
};

---

Phase 4: TypeScript Declarations

Files: 36 .d.ts files in types/

• Update types/index.d.ts for ESM:

declare module 'rclnodejs' {
  export const Node: typeof import('./node').Node;
  // ... named exports
  export default rcl;
}

---

Phase 5: Generator/Parser Modules (Optional)

Files: 16 .cjs files in rosidl_gen/, rosidl_parser/, rostsd_gen/

Optionally convert generator modules from CJS to ESM:

• Convert .cjs files to ESM syntax
• Rename back to .js
• Update scripts/*.cjs to use dynamic import() for these modules
• Keep generated output as CJS (templates still produce require() code)

💡 This phase is optional. The .cjs files work fine and can remain as CJS indefinitely.

---

Out of Scope (Deferred)

Category	Files	Notes
scripts/	8 JS files	Already renamed to .cjs, remain CJS
test/	All test files	Can convert later or use --experimental-vm-modules
example/	All examples	Update as documentation
third_party/ref-napi	2 files	Must stay CJS (native bindings)

---

Backward Compatibility Strategy

Consumer	Syntax	Works After Migration?
ESM	import rclnodejs from 'rclnodejs'	✅ Yes (native)
ESM	import { Node } from 'rclnodejs'	✅ Yes (native)
CJS	const rclnodejs = require('rclnodejs')	✅ Yes (via bundled CJS)
Generated	require('./generated/...')	✅ Yes (stays CJS)

---

Risks & Mitigations

Risk	Likelihood	Mitigation
FFI packages break	Medium	Wrap with createRequire(), test thoroughly
Circular deps cause runtime errors	Medium	Use dynamic import(), refactor if needed
Generated code compatibility	Low	Keep as CJS, test message serialization
Type definitions break	Low	Run tsd after each phase
Native addon loading fails	Low	Test across Node 18/20/22
Bundler output issues	Low	Test require('rclnodejs') in CJS project

---

Success Criteria

• import rclnodejs from 'rclnodejs' works
• import { Node, Clock } from 'rclnodejs' works
• const rclnodejs = require('rclnodejs') works (backward compat)
• All existing tests pass
• Generated message files load correctly
• TypeScript types resolve properly
• Examples run successfully

---

Summary of Changes from Original Plan

1. Combined Phases - Convert lib/ and index.js together for testability
2. Added temporary CJS isolation - Rename generator modules to .cjs
3. Added testability checkpoints - Each phase can be tested independently
4. Reordered phases - Convert source FIRST, bundler LAST

[minggangw]

Thanks for listing the steps for migration, it's smooth. BTW, how will we keep the backward compatibility for CommonJS, are we going to leverage bundle tool, like tsup?

[mahmoud-ghalayini]

Actually backward compatibility was for generated files, since we generate .js files - we can change the generator to output .cjs files later if needed.
But if you want to provide both syntaxes we can do it natively without any bundling tool using Node.js dual package exports:

{
  "type": "module",  // Makes .js files ESM by default
  "main": "./index.cjs",  // Fallback for older Node.js
  "exports": {
    ".": {
      "import": "./index.js",    // ESM users get this
      "require": "./index.cjs",  // CJS users get this
      "types": "./types/index.d.ts"
    },
    "./generated/*": "./generated/*"  // Generated CJS files
  }
}

• import rclnodejs from 'rclnodejs' → loads index.js (ESM)
• const rclnodejs = require('rclnodejs') → loads index.cjs (CJS)

This file must use .cjs extension (or be in a folder with package.json with "type": "commonjs"). It dynamically imports the ESM module and re-exports everything.

// index.cjs - CJS wrapper
let rclnodejs = null;
const initPromise = (async () => {
  const mod = await import('./index.js');
  rclnodejs = mod.default || mod;
  return rclnodejs;
})();

// Export a proxy that handles async loading
module.exports = new Proxy({}, {
  get(target, prop) {
    if (rclnodejs) {
      return rclnodejs[prop];
    }
    // If not loaded yet, return a promise-returning getter
    return initPromise.then(mod => mod[prop]);
  }
});

Note: This makes property access async for CJS users on first load. If you need fully synchronous CJS support, we'd need a bundler to create a true CJS build. But for most use cases, this native approach should work fine.

[minggangw]

So after the migration is done, will

const rclnodejs = require('rclnodejs');

still work?

[mahmoud-ghalayini]

Good question - I need to be clear here because my earlier explanation was misleading.

Will const rclnodejs = require('rclnodejs'); still work? Not without some extra work.

Here's the reality: once you convert the source code to ESM, CommonJS's require() can't load it synchronously anymore. It's just a limitation of how Node.js works. So we have a few paths forward:

Use a bundler (my recommendation)

This is what most libraries do - tools like tsup or esbuild take your ESM source code and generate a separate CommonJS build. It's a one-time setup that adds a build step, but then everything just works. Users on Node 16, 18, 20, 22... they all get backward compatibility. require('rclnodejs') keeps working exactly as it does today.

Make it a breaking change

Release v2.0.0, we could embrace the breaking change. Users would need to switch from require() to import, which honestly isn't that big of a deal. The upside is no bundler, simpler build process. The downside is it breaks existing code.

Require Node.js 22.12+ (not recommended)

Node 22.12 added an experimental flag that lets you require() ESM modules, but it's still experimental and forces everyone to upgrade to the latest Node version. I wouldn't go this route for a production library.

So realistically, it's between using a bundler or accepting the breaking change. Given that rclnodejs is infrastructure-level code that a lot of projects depend on, I'd lean toward the bundler approach for smoother migration. But if you're okay with the v2.0 breaking change, that's valid too.

What sounds better for you?

[minggangw]

Use a bundler (my recommendation)

This is what most libraries do - tools like tsup or esbuild take your ESM source code and generate a separate CommonJS build. It's a one-time setup that adds a build step, but then everything just works. Users on Node 16, 18, 20, 22... they all get backward compatibility. require('rclnodejs') keeps working exactly as it does today.

I also prefer this solution, it seems mature and feasible, not breaking the projects that leverage the rclnodejs prior to the migration. I'm not clear about the effort using the bundler, it seems not so big.

[mahmoud-ghalayini]

I did update the migration plan, actually I wanted to make it fully testable in every steps seems doable, but let's see

[minggangw]

I did update the migration plan, actually I wanted to make it fully testable in every steps seems doable, but let's see

Thanks for the update, I will take a look.

[mahmoud-ghalayini]

One important finding 🥲
const rclnodejs = require('rclnodejs'); will work using tsup, but this syntax won't work inside the project itself. for example, we will need to update the tests to use import rclnodejs from '../index.js; instead of const rclnodejs = require('../index.js'); etc...
I couldn't find a proper work around for the tests, but I found a work around for rosidl_convertor, rosidl_gen, ros_parser, and ref-napi