Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Nov 24, 2025

Declares the C extension as GIL-optional for CPython 3.13+ free-threaded builds and enables cibuildwheel to build free-threaded wheels.

Changes

C Extension (src/numpy_quaternion.c)

  • Added guarded PyUnstable_Module_SetGIL(module, Py_MOD_GIL_NOT_USED) call in module initialization
  • Only active when Py_GIL_DISABLED is defined (free-threaded builds)
  • Includes error handling for initialization failures
#ifdef Py_GIL_DISABLED
  /* Indicate this extension does NOT require the GIL in free-threaded builds. */
  if (PyUnstable_Module_SetGIL(module, Py_MOD_GIL_NOT_USED) < 0) {
    INITERROR;
  }
#endif

CI Workflow (.github/workflows/build.yml)

  • Added CIBW_ENABLE: "cpython-freethreading" to enable building 't' ABI wheels for Python 3.13+

Thread Safety Considerations

This declares the extension GIL-optional but does not guarantee thread safety. Areas requiring review:

  • pyquaternion_get_vec and pyquaternion_get_components return NumPy array views directly into PyQuaternion structs
  • Borrowed reference patterns may need protection under concurrent access
Original prompt

Goal

Make the moble/quaternion C extension declare itself safe for free-threaded CPython builds and enable cibuildwheel to build free-threading wheels. This requires a small guarded change to the module initialization in src/numpy_quaternion.c and a CI update so cibuildwheel builds the free-threading 't' wheels.

Background

  • The project defines a C extension at src/numpy_quaternion.c using single-phase initialization (PyModule_Create in PyInit_numpy_quaternion). On free-threaded CPython builds (GIL disabled) an extension must explicitly indicate it is GIL-optional; otherwise importing it will re-enable the GIL at runtime and print a warning.
  • CPython provides PyUnstable_Module_SetGIL(module, Py_MOD_GIL_NOT_USED) in free-threaded builds; it only exists when Py_GIL_DISABLED is defined. The recommended approach for single-phase modules is to call PyUnstable_Module_SetGIL guarded by #ifdef Py_GIL_DISABLED.
  • The repository uses GitHub Actions + cibuildwheel in .github/workflows/build.yml to build wheels. To build free-threaded wheels for CPython 3.13, cibuildwheel must be told to enable the cpython-freethreading build via the CIBW_ENABLE environment variable (e.g. CIBW_ENABLE=cpython-freethreading). On CPython 3.14 free-threaded builds are enabled by default in many toolchains, but enabling via CIBW_ENABLE is harmless.

Required changes (detailed and actionable)

  1. src/numpy_quaternion.c
  • After creating the module object (module = PyModule_Create(&moduledef);) and checking for NULL, insert a guarded call to PyUnstable_Module_SetGIL(module, Py_MOD_GIL_NOT_USED) using #ifdef Py_GIL_DISABLED. This is necessary so free-threaded interpreters do not re-enable the GIL when importing this extension.
  • The call must be placed before the function continues to register NumPy types/ufuncs and before returning the module object.
  • Example insertion (exact):
  if(module==NULL) {
    INITERROR;
  }

#ifdef Py_GIL_DISABLED
  /* Indicate this extension does NOT require the GIL in free-threaded builds. */
  PyUnstable_Module_SetGIL(module, Py_MOD_GIL_NOT_USED);
#endif
  1. .github/workflows/build.yml
  • Add CIBW_ENABLE=cpython-freethreading to the build_wheels job environment so cibuildwheel will build free-threaded wheels where supported (notably Python 3.13). The existing job already sets many CIBW_ environment variables; add CIBW_ENABLE to that env map.
  • Example change in the build_wheels job env block:
    env:
      CIBW_ARCHS: ${{matrix.archs}}
      CIBW_ARCHS_MACOS: "x86_64 universal2 arm64"
      CIBW_MANYLINUX_AARCH64_IMAGE: "manylinux_2_28"
      CIBW_BEFORE_BUILD: python -c "print(('#'*130+'\n')*10)" && python -m pip install "numpy>=2.0,<3"
      CIBW_TEST_REQUIRES: pytest pytest-cov
      CIBW_TEST_COMMAND: "pytest {project}/tests --durations=0"
      CIBW_SKIP: "*-win32 *-manylinux_i686 cp36-* cp37-* cp38-* cp39-* pp* *-musllinux*"
      CIBW_ENABLE: "cpython-freethreading"
      ...

Additional notes and follow-up recommendations

  • The guarded call only tells the interpreter the extension claims it doesn't need the GIL. You must still ensure the code is thread-safe. I inspected the repository and identified patterns that need careful review:

    • pyquaternion_get_vec and pyquaternion_get_components create NumPy arrays that point directly to fields inside a PyQuaternion instance (PyArray_SimpleNewFromData using &q->x / &q->w). Returning such views with concurrent modifications may be unsafe. Consider returning copies instead or protecting access with critical sections.
    • Search for uses of borrowed-reference-returning APIs (PyList_GetItem, PyDict_GetItem, PyDict_Next, etc.) and replace with strong-reference variants where appropriate or guard with critical sections.
    • Verify no use of PyObject_Malloc for non-Python-object buffers (use PyMem_* for buffers).
    • If your build uses py_limited_api=True/stable ABI, note that free-threaded builds currently do not support the limited C API/stable ABI. I did not find py_limited_api=True in the project files you supplied, but adjust packaging if needed: py_limited_api = not sysconfig.get_config_var("Py_GIL_DISABLED")
  • CI: After this PR, run CI with an actual free-threaded Python (cibuildwheel will build free-threaded wheels on supported images when CIBW_ENABLE is set) and run tests, especially multi-threaded stress tests and scenarios that create NumPy views into PyQuaternion instances from different threads.

What I'll include in the PR

  • Modified src/numpy_quaternion.c with the guarded PyUnstable_Module_SetGIL call inserted in PyInit_numpy_quaternion (Python 3 init path) and in the Python 2 legacy init path if applicable.
  • Modified .github/workflows/build.yml with CIBW_ENABLE set to "cpython-freethreading" in the build_wheels job env.

Files to change

  • src/numpy_quaternion.c (add PyUnstable_Module_SetGIL guard)
  • .github/workflows/build.yml (add CIBW_ENABLE env)

Testing to run after merging

  • Confirm the wheel build matrix in CI pr...

This pull request was created as a result of the following prompt from Copilot chat.

Goal

Make the moble/quaternion C extension declare itself safe for free-threaded CPython builds and enable cibuildwheel to build free-threading wheels. This requires a small guarded change to the module initialization in src/numpy_quaternion.c and a CI update so cibuildwheel builds the free-threading 't' wheels.

Background

  • The project defines a C extension at src/numpy_quaternion.c using single-phase initialization (PyModule_Create in PyInit_numpy_quaternion). On free-threaded CPython builds (GIL disabled) an extension must explicitly indicate it is GIL-optional; otherwise importing it will re-enable the GIL at runtime and print a warning.
  • CPython provides PyUnstable_Module_SetGIL(module, Py_MOD_GIL_NOT_USED) in free-threaded builds; it only exists when Py_GIL_DISABLED is defined. The recommended approach for single-phase modules is to call PyUnstable_Module_SetGIL guarded by #ifdef Py_GIL_DISABLED.
  • The repository uses GitHub Actions + cibuildwheel in .github/workflows/build.yml to build wheels. To build free-threaded wheels for CPython 3.13, cibuildwheel must be told to enable the cpython-freethreading build via the CIBW_ENABLE environment variable (e.g. CIBW_ENABLE=cpython-freethreading). On CPython 3.14 free-threaded builds are enabled by default in many toolchains, but enabling via CIBW_ENABLE is harmless.

Required changes (detailed and actionable)

  1. src/numpy_quaternion.c
  • After creating the module object (module = PyModule_Create(&moduledef);) and checking for NULL, insert a guarded call to PyUnstable_Module_SetGIL(module, Py_MOD_GIL_NOT_USED) using #ifdef Py_GIL_DISABLED. This is necessary so free-threaded interpreters do not re-enable the GIL when importing this extension.
  • The call must be placed before the function continues to register NumPy types/ufuncs and before returning the module object.
  • Example insertion (exact):
  if(module==NULL) {
    INITERROR;
  }

#ifdef Py_GIL_DISABLED
  /* Indicate this extension does NOT require the GIL in free-threaded builds. */
  PyUnstable_Module_SetGIL(module, Py_MOD_GIL_NOT_USED);
#endif
  1. .github/workflows/build.yml
  • Add CIBW_ENABLE=cpython-freethreading to the build_wheels job environment so cibuildwheel will build free-threaded wheels where supported (notably Python 3.13). The existing job already sets many CIBW_ environment variables; add CIBW_ENABLE to that env map.
  • Example change in the build_wheels job env block:
    env:
      CIBW_ARCHS: ${{matrix.archs}}
      CIBW_ARCHS_MACOS: "x86_64 universal2 arm64"
      CIBW_MANYLINUX_AARCH64_IMAGE: "manylinux_2_28"
      CIBW_BEFORE_BUILD: python -c "print(('#'*130+'\n')*10)" && python -m pip install "numpy>=2.0,<3"
      CIBW_TEST_REQUIRES: pytest pytest-cov
      CIBW_TEST_COMMAND: "pytest {project}/tests --durations=0"
      CIBW_SKIP: "*-win32 *-manylinux_i686 cp36-* cp37-* cp38-* cp39-* pp* *-musllinux*"
      CIBW_ENABLE: "cpython-freethreading"
      ...

Additional notes and follow-up recommendations

  • The guarded call only tells the interpreter the extension claims it doesn't need the GIL. You must still ensure the code is thread-safe. I inspected the repository and identified patterns that need careful review:

    • pyquaternion_get_vec and pyquaternion_get_components create NumPy arrays that point directly to fields inside a PyQuaternion instance (PyArray_SimpleNewFromData using &q->x / &q->w). Returning such views with concurrent modifications may be unsafe. Consider returning copies instead or protecting access with critical sections.
    • Search for uses of borrowed-reference-returning APIs (PyList_GetItem, PyDict_GetItem, PyDict_Next, etc.) and replace with strong-reference variants where appropriate or guard with critical sections.
    • Verify no use of PyObject_Malloc for non-Python-object buffers (use PyMem_* for buffers).
    • If your build uses py_limited_api=True/stable ABI, note that free-threaded builds currently do not support the limited C API/stable ABI. I did not find py_limited_api=True in the project files you supplied, but adjust packaging if needed: py_limited_api = not sysconfig.get_config_var("Py_GIL_DISABLED")
  • CI: After this PR, run CI with an actual free-threaded Python (cibuildwheel will build free-threaded wheels on supported images when CIBW_ENABLE is set) and run tests, especially multi-threaded stress tests and scenarios that create NumPy views into PyQuaternion instances from different threads.

What I'll include in the PR

  • Modified src/numpy_quaternion.c with the guarded PyUnstable_Module_SetGIL call inserted in PyInit_numpy_quaternion (Python 3 init path) and in the Python 2 legacy init path if applicable.
  • Modified .github/workflows/build.yml with CIBW_ENABLE set to "cpython-freethreading" in the build_wheels job env.

Files to change

  • src/numpy_quaternion.c (add PyUnstable_Module_SetGIL guard)
  • .github/workflows/build.yml (add CIBW_ENABLE env)

Testing to run after merging

  • Confirm the wheel build matrix in CI produces free-threaded wheel artifacts (they will have a "t" tag for manylinux/python ABI). Verify there are no warnings about non-free-threaded extensions re-enabling the GIL.
  • Run unit tests under a free-threaded interpreter (3.13 built with --disable-gil) or rely on cibuildwheel test stage for those wheel builds.

Please create a branch and PR that implements these changes. If you want I can also add a unit test and a short note in the README/CHANGES about free-threading support and the caveats (views into extension-owned memory).


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 3 commits November 24, 2025 14:15
- Add PyUnstable_Module_SetGIL call in src/numpy_quaternion.c
- Add CIBW_ENABLE=cpython-freethreading in build workflow

Co-authored-by: moble <[email protected]>
Check return value and handle errors properly

Co-authored-by: moble <[email protected]>
Copilot AI changed the title [WIP] Add free-threading support for numpy quaternion extension Enable free-threaded CPython support Nov 24, 2025
Copilot AI requested a review from moble November 24, 2025 14:20
Copilot finished work on behalf of moble November 24, 2025 14:20
@moble moble force-pushed the copilot/enable-free-threading-support branch from ca42aad to 7a699bd Compare November 24, 2025 15:02
@moble moble force-pushed the copilot/enable-free-threading-support branch from 7a699bd to 7986f11 Compare November 24, 2025 15:03
@moble moble merged commit 7986f11 into main Nov 24, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants