From ebcfa8e1d4e5bf0a68fcf3a327b4942bcf50db3f Mon Sep 17 00:00:00 2001 From: jaimergp Date: Tue, 17 Dec 2024 22:22:27 +0100 Subject: [PATCH 01/26] Write CEP about virtual packages --- cep-????.md | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+) create mode 100644 cep-????.md diff --git a/cep-????.md b/cep-????.md new file mode 100644 index 00000000..0d830b8c --- /dev/null +++ b/cep-????.md @@ -0,0 +1,100 @@ +# CEP ???? - Virtual packages + + + + + + + + + +
Title Virtual packages
Status Draft
Author(s) Jaime Rodríguez-Guerra <jaime.rogue@gmail.com>
Created Dec 17, 2024
Updated Dec 17, 2024
Discussion NA
Implementation Several
+ +> The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", + "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as + described in [RFC2119][RFC2119] when, and only when, they appear in all capitals, as shown here. + +## Abstract + +This CEP standardizes which virtual packages MUST be offered by conda solvers. + +## Specification + +A virtual package is defined as a package record with three fields: name, version and build string. +The name MUST start with double underscore. The version and build string MUST follow the same semantics as in regular package records. + +### List of virtual packages + +In alphabetical order, every conda client MUST support the following virtual packages: + +- `__archspec` +- `__cuda` +- `__glibc` +- `__linux` +- `__osx` +- `__unix` +- `__win` + +#### `__archspec` + +This virtual package MUST be always present, with the version set to `1`. The build string SHOULD reflect the detected CPU microarchitecture. If it cannot be detected, the build string SHOULD be `0`. + +The build string MUST be overridable with the `CONDA_OVERRIDE_ARCHSPEC` environment variable, if set to a non-empty value. + +#### `__cuda` + +This virtual package MUST be present when the system exhibits GPU drivers compatible with the CUDA runtimes. When available, the version value MUST be set to the oldest CUDA version supported by the detected drivers (i.e. the formatted value of `libcuda.cuDriverGetVersion()`), constrained to the first two components (major and minor) and formatted as `{major}.{minor}`. The build string MUST be `0`. + +The version MUST be overridable with the `CONDA_OVERRIDE_CUDA` environment variable, if set to a non-empty value. + +#### `__glibc` + +This virtual package MUST be present when the native platform is `linux-*`. Its version value MUST be set to the system `libc` version, constrained to the first two components (major and minor) formatted as `{major}.{minor}`. The build string MUST be `0`. + +The version MUST be overridable with the `CONDA_OVERRIDE_GLIBC` environment variable, if set to a non-empty value. + +If the `libc` version could not be estimated (e.g. the tool is not running on Linux), the tool SHOULD provide a default value (e.g. `2.17`) and inform the user of that choice and its possible overrides; e.g. via `CONDA_OVERRIDE_GLIBC`, a CLI flag or a configuration file. The environment variable MUST be ignored when the target platform is not `linux~-*`. + +#### `__linux` + +This virtual package MUST be present when the target platform is `linux-*`. Its version value MUST be set to the Linux kernel version, constrained to two to four numeric components formatted as `{major}.{minor}.{micro}.{patch}`. If the version cannot be estimated (e.g. because the native platform is not Linux), the fallback value MUST be set to `0`. The build string MUST be `0`. + +The version MUST be overridable with the `CONDA_OVERRIDE_LINUX` environment variable, if set to a non-empty value that matches the regex `"\d+\.\d+(\.\d+)?(\.\d+)?"`. The environment variable MUST be ignored when the target platform is not `linux-*`. + +#### `__osx` + +This virtual package MUST be present when the target platform is `osx-*`. Its version value MUST be set to the first two numeric components of macOS version formatted as `{major}[.{minor}]`. If the version cannot be estimated (e.g. because the native platform is not macOS), the fallback value MUST be set to `0`. The build string MUST be `0`. + +The version MUST be overridable with the `CONDA_OVERRIDE_OSX` environment variable. If this environment variable is set to the empty string `""`, then the `__osx` virtual package MUST NOT be present. The environment variable MUST be ignored when the target platform is not `osx-*`. + +#### `__unix` + +This virtual package MUST be present when the target platform is `linux-*`, `osx-*` or `freebsd-*`. The version and build string fields MUST be set to `0`. + +The version or build string fields MUST NOT be overriden by the `CONDA_OVERRIDE_UNIX` environment variable. However, if this environment variable is set to a non-empty value, the `__unix` virtual package MUST be present if otherwise if would not have been. The environment variable MUST be ignored when the target platform is not `linux-*`, `osx-*` or `freebsd-*`. + +#### `__win` + +This virtual package MUST be present when the target platform is `win-*`. The version MUST be set to the first three numeric components of the version reported by CMD's `ver` command, formatted as `{major}.{minor}.{build}`. If the version cannot be estimated (e.g. because the target platform does not match the native platform), the fallback value MUST be set to `0`. The build string MUST be `0`. + +The version MUST be overridable with the `CONDA_OVERRIDE_WIN` environment variable. If this environment variable is set to the empty string `""`, then the `__win` virtual package MUST NOT be present. The environment variable MUST be ignored when the target platform is not `win-*`. + +## Motivation + +Virtual packages are used to expose details of the system configuration to a conda client. They are commonly used as dependencies in regular packages to constrain on which systems they can be installed. Some examples include: + +* On Linux, the minimum `libc` version that must be available in the system via the `__glibc` virtual package. +* The oldest macOS version compatible with the package via the `__osx` virtual package. +* Whether a `noarch` package should be constrained to a single operating system via the `__linux`, `__osx` or `__win` virtual packages (often with no version). +* The minimum CPU microarchitecture level that the binaries require via the `__archspec` virtual package. +* The lowest CUDA version the GPU driver is compatible with via `__cuda`. + +## References + +* [Virtual packages implementation in `conda/conda` 24.11.1](https://github.com/conda/conda/tree/24.11.1/conda/plugins/virtual_packages) +* [ENH: make `__win` version usable for package metadata (conda/conda#14443)](https://github.com/conda/conda/issues/14443) +* [Drop `CONDA_OVERRIDE_WIN` environment variable (mamba-org/mamba#2815)](https://github.com/mamba-org/mamba/pull/2815) + +## Copyright + +All CEPs are explicitly [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/). From cda0a8c9a6e38ed9d924e272cd20fe2e4443dc49 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Tue, 17 Dec 2024 22:40:07 +0100 Subject: [PATCH 02/26] Add some Windows methods --- cep-????.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/cep-????.md b/cep-????.md index 0d830b8c..0a9cd20a 100644 --- a/cep-????.md +++ b/cep-????.md @@ -75,7 +75,13 @@ The version or build string fields MUST NOT be overriden by the `CONDA_OVERRIDE_ #### `__win` -This virtual package MUST be present when the target platform is `win-*`. The version MUST be set to the first three numeric components of the version reported by CMD's `ver` command, formatted as `{major}.{minor}.{build}`. If the version cannot be estimated (e.g. because the target platform does not match the native platform), the fallback value MUST be set to `0`. The build string MUST be `0`. +This virtual package MUST be present when the target platform is `win-*`. The version MUST be set to the first three numeric components of the Windows build version, formatted as `{major}.{minor}.{build}`. If the version cannot be estimated (e.g. because the target platform does not match the native platform), the fallback value MUST be set to `0`. The build string MUST be `0`. + +The string `{major}.{minor}.{build}` can be obtained from: + +- Python's `platform.win32_ver()` +- CMD's `ver` +- Powershell's `[System.Environment]::OSVersion.Version` or `(Get-CimInstance Win32_OperatingSystem).version` The version MUST be overridable with the `CONDA_OVERRIDE_WIN` environment variable. If this environment variable is set to the empty string `""`, then the `__win` virtual package MUST NOT be present. The environment variable MUST be ignored when the target platform is not `win-*`. From 6f2d60fe539e8deeebdd7f4f3f0e3796e0895eb5 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Tue, 17 Dec 2024 22:47:36 +0100 Subject: [PATCH 03/26] some more --- cep-????.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/cep-????.md b/cep-????.md index 0a9cd20a..823dc0db 100644 --- a/cep-????.md +++ b/cep-????.md @@ -81,7 +81,8 @@ The string `{major}.{minor}.{build}` can be obtained from: - Python's `platform.win32_ver()` - CMD's `ver` -- Powershell's `[System.Environment]::OSVersion.Version` or `(Get-CimInstance Win32_OperatingSystem).version` +- Powershell's `[System.Environment]::OSVersion.Version`, `(Get-CimInstance Win32_OperatingSystem).version` +- The command `wmic os get version` The version MUST be overridable with the `CONDA_OVERRIDE_WIN` environment variable. If this environment variable is set to the empty string `""`, then the `__win` virtual package MUST NOT be present. The environment variable MUST be ignored when the target platform is not `win-*`. From 9a793b93ac8f64f976f3c5c21e6fbca552588c75 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Tue, 17 Dec 2024 23:06:13 +0100 Subject: [PATCH 04/26] add details of glibc, linux and osx --- cep-????.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/cep-????.md b/cep-????.md index 823dc0db..24ace81b 100644 --- a/cep-????.md +++ b/cep-????.md @@ -51,6 +51,13 @@ The version MUST be overridable with the `CONDA_OVERRIDE_CUDA` environment varia This virtual package MUST be present when the native platform is `linux-*`. Its version value MUST be set to the system `libc` version, constrained to the first two components (major and minor) formatted as `{major}.{minor}`. The build string MUST be `0`. +The `libc` version can be computed via: + +- Python's `os.confstr("CS_GNU_LIBC_VERSION")` +- `getconf GNU_LIBC_VERSION` +- `ldd --version` (in GLIBC distros) +- System's `libc.so` (in MUSL distros, location not standardized) + The version MUST be overridable with the `CONDA_OVERRIDE_GLIBC` environment variable, if set to a non-empty value. If the `libc` version could not be estimated (e.g. the tool is not running on Linux), the tool SHOULD provide a default value (e.g. `2.17`) and inform the user of that choice and its possible overrides; e.g. via `CONDA_OVERRIDE_GLIBC`, a CLI flag or a configuration file. The environment variable MUST be ignored when the target platform is not `linux~-*`. @@ -59,12 +66,25 @@ If the `libc` version could not be estimated (e.g. the tool is not running on Li This virtual package MUST be present when the target platform is `linux-*`. Its version value MUST be set to the Linux kernel version, constrained to two to four numeric components formatted as `{major}.{minor}.{micro}.{patch}`. If the version cannot be estimated (e.g. because the native platform is not Linux), the fallback value MUST be set to `0`. The build string MUST be `0`. +The Linux kernel version can be obtained via: + +* Python's `platform.release()` +* `uname -r` +* `cat /proc/version` + The version MUST be overridable with the `CONDA_OVERRIDE_LINUX` environment variable, if set to a non-empty value that matches the regex `"\d+\.\d+(\.\d+)?(\.\d+)?"`. The environment variable MUST be ignored when the target platform is not `linux-*`. #### `__osx` This virtual package MUST be present when the target platform is `osx-*`. Its version value MUST be set to the first two numeric components of macOS version formatted as `{major}[.{minor}]`. If the version cannot be estimated (e.g. because the native platform is not macOS), the fallback value MUST be set to `0`. The build string MUST be `0`. +The macOS version can be ontained via: + +- Python's `platform.mac_ver()[0]` +- `sw_vers -productVersion` + +> If applicable, the `SYSTEM_VERSION_COMPAT` workaround MUST NOT be enabled; e.g. the version reported for Big Sur should be 11.x and not 10.16. + The version MUST be overridable with the `CONDA_OVERRIDE_OSX` environment variable. If this environment variable is set to the empty string `""`, then the `__osx` virtual package MUST NOT be present. The environment variable MUST be ignored when the target platform is not `osx-*`. #### `__unix` From 9b4b7a7c919248305801ce3fc076f939becdff53 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Tue, 17 Dec 2024 23:15:12 +0100 Subject: [PATCH 05/26] pre-commit --- cep-????.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/cep-????.md b/cep-????.md index 24ace81b..e17766b9 100644 --- a/cep-????.md +++ b/cep-????.md @@ -76,9 +76,9 @@ The version MUST be overridable with the `CONDA_OVERRIDE_LINUX` environment vari #### `__osx` -This virtual package MUST be present when the target platform is `osx-*`. Its version value MUST be set to the first two numeric components of macOS version formatted as `{major}[.{minor}]`. If the version cannot be estimated (e.g. because the native platform is not macOS), the fallback value MUST be set to `0`. The build string MUST be `0`. +This virtual package MUST be present when the target platform is `osx-*`. Its version value MUST be set to the first two numeric components of macOS version formatted as `{major}[.{minor}]`. If the version cannot be estimated (e.g. because the native platform is not macOS), the fallback value MUST be set to `0`. The build string MUST be `0`. -The macOS version can be ontained via: +The macOS version can be contained via: - Python's `platform.mac_ver()[0]` - `sw_vers -productVersion` @@ -91,11 +91,11 @@ The version MUST be overridable with the `CONDA_OVERRIDE_OSX` environment variab This virtual package MUST be present when the target platform is `linux-*`, `osx-*` or `freebsd-*`. The version and build string fields MUST be set to `0`. -The version or build string fields MUST NOT be overriden by the `CONDA_OVERRIDE_UNIX` environment variable. However, if this environment variable is set to a non-empty value, the `__unix` virtual package MUST be present if otherwise if would not have been. The environment variable MUST be ignored when the target platform is not `linux-*`, `osx-*` or `freebsd-*`. +The version or build string fields MUST NOT be overridden by the `CONDA_OVERRIDE_UNIX` environment variable. However, if this environment variable is set to a non-empty value, the `__unix` virtual package MUST be present if otherwise if would not have been. The environment variable MUST be ignored when the target platform is not `linux-*`, `osx-*` or `freebsd-*`. #### `__win` -This virtual package MUST be present when the target platform is `win-*`. The version MUST be set to the first three numeric components of the Windows build version, formatted as `{major}.{minor}.{build}`. If the version cannot be estimated (e.g. because the target platform does not match the native platform), the fallback value MUST be set to `0`. The build string MUST be `0`. +This virtual package MUST be present when the target platform is `win-*`. The version MUST be set to the first three numeric components of the Windows build version, formatted as `{major}.{minor}.{build}`. If the version cannot be estimated (e.g. because the target platform does not match the native platform), the fallback value MUST be set to `0`. The build string MUST be `0`. The string `{major}.{minor}.{build}` can be obtained from: From 59462edb7c8b96d1c03410b9bf89b9e5b64da56b Mon Sep 17 00:00:00 2001 From: jaimergp Date: Tue, 17 Dec 2024 23:19:19 +0100 Subject: [PATCH 06/26] add discussion, implementation --- cep-????.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/cep-????.md b/cep-????.md index e17766b9..88e112e4 100644 --- a/cep-????.md +++ b/cep-????.md @@ -6,8 +6,8 @@ Author(s) Jaime Rodríguez-Guerra <jaime.rogue@gmail.com> Created Dec 17, 2024 Updated Dec 17, 2024 - Discussion NA - Implementation Several + Discussion https://github.com/conda/ceps/pull/103 + Implementation https://github.com/conda/conda/tree/24.11.1/conda/plugins/virtual_packages, https://github.com/mamba-org/mamba/blob/libmamba-2.0.5/libmamba/src/core/virtual_packages.cpp, https://github.com/conda/rattler/tree/rattler-v0.28.8/crates/rattler_virtual_packages/src > The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", @@ -119,6 +119,8 @@ Virtual packages are used to expose details of the system configuration to a con ## References * [Virtual packages implementation in `conda/conda` 24.11.1](https://github.com/conda/conda/tree/24.11.1/conda/plugins/virtual_packages) +* [Virtual packages implementation in `libmamba` 2.0.5](https://github.com/mamba-org/mamba/blob/libmamba-2.0.5/libmamba/src/core/virtual_packages.cpp) +* [Virtual packages implementation in `rattler` 0.28.8](https://github.com/conda/rattler/tree/rattler-v0.28.8/crates/rattler_virtual_packages/src) * [ENH: make `__win` version usable for package metadata (conda/conda#14443)](https://github.com/conda/conda/issues/14443) * [Drop `CONDA_OVERRIDE_WIN` environment variable (mamba-org/mamba#2815)](https://github.com/mamba-org/mamba/pull/2815) From cedde6a117cdcecf1fbbb2f10b8473dd7c20742c Mon Sep 17 00:00:00 2001 From: jaimergp Date: Tue, 17 Dec 2024 23:48:27 +0100 Subject: [PATCH 07/26] add general paragraph about CONDA_OVERRIDE_* --- cep-????.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cep-????.md b/cep-????.md index 88e112e4..00d6e63e 100644 --- a/cep-????.md +++ b/cep-????.md @@ -23,6 +23,8 @@ This CEP standardizes which virtual packages MUST be offered by conda solvers. A virtual package is defined as a package record with three fields: name, version and build string. The name MUST start with double underscore. The version and build string MUST follow the same semantics as in regular package records. +In general, the version or build string of a virtual package MAY be overridden by the value of `CONDA_OVERRIDE_{NAME}` environment variable, with `{NAME}` being the uppercased name of the virtual package. Many exceptions apply so please observe the details in the section below. + ### List of virtual packages In alphabetical order, every conda client MUST support the following virtual packages: From df9d61d1e913dc3c0f8286ed4bbb7ca0d7101be5 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Tue, 17 Dec 2024 23:54:18 +0100 Subject: [PATCH 08/26] Apply suggestions from code review Co-authored-by: jakirkham --- cep-????.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cep-????.md b/cep-????.md index 00d6e63e..059dd4a9 100644 --- a/cep-????.md +++ b/cep-????.md @@ -51,9 +51,9 @@ The version MUST be overridable with the `CONDA_OVERRIDE_CUDA` environment varia #### `__glibc` -This virtual package MUST be present when the native platform is `linux-*`. Its version value MUST be set to the system `libc` version, constrained to the first two components (major and minor) formatted as `{major}.{minor}`. The build string MUST be `0`. +This virtual package MUST be present when the native platform is `linux-*`. Its version value MUST be set to the system GNU `libc` version, constrained to the first two components (major and minor) formatted as `{major}.{minor}`. The build string MUST be `0`. -The `libc` version can be computed via: +The GNU `libc` version can be computed via: - Python's `os.confstr("CS_GNU_LIBC_VERSION")` - `getconf GNU_LIBC_VERSION` From 72022f488de4d3690ca86b4d0cf98ff7930ba982 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Wed, 18 Dec 2024 00:43:55 +0100 Subject: [PATCH 09/26] more details about archspec --- cep-????.md | 62 +++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 43 insertions(+), 19 deletions(-) diff --git a/cep-????.md b/cep-????.md index 00d6e63e..a40499d2 100644 --- a/cep-????.md +++ b/cep-????.md @@ -39,7 +39,30 @@ In alphabetical order, every conda client MUST support the following virtual pac #### `__archspec` -This virtual package MUST be always present, with the version set to `1`. The build string SHOULD reflect the detected CPU microarchitecture. If it cannot be detected, the build string SHOULD be `0`. +This virtual package MUST be always present, with the version set to `1`. The build string SHOULD reflect the detected CPU microarchitecture when the target platform matches the native platform, +as provided by the generic values in the [`archspec/archspec-json` database](https://github.com/archspec/archspec-json/blob/v0.2.5/cpu/microarchitectures.json), plus some exceptions. For example, `conda/conda` reports its generic values for Intel/AMD and `m*` names for Apple: + +- Generic: `x86_64_v1`, `x86_64_v2`, `x86_64_v3`, `x86_64_v4`, `arm`, `ppc`... +- Apple: For M1, the value is `m1`. M2 and M3 are reported as `m2` and `m3`, respectively. + +If the microarchitecture cannot be detected or the target platform does not match the native platform, the build string MUST be set to the second component of the target platform, mapped with these rules: + +| Target platform | Reported `archspec` build string | +|-----------------|----------------------------------| +| `*-32` | `x86` | +| `*-64` | `x86_64` | +| `*-armv6l` | `armv6l` | +| `*-armv7l` | `armv7l` | +| `*-aarch64` | `aarch64` | +| `*-arm64` | `arm64` | +| `*-ppc64` | `ppc64` | +| `*-ppc64le` | `ppc64le` | +| `*-riscv64` | `riscv64` | +| `*-s390x` | `s390x` | +| `zos-z` | `0` | +| Any other value | `0` | + + The build string MUST be overridable with the `CONDA_OVERRIDE_ARCHSPEC` environment variable, if set to a non-empty value. @@ -53,42 +76,43 @@ The version MUST be overridable with the `CONDA_OVERRIDE_CUDA` environment varia This virtual package MUST be present when the native platform is `linux-*`. Its version value MUST be set to the system `libc` version, constrained to the first two components (major and minor) formatted as `{major}.{minor}`. The build string MUST be `0`. -The `libc` version can be computed via: - -- Python's `os.confstr("CS_GNU_LIBC_VERSION")` -- `getconf GNU_LIBC_VERSION` -- `ldd --version` (in GLIBC distros) -- System's `libc.so` (in MUSL distros, location not standardized) - The version MUST be overridable with the `CONDA_OVERRIDE_GLIBC` environment variable, if set to a non-empty value. If the `libc` version could not be estimated (e.g. the tool is not running on Linux), the tool SHOULD provide a default value (e.g. `2.17`) and inform the user of that choice and its possible overrides; e.g. via `CONDA_OVERRIDE_GLIBC`, a CLI flag or a configuration file. The environment variable MUST be ignored when the target platform is not `linux~-*`. +> The `libc` version can be computed via: +> +> - Python's `os.confstr("CS_GNU_LIBC_VERSION")` +> - `getconf GNU_LIBC_VERSION` +> - `ldd --version`. Note this only applies to GLIBC distros. In MUSL distros this will return the MUSL version instead. In these systems, you might need to locate the gcompat `libc.so` library and call it directly. + #### `__linux` This virtual package MUST be present when the target platform is `linux-*`. Its version value MUST be set to the Linux kernel version, constrained to two to four numeric components formatted as `{major}.{minor}.{micro}.{patch}`. If the version cannot be estimated (e.g. because the native platform is not Linux), the fallback value MUST be set to `0`. The build string MUST be `0`. -The Linux kernel version can be obtained via: - -* Python's `platform.release()` -* `uname -r` -* `cat /proc/version` - The version MUST be overridable with the `CONDA_OVERRIDE_LINUX` environment variable, if set to a non-empty value that matches the regex `"\d+\.\d+(\.\d+)?(\.\d+)?"`. The environment variable MUST be ignored when the target platform is not `linux-*`. +> The Linux kernel version can be obtained via: +> +> - Python's `platform.release()` +> - `uname -r` +> - `cat /proc/version` + #### `__osx` This virtual package MUST be present when the target platform is `osx-*`. Its version value MUST be set to the first two numeric components of macOS version formatted as `{major}[.{minor}]`. If the version cannot be estimated (e.g. because the native platform is not macOS), the fallback value MUST be set to `0`. The build string MUST be `0`. -The macOS version can be contained via: -- Python's `platform.mac_ver()[0]` -- `sw_vers -productVersion` - -> If applicable, the `SYSTEM_VERSION_COMPAT` workaround MUST NOT be enabled; e.g. the version reported for Big Sur should be 11.x and not 10.16. The version MUST be overridable with the `CONDA_OVERRIDE_OSX` environment variable. If this environment variable is set to the empty string `""`, then the `__osx` virtual package MUST NOT be present. The environment variable MUST be ignored when the target platform is not `osx-*`. +> The macOS version can be obtained via: +> +> - Python's `platform.mac_ver()[0]` +> - `sw_vers -productVersion` +> +> If applicable, the `SYSTEM_VERSION_COMPAT` workaround MUST NOT be enabled; e.g. the version reported for Big Sur should be 11.x and not 10.16. + #### `__unix` This virtual package MUST be present when the target platform is `linux-*`, `osx-*` or `freebsd-*`. The version and build string fields MUST be set to `0`. From e6fcda51c2686353cd6c84b76e803f4127fffba4 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Wed, 18 Dec 2024 11:21:58 +0100 Subject: [PATCH 10/26] link to existing issue --- cep-????.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/cep-????.md b/cep-????.md index 16775218..1855fc02 100644 --- a/cep-????.md +++ b/cep-????.md @@ -143,7 +143,7 @@ Virtual packages are used to expose details of the system configuration to a con This CEP attempts to describe and standardize the existing implementations of the currently available virtual packages. The following items are deliberately kept out of scope and recommended for discussion in future CEPs: - Additional OSes, like `__freebsd` or `__netbsd`. -- Architectures, like `__x86_64` or `__arm64`. +- Architectures, like `__x86_64` or `__arm64`, or, more generally, [`__arch`](https://github.com/conda/conda/issues/13420). - More `libc` implementations, like `__musl`. ## References @@ -153,6 +153,7 @@ This CEP attempts to describe and standardize the existing implementations of th * [Virtual packages implementation in `rattler` 0.28.8](https://github.com/conda/rattler/tree/rattler-v0.28.8/crates/rattler_virtual_packages/src) * [ENH: make `__win` version usable for package metadata (conda/conda#14443)](https://github.com/conda/conda/issues/14443) * [Drop `CONDA_OVERRIDE_WIN` environment variable (mamba-org/mamba#2815)](https://github.com/mamba-org/mamba/pull/2815) +* [`__arch` feature request](https://github.com/conda/conda/issues/13420) ## Copyright From 40b444219ea89ae7efc64a55b985d80930a4e0e5 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 19 Dec 2024 11:35:59 +0100 Subject: [PATCH 11/26] Apply suggestions from code review Co-authored-by: jakirkham --- cep-????.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/cep-????.md b/cep-????.md index 1855fc02..411baf59 100644 --- a/cep-????.md +++ b/cep-????.md @@ -21,7 +21,9 @@ This CEP standardizes which virtual packages MUST be offered by conda solvers. ## Specification A virtual package is defined as a package record with three fields: name, version and build string. -The name MUST start with double underscore. The version and build string MUST follow the same semantics as in regular package records. +The name MUST start with double underscore (`__`). The version and build string MUST follow the same semantics as in regular package records. + +The build string MAY be zero (`0`). Some exceptions apply. See below. In general, the version or build string of a virtual package MAY be overridden by the value of `CONDA_OVERRIDE_{NAME}` environment variable, with `{NAME}` being the uppercased name of the virtual package. Many exceptions apply so please observe the details in the section below. @@ -82,7 +84,7 @@ If the GNU `libc` version could not be estimated (e.g. the tool is not running o > > - Python's `os.confstr("CS_GNU_LIBC_VERSION")` > - `getconf GNU_LIBC_VERSION` -> - `ldd --version`. Note this only applies to GLIBC distros. In MUSL distros this will return the MUSL version instead. In these systems, you might need to locate the GNU `libc.so` library and call it directly. +> - `ldd --version`. Please verify that it references GNU libc or GLIBC. For non-standard installs, using a GLIBC compatibility layer, this may require locating the implementation and directly querying. #### `__linux` @@ -138,12 +140,14 @@ Virtual packages are used to expose details of the system configuration to a con * The minimum CPU microarchitecture level that the binaries require via the `__archspec` virtual package. * The lowest CUDA version the GPU driver is compatible with via `__cuda`. -## Rationale +## Potential future work + +This CEP focuses on the standardization of existing virtual package implementations. -This CEP attempts to describe and standardize the existing implementations of the currently available virtual packages. The following items are deliberately kept out of scope and recommended for discussion in future CEPs: +The following items are not considered here. Though would be open for discussion in future CEP work: - Additional OSes, like `__freebsd` or `__netbsd`. -- Architectures, like `__x86_64` or `__arm64`, or, more generally, [`__arch`](https://github.com/conda/conda/issues/13420). +- Coarse grain architecture information, like `__x86_64` or `__arm64`, or, more generally, [`__arch`](https://github.com/conda/conda/issues/13420). - More `libc` implementations, like `__musl`. ## References From bc6d6a6f66386db02e4c0a68301148c358de03de Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 19 Dec 2024 11:37:55 +0100 Subject: [PATCH 12/26] Move motivation further up --- cep-????.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/cep-????.md b/cep-????.md index 411baf59..1a7c3771 100644 --- a/cep-????.md +++ b/cep-????.md @@ -16,7 +16,17 @@ ## Abstract -This CEP standardizes which virtual packages MUST be offered by conda solvers. +This CEP standardizes which virtual packages MUST be offered by conda install tools. + +## Motivation + +Virtual packages are used to expose details of the system configuration to a conda client. They are commonly used as dependencies in regular packages to constrain on which systems they can be installed. Some examples include: + +* On Linux, the minimum GNU `libc` version that must be available in the system via the `__glibc` virtual package. +* The oldest macOS version compatible with the package via the `__osx` virtual package. +* Whether a `noarch` package should be constrained to a single operating system via the `__linux`, `__osx` or `__win` virtual packages (often with no version constraint). +* The minimum CPU microarchitecture level that the binaries require via the `__archspec` virtual package. +* The lowest CUDA version the GPU driver is compatible with via `__cuda`. ## Specification @@ -130,16 +140,6 @@ The string `{major}.{minor}.{build}` can be obtained from: The version MUST be overridable with the `CONDA_OVERRIDE_WIN` environment variable. If this environment variable is set to the empty string `""`, then the `__win` virtual package MUST NOT be present. The environment variable MUST be ignored when the target platform is not `win-*`. -## Motivation - -Virtual packages are used to expose details of the system configuration to a conda client. They are commonly used as dependencies in regular packages to constrain on which systems they can be installed. Some examples include: - -* On Linux, the minimum `libc` version that must be available in the system via the `__glibc` virtual package. -* The oldest macOS version compatible with the package via the `__osx` virtual package. -* Whether a `noarch` package should be constrained to a single operating system via the `__linux`, `__osx` or `__win` virtual packages (often with no version). -* The minimum CPU microarchitecture level that the binaries require via the `__archspec` virtual package. -* The lowest CUDA version the GPU driver is compatible with via `__cuda`. - ## Potential future work This CEP focuses on the standardization of existing virtual package implementations. From 5bf10f15045403923ca2afd9adc0561c2ebe8d9b Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 19 Dec 2024 12:36:37 +0100 Subject: [PATCH 13/26] Rework `__glibc` specification --- cep-????.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/cep-????.md b/cep-????.md index 1a7c3771..39628487 100644 --- a/cep-????.md +++ b/cep-????.md @@ -84,17 +84,22 @@ The version MUST be overridable with the `CONDA_OVERRIDE_CUDA` environment varia #### `__glibc` -This virtual package MUST be present when the native platform is `linux-*`. Its version value MUST be set to the system GNU `libc` version, constrained to the first two components (major and minor) formatted as `{major}.{minor}`. The build string MUST be `0`. +This virtual package MUST NOT be present if the target platform is not `linux-*`. -The version MUST be overridable with the `CONDA_OVERRIDE_GLIBC` environment variable, if set to a non-empty value. +This virtual package MUST be present when the native and target platforms are both the same type of `linux-*` and GNU `libc` is installed in the system. The version value MUST be set to the system GNU `libc` version, constrained to the first two components (major and minor) formatted as `{major}.{minor}`. If the version cannot be estimated, the tool MUST set the version to a default value (e.g. `2.17`). + +If the native platform does not match the target platform, the tool MAY export `__glibc` with its `version` field set to a default value (e.g. `2.17`) of its choice. + +If the `CONDA_OVERRIDE_GLIBC` environment variable if set to a non-empty value that complies to the version string specification, the tool MUST export `__glibc` with its version value set to the value of the environemnt variable. + +The build string MUST always be `0`. -If the GNU `libc` version could not be estimated (e.g. the tool is not running on Linux), the tool SHOULD provide a default value (e.g. `2.17`) and inform the user of that choice and its possible overrides; e.g. via `CONDA_OVERRIDE_GLIBC`, a CLI flag or a configuration file. The environment variable MUST be ignored when the target platform is not `linux~-*`. > The GNU `libc` version can be computed via: > > - Python's `os.confstr("CS_GNU_LIBC_VERSION")` > - `getconf GNU_LIBC_VERSION` -> - `ldd --version`. Please verify that it references GNU libc or GLIBC. For non-standard installs, using a GLIBC compatibility layer, this may require locating the implementation and directly querying. +> - `ldd --version`. Please verify that it references GNU `libc` or GLIBC. For non-standard installs, using a GLIBC compatibility layer, this may require locating the implementation and directly querying. #### `__linux` From 7a2c7e5c081aa991f81b5ae1c363f2f7f8d94247 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 19 Dec 2024 12:37:19 +0100 Subject: [PATCH 14/26] Forbid `CONDA_OVERRIDE_UNIX` --- cep-????.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cep-????.md b/cep-????.md index 39628487..ebcd596c 100644 --- a/cep-????.md +++ b/cep-????.md @@ -130,7 +130,7 @@ The version MUST be overridable with the `CONDA_OVERRIDE_OSX` environment variab This virtual package MUST be present when the target platform is `linux-*`, `osx-*` or `freebsd-*`. The version and build string fields MUST be set to `0`. -The version or build string fields MUST NOT be overridden by the `CONDA_OVERRIDE_UNIX` environment variable. However, if this environment variable is set to a non-empty value, the `__unix` virtual package MUST be present if otherwise if would not have been. The environment variable MUST be ignored when the target platform is not `linux-*`, `osx-*` or `freebsd-*`. +The `CONDA_OVERRIDE_UNIX` environment variable MUST NOT have any effect. #### `__win` From cd06910868c8ff22d0df825712e3584b6d3062c9 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 19 Dec 2024 12:37:47 +0100 Subject: [PATCH 15/26] format table --- cep-????.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/cep-????.md b/cep-????.md index ebcd596c..dfcee5e8 100644 --- a/cep-????.md +++ b/cep-????.md @@ -60,19 +60,19 @@ as provided by the generic values in the [`archspec/archspec-json` database](htt If the microarchitecture cannot be detected or the target platform does not match the native platform, the build string MUST be set to the second component of the target platform, mapped with these rules: | Target platform | Reported `archspec` build string | -|-----------------|----------------------------------| -| `*-32` | `x86` | -| `*-64` | `x86_64` | -| `*-armv6l` | `armv6l` | -| `*-armv7l` | `armv7l` | -| `*-aarch64` | `aarch64` | -| `*-arm64` | `arm64` | -| `*-ppc64` | `ppc64` | -| `*-ppc64le` | `ppc64le` | -| `*-riscv64` | `riscv64` | -| `*-s390x` | `s390x` | -| `zos-z` | `0` | -| Any other value | `0` | +| --------------- | -------------------------------- | +| `*-32` | `x86` | +| `*-64` | `x86_64` | +| `*-armv6l` | `armv6l` | +| `*-armv7l` | `armv7l` | +| `*-aarch64` | `aarch64` | +| `*-arm64` | `arm64` | +| `*-ppc64` | `ppc64` | +| `*-ppc64le` | `ppc64le` | +| `*-riscv64` | `riscv64` | +| `*-s390x` | `s390x` | +| `zos-z` | `0` | +| Any other value | `0` | The build string MUST be overridable with the `CONDA_OVERRIDE_ARCHSPEC` environment variable, if set to a non-empty value. From e5203368a96052b498702f1ca8628e351ec5c160 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 19 Dec 2024 12:38:07 +0100 Subject: [PATCH 16/26] Reword general considerations --- cep-????.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/cep-????.md b/cep-????.md index dfcee5e8..54d39fc5 100644 --- a/cep-????.md +++ b/cep-????.md @@ -33,9 +33,11 @@ Virtual packages are used to expose details of the system configuration to a con A virtual package is defined as a package record with three fields: name, version and build string. The name MUST start with double underscore (`__`). The version and build string MUST follow the same semantics as in regular package records. -The build string MAY be zero (`0`). Some exceptions apply. See below. +Some general considerations: -In general, the version or build string of a virtual package MAY be overridden by the value of `CONDA_OVERRIDE_{NAME}` environment variable, with `{NAME}` being the uppercased name of the virtual package. Many exceptions apply so please observe the details in the section below. +- The version or build string of a virtual package MAY be overridden by the value of `CONDA_OVERRIDE_{NAME}` environment variable, with `{NAME}` being the uppercased name of the virtual package. If the field being overridden is `version`, the value MUST be parsable as a version string. Many exceptions apply so please observe the details in the section below. +- The build string MAY be zero (`0`). Some exceptions apply. See below. +- When the tool used a fallback default value instead of a computed one, it SHOULD also inform the user of that choice and its possible override options (e.g. `CONDA_OVERRIDE_{NAME}` variables, CLI flags, configuration file, etc). ### List of virtual packages From 1b011825c838d1cab43843d663a874125a8afefa Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 19 Dec 2024 12:40:39 +0100 Subject: [PATCH 17/26] Be stricter with override values and more lenient with fallback values --- cep-????.md | 36 +++++++++++++++++++----------------- 1 file changed, 19 insertions(+), 17 deletions(-) diff --git a/cep-????.md b/cep-????.md index 54d39fc5..d1f02d30 100644 --- a/cep-????.md +++ b/cep-????.md @@ -31,11 +31,11 @@ Virtual packages are used to expose details of the system configuration to a con ## Specification A virtual package is defined as a package record with three fields: name, version and build string. -The name MUST start with double underscore (`__`). The version and build string MUST follow the same semantics as in regular package records. +The name MUST start with double underscore (`__`). The version and build string MUST follow the same semantics as in regular package records. More specifically, the version field MUST follow the version string specifications, regardless its origin (computed from a system property, overridden by the user or configuration, or provided by default by the tool). Some general considerations: -- The version or build string of a virtual package MAY be overridden by the value of `CONDA_OVERRIDE_{NAME}` environment variable, with `{NAME}` being the uppercased name of the virtual package. If the field being overridden is `version`, the value MUST be parsable as a version string. Many exceptions apply so please observe the details in the section below. +- The version or build string of a virtual package MAY be overridden by the value of `CONDA_OVERRIDE_{NAME}` environment variable, with `{NAME}` being the uppercased name of the virtual package. Many exceptions apply so please observe the details in the section below. - The build string MAY be zero (`0`). Some exceptions apply. See below. - When the tool used a fallback default value instead of a computed one, it SHOULD also inform the user of that choice and its possible override options (e.g. `CONDA_OVERRIDE_{NAME}` variables, CLI flags, configuration file, etc). @@ -54,7 +54,9 @@ In alphabetical order, every conda client MUST support the following virtual pac #### `__archspec` This virtual package MUST be always present, with the version set to `1`. The build string SHOULD reflect the detected CPU microarchitecture when the target platform matches the native platform, -as provided by the generic values in the [`archspec/archspec-json` database](https://github.com/archspec/archspec-json/blob/v0.2.5/cpu/microarchitectures.json), plus some exceptions. For example, `conda/conda` reports its generic values for Intel/AMD and `m*` names for Apple: +as provided by the generic values in the [`archspec/archspec-json` database](https://github.com/archspec/archspec-json/blob/v0.2.5/cpu/microarchitectures.json), plus some exceptions. + +For example, `conda/conda` reports its generic values for Intel/AMD and vendor-specific names for Apple: - Generic: `x86_64_v1`, `x86_64_v2`, `x86_64_v3`, `x86_64_v4`, `arm`, `ppc`... - Apple: For M1, the value is `m1`. M2 and M3 are reported as `m2` and `m3`, respectively. @@ -82,7 +84,7 @@ The build string MUST be overridable with the `CONDA_OVERRIDE_ARCHSPEC` environm This virtual package MUST be present when the system exhibits GPU drivers compatible with the CUDA runtimes. When available, the version value MUST be set to the oldest CUDA version supported by the detected drivers (i.e. the formatted value of `libcuda.cuDriverGetVersion()`), constrained to the first two components (major and minor) and formatted as `{major}.{minor}`. The build string MUST be `0`. -The version MUST be overridable with the `CONDA_OVERRIDE_CUDA` environment variable, if set to a non-empty value. +The version MUST be overridable with the `CONDA_OVERRIDE_CUDA` environment variable, if set to a non-empty value that can be parsed as a version string. #### `__glibc` @@ -105,7 +107,7 @@ The build string MUST always be `0`. #### `__linux` -This virtual package MUST be present when the target platform is `linux-*`. Its version value MUST be set to the Linux kernel version, constrained to two to four numeric components formatted as `{major}.{minor}.{micro}.{patch}`. If the version cannot be estimated (e.g. because the native platform is not Linux), the fallback value MUST be set to `0`. The build string MUST be `0`. +This virtual package MUST be present when the target platform is `linux-*`. Its version value MUST be set to the Linux kernel version, constrained to the first two to four numeric components formatted as `{major}.{minor}.{micro}.{patch}`. If the version cannot be estimated (e.g. because the native platform is not Linux), the tool MUST set `version` to a fallback value of its choice. The build string MUST be `0`. The version MUST be overridable with the `CONDA_OVERRIDE_LINUX` environment variable, if set to a non-empty value that matches the regex `"\d+\.\d+(\.\d+)?(\.\d+)?"`. The environment variable MUST be ignored when the target platform is not `linux-*`. @@ -117,16 +119,14 @@ The version MUST be overridable with the `CONDA_OVERRIDE_LINUX` environment vari #### `__osx` -This virtual package MUST be present when the target platform is `osx-*`. Its version value MUST be set to the first two numeric components of macOS version formatted as `{major}[.{minor}]`. If the version cannot be estimated (e.g. because the native platform is not macOS), the fallback value MUST be set to `0`. The build string MUST be `0`. +This virtual package MUST be present when the target platform is `osx-*`. Its version value MUST be set to the first two numeric components of macOS version formatted as `{major}.{minor}`. If applicable, the `SYSTEM_VERSION_COMPAT` environment variable workaround MUST NOT be enabled; e.g. the version reported for Big Sur should be 11.x and not 10.16. If the version cannot be estimated (e.g. because the native platform is not macOS), the fallback value MUST be set to `0`. The build string MUST be `0`. -The version MUST be overridable with the `CONDA_OVERRIDE_OSX` environment variable. If this environment variable is set to the empty string `""`, then the `__osx` virtual package MUST NOT be present. The environment variable MUST be ignored when the target platform is not `osx-*`. +The version MUST be overridable with the `CONDA_OVERRIDE_OSX` environment variable if set to a non-empty value that can be parsed as a version string. The environment variable MUST be ignored when the target platform is not `osx-*`. > The macOS version can be obtained via: > > - Python's `platform.mac_ver()[0]` -> - `sw_vers -productVersion` -> -> If applicable, the `SYSTEM_VERSION_COMPAT` workaround MUST NOT be enabled; e.g. the version reported for Big Sur should be 11.x and not 10.16. +> - `SYSTEM_VERSION_COMPAT=0 sw_vers -productVersion` #### `__unix` @@ -136,16 +136,18 @@ The `CONDA_OVERRIDE_UNIX` environment variable MUST NOT have any effect. #### `__win` -This virtual package MUST be present when the target platform is `win-*`. The version MUST be set to the first three numeric components of the Windows build version, formatted as `{major}.{minor}.{build}`. If the version cannot be estimated (e.g. because the target platform does not match the native platform), the fallback value MUST be set to `0`. The build string MUST be `0`. +This virtual package MUST be present when the target platform is `win-*`. The version MUST be set to the first three numeric components of the Windows build version, formatted as `{major}.{minor}.{build}`. If the version cannot be estimated (e.g. because the target platform does not match the native platform), the tool MUST set the version to a default value of its choice. -The string `{major}.{minor}.{build}` can be obtained from: +The version MUST be overridable with the `CONDA_OVERRIDE_WIN` environment variable if set to a non-empty value that can be parsed as a version string. The environment variable MUST be ignored when the target platform is not `win-*`. -- Python's `platform.win32_ver()` -- CMD's `ver` -- Powershell's `[System.Environment]::OSVersion.Version`, `(Get-CimInstance Win32_OperatingSystem).version` -- The command `wmic os get version` +The build string MUST be `0`. -The version MUST be overridable with the `CONDA_OVERRIDE_WIN` environment variable. If this environment variable is set to the empty string `""`, then the `__win` virtual package MUST NOT be present. The environment variable MUST be ignored when the target platform is not `win-*`. +> The version string `{major}.{minor}.{build}` can be obtained from: +> +> - Python's `platform.win32_ver()` +> - CMD's `ver` +- Powershell's `[System.Environment]::OSVersion.Version`, `(Get-CimInstance Win32_OperatingSystem).version` +> - The command `wmic os get version` ## Potential future work From 21486542e3c82e7c19db74b6260c7692d4827cee Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 19 Dec 2024 13:06:21 +0100 Subject: [PATCH 18/26] reword __archspec --- cep-????.md | 46 ++++++++++++++++++++++------------------------ 1 file changed, 22 insertions(+), 24 deletions(-) diff --git a/cep-????.md b/cep-????.md index d1f02d30..2590a44a 100644 --- a/cep-????.md +++ b/cep-????.md @@ -53,30 +53,28 @@ In alphabetical order, every conda client MUST support the following virtual pac #### `__archspec` -This virtual package MUST be always present, with the version set to `1`. The build string SHOULD reflect the detected CPU microarchitecture when the target platform matches the native platform, -as provided by the generic values in the [`archspec/archspec-json` database](https://github.com/archspec/archspec-json/blob/v0.2.5/cpu/microarchitectures.json), plus some exceptions. - -For example, `conda/conda` reports its generic values for Intel/AMD and vendor-specific names for Apple: - -- Generic: `x86_64_v1`, `x86_64_v2`, `x86_64_v3`, `x86_64_v4`, `arm`, `ppc`... -- Apple: For M1, the value is `m1`. M2 and M3 are reported as `m2` and `m3`, respectively. - -If the microarchitecture cannot be detected or the target platform does not match the native platform, the build string MUST be set to the second component of the target platform, mapped with these rules: - -| Target platform | Reported `archspec` build string | -| --------------- | -------------------------------- | -| `*-32` | `x86` | -| `*-64` | `x86_64` | -| `*-armv6l` | `armv6l` | -| `*-armv7l` | `armv7l` | -| `*-aarch64` | `aarch64` | -| `*-arm64` | `arm64` | -| `*-ppc64` | `ppc64` | -| `*-ppc64le` | `ppc64le` | -| `*-riscv64` | `riscv64` | -| `*-s390x` | `s390x` | -| `zos-z` | `0` | -| Any other value | `0` | +This virtual package MUST be always present, with the version set to `1`. + +The build string MUST reflect one of: + +- The detected CPU microarchitecture when the target platform matches the native platform, as specified in the [`archspec/archspec-json` database](https://github.com/archspec/archspec-json/blob/v0.2.5/cpu/microarchitectures.json). For Apple Silicon, the reported values MUST correspond to the `vendor == Apple` entries; e.g. `m1`, `m2`, etc. For any other vendors, the reported values MUST correspond to the `vendor == generic` entries; e.g. `x86_64_v1`, `x86_64_v2`, `x86_64_v3`, `x86_64_v4`, `arm`, `ppc`, etc. + +- The target platform architecture (second component of the platform string), mapped as: + + | Target platform | Reported `archspec` build string | + | --------------- | -------------------------------- | + | `*-32` | `x86` | + | `*-64` | `x86_64` | + | `*-armv6l` | `armv6l` | + | `*-armv7l` | `armv7l` | + | `*-aarch64` | `aarch64` | + | `*-arm64` | `arm64` | + | `*-ppc64` | `ppc64` | + | `*-ppc64le` | `ppc64le` | + | `*-riscv64` | `riscv64` | + | `*-s390x` | `s390x` | + | `zos-z` | `0` | + | Any other value | `0` | The build string MUST be overridable with the `CONDA_OVERRIDE_ARCHSPEC` environment variable, if set to a non-empty value. From eaeb2223ee901438e0d3092114473e9f9e6dc46d Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 19 Dec 2024 14:04:01 +0100 Subject: [PATCH 19/26] discard generic CPU rules --- cep-????.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cep-????.md b/cep-????.md index 2590a44a..6dcf71ff 100644 --- a/cep-????.md +++ b/cep-????.md @@ -57,7 +57,7 @@ This virtual package MUST be always present, with the version set to `1`. The build string MUST reflect one of: -- The detected CPU microarchitecture when the target platform matches the native platform, as specified in the [`archspec/archspec-json` database](https://github.com/archspec/archspec-json/blob/v0.2.5/cpu/microarchitectures.json). For Apple Silicon, the reported values MUST correspond to the `vendor == Apple` entries; e.g. `m1`, `m2`, etc. For any other vendors, the reported values MUST correspond to the `vendor == generic` entries; e.g. `x86_64_v1`, `x86_64_v2`, `x86_64_v3`, `x86_64_v4`, `arm`, `ppc`, etc. +- If the target platform matches the native platform, the best fitting CPU microarchitecture in the [`archspec/archspec-json` database](https://github.com/archspec/archspec-json/blob/v0.2.5/cpu/microarchitectures.json). The reference CPU detection implementation is [`archspec.cpu.detect.host()`](https://github.com/archspec/archspec/blob/v0.2.5/archspec/cpu/detect.py#L338). - The target platform architecture (second component of the platform string), mapped as: From 9a95c6ba3e545123de399f5e6b4525a155a57f0e Mon Sep 17 00:00:00 2001 From: jaimergp Date: Sat, 21 Dec 2024 01:44:07 +0100 Subject: [PATCH 20/26] clarify env var syntax --- cep-????.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cep-????.md b/cep-????.md index 6dcf71ff..493dc093 100644 --- a/cep-????.md +++ b/cep-????.md @@ -35,7 +35,7 @@ The name MUST start with double underscore (`__`). The version and build string Some general considerations: -- The version or build string of a virtual package MAY be overridden by the value of `CONDA_OVERRIDE_{NAME}` environment variable, with `{NAME}` being the uppercased name of the virtual package. Many exceptions apply so please observe the details in the section below. +- The version or build string of a virtual package MAY be overridden by the value of `CONDA_OVERRIDE_{NAME}` environment variable, with `{NAME}` being the uppercased name of the virtual package (excluding the leading underscores). Many exceptions apply so please observe the details in the section below. - The build string MAY be zero (`0`). Some exceptions apply. See below. - When the tool used a fallback default value instead of a computed one, it SHOULD also inform the user of that choice and its possible override options (e.g. `CONDA_OVERRIDE_{NAME}` variables, CLI flags, configuration file, etc). From 6a9f82e19411a66272ec457981d26387c58538b5 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Wed, 12 Feb 2025 15:56:58 +0100 Subject: [PATCH 21/26] Apply Cheng's suggestions --- cep-????.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cep-????.md b/cep-????.md index 493dc093..3d5aa5ab 100644 --- a/cep-????.md +++ b/cep-????.md @@ -105,7 +105,7 @@ The build string MUST always be `0`. #### `__linux` -This virtual package MUST be present when the target platform is `linux-*`. Its version value MUST be set to the Linux kernel version, constrained to the first two to four numeric components formatted as `{major}.{minor}.{micro}.{patch}`. If the version cannot be estimated (e.g. because the native platform is not Linux), the tool MUST set `version` to a fallback value of its choice. The build string MUST be `0`. +This virtual package MUST be present when the target platform is `linux-*`. Its version value MUST be set to the upstream (mainline) Linux kernel version, but it MUST exclude any and all [distribution-specific components](https://www.kernel.org/releases.html#distribution-kernels) of the kernel version. If the version cannot be estimated (e.g. because the native platform is not Linux), the tool MUST set `version` to a fallback value of its choice. The build string MUST be `0`. The version MUST be overridable with the `CONDA_OVERRIDE_LINUX` environment variable, if set to a non-empty value that matches the regex `"\d+\.\d+(\.\d+)?(\.\d+)?"`. The environment variable MUST be ignored when the target platform is not `linux-*`. From d4b32f991d60760f87de13dfaa89a5c2e3a8bae5 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Wed, 12 Feb 2025 15:57:28 +0100 Subject: [PATCH 22/26] Add riscv32 and sort table --- cep-????.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/cep-????.md b/cep-????.md index 3d5aa5ab..f4145038 100644 --- a/cep-????.md +++ b/cep-????.md @@ -65,12 +65,13 @@ The build string MUST reflect one of: | --------------- | -------------------------------- | | `*-32` | `x86` | | `*-64` | `x86_64` | - | `*-armv6l` | `armv6l` | - | `*-armv7l` | `armv7l` | | `*-aarch64` | `aarch64` | | `*-arm64` | `arm64` | + | `*-armv6l` | `armv6l` | + | `*-armv7l` | `armv7l` | | `*-ppc64` | `ppc64` | | `*-ppc64le` | `ppc64le` | + | `*-riscv32` | `riscv32` | | `*-riscv64` | `riscv64` | | `*-s390x` | `s390x` | | `zos-z` | `0` | From 30a60301e2ae619f3fc5338f07d6210c5d6e9834 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Fri, 26 Sep 2025 18:57:21 +0200 Subject: [PATCH 23/26] Rename --- cep-????.md => cep-XXXX.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename cep-????.md => cep-XXXX.md (100%) diff --git a/cep-????.md b/cep-XXXX.md similarity index 100% rename from cep-????.md rename to cep-XXXX.md From 6b6714e45cd0098b4caf390693153d1ab60d9d00 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Fri, 26 Sep 2025 18:58:27 +0200 Subject: [PATCH 24/26] pre-commit --- cep-XXXX.md | 81 +++++++++++++++++++++++++++-------------------------- 1 file changed, 42 insertions(+), 39 deletions(-) diff --git a/cep-XXXX.md b/cep-XXXX.md index f4145038..1dbc4600 100644 --- a/cep-XXXX.md +++ b/cep-XXXX.md @@ -1,7 +1,7 @@ -# CEP ???? - Virtual packages +# CEP XXXX - Virtual packages - + @@ -33,33 +33,33 @@ Virtual packages are used to expose details of the system configuration to a con A virtual package is defined as a package record with three fields: name, version and build string. The name MUST start with double underscore (`__`). The version and build string MUST follow the same semantics as in regular package records. More specifically, the version field MUST follow the version string specifications, regardless its origin (computed from a system property, overridden by the user or configuration, or provided by default by the tool). -Some general considerations: +Some general considerations: -- The version or build string of a virtual package MAY be overridden by the value of `CONDA_OVERRIDE_{NAME}` environment variable, with `{NAME}` being the uppercased name of the virtual package (excluding the leading underscores). Many exceptions apply so please observe the details in the section below. -- The build string MAY be zero (`0`). Some exceptions apply. See below. -- When the tool used a fallback default value instead of a computed one, it SHOULD also inform the user of that choice and its possible override options (e.g. `CONDA_OVERRIDE_{NAME}` variables, CLI flags, configuration file, etc). +* The version or build string of a virtual package MAY be overridden by the value of `CONDA_OVERRIDE_{NAME}` environment variable, with `{NAME}` being the uppercased name of the virtual package (excluding the leading underscores). Many exceptions apply so please observe the details in the section below. +* The build string MAY be zero (`0`). Some exceptions apply. See below. +* When the tool used a fallback default value instead of a computed one, it SHOULD also inform the user of that choice and its possible override options (e.g. `CONDA_OVERRIDE_{NAME}` variables, CLI flags, configuration file, etc). ### List of virtual packages In alphabetical order, every conda client MUST support the following virtual packages: -- `__archspec` -- `__cuda` -- `__glibc` -- `__linux` -- `__osx` -- `__unix` -- `__win` +* `__archspec` +* `__cuda` +* `__glibc` +* `__linux` +* `__osx` +* `__unix` +* `__win` #### `__archspec` -This virtual package MUST be always present, with the version set to `1`. +This virtual package MUST be always present, with the version set to `1`. The build string MUST reflect one of: -- If the target platform matches the native platform, the best fitting CPU microarchitecture in the [`archspec/archspec-json` database](https://github.com/archspec/archspec-json/blob/v0.2.5/cpu/microarchitectures.json). The reference CPU detection implementation is [`archspec.cpu.detect.host()`](https://github.com/archspec/archspec/blob/v0.2.5/archspec/cpu/detect.py#L338). +* If the target platform matches the native platform, the best fitting CPU microarchitecture in the [`archspec/archspec-json` database](https://github.com/archspec/archspec-json/blob/v0.2.5/cpu/microarchitectures.json). The reference CPU detection implementation is [`archspec.cpu.detect.host()`](https://github.com/archspec/archspec/blob/v0.2.5/archspec/cpu/detect.py#L338). -- The target platform architecture (second component of the platform string), mapped as: +* The target platform architecture (second component of the platform string), mapped as: | Target platform | Reported `archspec` build string | | --------------- | -------------------------------- | @@ -89,20 +89,19 @@ The version MUST be overridable with the `CONDA_OVERRIDE_CUDA` environment varia This virtual package MUST NOT be present if the target platform is not `linux-*`. -This virtual package MUST be present when the native and target platforms are both the same type of `linux-*` and GNU `libc` is installed in the system. The version value MUST be set to the system GNU `libc` version, constrained to the first two components (major and minor) formatted as `{major}.{minor}`. If the version cannot be estimated, the tool MUST set the version to a default value (e.g. `2.17`). +This virtual package MUST be present when the native and target platforms are both the same type of `linux-*` and GNU `libc` is installed in the system. The version value MUST be set to the system GNU `libc` version, constrained to the first two components (major and minor) formatted as `{major}.{minor}`. If the version cannot be estimated, the tool MUST set the version to a default value (e.g. `2.17`). -If the native platform does not match the target platform, the tool MAY export `__glibc` with its `version` field set to a default value (e.g. `2.17`) of its choice. +If the native platform does not match the target platform, the tool MAY export `__glibc` with its `version` field set to a default value (e.g. `2.17`) of its choice. -If the `CONDA_OVERRIDE_GLIBC` environment variable if set to a non-empty value that complies to the version string specification, the tool MUST export `__glibc` with its version value set to the value of the environemnt variable. - -The build string MUST always be `0`. +If the `CONDA_OVERRIDE_GLIBC` environment variable if set to a non-empty value that complies to the version string specification, the tool MUST export `__glibc` with its version value set to the value of the environment variable. +The build string MUST always be `0`. > The GNU `libc` version can be computed via: > -> - Python's `os.confstr("CS_GNU_LIBC_VERSION")` -> - `getconf GNU_LIBC_VERSION` -> - `ldd --version`. Please verify that it references GNU `libc` or GLIBC. For non-standard installs, using a GLIBC compatibility layer, this may require locating the implementation and directly querying. +> * Python's `os.confstr("CS_GNU_LIBC_VERSION")` +> * `getconf GNU_LIBC_VERSION` +> * `ldd --version`. Please verify that it references GNU `libc` or GLIBC. For non-standard installs, using a GLIBC compatibility layer, this may require locating the implementation and directly querying. #### `__linux` @@ -111,10 +110,10 @@ This virtual package MUST be present when the target platform is `linux-*`. Its The version MUST be overridable with the `CONDA_OVERRIDE_LINUX` environment variable, if set to a non-empty value that matches the regex `"\d+\.\d+(\.\d+)?(\.\d+)?"`. The environment variable MUST be ignored when the target platform is not `linux-*`. > The Linux kernel version can be obtained via: -> -> - Python's `platform.release()` -> - `uname -r` -> - `cat /proc/version` +> +> * Python's `platform.release()` +> * `uname -r` +> * `cat /proc/version` #### `__osx` @@ -123,9 +122,9 @@ This virtual package MUST be present when the target platform is `osx-*`. Its ve The version MUST be overridable with the `CONDA_OVERRIDE_OSX` environment variable if set to a non-empty value that can be parsed as a version string. The environment variable MUST be ignored when the target platform is not `osx-*`. > The macOS version can be obtained via: -> -> - Python's `platform.mac_ver()[0]` -> - `SYSTEM_VERSION_COMPAT=0 sw_vers -productVersion` +> +> * Python's `platform.mac_ver()[0]` +> * `SYSTEM_VERSION_COMPAT=0 sw_vers -productVersion` #### `__unix` @@ -142,11 +141,13 @@ The version MUST be overridable with the `CONDA_OVERRIDE_WIN` environment variab The build string MUST be `0`. > The version string `{major}.{minor}.{build}` can be obtained from: -> -> - Python's `platform.win32_ver()` -> - CMD's `ver` -- Powershell's `[System.Environment]::OSVersion.Version`, `(Get-CimInstance Win32_OperatingSystem).version` -> - The command `wmic os get version` +> +> * Python's `platform.win32_ver()` +> * CMD's `ver` +> +* Powershell's `[System.Environment]::OSVersion.Version`, `(Get-CimInstance Win32_OperatingSystem).version` +> +> * The command `wmic os get version` ## Potential future work @@ -154,9 +155,9 @@ This CEP focuses on the standardization of existing virtual package implementati The following items are not considered here. Though would be open for discussion in future CEP work: -- Additional OSes, like `__freebsd` or `__netbsd`. -- Coarse grain architecture information, like `__x86_64` or `__arm64`, or, more generally, [`__arch`](https://github.com/conda/conda/issues/13420). -- More `libc` implementations, like `__musl`. +* Additional OSes, like `__freebsd` or `__netbsd`. +* Coarse grain architecture information, like `__x86_64` or `__arm64`, or, more generally, [`__arch`](https://github.com/conda/conda/issues/13420). +* More `libc` implementations, like `__musl`. ## References @@ -170,3 +171,5 @@ The following items are not considered here. Though would be open for discussion ## Copyright All CEPs are explicitly [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/). + +[RFC2119]: https://datatracker.ietf.org/doc/html/rfc2119 From 83ea61eae466559aadc25383ff43f9fc76abb687 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Fri, 26 Sep 2025 18:59:36 +0200 Subject: [PATCH 25/26] more pre-commit --- cep-XXXX.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/cep-XXXX.md b/cep-XXXX.md index 1dbc4600..051721a7 100644 --- a/cep-XXXX.md +++ b/cep-XXXX.md @@ -117,7 +117,9 @@ The version MUST be overridable with the `CONDA_OVERRIDE_LINUX` environment vari #### `__osx` -This virtual package MUST be present when the target platform is `osx-*`. Its version value MUST be set to the first two numeric components of macOS version formatted as `{major}.{minor}`. If applicable, the `SYSTEM_VERSION_COMPAT` environment variable workaround MUST NOT be enabled; e.g. the version reported for Big Sur should be 11.x and not 10.16. If the version cannot be estimated (e.g. because the native platform is not macOS), the fallback value MUST be set to `0`. The build string MUST be `0`. +This virtual package MUST be present when the target platform is `osx-*`. Its version value MUST be set to the first two numeric components of macOS version formatted as `{major}.{minor}`. If applicable, the `SYSTEM_VERSION_COMPAT` environment variable workaround MUST NOT be enabled; e.g. the version reported for Big Sur should be 11.x and not 10.16. + +If the version cannot be estimated (e.g. because the native platform is not macOS), the fallback value MUST be set to `0`. The build string MUST be `0`. The version MUST be overridable with the `CONDA_OVERRIDE_OSX` environment variable if set to a non-empty value that can be parsed as a version string. The environment variable MUST be ignored when the target platform is not `osx-*`. From be15de88e7d7ea4c516dc94536462c7db53ebe1f Mon Sep 17 00:00:00 2001 From: jaimergp Date: Fri, 24 Oct 2025 14:23:13 +0200 Subject: [PATCH 26/26] Update references to existing CEPs or drafts --- cep-XXXX.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/cep-XXXX.md b/cep-XXXX.md index 051721a7..e4354e71 100644 --- a/cep-XXXX.md +++ b/cep-XXXX.md @@ -8,6 +8,8 @@ + +
Title Virtual packages
Title CEP XXXX - Virtual packages
Status Draft
Author(s) Jaime Rodríguez-Guerra <jaime.rogue@gmail.com>
Created Dec 17, 2024
Updated Dec 17, 2024
Discussion https://github.com/conda/ceps/pull/103
Implementation https://github.com/conda/conda/tree/24.11.1/conda/plugins/virtual_packages, https://github.com/mamba-org/mamba/blob/libmamba-2.0.5/libmamba/src/core/virtual_packages.cpp, https://github.com/conda/rattler/tree/rattler-v0.28.8/crates/rattler_virtual_packages/src
Requires https://github.com/conda/ceps/pull/132
> The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", @@ -31,7 +33,7 @@ Virtual packages are used to expose details of the system configuration to a con ## Specification A virtual package is defined as a package record with three fields: name, version and build string. -The name MUST start with double underscore (`__`). The version and build string MUST follow the same semantics as in regular package records. More specifically, the version field MUST follow the version string specifications, regardless its origin (computed from a system property, overridden by the user or configuration, or provided by default by the tool). +These fields MUST comply with [CEP 26](./cep-0026.md). More specifically, the version field MUST follow the version string specifications as discussed in [CEP PR #132](https://github.com/conda/ceps/pull/132), regardless its origin (computed from a system property, overridden by the user or configuration, or provided by default by the tool). Some general considerations: