CEP XXXX: Package metadata files served by conda channels #135

jaimergp · 2025-09-30T11:48:08Z

Checklist for submitter

I am submitting a new CEP: Package metadata files served by conda channels.
- I am using the CEP template by creating a copy cep-0000.md named cep-XXXX.md in the root level.
I am submitting modifications to CEP XX.
Something else: (add your description here).

Checklist for CEP approvals

The vote period has ended and the vote has passed the necessary quorum and approval thresholds.
A new CEP number has been minted. Usually, this is ${greatest-number-in-main} + 1.
The cep-XXXX.md file has been renamed accordingly.
The # CEP XXXX - header has been edited accordingly.
The CEP status in the table has been changed to approved.
The last modification date in the table has been updated accordingly.
The table in the README has been updated with the new CEP entry.
The pre-commit checks are passing.

soapy1 · 2025-10-29T22:19:28Z

cep-XXXX.md

+
+### `channeldata.json`
+
+Deprecated.


Is this really deprecated? I couldn't find any other notes that says that. And it looks like conda-build and conda-index still fully support features that rely on (or build) channeldata.

AFAIK this is the first public attempt to deprecate it, but it has come up in meetings and other conversations. For example, repo.prefix.dev does not generate it and seems to be perfectly functional as a channel. I'd need to check with conda-build, but AFAICR, it was only needed for run_exports and that's why we added CEP 12 (among other reasons). conda-index may still generate it for backwards compatibility.

I would prefer we document first (or as I said above move the channeldata.json definition into a seperate CEP), and then deprecate in a separate step.

No non-CDN anaconda.org channel generates channeldata. There are lots of reasons why it should go away, including that it can be nondeterministic which subdir's package is representative in channeldata.

soapy1 · 2025-10-29T22:31:28Z

cep-XXXX.md

+Each entry in `packages` and `packages.conda`:
+
+- MUST follow the `index.json` schema (see [CEP PR#133](https://github.com/conda/ceps/pull/133)).
+- SHOULD report the same values as the artifact's `info/index.json` metadata. Small modifications MAY be introduced to apply metadata fixes (e.g. correct the constraints of a requirement in the `depends` field) without needing to rebuild the artifact.


Is there a rule, or recommendation for how consumers should resolve differences between the artifact's info/index.json and the repodata info for the package?

This clause is introduced mostly to support repodata patching. repodata.json has precedence at install time. See https://github.com/jaimergp/ceps/blob/conda-meta-json/cep-XXXX.md ("Management of a conda environment").

Small, medium or large modifications perhaps.

cep-XXXX.md

jezdez · 2025-10-30T13:35:34Z

cep-XXXX.md

+
+## Abstract
+
+This CEP standardizes the schema for the metadata files served in conda channels. Namely, `repodata.json` (and its variants) and `channeldata.json`.


I'm wondering if specifying channeldata.json in a separate CEP might make it easier to deprecate it?

Good point. Moved to #140.

jezdez · 2025-10-30T13:39:20Z

cep-XXXX.md

+
+A conda channel MAY serve additional `repodata.json` documents in each subdir. Their name SHOULD match the glob `*repodata*.json`, and their contents MUST follow the `repodata.json` schema. Common variants include `current_repodata.json`.
+
+Channels SHOULD serve compressed versions of every repodata file. The following compression schemes are recognized:


I'm not sure if that's a SHOULD tbh; I think it should be a MUST, since practically speaking, conda-index does produce both. I'd think, for the sake of running compatible channels, we should clearly dictate what the minimum operationally reasonable set of files is.

I think if you are creating a local channel on your machine you don't need the compressed repodata for any practical reason. And, I think that should still be a valid channel.

Does it make sense to have like an implementation recommendation section? That can information like:

It's practical to serve compressed versions of repodata

The channeldata.json file is not required
etc.

we should clearly dictate what the minimum operationally reasonable set of files is.

+1!

Right, the compression is an optimization of distribution-sized channels, which I guess sharding solves differently. I guess the source of truth is just the repodata.json, and having a "deployment" or "best practices" section is a good idea.

rattler-index allows to not write the zstd file: https://github.com/conda/rattler/blob/55014fa15730502bfea1f185c0c2c539672b9d55/crates/rattler_index/src/lib.rs#L999

i think it should be optional (but recommended) as i can see use cases for personal small channels where it's not needed.

and having a "deployment" or "best practices" section is a good idea.

I'd rather leave that to the aggregated conda.org documentation, because those best practices may change over time and there's no need to "vote" on them on CEP.

One reason to allow missing repodata.json.zst is to support old, frozen channels. Backwards compatibility is necessary.
How long until we have sharded-repodata-only channels that provide no repodata.json?

jezdez · 2025-10-30T13:39:56Z

cep-XXXX.md

+
+### `channeldata.json`
+
+Deprecated.


I would prefer we document first (or as I said above move the channeldata.json definition into a seperate CEP), and then deprecate in a separate step.

jezdez · 2025-10-30T13:41:15Z

cep-XXXX.md

+The `channeldata.json` file is considered deprecated because the listed metadata may be unreliable. It assumes that all the artifacts for a given package name will always have a homogeneous composition, but this is not necessarily true. Some examples:
+
+- Some artifacts may contain activation scripts on some platforms, but not on others.
+- Prefix replacement may only be needed from a certain point in the lifetime of the project (e.g. the maintainers add a compiled extension for performance).
+- The website or license may change during the project lifetime.


Valid reasons, but out of scope of the CEP

Standardize repodata.json and friends

6524dad

jaimergp linked an issue Sep 30, 2025 that may be closed by this pull request

CEP: Repodata Quansight-Labs/conda-ecosystem-sta-mgmt#34

Open

jaimergp added 3 commits September 30, 2025 19:08

Do not assume filesystem

766303b

Finish info schema, repodata subsets and patching

4b3d47b

Document the legacy_bz2_* keys, cover additional keys

64053cf

danyeaw added this to conda Roadmap and Sprint Planning Oct 15, 2025

github-project-automation bot moved this to Sorting ⚙️ in conda Roadmap and Sprint Planning Oct 15, 2025

danyeaw moved this from Sorting ⚙️ to TODO 📬 in conda Roadmap and Sprint Planning Oct 15, 2025

jaimergp assigned jaimergp and jezdez Oct 22, 2025

Add Requires

845a03f

danyeaw moved this from TODO 📬 to In review 🔍 in conda Roadmap and Sprint Planning Oct 29, 2025

soapy1 reviewed Oct 29, 2025

View reviewed changes

Comment on current_repodata and supported compression schemes

4543d60

jezdez reviewed Oct 30, 2025

View reviewed changes

jaimergp added 3 commits November 5, 2025 11:45

Split channeldata.json

c99fb5a

Some more details for current_repodata

f9398a2

Update title

03278dd

jaimergp changed the title ~~CEP XXXX: Metadata files served by conda channels~~ CEP XXXX: Package metadata files served by conda channels Nov 5, 2025

jaimergp mentioned this pull request Nov 5, 2025

Add "Learn> Specifications" section conda/conda-dot-org#256

Open

18 tasks

jaimergp added 2 commits November 5, 2025 16:55

Add a bit of info about signatures metadata

e2307a6

Move signatures to appendix

12edd25

jaimergp mentioned this pull request Nov 5, 2025

CEP: Package signing (conda-content-trust) Quansight-Labs/conda-ecosystem-sta-mgmt#35

Closed

pre-commit

1e8f5bb

This was referenced Nov 11, 2025

Status Report (2025-11-07) Quansight-Labs/conda-ecosystem-sta-mgmt#94

Closed

CEP for the next evolution of Repodata (v2) and MatchSpec #111

Open


		## Abstract

		This CEP standardizes the schema for the metadata files served in conda channels. Namely, `repodata.json` (and its variants) and `channeldata.json`.


		A conda channel MAY serve additional `repodata.json` documents in each subdir. Their name SHOULD match the glob `repodata.json`, and their contents MUST follow the `repodata.json` schema. Common variants include `current_repodata.json`.

		Channels SHOULD serve compressed versions of every repodata file. The following compression schemes are recognized:


		### `channeldata.json`

		Deprecated.


		### `channeldata.json`

		Deprecated.

CEP XXXX: Package metadata files served by conda channels #135

Are you sure you want to change the base?

CEP XXXX: Package metadata files served by conda channels #135

Uh oh!

Conversation

jaimergp commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist for submitter

Checklist for CEP approvals

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dholth Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

jaimergp commented Sep 30, 2025 •

edited

Loading

dholth Nov 5, 2025 •

edited

Loading