Blosc2

[mkitti]

I see that you are primarily using c-blosc 1. Have you considered c-blosc 2? It is being much more actively developed than c-blosc 1 and corrects some significant deficiencies.
https://github.com/Blosc/c-blosc2

We'll probably need zarr-developers/numcodecs#413 , but this is overdue.

[aliddell]

We had started out with C-Blosc 2 but quickly ran into exactly what you noted: zarr-python (actually numcodecs) couldn't read what we wrote, which is a non-starter. We're definitely interested in upgrading and we're following that issue.

[mkitti]

I just got c-blosc 1 to address a known corruption issue in ZSTD, so please use c-blosc version 1.21.4 or greater to avoid that.
Blosc/c-blosc#364

[joshmoore]

I'll add that in general there's also the consideration of whether or not the generated data can be accessed beyond zarr-python.

With Zarr v3, that will be covered by the individual ZEPs covering different codecs.

[mkitti]

A Blosc 2 PR is now pending as developed at the NumFocus Summit: zarr-developers/numcodecs#463