Skip to content

Conversation

@lhoupert
Copy link
Contributor

@lhoupert lhoupert commented Dec 11, 2025

This pull request introduces utilities and enhancements for managing and annotating S3 storage tier metadata in STAC items, focusing on OVHcloud storage classes. The main changes include adding a new shared utility module for storage tier operations, updating the STAC registration workflow to use these utilities, and improving documentation and metadata handling for storage tiers.

New storage tier utilities:

  • Added a new module storage_tier_utils.py providing functions to:
    • Extract the OVHcloud region from an S3 endpoint.
    • Query and validate S3 storage classes (including detection of mixed storage tiers in Zarr directories).
    • Return detailed tier distribution info for Zarr assets.

Enhancements to STAC registration:

  • Updated register_v1.py to use the new utility functions for extracting region and querying storage tier, replacing previous inline logic. [1] [2]
  • When registering assets, the code now annotates each asset's alternate.s3 with the detected storage tier (using the ovh:storage_tier field), preserving any existing alternate asset metadata.
  • Improved the registration workflow to clarify the order of steps and document the addition of storage tier metadata.

Documentation improvements:

  • Added a new README_update_storage_tier.md describing the update process, usage examples, expected outputs, and related scripts for managing storage tier metadata.

These changes make storage tier handling more robust, reusable, and transparent for both new and legacy STAC items.

Contributes to implementation of the 2nd phase 2. Storage Tier Transitions of https://github.com/EOPF-Explorer/coordination/issues/182, as discussed in ADR https://github.com/EOPF-Explorer/coordination/pull/179

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants