feat(ilamb): add CMIP7 data support to ILAMB diagnostics#535
feat(ilamb): add CMIP7 data support to ILAMB diagnostics#535lewisjared merged 7 commits intomainfrom
Conversation
Codecov Report❌ Patch coverage is
🚀 New features to boost your workflow:
|
Add dual CMIP6/CMIP7 data requirements to ILAMBStandard so diagnostics can run against either MIP era. Includes branded variable name lookups via the Data Request, dynamic source type detection at execution time, and test data specifications for both CMIP6 and CMIP7. Also extends the variable map with new land (gpp, lai, mrro, mrsos, nbp, cSoil, mrsol, burntFractionAll, snc), ocean (so, sos, thetao, msftmz), and land-ice (snc) mappings.
…th present When CMIP7 model data includes both a 3D variable (e.g. thetao, 1850-1860) and its 2D surface alternate (e.g. tos, 1850-2014), ilamb3's merge creates a dataset where time_bnds is NaN-filled for the shorter variable's missing steps. This corrupts the time frequency calculation and causes a NotImplementedError in handle_timescale_mismatch. Drop the primary 3D variable when an alternate is available, letting ilamb3 use the surface variable which has full temporal coverage and is equivalent after select_depth. Also adds per-diagnostic test_source_id override for test data fetching.
CanESM5 only publishes volcello for p2 physics members (e.g. r26i1p2f1), not the r1i1p1f1 used by the main test data request. Previously, volcello was bundled with areacello/sftof in a single request where remove_ensembles picked a member lacking volcello. Also collect branded variable names from all matching DReq tables instead of stopping at the first match, so volcello gets both its Omon and Ofx branded names.
e3c8718 to
20fac01
Compare
There was a problem hiding this comment.
Pull request overview
Adds CMIP7 support to the ILAMB diagnostic provider so ILAMB diagnostics can run against either CMIP6 catalogs or CMIP7 catalogs (via branded variable name filtering and CMIP7-specific facet handling).
Changes:
- Extend the CMIP6→CMIP7 Data Request subset with additional ILAMB-relevant variables and update the extraction script’s included variable list.
- Update ILAMB Standard diagnostic wiring to support dual CMIP6/CMIP7
DataRequirements, branded variable lookups, and CMIP7 facet normalization (variant_label↔member_id). - Add/adjust ILAMB configuration and a changelog entry (including per-diagnostic test
source_idoverride forburntFractionAll-GFED).
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| scripts/extract-data-request-mappings.py | Expands the included variable IDs used when generating the bundled CMIP6→CMIP7 mapping subset. |
| packages/climate-ref-ilamb/src/climate_ref_ilamb/standard.py | Implements CMIP7-aware ILAMB requirements, branded-name lookup, selector normalization, and CMIP7 test cases. |
| packages/climate-ref-ilamb/src/climate_ref_ilamb/configure/ilamb.yaml | Adds a per-diagnostic test_source_id override for the burntFractionAll case. |
| packages/climate-ref-core/src/climate_ref_core/data/cmip6_cmip7_variable_map.json | Adds new Data Request mapping entries for ILAMB-used variables (land + ocean). |
| changelog/535.feature.md | Documents the user-facing CMIP7 support addition for ILAMB diagnostics. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| branded.append(entry.branded_variable_name) | ||
| found = True | ||
| except KeyError: | ||
| continue | ||
| if not found: | ||
| logger.debug(f"No CMIP7 branded variable name found for {var_id}") | ||
|
|
||
| return tuple(branded) |
There was a problem hiding this comment.
In _get_branded_variable_names, once a mapping is found for a variable in the preferred table order, the loop continues through the remaining tables and can append additional branded names for the same var_id. This can lead to duplicate/mixed branded variable filters and may match the wrong branding if a variable exists in multiple tables. Consider breaking out of the inner loop after the first successful get_dreq_entry (and optionally deduplicating the final list).
| branded.append(entry.branded_variable_name) | |
| found = True | |
| except KeyError: | |
| continue | |
| if not found: | |
| logger.debug(f"No CMIP7 branded variable name found for {var_id}") | |
| return tuple(branded) | |
| except KeyError: | |
| continue | |
| branded.append(entry.branded_variable_name) | |
| found = True | |
| # Once a mapping is found in the preferred table order, | |
| # stop searching additional tables for this variable. | |
| break | |
| if not found: | |
| logger.debug(f"No CMIP7 branded variable name found for {var_id}") | |
| # Deduplicate while preserving order, in case multiple inputs map | |
| # to the same branded variable name. | |
| return tuple(dict.fromkeys(branded)) |
* origin/main: refactor: replace _add_branded_variable_name with hybrid_property and rename to branded_variable fix(esmvaltool): pick latest recipe run when multiple executions exist fix(esmvaltool): preserve year coordinate in annual_statistics preprocessor fix(esmvaltool): add realm facets, fix test data source_ids, and handle CMIP7 per-variable datasets fix(core): sanitize activity_id and add version attr in CMIP6-to-CMIP7 conversion chore: rename changelog to match PR number feat: add file dimensions to all ESMValTool diagnostics feat: add file dimensions to ESMValTool diagnostics
Description
Add CMIP7 data support to ILAMB diagnostic providers, enabling diagnostics to run against either CMIP6 or CMIP7 datasets.
CMIP7 Test Case Status
ILAMB
Checklist
Please confirm that this pull request has done the following:
changelog/