[ZFDB] Feature/zfdb/fdb 678 2nd step - refactor tests and zarr interface alignment

[tbkr]

Description

Further refactoring and Zarr API conformity alignment. Also restructured tests, as they became a bit disordered.

Contributor Declaration

By opening this pull request, I affirm the following:

• All authors agree to the Contributor License Agreement.
• The code follows the project's coding standards.
• I have performed self-review and added comments where needed.
• I have added or updated tests to verify that my changes are effective and functional.
• I have run all existing tests and confirmed they pass.

🌈🌦️📖🚧 Documentation FDB 🚧📖🌦️🌈
https://sites.ecmwf.int/docs/fdb/pull-requests/PR-302

[codecov-commenter]

Codecov Report

❌ Patch coverage is 6.25000% with 45 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.99%. Comparing base (4c75f61) to head (0b89d13).

Files with missing lines	Patch %	Lines
src/chunked_data_view/GribExtractor.cc	0.00%	19 Missing ⚠️
src/chunked_data_view/Axis.cc	0.00%	12 Missing ⚠️
src/chunked_data_view/RequestManipulation.cc	0.00%	10 Missing ⚠️
src/chunked_data_view/ChunkedDataViewBuilder.cc	42.85%	4 Missing ⚠️

Additional details and impacted files

@@                    Coverage Diff                    @@
##           feature/zfdb/refactor     #302      +/-   ##
=========================================================
- Coverage                  72.08%   71.99%   -0.10%     
=========================================================
  Files                        378      378              
  Lines                      23901    23946      +45     
  Branches                    2504     2513       +9     
=========================================================
+ Hits                       17230    17240      +10     
- Misses                      6671     6706      +35     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This PR continues the Zarr v3 alignment for z3fdb by updating the internal Zarr store implementation (notably metadata/consolidation behavior and key handling) and substantially reorganizing/expanding the Python test suite into clearer categories.

Changes:

• Updated FdbZarrStore/FdbZarrGroup/FdbZarrArray to use string keys and added root zarr.json generation with inline consolidated metadata for group roots.
• Restructured tests into unit/, permutation_tests/, and zarr_interface_conformity/, adding deeper store-path and consolidated-metadata conformity coverage.
• Updated CMake test registration to reflect the new test layout and added a minimal Ruff configuration.

Reviewed changes

Copilot reviewed 18 out of 20 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
tests/z3fdb/zarr_interface_conformity/test_z3fdb_store_path_correctness.py	New deep-hierarchy path/listing/get/getitem correctness coverage.
tests/z3fdb/zarr_interface_conformity/test_z3fdb_store_interface.py	Adds Zarr StoreTests-based interface checks for FdbZarrStore.
tests/z3fdb/zarr_interface_conformity/test_z3fdb_consolidated_metadata.py	Verifies inline consolidated metadata structure and Zarr API navigation.
tests/z3fdb/zarr_interface_conformity/conftest.py	Provides shared fixtures for deep store/group and configures asyncio behavior.
tests/z3fdb/zarr_interface_conformity/_mocks.py	Adds mock chunked data view + deep store factory helpers.
tests/z3fdb/unit/test_store_v3.py	Moves/extends basic v3 access tests into unit/.
tests/z3fdb/unit/test_store_v3_errors.py	Adds request-format and axis-extension error tests.
tests/z3fdb/test_store_v3.py	Removed (tests relocated into tests/z3fdb/unit/).
tests/z3fdb/test_store_v3_random_axis.py	Removed (coverage moved to new permutation tests).
tests/z3fdb/test_store_v3_pattern.py	Removed (coverage moved to new permutation tests).
tests/z3fdb/permutation_tests/test_store_multiple_parts.py	Refactors request strings into constants and reuses them across scenarios.
tests/z3fdb/permutation_tests/test_store_missing_values.py	Adds online test coverage for bitmap-missing handling and fill_value propagation.
tests/z3fdb/permutation_tests/test_store_individual_chunking.py	Formatting refactor; keeps chunking behavior checks.
tests/z3fdb/permutation_tests/test_scrambled_request_ordering.py	New tests for scrambled request value ordering correctness.
tests/z3fdb/permutation_tests/test_axis_definition_permutations.py	New exhaustive permutation sweep tests for axis definition ordering/chunking.
tests/z3fdb/permutation_tests/test_axis_definition_ordering.py	New targeted axis-ordering/chunking scenario tests.
tests/z3fdb/CMakeLists.txt	Re-registers tests by category and adjusts online test invocation.
src/z3fdb/simple_store_builder.py	Narrows build() return type to FdbZarrStore.
src/z3fdb/_internal/zarr.py	Implements string-key access, consolidated root zarr.json, and updated Store methods.
ruff.toml	Adds Ruff target-version/line-length configuration.

Comments suppressed due to low confidence (1)

tests/z3fdb/CMakeLists.txt:66

• The standalone online test uses ENVIRONMENT "${_env}", but _env is defined inside the add_py_test() function (function scope), so it will be empty here. That likely breaks imports (PYTHONPATH/FDB5_DIR/ECCODES_DIR) when running ctest -L z3fdb_online. Define _env in this scope (or move it to a global variable shared by both paths).

set_tests_properties(pychunked_data_view_${online_stem}
    PROPERTIES
    RESOURCE_LOCK pytest
    ENVIRONMENT "${_env}"
    LABELS "z3fdb;z3fdb_online"

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 30 out of 34 changed files in this pull request and generated 4 comments.