Skip to content

Sorcha/dev/1696 - #2630

Open
enssow wants to merge 34 commits into
developfrom
sorcha/dev/1696
Open

Sorcha/dev/1696#2630
enssow wants to merge 34 commits into
developfrom
sorcha/dev/1696

Conversation

@enssow

@enssow enssow commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Description

Read the Docs working implementation, to be moved to ecmwf/develop so it can be udpated when eval code is changed.

Issue Number

Closes #1696
Closes #2611

Checklist before asking for review

  • I have performed a self-review of my code
  • My changes comply with basic sanity checks:
    • I have fixed formatting issues with ./scripts/actions.sh lint
    • I have run unit tests with ./scripts/actions.sh unit-test
    • I have documented my code and I have updated the docstrings.
    • I have added unit tests, if relevant
  • I have tried my changes with data and code:
    • I have run the integration tests with ./scripts/actions.sh integration-test
    • (bigger changes) I have run a full training and I have written in the comment the run_id(s): launch-slurm.py --time 60
    • (bigger changes and experiments) I have shared a hegdedoc in the github issue with all the configurations and runs for this experiments
  • I have informed and aligned with people impacted by my change:
    • for config changes: the MatterMost channels and/or a design doc
    • for changes of dependencies: the MatterMost software development channel

@github-actions github-actions Bot added the eval anything related to the model evaluation pipeline label Jul 13, 2026

@tjhunter tjhunter left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@enssow (cc @iluise ): the docs look good, thank you for putting it together. All the important pieces are there.

One change breaks the uv packaging (moving README), the other comments are more details. As you noted, it is hard to use without an example, but I would not prioritize it: the fast-eval package is tied to running weathergen, and only the people in the consortium can currently run weathergen.

I would focus first on:

  • more general explanations on running end to end weathergen before more efforts on the eval doc.
  • automating checks on the json/yaml config schema: it is becoming long and complex, and the doc will fast misalign with the code.

name: readthedocs
locale: en
highlightjs: true
# font:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove the commented parts

build:
os: "ubuntu-22.04"
tools:
python: "3.11"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you make it python 3.12 like the rest of the project?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can it be symlink'ed from the root of the project rather than copied?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am confused why we still have this file, if the rest is moved inside package/evaluate

## Input

The `WeatherGenerator-FastEvaluation` package supports multiple inputs:
- The WeatherGenerator `.zarr` files generated by the inference step of the WeatherGenerator chain. It reads both `Target` and `Prediction` from local files (`WeatherGenReader`).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing new line, this is not rendering as a list


The default config file is at: `WeatherGenerator/configs/evaluate/eval_config.yml`

More instructions can be found here: https://gitlab.jsc.fz-juelich.de/esde/WeatherGenerator-private/-/wikis/home/Common-workflows/fast-evaluation

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this link is not a link in the final doc. Also, this is for the private wiki, it does not fit in public documentation (or it should be marked as private)


nav:
- "Introduction": README.md
- "Configuration File": guide/config.md

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Configuration reference"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you cannot move the file, it is required by uv to make a package (it is a python package requriment). However, you can create a symbolic link, which git will properly register.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sure to have an operating procedure with the command that you ran to update this file.

@github-project-automation github-project-automation Bot moved this to In Progress in WeatherGen-dev Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eval anything related to the model evaluation pipeline

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

Investigate readthedocs for code documentation convert .rst files to .md for readthedocs

4 participants