fix(skill-creator): handle eval_id=None crash in eval-viewer sort#1316
Open
Alex-ai-future wants to merge 1 commit into
Open
fix(skill-creator): handle eval_id=None crash in eval-viewer sort#1316Alex-ai-future wants to merge 1 commit into
Alex-ai-future wants to merge 1 commit into
Conversation
Both Python and JS sort logic assumed eval_id is always an int. When eval_metadata.json is missing, build_run() returns eval_id=None which is a valid output. The sort code now explicitly handles None by placing those runs last. - generate_review.py: r.get() doesn't work when key exists with None value - viewer.html: a - b produces NaN when either operand is null Signed-off-by: Alex <alex.tech.lab@outlook.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both Python and JS sort logic in the skill-creator eval-viewer crash when
eval_idisNone/null. This fix makes the sort handle that case gracefully by placing those runs last.Bug description
When the skill-creator eval-viewer scans a workspace,
build_run()returnseval_id=Nonefor any run directory that does not have aneval_metadata.jsonfile (or the file exists but lacks aneval_idfield). The sort code then crashes:Python (
skills/skill-creator/eval-viewer/generate_review.py):r.get("eval_id", float("inf"))returnsNone— notinf— because.get()only falls back to the default when the key is absent, not when it exists with aNonevalue. Comparing(1, "...") < (None, "...")raisesTypeError: '<' not supported between instances of 'int' and 'NoneType'.JS (
skills/skill-creator/eval-viewer/viewer.html):null - intproducesNaN, making the sort order unpredictable.Why this is not a dirty-data problem
1NoneNoneNoneThe producer (
build_run()) and consumer (find_runs()) are in the same file (generate_review.py).build_run()was explicitly designed to returneval_id=Noneas a valid output rather than raising or skipping — the author chose to treat runs without metadata as legitimate entries. This is an internal contract: one module producesNone, another module does not handle it.The fix is to make the sort defensive: handle
Noneexplicitly instead of assuming alleval_idvalues are integers.Fix
skills/skill-creator/eval-viewer/generate_review.py: Changed tor["eval_id"] if r["eval_id"] is not None else float("inf")skills/skill-creator/eval-viewer/viewer.html: Changed to explicit null-aware comparator that placesnullvalues last