Remove NRL docs and Perflab baseline reporting - #2432
Conversation
3df166a to
f6ffc92
Compare
Greptile SummaryThe PR removes the local NRL harness documentation and Perflab baseline comparison implementation while redirecting benchmark guidance to NRB.
|
| Filename | Overview |
|---|---|
| nemo_retriever/src/nemo_retriever/harness/cli.py | Removes baseline loading but also removes the deprecated CLI compatibility option, leaving existing automation unable to invoke post-slack. |
| nemo_retriever/src/nemo_retriever/harness/slack.py | Removes baseline rendering and the compatibility keyword accepted by existing Python callers. |
| nemo_retriever/src/nemo_retriever/harness/release_reference.py | Deletes the obsolete release-reference parser as part of removing Perflab comparison support. |
| ops/retriever-nightly/run-nightly.sh | Removes release-reference propagation while retaining ordinary nightly execution and Slack posting. |
| nemo_retriever/README.md | Replaces deleted harness guidance with a concrete hyperlink to the NRB repository. |
| nemo_retriever/docs/cli/benchmarking.md | Redirects end-to-end benchmarking guidance to NRB while retaining local stage-microbenchmark documentation. |
Comments Outside Diff (1)
-
nemo_retriever/src/nemo_retriever/harness/cli.py, line 376-383 (link)Deprecated reference inputs rejected
When existing automation passes
post-slack --reference-file, Typer rejects the removed option; existing Python callers passingrelease_referencestobuild_slack_payloadorpost_report_to_slackreceive an unexpected-keywordTypeError. This breaks the compatibility contract for inputs that are intended to remain accepted while their data is ignored.Rule Used: Changes to public API surfaces (FastAPI endpoints,... (source)
Knowledge Base Used: Benchmark Harness
Prompt To Fix With AI
This is a comment left during a code review. Path: nemo_retriever/src/nemo_retriever/harness/cli.py Line: 376-383 Comment: **Deprecated reference inputs rejected** When existing automation passes `post-slack --reference-file`, Typer rejects the removed option; existing Python callers passing `release_references` to `build_slack_payload` or `post_report_to_slack` receive an unexpected-keyword `TypeError`. This breaks the compatibility contract for inputs that are intended to remain accepted while their data is ignored. **Rule Used:** Changes to public API surfaces (FastAPI endpoints,... ([source](.greptile)) **Knowledge Base Used:** [Benchmark Harness](https://app.greptile.com/nvidia-public-github/-/custom-context/knowledge-base/nvidia/nemo-retriever/-/docs/benchmark-harness.md) --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
Prompt To Fix All With AI
### Issue 1
nemo_retriever/src/nemo_retriever/harness/cli.py:376-383
**Deprecated reference inputs rejected**
When existing automation passes `post-slack --reference-file`, Typer rejects the removed option; existing Python callers passing `release_references` to `build_slack_payload` or `post_report_to_slack` receive an unexpected-keyword `TypeError`. This breaks the compatibility contract for inputs that are intended to remain accepted while their data is ignored.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Reviews (3): Last reviewed commit: "Purge deprecated reference compatibility" | Re-trigger Greptile
Summary
--reference-fileandrelease_referencescompatibility surfaces entirelyWhy
Perflab is the canonical home for performance results and comparisons. This removes the ad hoc baseline presentation from NRL while preserving the harness implementation for the separate code decision.
Validation
git diff --checkpasses[[ -v "$name" ]]construct is rejected by system Bash 3.2