Enforce the DL API contract in CI; drop the dead fields it caught - #179
Open
Alvin-Nahabwe wants to merge 1 commit into
Open
Enforce the DL API contract in CI; drop the dead fields it caught#179Alvin-Nahabwe wants to merge 1 commit into
Alvin-Nahabwe wants to merge 1 commit into
Conversation
scripts/check_api_contract.py parses server/deep_learning.R and verifies every field sent by dl_request()/req_body_multipart() pipelines exists in the vendored API contract (scripts/dl_api_contract.json, generated by no-code-transformers' scripts/dump_contract.py from the FastAPI OpenAPI schema). A new CI workflow runs it on every push/PR, so the drift class of bug that broke the platform (weight_decay vs weight_decay_hf, data_zip vs data_file) now fails the build instead of production. Running the checker immediately caught real drift: all three training calls sent fields the API ignores (num_proc, fp16, push_to_hub, hub_user_id, log_to_wandb, wandb_project, wandb_entity; OD also max_grad_norm and force_preprocess - 23 fields total). Removed them from the request bodies, and removed the UI controls that fed them (the three Hub & Logging panels plus the fp16 / num_proc / max-grad-norm / force-preprocess inputs): a control whose value is silently discarded is worse than no control. This also resolves the open question on the ASR PR about the dead W&B/Hub panels. Reinstate them if and when the API implements those features. Verified: checker passes on the cleaned client (8 requests), catches the historical weight_decay bug when deliberately reintroduced (exit 1), and both R files parse. The checker is stdlib-only Python. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
CI now proves the R client only sends fields the DL API accepts.
scripts/dl_api_contract.json— vendored contract (per endpoint, the exact accepted form fields), generated from the API's OpenAPI schema byno-code-transformers/scripts/dump_contract.py.scripts/check_api_contract.py— stdlib-only checker that parsesserver/deep_learning.R, extracts everydl_request() %>% req_body_multipart()field, and fails if any isn't in the contract..github/workflows/contract.yml— runs it on every push/PR. (Separate file from CI: R parse check and secret scan #176'sci.ymlto avoid a conflict; they can be merged later.)This is the permanent gate for the drift class of bug that broke the platform (
weight_decayvsweight_decay_hf,data_zipvsdata_file, the dead ASR surface): it now fails the build, not production.It immediately caught real drift
On first run the checker flagged 23 dead fields across all three training calls —
num_proc,fp16,push_to_hub,hub_user_id,log_to_wandb,wandb_project,wandb_entity(+ OD'smax_grad_norm,force_preprocess). The API silently ignores every one.Resolution: removed them from the request bodies and removed the UI controls that fed them (three "Hub & Logging" panels, fp16/num_proc/max-grad-norm/force-preprocess inputs). A control whose value is silently discarded misleads users. This also answers the open question from #174 about the dead W&B/Hub panels — reinstate when the API actually implements those features. Net: −74/+7 in the DL module, meaningfully decluttering the training sidebar.
Verification
weight_decaybug → caught, exit 1 (regression-proof)parse()clean; workflow YAML validKeeping the contract fresh
The API repo commits
dl_api_contract.jsonand CI there fails if it's stale. When the API changes its fields: regenerate there (python scripts/dump_contract.py > dl_api_contract.json), copy here, and this CI tells you if the client needs updating.🤖 Generated with Claude Code