Skip to content

Clarify model export and runtime ownership - #500

Merged
justinchuby merged 4 commits into
mainfrom
justinchuby-separate-no-gate-skills
Aug 15, 2026
Merged

Clarify model export and runtime ownership#500
justinchuby merged 4 commits into
mainfrom
justinchuby-separate-no-gate-skills

Conversation

@justinchuby

Copy link
Copy Markdown
Member

Summary

  • extract the model-conversion lessons from Add NVIDIA Nemotron 3.5 Lightning export support #487 into a standalone skills-only change
  • require faithful graph-derived metadata without gating Mobius artifacts on the current ORT GenAI registry, version, topology, or cache executor
  • clarify cached-decode diagnosis, standard-ONNX controls before blaming Olive, and provider isolation for weight-only quantization
  • preserve the quantized-by-default and Direct GGUF guidance merged in Make GGUF imports quantized by default #496

Scope

Only the seven intended SKILL.md files are changed. Model code, tests, workflows, examples, and .agents/skills/moe-models/SKILL.md are intentionally excluded.

Validation

  • python -m pytest tests/yaml_schema_test.py -q --tb=short (246 passed)
  • all 19 skill frontmatter blocks parsed and matched their directory names
  • lintrunner -a --output oneline
  • git diff --check
  • independent code review completed; its optional-runtime waiver contradiction was resolved

Document that Mobius emits faithful artifacts without gating on downstream ORT GenAI capabilities. Strengthen cache, quantization, provider isolation, and optional runtime validation guidance while preserving direct GGUF defaults.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates Mobius’s internal Skills documentation to clarify that Mobius is responsible for faithful, graph-derived export metadata, while ORT GenAI / Foundry Local runtime acceptance is optional downstream evidence that must not gate artifact emission.

Changes:

  • Refines the Definition-of-Done guidance to distinguish required metadata correctness from optional downstream runtime probes.
  • Adds/adjusts diagnostic guidance for cached-decode failures and for validating standard-ONNX controls before attributing issues to Olive/runtime.
  • Documents operational guidance around provider isolation for weight-only quantization workflows.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
.agents/skills/writing-tests/SKILL.md Adds guidance on reduced real-weight fixtures and cached-decode parity checks.
.agents/skills/quality-checklist/SKILL.md Reframes checklist items around metadata correctness and optional downstream runtime probes.
.agents/skills/ort-genai-config/SKILL.md Clarifies Mobius metadata ownership vs ORT GenAI runtime capability decisions.
.agents/skills/onnx-export-quantization/SKILL.md Adds provider-isolation guidance and reframes ORT GenAI quantized smoke tests as optional.
.agents/skills/multimodal-models/SKILL.md Aligns multimodal guidance with “emit faithful metadata; runtime is optional probe”.
.agents/skills/diffusion-models/SKILL.md Aligns diffusion guidance to avoid runtime limitations becoming export gates.
.agents/skills/adding-a-new-model/SKILL.md Updates the checklist to emphasize tested graph-derived metadata and optional downstream probes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .agents/skills/writing-tests/SKILL.md Outdated
Comment thread .agents/skills/ort-genai-config/SKILL.md
@github-actions

github-actions Bot commented Aug 15, 2026

Copy link
Copy Markdown

Performance Comparison

Comparing 1299f3987e57e7

Model Metric Baseline Current Delta
bert (feature-extraction) model_size_bytes 359 KB 359 KB +0.0%
bert (feature-extraction) num_nodes 60 60 +0.0%
falcon model_size_bytes 364 KB 364 KB +0.0%
falcon num_nodes 66 66 +0.0%
gemma2 model_size_bytes 428 KB 428 KB +0.0%
gemma2 num_nodes 105 105 +0.0%
gpt2 model_size_bytes 388 KB 388 KB +0.0%
gpt2 num_nodes 54 54 +0.0%
llama model_size_bytes 425 KB 425 KB +0.0%
llama num_nodes 60 60 +0.0%
llama (static-cache) model_size_bytes 425 KB 425 KB +0.0%
llama (static-cache) num_nodes 56 56 +0.0%
mamba (ssm-text-generation) model_size_bytes 296 KB 296 KB +0.0%
mamba (ssm-text-generation) num_nodes 94 94 +0.0%
phi3 model_size_bytes 421 KB 421 KB +0.0%
phi3 num_nodes 58 58 +0.0%
phi3 (static-cache) model_size_bytes 421 KB 421 KB +0.0%
phi3 (static-cache) num_nodes 54 54 +0.0%
qwen2 model_size_bytes 425 KB 425 KB +0.0%
qwen2 num_nodes 60 60 +0.0%
qwen2 (static-cache) model_size_bytes 425 KB 425 KB +0.0%
qwen2 (static-cache) num_nodes 56 56 +0.0%
qwen3_5_moe (hybrid-text-generation) model_size_bytes 506 KB 506 KB +0.0%
qwen3_5_moe (hybrid-text-generation) num_nodes 264 264 +0.0%
qwen3_5_text (hybrid-text-generation) model_size_bytes 458 KB 458 KB +0.0%
qwen3_5_text (hybrid-text-generation) num_nodes 126 126 +0.0%
qwen3_5_vl (hybrid-qwen-vl) model_size_bytes 977 KB 977 KB +0.0%
qwen3_5_vl (hybrid-qwen-vl) num_nodes 428 428 +0.0%
t5 (seq2seq) model_size_bytes 836 KB 836 KB +0.0%
t5 (seq2seq) num_nodes 166 166 +0.0%
whisper (speech-to-text) model_size_bytes 1008 KB 1008 KB +0.0%
whisper (speech-to-text) num_nodes 128 128 +0.0%

No performance regressions.

justinchuby and others added 2 commits August 14, 2026 23:21
Require direct ONNX Runtime validation for GGUF imports while keeping downstream GenAI and Foundry probes optional. Clarify reduced-fixture and metadata wording from review.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>
Move dual-head-dimension runtime configuration guidance into optional downstream evidence and state that omitted probes require no waiver or TODO.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>
justinchuby added a commit that referenced this pull request Aug 15, 2026
Remove the seven cross-cutting skill updates from the Nemotron implementation branch now that they are published independently in PR #500. Retain Nemotron-specific MoE guidance and all implementation, test, example, and workflow changes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>
Comment thread .agents/skills/ort-genai-config/SKILL.md Outdated
Comment thread .agents/skills/quality-checklist/SKILL.md Outdated
Comment thread .agents/skills/quality-checklist/SKILL.md Outdated
Comment thread .agents/skills/quality-checklist/SKILL.md Outdated
Comment thread .agents/skills/quality-checklist/SKILL.md Outdated
Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
@justinchuby
justinchuby merged commit 8143a7c into main Aug 15, 2026
18 of 19 checks passed
@justinchuby
justinchuby deleted the justinchuby-separate-no-gate-skills branch August 15, 2026 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants