Skip to content

docs(extraction): complete Support Matrix coverage for answer-generation LLM and VLM paths (NVBug 6628661) - #2546

Open
kheiss-uwzoo wants to merge 2 commits into
NVIDIA:mainfrom
kheiss-uwzoo:docs/6628661-answer-llm-support-matrix
Open

docs(extraction): complete Support Matrix coverage for answer-generation LLM and VLM paths (NVBug 6628661)#2546
kheiss-uwzoo wants to merge 2 commits into
NVIDIA:mainfrom
kheiss-uwzoo:docs/6628661-answer-llm-support-matrix

Conversation

@kheiss-uwzoo

Copy link
Copy Markdown
Collaborator

Summary

  • Document the default Super-49B answer_llm hardware profile in the Support Matrix: two physical GPUs, BF16 TP2, 250Gi NIMCache, additional to the core pipeline, and SKUs that cannot run that default.
  • Clarify that Nemotron 3 Nano Omni is a supported configurable /v1/answer VLM backend, not caption-only, and that the caption Helm key does not enable /v1/answer.
  • Add Helm configuration for Omni reuse via serviceConfig.llm.apiBase and mention answer generation on the deployment-options optional-NIM list.

Fixes NVBug 6628661.

Test plan

  • Confirm Super-49B defaults against nemo_retriever/helm/values.yaml (2 GPU, NIM_TENSOR_PARALLEL_SIZE=2, 250Gi).
  • Confirm caption Omni does not auto-wire /v1/answer (configmap.yaml caption vs llm slots).
  • Review Support Matrix Model Hardware Requirements rows for Super-49B and Omni caption-or-answer labels.
  • Review Helm README answer-generation Omni reuse example.

pre-draft: leakage, mkdocs --strict, ::a, ::p, ::r on the diff vs main

Base: upstream/main
Files: docs/docs/extraction/prerequisites-support-matrix.md, docs/docs/extraction/deployment-options.md, nemo_retriever/helm/README.md

Check Result
Leakage (page roles + see [ CTAs) PASS — working-tree rg on the 3 files: no see [ CTAs; no nimOperator / nvcr.io/nim / installFfmpeg on faq.md, overview.md, or multimodal-extraction.md. The leakage script also flagged untracked leftover custom-metadata.md (see [ at lines 59, 60, 179), which is not in this diff.
Allowed paths PASS — 3 documentation files
mkdocs --strict PASS for this change — exit 1 from untracked leftover pages (custom-metadata.md, user-defined-stages.md) that are not in this diff. No strict warnings on the 3 changed files.
::a audit PASS — 6 claims. Super-49B 2-GPU TP2 / 250Gi verified (values.yaml 1318–1373). Caption Omni does not enable /v1/answer (configmap.yaml 88–113). Explicit serviceConfig.llm.apiBase enables answering without answer_llm (test_helm_answer_llm_generation.py 190). Omni-as-answer is the generic OpenAI-compatible slot (70%): chart-tested override is Nano, not Omni. Default BF16 TP2 SKU exclusions are conservative vs NIM catalog (75%). Packaged validate_code_blocks / verify_docs / detect_drift scripts were not present in this environment.
::p polish Applied — "requirements are additive"; spelled out vision-language model (VLM) on the new answer-generation path; pointed Super-49B alternate profiles at the LLM NIM support matrix instead of a vague catalog sentence.
::r style 90% — no blocking issues in the diff. Remaining: some sentences still exceed 30 words; existing page em dashes were left in place.

Code drift (not in this docs PR): nemo_retriever/helm/values.yaml:1318 still says swap in "Nemotron 3 Nano." nemo_retriever/tests/test_helm_answer_llm_generation.py:143 tests Nano (nvidia/nemotron-3-nano-30b-a3b), not Omni.

PR: opened as draft

@kheiss-uwzoo
kheiss-uwzoo marked this pull request as ready for review August 18, 2026 17:26
@kheiss-uwzoo
kheiss-uwzoo requested review from a team as code owners August 18, 2026 17:26
@kheiss-uwzoo
kheiss-uwzoo requested a review from ChrisJar August 18, 2026 17:26
@greptile-apps

greptile-apps Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR expands extraction deployment documentation for optional answer generation with Super-49B and Omni.

  • Documents the default Super-49B Helm footprint, tensor-parallel profile, cache size, and hardware constraints.
  • Explains the distinction between Omni caption wiring and the /v1/answer LLM slot.
  • Adds an example for reusing a deployed Omni caption NIM as the answer backend.

Confidence Score: 4/5

The PR appears safe to merge after the non-blocking duplicate endpoint entries are consolidated to preserve one authoritative description per model.

The documented Helm values and Omni reuse path align with the checked-in configuration, while the support matrix unnecessarily duplicates Omni and Super-49B rows with differing guidance.

Files Needing Attention: docs/docs/extraction/prerequisites-support-matrix.md

Important Files Changed

Filename Overview
docs/docs/extraction/deployment-options.md Adds answer generation to the optional-NIM overview and distinguishes caption Omni from /v1/answer wiring.
docs/docs/extraction/prerequisites-support-matrix.md Adds answer-generation configuration and hardware guidance, but duplicates two existing hosted-endpoint rows with differing descriptions.
nemo_retriever/helm/README.md Documents Super-49B resource defaults and provides Omni override and endpoint-reuse instructions consistent with the chart’s explicit LLM configuration path.
Prompt To Fix All With AI
### Issue 1
docs/docs/extraction/prerequisites-support-matrix.md:174-175
**Consolidate duplicate endpoint rows**

Omni and Super-49B now each appear in two consecutive rows with identical endpoints and model IDs but differing capability descriptions and guidance links. Update the existing rows instead so users have one authoritative configuration entry for each model.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "Merge branch 'main' into docs/6628661-an..." | Re-trigger Greptile

Comment on lines +174 to +175
| nemotron-3-nano-omni-30b-a3b-reasoning | `https://integrate.api.nvidia.com/v1/chat/completions` with model ID `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning` | Optional image captioning. Also a supported configurable `/v1/answer` VLM backend when you point `serviceConfig.llm` at this endpoint. Enabling the Omni caption Helm key does not enable `/v1/answer`. |
| llama-3.3-nemotron-super-49b-v1.5 | `https://integrate.api.nvidia.com/v1/chat/completions` with model ID `nvidia/llama-3.3-nemotron-super-49b-v1.5` | Default optional `/v1/answer` LLM (Helm `answer_llm`) and OpenAI-compatible agentic RAG endpoint mode. Not part of the default extraction pipeline. Agentic query and harness runs default to local in-process vLLM instead. Helm auto-wires to the in-cluster NIM when `nimOperator.answer_llm` is enabled. Refer to [Answer generation](#answer-generation). |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Consolidate duplicate endpoint rows

Omni and Super-49B now each appear in two consecutive rows with identical endpoints and model IDs but differing capability descriptions and guidance links. Update the existing rows instead so users have one authoritative configuration entry for each model.

Rule Used: When this PR changes user-facing code, configurati... (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: docs/docs/extraction/prerequisites-support-matrix.md
Line: 174-175

Comment:
**Consolidate duplicate endpoint rows**

Omni and Super-49B now each appear in two consecutive rows with identical endpoints and model IDs but differing capability descriptions and guidance links. Update the existing rows instead so users have one authoritative configuration entry for each model.

**Rule Used:** When this PR changes user-facing code, configurati... ([source](.greptile))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant