docs(extraction): complete Support Matrix coverage for answer-generation LLM and VLM paths (NVBug 6628661) - #2546
Conversation
…the Support Matrix (NVBug 6628661)
Greptile SummaryThe PR expands extraction deployment documentation for optional answer generation with Super-49B and Omni.
|
| Filename | Overview |
|---|---|
| docs/docs/extraction/deployment-options.md | Adds answer generation to the optional-NIM overview and distinguishes caption Omni from /v1/answer wiring. |
| docs/docs/extraction/prerequisites-support-matrix.md | Adds answer-generation configuration and hardware guidance, but duplicates two existing hosted-endpoint rows with differing descriptions. |
| nemo_retriever/helm/README.md | Documents Super-49B resource defaults and provides Omni override and endpoint-reuse instructions consistent with the chart’s explicit LLM configuration path. |
Prompt To Fix All With AI
### Issue 1
docs/docs/extraction/prerequisites-support-matrix.md:174-175
**Consolidate duplicate endpoint rows**
Omni and Super-49B now each appear in two consecutive rows with identical endpoints and model IDs but differing capability descriptions and guidance links. Update the existing rows instead so users have one authoritative configuration entry for each model.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Reviews (1): Last reviewed commit: "Merge branch 'main' into docs/6628661-an..." | Re-trigger Greptile
| | nemotron-3-nano-omni-30b-a3b-reasoning | `https://integrate.api.nvidia.com/v1/chat/completions` with model ID `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning` | Optional image captioning. Also a supported configurable `/v1/answer` VLM backend when you point `serviceConfig.llm` at this endpoint. Enabling the Omni caption Helm key does not enable `/v1/answer`. | | ||
| | llama-3.3-nemotron-super-49b-v1.5 | `https://integrate.api.nvidia.com/v1/chat/completions` with model ID `nvidia/llama-3.3-nemotron-super-49b-v1.5` | Default optional `/v1/answer` LLM (Helm `answer_llm`) and OpenAI-compatible agentic RAG endpoint mode. Not part of the default extraction pipeline. Agentic query and harness runs default to local in-process vLLM instead. Helm auto-wires to the in-cluster NIM when `nimOperator.answer_llm` is enabled. Refer to [Answer generation](#answer-generation). | |
There was a problem hiding this comment.
Consolidate duplicate endpoint rows
Omni and Super-49B now each appear in two consecutive rows with identical endpoints and model IDs but differing capability descriptions and guidance links. Update the existing rows instead so users have one authoritative configuration entry for each model.
Rule Used: When this PR changes user-facing code, configurati... (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: docs/docs/extraction/prerequisites-support-matrix.md
Line: 174-175
Comment:
**Consolidate duplicate endpoint rows**
Omni and Super-49B now each appear in two consecutive rows with identical endpoints and model IDs but differing capability descriptions and guidance links. Update the existing rows instead so users have one authoritative configuration entry for each model.
**Rule Used:** When this PR changes user-facing code, configurati... ([source](.greptile))
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Summary
answer_llmhardware profile in the Support Matrix: two physical GPUs, BF16 TP2, 250Gi NIMCache, additional to the core pipeline, and SKUs that cannot run that default./v1/answerVLM backend, not caption-only, and that the caption Helm key does not enable/v1/answer.serviceConfig.llm.apiBaseand mention answer generation on the deployment-options optional-NIM list.Fixes NVBug 6628661.
Test plan
nemo_retriever/helm/values.yaml(2 GPU,NIM_TENSOR_PARALLEL_SIZE=2, 250Gi)./v1/answer(configmap.yamlcaption vsllmslots).pre-draft: leakage, mkdocs --strict, ::a, ::p, ::r on the diff vs main
Base: upstream/main
Files:
docs/docs/extraction/prerequisites-support-matrix.md,docs/docs/extraction/deployment-options.md,nemo_retriever/helm/README.mdsee [CTAs)see [CTAs; nonimOperator/nvcr.io/nim/installFfmpegonfaq.md,overview.md, ormultimodal-extraction.md. The leakage script also flagged untracked leftovercustom-metadata.md(see [at lines 59, 60, 179), which is not in this diff.custom-metadata.md,user-defined-stages.md) that are not in this diff. No strict warnings on the 3 changed files.values.yaml1318–1373). Caption Omni does not enable/v1/answer(configmap.yaml88–113). ExplicitserviceConfig.llm.apiBaseenables answering withoutanswer_llm(test_helm_answer_llm_generation.py190). Omni-as-answer is the generic OpenAI-compatible slot (70%): chart-tested override is Nano, not Omni. Default BF16 TP2 SKU exclusions are conservative vs NIM catalog (75%). Packagedvalidate_code_blocks/verify_docs/detect_driftscripts were not present in this environment.Code drift (not in this docs PR):
nemo_retriever/helm/values.yaml:1318still says swap in "Nemotron 3 Nano."nemo_retriever/tests/test_helm_answer_llm_generation.py:143tests Nano (nvidia/nemotron-3-nano-30b-a3b), not Omni.PR: opened as draft