Skip to content

docs: add OrcaRouter as an OpenAI-compatible gateway example - #2509

Open
XiaoHuo888-hue wants to merge 1 commit into
NVIDIA:mainfrom
XiaoHuo888-hue:docs-orcarouter-agentic-and-generation
Open

docs: add OrcaRouter as an OpenAI-compatible gateway example#2509
XiaoHuo888-hue wants to merge 1 commit into
NVIDIA:mainfrom
XiaoHuo888-hue:docs-orcarouter-agentic-and-generation

Conversation

@XiaoHuo888-hue

Copy link
Copy Markdown

Description

Adds OrcaRouter as a named OpenAI-compatible gateway example in the documentation. With one API key, users get 150+ models from OpenAI, Anthropic, Google, DeepSeek, Qwen, MiniMax and xAI behind a single endpoint. Because NeMo Retriever's OpenAI-compatible client paths are a thin base-URL swap, any NeMo Retriever pipeline or agent that uses them also inherits OrcaRouter's gateway-level, zero-trust security controls for AI agents — with no application code changes. The gateway screens every prompt and response and governs every tool call on a default-deny basis, across four layers:

  • Scoped keys — bind a key to specific models, IPs, spend caps, and expiry.
  • Guardrails — screen for PII, secret leakage, prompt injection, and unsafe output.
  • Agent firewall — tool allow-lists with per-argument validation.
  • Audit trail — a record of every match, verdict, and approval decision.

What changed

Documentation only — three pages under docs/docs/extraction/:

  • workflow-agentic-retrieval.md — added a named agentic-service configuration example routing llm_model / invoke_url through OrcaRouter, plus the ORCAROUTER_API_KEY environment note.
  • nemo-retriever-api-reference.md — added a TextGenerationParams.from_kwargs(...) example using openai/orcarouter/auto with api_base="https://api.orcarouter.ai/v1".
  • api-keys.md — noted os.environ/ORCAROUTER_API_KEY as the credential reference for OpenAI-compatible gateways.

No code, configuration, or dependency changes. The examples follow the existing OpenAI-compatible wiring already documented: LiteLLM openai/<model> + api_base for the Python library, and invoke_url + llm_model for agentic service mode.

Verification

Both documented patterns were exercised live against https://api.orcarouter.ai/v1 before writing the examples:

  • LiteLLM openai/orcarouter/auto (and the namespaced openai/anthropic/claude-sonnet-4.6, openai/deepseek/deepseek-v4-pro) returned successful completions.
  • A direct OpenAI-compatible client pointed at the same endpoint (the agentic service pattern) returned successful completions.

I'm an engineer on the OrcaRouter team.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

Signed-off-by: XiaoHuo888 <sjh00112233@outlook.com>
@XiaoHuo888-hue
XiaoHuo888-hue requested review from a team as code owners August 17, 2026 13:29
@copy-pr-bot

copy-pr-bot Bot commented Aug 17, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@greptile-apps

greptile-apps Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds OrcaRouter as an OpenAI-compatible gateway example across credential, Python-library, and agentic-service documentation.

  • Documents ORCAROUTER_API_KEY as an environment-backed credential reference.
  • Adds a LiteLLM-compatible SummarizationOperator example using OrcaRouter.
  • Adds agentic-service model and endpoint configuration, although its stated credential variable is not consumed by that service path.

Confidence Score: 4/5

The PR should not merge until the agentic-service credential instructions are aligned with a credential path the service actually supports.

The Python-library example is consistent with current APIs, but the service example directs users to set ORCAROUTER_API_KEY while the service sends the embedding/NVIDIA credential—or no credential—to the configured gateway.

Files Needing Attention: docs/docs/extraction/workflow-agentic-retrieval.md

Important Files Changed

Filename Overview
docs/docs/extraction/api-keys.md Adds an environment-reference example that is compatible with the library credential resolver.
docs/docs/extraction/nemo-retriever-api-reference.md Adds a Python OrcaRouter example consistent with the current generation parameter and operator APIs.
docs/docs/extraction/workflow-agentic-retrieval.md Adds valid model and endpoint settings but documents an API-key environment variable that the agentic service does not resolve.
Prompt To Fix All With AI
### Issue 1
docs/docs/extraction/workflow-agentic-retrieval.md:71-73
**Agentic gateway key is ignored**

When users follow this example and export `ORCAROUTER_API_KEY`, the agentic service does not read that variable; it reuses the embedding credential resolved from `NVIDIA_API_KEY` or `NGC_API_KEY`. OrcaRouter consequently receives an empty or NVIDIA bearer token and rejects the chat-completion request.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "docs: add OrcaRouter as an OpenAI-compat..." | Re-trigger Greptile

Comment on lines +71 to +73
`ORCAROUTER_API_KEY` environment variable, and its keys start with `sk-orca-`.

```yaml

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Agentic gateway key is ignored

When users follow this example and export ORCAROUTER_API_KEY, the agentic service does not read that variable; it reuses the embedding credential resolved from NVIDIA_API_KEY or NGC_API_KEY. OrcaRouter consequently receives an empty or NVIDIA bearer token and rejects the chat-completion request.

Rule Used: When this PR changes user-facing code, configurati... (source)

Knowledge Base Used: Query Pipeline

Prompt To Fix With AI
This is a comment left during a code review.
Path: docs/docs/extraction/workflow-agentic-retrieval.md
Line: 71-73

Comment:
**Agentic gateway key is ignored**

When users follow this example and export `ORCAROUTER_API_KEY`, the agentic service does not read that variable; it reuses the embedding credential resolved from `NVIDIA_API_KEY` or `NGC_API_KEY`. OrcaRouter consequently receives an empty or NVIDIA bearer token and rejects the chat-completion request.

**Rule Used:** When this PR changes user-facing code, configurati... ([source](.greptile))

**Knowledge Base Used:** [Query Pipeline](https://app.greptile.com/nvidia-public-github/-/custom-context/knowledge-base/nvidia/nemo-retriever/-/docs/query-pipeline.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant