From e4ff46b657c6026021968014824d3fcada102501 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alperen=20K=C3=B6m=C3=BCrc=C3=BC?= Date: Fri, 17 Jul 2026 12:49:21 +0200 Subject: [PATCH] chore(docs): update docs and release notes --- packages/gen/RELEASE_NOTES.md | 14 + .../gen_ai_hub/examples/batch-service.ipynb | 338 +++++++++++++++ .../examples/document-grounding.ipynb | 2 +- .../examples/document-grounding2.ipynb | 394 ++++++++++++++++++ .../docs/gen_ai_hub/examples/gen_ai_hub.ipynb | 60 --- 5 files changed, 747 insertions(+), 61 deletions(-) create mode 100644 packages/gen/docs/gen_ai_hub/examples/batch-service.ipynb create mode 100644 packages/gen/docs/gen_ai_hub/examples/document-grounding2.ipynb diff --git a/packages/gen/RELEASE_NOTES.md b/packages/gen/RELEASE_NOTES.md index 7da08e0..18bcbdb 100644 --- a/packages/gen/RELEASE_NOTES.md +++ b/packages/gen/RELEASE_NOTES.md @@ -1,4 +1,18 @@ # Release Notes +## 7.2.0 + +### Features +- Added Support for LLM Batch Service, see [](batch_service) + +### Bugfixes +- Upgraded langchain +- Upgraded langchain-aws +- Upgraded langchain-google-genai +- Upgraded langchain-openai +- Upgraded google-genai to v2 +- Fixed OrchestrationV2 response handling +- Removed hard-coded model lists. New versions of existing models will be automatically supported. + ## 6.10.0 ### Features diff --git a/packages/gen/docs/gen_ai_hub/examples/batch-service.ipynb b/packages/gen/docs/gen_ai_hub/examples/batch-service.ipynb new file mode 100644 index 0000000..ee7d21a --- /dev/null +++ b/packages/gen/docs/gen_ai_hub/examples/batch-service.ipynb @@ -0,0 +1,338 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "(batch_service)=\n", + "# LLM Batch Service\n", + "\n", + "The LLM Batch Service lets you process large volumes of LLM requests asynchronously. Instead of making individual synchronous inference calls, you submit a collection of requests as a single JSONL input file. The service processes them in the background and writes the results to your object store.\n", + "\n", + "> **Note:** Batch consumption only supports native LLM calls. Orchestration requests are not supported.\n", + ">\n", + "> Batch consumption is available in EU and US regions, except `prod-euonly` and sovereign cloud deployments.\n", + "\n", + "**Key capabilities:**\n", + "- Process hundreds or thousands of LLM requests in a single submission\n", + "- Reduced cost compared to synchronous inference calls\n", + "- Automatic retry handling for transient provider errors\n", + "- No rate-limit management required on the client side" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Prerequisites\n", + "\n", + "Before using the SDK, complete the setup steps described in the [SAP Help Portal — Batch Consumption](https://help.sap.com/docs/sap-ai-core/generative-ai/batch-consumption):\n", + "\n", + "1. Ensure you have a valid SAP AI Core service instance with access to the generative AI hub.\n", + "2. Register an object store secret for one of the supported providers (Amazon S3, Azure Blob Storage, Google Cloud Storage, Alibaba Cloud OSS, or SAP HANA Cloud, Data Lake).\n", + "3. Prepare your input file as a JSON Lines (`.jsonl`) file and upload it to your object store, following the input file specification and upload instructions on the Help Portal.\n", + "\n", + "Once the input file is uploaded, note down its `ai://` URI and the `ai://` URI of the output directory — you will need both below.\n", + "\n", + "The `BatchService` initialises a `GenAIHubProxyClient` automatically, which reads credentials from configuration files or environment variables. You can also pass a custom `proxy_client` instance." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from gen_ai_hub.batch_service import BatchService\n", + "\n", + "# Initialise using credentials from the environment / config file\n", + "batch_service = BatchService()\n", + "\n", + "# Or provide a proxy client explicitly:\n", + "# from gen_ai_hub.proxy import get_proxy_client\n", + "# proxy_client = get_proxy_client('gen-ai-hub')\n", + "# batch_service = BatchService(proxy_client=proxy_client)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Configure these values before running the workflow cells\n", + "INPUT_URI = \"ai:////input.jsonl\"\n", + "OUTPUT_URI = \"ai:////\"\n", + "MODEL = \"gpt-4.1\" # Must match the model used in the input file\n", + "PROVIDER = \"azure-openai\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Create a Batch Job\n", + "\n", + "Submit the batch job by calling `batch_service.create()`. Provide:\n", + "\n", + "| Parameter | Description |\n", + "|---|---|\n", + "| `type` | Batch processing type. Only `\"llm-native\"` is supported. |\n", + "| `input_uri` | `ai://` URI of the input `.jsonl` file. |\n", + "| `output_uri` | `ai://` URI of the output directory. Must end with `/`. |\n", + "| `provider` | LLM provider (e.g. `\"azure-openai\"`). |\n", + "| `model` | Model name — must match the value used in the input file. |\n", + "\n", + "The service schedules the job and returns a unique batch ID." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "create_response = batch_service.create(\n", + " type=\"llm-native\",\n", + " input_uri=INPUT_URI,\n", + " output_uri=OUTPUT_URI,\n", + " provider=PROVIDER,\n", + " model=MODEL,\n", + ")\n", + "\n", + "BATCH_ID = create_response.id\n", + "print(f\"Batch ID: {BATCH_ID}\")\n", + "print(f\"Status: {create_response.status}\")\n", + "print(f\"Message: {create_response.message}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Check Batch Status\n", + "\n", + "Batches are processed **asynchronously**. Poll the status endpoint until the job reaches a terminal state (`COMPLETED`, `FAILED`, or `CANCELLED`)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import time\n", + "\n", + "TERMINAL_STATUSES = {\"COMPLETED\", \"FAILED\", \"CANCELLED\"}\n", + "POLLING_INTERVAL = 30 # seconds\n", + "\n", + "while True:\n", + " status_response = batch_service.get_status(BATCH_ID)\n", + " current = status_response.current_status\n", + " print(f\"[{time.strftime('%H:%M:%S')}] Status: {current} → target: {status_response.target_status}\")\n", + "\n", + " if current in TERMINAL_STATUSES:\n", + " break\n", + "\n", + " time.sleep(POLLING_INTERVAL)\n", + "\n", + "print(f\"\\nFinal status: {current}\")\n", + "if status_response.message:\n", + " print(f\"Message: {status_response.message}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## List Batch Jobs and Get Batch Details\n", + "\n", + "### List all batch jobs\n", + "\n", + "Retrieve a summary of all batch jobs for the current resource group." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "list_response = batch_service.list()\n", + "\n", + "print(f\"Total batch jobs: {list_response.count}\")\n", + "for job in list_response.resources or []:\n", + " print(f\" {job.id} | {job.provider:<15} | {job.created_at} | {job.status}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Get full details of a batch job\n", + "\n", + "Retrieve the complete details of a specific batch job: input/output URIs, provider, model, and current status." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "detail = batch_service.get(BATCH_ID)\n", + "\n", + "print(f\"ID: {detail.id}\")\n", + "print(f\"Type: {detail.type}\")\n", + "print(f\"Provider: {detail.provider}\")\n", + "print(f\"Created at: {detail.created_at}\")\n", + "print(f\"Input URI: {detail.input.uri}\")\n", + "print(f\"Output URI: {detail.output.uri}\")\n", + "print(f\"Model: {detail.spec.get('model') if detail.spec else None}\")\n", + "print(f\"Status: {detail.status.current_status}\")\n", + "print(f\"Updated at: {detail.status.updated_at}\")\n", + "if detail.status.message:\n", + " print(f\"Message: {detail.status.message}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Cancel a Batch Job\n", + "\n", + "You can cancel a batch job that is in a non-terminal state (e.g. `PENDING` or `IN_PROGRESS`). Cancellation is **asynchronous** — the job transitions to `CANCELLING` and then `CANCELLED` after any in-flight provider requests have been terminated.\n", + "\n", + "Use `get_status()` to track the cancellation progress." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Cancel the batch job — only meaningful if the job is still in a non-terminal state.\n", + "# Skip this cell if the job has already completed.\n", + "\n", + "cancel_response = batch_service.cancel(BATCH_ID)\n", + "print(f\"ID: {cancel_response.id}\")\n", + "print(f\"Message: {cancel_response.message}\")\n", + "\n", + "# Poll until the cancellation is confirmed\n", + "while True:\n", + " status_response = batch_service.get_status(BATCH_ID)\n", + " print(f\"Status: {status_response.current_status}\")\n", + " if status_response.current_status in TERMINAL_STATUSES:\n", + " break\n", + " time.sleep(10)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Delete a Batch Job\n", + "\n", + "Deleting a job removes its **metadata** from the service. Output files in the object store are **not** affected.\n", + "\n", + "> **Restriction:** Only batch jobs in `COMPLETED`, `FAILED`, or `CANCELLED` state can be deleted." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Confirm the job is in a deletable state before deleting\n", + "status_response = batch_service.get_status(BATCH_ID)\n", + "deletable = {\"COMPLETED\", \"FAILED\", \"CANCELLED\"}\n", + "\n", + "if status_response.current_status not in deletable:\n", + " print(f\"Cannot delete: job is in state '{status_response.current_status}'. Cancel it first.\")\n", + "else:\n", + " delete_response = batch_service.delete(BATCH_ID)\n", + " print(f\"Deleted batch job: {delete_response.id}\")\n", + " print(f\"Message: {delete_response.message}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Retrieving Results\n", + "\n", + "Once the job status is `COMPLETED`, the results are available in your object store under the output URI specified at creation, in a subdirectory named after the batch ID. Successful responses are written to `output.jsonl` and any failed individual requests to `error.jsonl`, each line matched to its input by `custom_id`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Async Support\n", + "\n", + "All `BatchService` methods have async counterparts: `acreate`, `alist`, `aget`, `aget_status`, `acancel`, `adelete`. The example below runs the full workflow asynchronously." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import asyncio\n", + "\n", + "async def run_batch_async():\n", + " service = BatchService()\n", + "\n", + " # Create\n", + " resp = await service.acreate(\n", + " type=\"llm-native\",\n", + " input_uri=INPUT_URI,\n", + " output_uri=OUTPUT_URI,\n", + " provider=PROVIDER,\n", + " model=MODEL,\n", + " )\n", + " batch_id = resp.id\n", + " print(f\"Created batch: {batch_id} ({resp.status})\")\n", + "\n", + " # Poll until terminal\n", + " while True:\n", + " status = await service.aget_status(batch_id)\n", + " print(f\"Status: {status.current_status}\")\n", + " if status.current_status in TERMINAL_STATUSES:\n", + " break\n", + " await asyncio.sleep(30)\n", + "\n", + " # Get full details\n", + " detail = await service.aget(batch_id)\n", + " print(f\"Completed. Output at: {detail.output.uri}{batch_id}/output.jsonl\")\n", + "\n", + " await service.aclose_http_connection()\n", + "\n", + "await run_batch_async()" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "env", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "pygments_lexer": "ipython3", + "version": "3.10.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/packages/gen/docs/gen_ai_hub/examples/document-grounding.ipynb b/packages/gen/docs/gen_ai_hub/examples/document-grounding.ipynb index ec0ab33..6e116db 100644 --- a/packages/gen/docs/gen_ai_hub/examples/document-grounding.ipynb +++ b/packages/gen/docs/gen_ai_hub/examples/document-grounding.ipynb @@ -41,7 +41,7 @@ "source": [ "(document_storage)=\n", "### Create a Vector knowledge base\n", - "In this example, we will use an S3 data storage, which has been created by the user. The user hase uploaded a set of documents to the S3 bucket.\n", + "In this example, we will use an S3 data storage, which has been created by the user. The user has uploaded a set of documents to the S3 bucket.\n", "\n", "Check if\n", " - [Document Grounding is enabled](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-resource-group-for-ai-data-management?q=document%20grounding) and\n", diff --git a/packages/gen/docs/gen_ai_hub/examples/document-grounding2.ipynb b/packages/gen/docs/gen_ai_hub/examples/document-grounding2.ipynb new file mode 100644 index 0000000..de28327 --- /dev/null +++ b/packages/gen/docs/gen_ai_hub/examples/document-grounding2.ipynb @@ -0,0 +1,394 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "01f95a5e", + "metadata": {}, + "source": [ + "# Document Grounding SDK – Vector, Retrieval и Pipelines\n", + "\n", + "This notebook demonstrates the changes introduced to the SDK:\n", + "\n", + "- **Vector API**: management of collections/documents and semantic search.\n", + "- **Retrieval API**: search across configured repositories (vector / help.sap.com, etc.).\n", + "- **Pipelines API**: added missing methods (search, executions, documents, trigger, etc.).\n", + "\n", + "> The examples below are written so they can be executed in environment with access configured to SAP GenAI Hub / AI Core.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "05dd95f1", + "metadata": {}, + "outputs": [], + "source": [ + "from __future__ import annotations\n", + "\n", + "import json\n", + "from typing import Any\n", + "\n", + "def as_json(obj: Any) -> str:\n", + " \"\"\"Helper pretty-print for Pydantic models\"\"\"\n", + " if hasattr(obj, 'model_dump'):\n", + " data = obj.model_dump(by_alias=True, exclude_none=True)\n", + " else:\n", + " data = obj\n", + " return json.dumps(data, ensure_ascii=False, indent=2)\n" + ] + }, + { + "cell_type": "markdown", + "id": "bc2278c1", + "metadata": {}, + "source": [ + "## Client Initialization\n", + "\n", + "Separate clients were added under `gen_ai_hub.document_grounding.clients.*`, while backward compatibility is preserved via re-exports in `gen_ai_hub.document_grounding.client`.\n", + "\n", + "### 1) List and search pipelines" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "67699372", + "metadata": {}, + "outputs": [], + "source": [ + "from gen_ai_hub.proxy import get_proxy_client\n", + "\n", + "# Backwards compatible imports\n", + "from gen_ai_hub.document_grounding.client import PipelineAPIClient, VectorAPIClient, RetrievalAPIClient\n", + "\n", + "proxy_client = get_proxy_client(proxy_version='gen-ai-hub')\n", + "\n", + "pipelines = PipelineAPIClient(proxy_client)\n", + "vector = VectorAPIClient(proxy_client)\n", + "retrieval = RetrievalAPIClient(proxy_client)\n", + "\n", + "print('Clients initialized')\n" + ] + }, + { + "cell_type": "markdown", + "id": "995eb8d3", + "metadata": {}, + "source": [ + "## Pipelines API\n", + "\n", + "Below are examples of the main operations. There were introduced the following methods:\n", + "\n", + "- `search_pipelines(...)`\n", + "- `get_pipeline_executions(...)`, `get_pipeline_execution_by_id(...)`\n", + "- `get_execution_documents(...)`, `get_execution_document_by_id(...)`\n", + "- `get_pipeline_documents(...)`, `get_pipeline_document_by_id(...)`\n", + "- `trigger_pipeline(...)`\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "04cdff3c", + "metadata": {}, + "outputs": [], + "source": [ + "pipelines_list = pipelines.get_pipelines(top=10)\n", + "print(as_json(pipelines_list))\n", + "\n", + "from gen_ai_hub.document_grounding.models.pipeline import SearchPipelineRequest, SearchPipelineData\n", + "\n", + "search_req = SearchPipelineRequest(data=SearchPipelineData(search=''))\n", + "search_res = pipelines.search_pipelines(search_req)\n", + "print(as_json(search_res))" + ] + }, + { + "cell_type": "markdown", + "id": "aa38c7ee", + "metadata": {}, + "source": [ + "### 2) Pipeline status and manual trigger\n", + "\n", + "To start a pipeline, use `trigger_pipeline` (Manual Trigger)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "32e3d4d9", + "metadata": {}, + "outputs": [], + "source": [ + "from gen_ai_hub.document_grounding.models.pipeline import ManualPipelineTrigger\n", + "\n", + "pipeline_id = ''\n", + "\n", + "status = pipelines.get_pipeline_status(pipeline_id)\n", + "print('Status:')\n", + "print(as_json(status))\n", + "\n", + "trigger_req = ManualPipelineTrigger()\n", + "trigger_res = pipelines.trigger_pipeline(pipeline_id, trigger_req)\n", + "print('Trigger response:')\n", + "print(as_json(trigger_res))\n" + ] + }, + { + "cell_type": "markdown", + "id": "e5f57b5b", + "metadata": {}, + "source": [ + "### 3) Executions and Documents\n", + "\n", + "There were added coverage for working with pipeline executions and documents.\n", + "This is useful for diagnostics: which documents were processed, what errors occurred, etc.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5a4c4d48", + "metadata": {}, + "outputs": [], + "source": [ + "execs = pipelines.get_pipeline_executions(pipeline_id, top=20)\n", + "print(as_json(execs))\n", + "\n", + "execution_id = ''\n", + "execution = pipelines.get_pipeline_execution_by_id(pipeline_id, execution_id)\n", + "print(as_json(execution))\n", + "\n", + "\n", + "docs = pipelines.get_execution_documents(pipeline_id, execution_id, top=50)\n", + "print(as_json(docs))\n", + "\n", + "\n", + "document_id = ''\n", + "doc = pipelines.get_execution_document_by_id(pipeline_id, execution_id, document_id)\n", + "print(as_json(doc))\n", + "\n", + "\n", + "pipeline_docs = pipelines.get_pipeline_documents(pipeline_id, top=50)\n", + "print(as_json(pipeline_docs))\n", + "\n", + "\n", + "pipeline_doc = pipelines.get_pipeline_document_by_id(pipeline_id, document_id)\n", + "print(as_json(pipeline_doc))\n" + ] + }, + { + "cell_type": "markdown", + "id": "59a80408", + "metadata": {}, + "source": [ + "## Vector API\n", + "\n", + "The Vector API is designed for managing **collections** and **documents**, as well as performing search.\n", + "\n", + "### 1) List collections" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "47bc724c", + "metadata": {}, + "outputs": [], + "source": [ + "collections = vector.get_collections(top=50)\n", + "print(as_json(collections))\n" + ] + }, + { + "cell_type": "markdown", + "id": "524d3921", + "metadata": {}, + "source": [ + "### 2) Create a collection\n", + "\n", + "In `CollectionCreateRequest`, the `embeddingConfig` field is required.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "75247b6f", + "metadata": {}, + "outputs": [], + "source": [ + "from gen_ai_hub.document_grounding.models.vector import CollectionCreateRequest, EmbeddingConfig\n", + "\n", + "create_req = CollectionCreateRequest(\n", + " title='My SDK Demo Collection',\n", + " embeddingConfig=EmbeddingConfig(modelName='text-embedding-3-large'),\n", + " metadata=[],\n", + ")\n", + "\n", + "create_res = vector.create_collection(create_req)\n", + "print(as_json(create_res))\n" + ] + }, + { + "cell_type": "markdown", + "id": "f58b9e9c", + "metadata": {}, + "source": "### 3) Add / update / delete documents" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b3492414", + "metadata": {}, + "outputs": [], + "source": [ + "from gen_ai_hub.document_grounding.models.vector import (\n", + " DocumentsCreateRequest, DocumentsUpdateRequest,\n", + " TextOnlyBaseChunk, BaseDocument, VectorKeyValueListPair\n", + ")\n", + "\n", + "collection_id = ''\n", + "\n", + "doc = BaseDocument(\n", + " chunks=[TextOnlyBaseChunk(content='Hello from SDK Vector API', metadata=[])],\n", + " metadata=[VectorKeyValueListPair(key='source', value=['notebook'])],\n", + ")\n", + "\n", + "create_docs_req = DocumentsCreateRequest(documents=[doc])\n", + "created = vector.create_documents(collection_id, create_docs_req)\n", + "print('Created:')\n", + "print(as_json(created))\n", + "\n", + "documents = vector.get_documents(collection_id, top=20)\n", + "print(as_json(documents))\n", + "document_id = documents.resources[0].id\n", + "\n", + "update_req = DocumentsUpdateRequest(documents=[ ... ])\n", + "updated = vector.update_documents(collection_id, update_req)\n", + "print(as_json(updated))\n", + "\n", + "vector.delete_document(collection_id, document_id)\n" + ] + }, + { + "cell_type": "markdown", + "id": "98a782bf", + "metadata": {}, + "source": [ + "### 4) Search across collections (Text Search)\n", + "\n", + "`TextSearchRequest` is used with `filters`, where `collectionIds` and limits for chunks/documents are specified." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "00e0e35e", + "metadata": {}, + "outputs": [], + "source": [ + "from gen_ai_hub.document_grounding.models.vector import (\n", + " TextSearchRequest, VectorSearchFilter, VectorSearchConfiguration\n", + ")\n", + "\n", + "search_req = TextSearchRequest(\n", + " query='Hello',\n", + " filters=[\n", + " VectorSearchFilter(\n", + " id='f1',\n", + " collectionIds=[collection_id],\n", + " configuration=VectorSearchConfiguration(maxChunkCount=5, maxDocumentCount=3),\n", + " documentMetadata=[],\n", + " chunkMetadata=[],\n", + " collectionMetadata=[],\n", + " )\n", + " ],\n", + ")\n", + "\n", + "search_res = vector.search(search_req)\n", + "print(as_json(search_res))\n" + ] + }, + { + "cell_type": "markdown", + "id": "09fead35", + "metadata": {}, + "source": [ + "## Retrieval API\n", + "\n", + "The Retrieval API provides a unified search interface across configured data repositories (e.g., `vector` collections or `help.sap.com`).\n", + "\n", + "### 1) List repositories" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "97363a83", + "metadata": {}, + "outputs": [], + "source": [ + "repos = retrieval.get_data_repositories(top=50)\n", + "print(as_json(repos))\n", + "\n", + "repo_id = ''\n", + "repo = retrieval.get_data_repository_by_id(repo_id)\n", + "print(as_json(repo))\n" + ] + }, + { + "cell_type": "markdown", + "id": "03ba6f4b", + "metadata": {}, + "source": [ + "### 2) Retrieval search\n", + "\n", + "In `RetrievalSearchFilter`, the `dataRepositoryType` is specified (e.g., `'vector'` or `'help.sap.com'`), along with optional constraints/metadata.\n", + "\n", + "The example below shows a typical request structure." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d99636c6", + "metadata": {}, + "outputs": [], + "source": [ + "from gen_ai_hub.document_grounding.models.retrieval import (\n", + " RetrievalSearchInput, RetrievalSearchFilter, RetrievalSearchConfiguration\n", + ")\n", + "\n", + "retrieval_req = RetrievalSearchInput(\n", + " query='How to configure Document Grounding?',\n", + " filters=[\n", + " RetrievalSearchFilter(\n", + " id='r1',\n", + " dataRepositoryType='help.sap.com',\n", + " searchConfiguration=RetrievalSearchConfiguration(maxChunkCount=5, maxDocumentCount=3),\n", + " dataRepositories=[],\n", + " )\n", + " ],\n", + ")\n", + "\n", + "retrieval_res = retrieval.search(retrieval_req)\n", + "print(as_json(retrieval_res))\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/packages/gen/docs/gen_ai_hub/examples/gen_ai_hub.ipynb b/packages/gen/docs/gen_ai_hub/examples/gen_ai_hub.ipynb index 458f7bb..2a6ecb3 100644 --- a/packages/gen/docs/gen_ai_hub/examples/gen_ai_hub.ipynb +++ b/packages/gen/docs/gen_ai_hub/examples/gen_ai_hub.ipynb @@ -898,66 +898,6 @@ "id": "7ec3315cb2c29efa", "outputs": [], "execution_count": null - }, - { - "metadata": {}, - "cell_type": "markdown", - "source": [ - "(unsupported_models)=\n", - "# Using New Models Before Official SDK Support\n", - "\n", - "You can use models via Gen AI Hub even before they are officially listed, provided their provider family (e.g., `Google`, `Amazon Bedrock`) is supported.\n", - "\n", - "1. **Native SDK Clients:**\n", - "\n", - " If using the provider's native SDK (like `boto3`, `google-genai`) through the Gen AI Hub proxy, you can often use the new model name/ID directly with existing client methods.\n", - "\n", - "2. **Langchain Integration (`init_llm`):**\n", - "\n", - " The `init_llm` helper simplifies creating Langchain LLM objects configured for the proxy.\n", - "\n", - " * **Alternative:** You can always bypass `init_llm` and instantiate the Langchain classes (e.g., `ChatGoogleGenerativeAI`, `ChatBedrock`, `ChatBedrockConverse`) directly.\n", - " * **Bedrock Specifics**:\n", - " * Requires `model_id` in addition to `model_name`. Find IDs [here](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html). `init_llm` automatically selects the appropriate Bedrock API (older Invoke via `ChatBedrock` or newer Converse via `ChatBedrockConverse`) based on known models.\n", - " * **Crucially:** For *new* Bedrock models or to force a specific API (Invoke/Converse), you must pass the corresponding initialization function (`init_chat_model` or `init_chat_converse_model`) to the `init_func` argument of `init_llm`.\n" - ], - "id": "5e85a5c1bfbfe321" - }, - { - "cell_type": "code", - "id": "f6676f9c4f07cb2d", - "metadata": {}, - "source": [ - "from gen_ai_hub.proxy.langchain import init_llm\n", - "# Import specific init functions for overriding Bedrock behavior\n", - "from gen_ai_hub.proxy.langchain.amazon import (\n", - " init_chat_model as amazon_init_invoke_model,\n", - " init_chat_converse_model as amazon_init_converse_model\n", - ")\n", - "from gen_ai_hub.proxy.langchain.google_genai import init_chat_model as google_genai_init_chat_model\n", - "\n", - "# --- Google Example ---\n", - "llm_google = init_llm(model_name='gemini-newer-version', init_func=google_genai_init_chat_model) # Often just needs model_name\n", - "\n", - "# --- Bedrock Example (New Model requiring Converse API) ---\n", - "model_name_amazon = 'anthropic--claude-newer-version'\n", - "model_id_amazon = 'anthropic.claude-newer-version-v1:0' # Use actual ID\n", - "\n", - "llm_amazon = init_llm(\n", - " model_name_amazon,\n", - " model_id=model_id_amazon,\n", - " init_func=amazon_init_converse_model # Explicitly select Converse API\n", - ")\n", - "\n", - "# --- Bedrock Example (Explicitly using older Invoke API) ---\n", - "# llm_amazon_invoke = init_llm(\n", - "# 'some-model-name',\n", - "# model_id='some-model-id',\n", - "# init_func=amazon_init_invoke_model # Explicitly select Invoke API\n", - "# )\n" - ], - "outputs": [], - "execution_count": null } ], "metadata": {