chore(docs): update docs and release notes

packages/gen/RELEASE_NOTES.md

@@ -1,4 +1,18 @@
 # Release Notes
+## 7.2.0
+
+### Features
+- Added Support for LLM Batch Service, see [](batch_service)
+
+### Bugfixes
+- Upgraded langchain
+- Upgraded langchain-aws
+- Upgraded langchain-google-genai
+- Upgraded langchain-openai
+- Upgraded google-genai to v2
+- Fixed OrchestrationV2 response handling
+- Removed hard-coded model lists. New versions of existing models will be automatically supported.
+
 ## 6.10.0
 
 ### Features

packages/gen/docs/gen_ai_hub/examples/batch-service.ipynb

@@ -0,0 +1,338 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+   "(batch_service)=\n",
+    "# LLM Batch Service\n",
+    "\n",
+    "The LLM Batch Service lets you process large volumes of LLM requests asynchronously. Instead of making individual synchronous inference calls, you submit a collection of requests as a single JSONL input file. The service processes them in the background and writes the results to your object store.\n",
+    "\n",
+    "> **Note:** Batch consumption only supports native LLM calls. Orchestration requests are not supported.\n",
+    ">\n",
+    "> Batch consumption is available in EU and US regions, except `prod-euonly` and sovereign cloud deployments.\n",
+    "\n",
+    "**Key capabilities:**\n",
+    "- Process hundreds or thousands of LLM requests in a single submission\n",
+    "- Reduced cost compared to synchronous inference calls\n",
+    "- Automatic retry handling for transient provider errors\n",
+    "- No rate-limit management required on the client side"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Prerequisites\n",
+    "\n",
+    "Before using the SDK, complete the setup steps described in the [SAP Help Portal — Batch Consumption](https://help.sap.com/docs/sap-ai-core/generative-ai/batch-consumption):\n",
+    "\n",
+    "1. Ensure you have a valid SAP AI Core service instance with access to the generative AI hub.\n",
+    "2. Register an object store secret for one of the supported providers (Amazon S3, Azure Blob Storage, Google Cloud Storage, Alibaba Cloud OSS, or SAP HANA Cloud, Data Lake).\n",
+    "3. Prepare your input file as a JSON Lines (`.jsonl`) file and upload it to your object store, following the input file specification and upload instructions on the Help Portal.\n",
+    "\n",
+    "Once the input file is uploaded, note down its `ai://` URI and the `ai://` URI of the output directory — you will need both below.\n",
+    "\n",
+    "The `BatchService` initialises a `GenAIHubProxyClient` automatically, which reads credentials from configuration files or environment variables. You can also pass a custom `proxy_client` instance."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from gen_ai_hub.batch_service import BatchService\n",
+    "\n",
+    "# Initialise using credentials from the environment / config file\n",
+    "batch_service = BatchService()\n",
+    "\n",
+    "# Or provide a proxy client explicitly:\n",
+    "# from gen_ai_hub.proxy import get_proxy_client\n",
+    "# proxy_client = get_proxy_client('gen-ai-hub')\n",
+    "# batch_service = BatchService(proxy_client=proxy_client)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Configure these values before running the workflow cells\n",
+    "INPUT_URI = \"ai://<object_store_secret_name>/<path>/input.jsonl\"\n",
+    "OUTPUT_URI = \"ai://<object_store_secret_name>/<output_folder>/\"\n",
+    "MODEL = \"gpt-4.1\"   # Must match the model used in the input file\n",
+    "PROVIDER = \"azure-openai\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Create a Batch Job\n",
+    "\n",
+    "Submit the batch job by calling `batch_service.create()`. Provide:\n",
+    "\n",
+    "| Parameter | Description |\n",
+    "|---|---|\n",
+    "| `type` | Batch processing type. Only `\"llm-native\"` is supported. |\n",
+    "| `input_uri` | `ai://` URI of the input `.jsonl` file. |\n",
+    "| `output_uri` | `ai://` URI of the output directory. Must end with `/`. |\n",
+    "| `provider` | LLM provider (e.g. `\"azure-openai\"`). |\n",
+    "| `model` | Model name — must match the value used in the input file. |\n",
+    "\n",
+    "The service schedules the job and returns a unique batch ID."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "create_response = batch_service.create(\n",
+    "    type=\"llm-native\",\n",
+    "    input_uri=INPUT_URI,\n",
+    "    output_uri=OUTPUT_URI,\n",
+    "    provider=PROVIDER,\n",
+    "    model=MODEL,\n",
+    ")\n",
+    "\n",
+    "BATCH_ID = create_response.id\n",
+    "print(f\"Batch ID: {BATCH_ID}\")\n",
+    "print(f\"Status:   {create_response.status}\")\n",
+    "print(f\"Message:  {create_response.message}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Check Batch Status\n",
+    "\n",
+    "Batches are processed **asynchronously**. Poll the status endpoint until the job reaches a terminal state (`COMPLETED`, `FAILED`, or `CANCELLED`)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import time\n",
+    "\n",
+    "TERMINAL_STATUSES = {\"COMPLETED\", \"FAILED\", \"CANCELLED\"}\n",
+    "POLLING_INTERVAL = 30  # seconds\n",
+    "\n",
+    "while True:\n",
+    "    status_response = batch_service.get_status(BATCH_ID)\n",
+    "    current = status_response.current_status\n",
+    "    print(f\"[{time.strftime('%H:%M:%S')}] Status: {current}  →  target: {status_response.target_status}\")\n",
+    "\n",
+    "    if current in TERMINAL_STATUSES:\n",
+    "        break\n",
+    "\n",
+    "    time.sleep(POLLING_INTERVAL)\n",
+    "\n",
+    "print(f\"\\nFinal status: {current}\")\n",
+    "if status_response.message:\n",
+    "    print(f\"Message: {status_response.message}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## List Batch Jobs and Get Batch Details\n",
+    "\n",
+    "### List all batch jobs\n",
+    "\n",
+    "Retrieve a summary of all batch jobs for the current resource group."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "list_response = batch_service.list()\n",
+    "\n",
+    "print(f\"Total batch jobs: {list_response.count}\")\n",
+    "for job in list_response.resources or []:\n",
+    "    print(f\"  {job.id}  |  {job.provider:<15}  |  {job.created_at}  |  {job.status}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Get full details of a batch job\n",
+    "\n",
+    "Retrieve the complete details of a specific batch job: input/output URIs, provider, model, and current status."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "detail = batch_service.get(BATCH_ID)\n",
+    "\n",
+    "print(f\"ID:           {detail.id}\")\n",
+    "print(f\"Type:         {detail.type}\")\n",
+    "print(f\"Provider:     {detail.provider}\")\n",
+    "print(f\"Created at:   {detail.created_at}\")\n",
+    "print(f\"Input URI:    {detail.input.uri}\")\n",
+    "print(f\"Output URI:   {detail.output.uri}\")\n",
+    "print(f\"Model:        {detail.spec.get('model') if detail.spec else None}\")\n",
+    "print(f\"Status:       {detail.status.current_status}\")\n",
+    "print(f\"Updated at:   {detail.status.updated_at}\")\n",
+    "if detail.status.message:\n",
+    "    print(f\"Message:      {detail.status.message}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Cancel a Batch Job\n",
+    "\n",
+    "You can cancel a batch job that is in a non-terminal state (e.g. `PENDING` or `IN_PROGRESS`). Cancellation is **asynchronous** — the job transitions to `CANCELLING` and then `CANCELLED` after any in-flight provider requests have been terminated.\n",
+    "\n",
+    "Use `get_status()` to track the cancellation progress."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Cancel the batch job — only meaningful if the job is still in a non-terminal state.\n",
+    "# Skip this cell if the job has already completed.\n",
+    "\n",
+    "cancel_response = batch_service.cancel(BATCH_ID)\n",
+    "print(f\"ID:      {cancel_response.id}\")\n",
+    "print(f\"Message: {cancel_response.message}\")\n",
+    "\n",
+    "# Poll until the cancellation is confirmed\n",
+    "while True:\n",
+    "    status_response = batch_service.get_status(BATCH_ID)\n",
+    "    print(f\"Status: {status_response.current_status}\")\n",
+    "    if status_response.current_status in TERMINAL_STATUSES:\n",
+    "        break\n",
+    "    time.sleep(10)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Delete a Batch Job\n",
+    "\n",
+    "Deleting a job removes its **metadata** from the service. Output files in the object store are **not** affected.\n",
+    "\n",
+    "> **Restriction:** Only batch jobs in `COMPLETED`, `FAILED`, or `CANCELLED` state can be deleted."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Confirm the job is in a deletable state before deleting\n",
+    "status_response = batch_service.get_status(BATCH_ID)\n",
+    "deletable = {\"COMPLETED\", \"FAILED\", \"CANCELLED\"}\n",
+    "\n",
+    "if status_response.current_status not in deletable:\n",
+    "    print(f\"Cannot delete: job is in state '{status_response.current_status}'. Cancel it first.\")\n",
+    "else:\n",
+    "    delete_response = batch_service.delete(BATCH_ID)\n",
+    "    print(f\"Deleted batch job: {delete_response.id}\")\n",
+    "    print(f\"Message: {delete_response.message}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Retrieving Results\n",
+    "\n",
+    "Once the job status is `COMPLETED`, the results are available in your object store under the output URI specified at creation, in a subdirectory named after the batch ID. Successful responses are written to `output.jsonl` and any failed individual requests to `error.jsonl`, each line matched to its input by `custom_id`."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Async Support\n",
+    "\n",
+    "All `BatchService` methods have async counterparts: `acreate`, `alist`, `aget`, `aget_status`, `acancel`, `adelete`. The example below runs the full workflow asynchronously."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import asyncio\n",
+    "\n",
+    "async def run_batch_async():\n",
+    "    service = BatchService()\n",
+    "\n",
+    "    # Create\n",
+    "    resp = await service.acreate(\n",
+    "        type=\"llm-native\",\n",
+    "        input_uri=INPUT_URI,\n",
+    "        output_uri=OUTPUT_URI,\n",
+    "        provider=PROVIDER,\n",
+    "        model=MODEL,\n",
+    "    )\n",
+    "    batch_id = resp.id\n",
+    "    print(f\"Created batch: {batch_id}  ({resp.status})\")\n",
+    "\n",
+    "    # Poll until terminal\n",
+    "    while True:\n",
+    "        status = await service.aget_status(batch_id)\n",
+    "        print(f\"Status: {status.current_status}\")\n",
+    "        if status.current_status in TERMINAL_STATUSES:\n",
+    "            break\n",
+    "        await asyncio.sleep(30)\n",
+    "\n",
+    "    # Get full details\n",
+    "    detail = await service.aget(batch_id)\n",
+    "    print(f\"Completed. Output at: {detail.output.uri}{batch_id}/output.jsonl\")\n",
+    "\n",
+    "    await service.aclose_http_connection()\n",
+    "\n",
+    "await run_batch_async()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "env",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

packages/gen/docs/gen_ai_hub/examples/document-grounding.ipynb

@@ -41,7 +41,7 @@
    "source": [
     "(document_storage)=\n",
     "### Create a Vector knowledge base\n",
-    "In this example, we will use an S3 data storage, which has been created by the user. The user hase uploaded a set of documents to the S3 bucket.\n",
+    "In this example, we will use an S3 data storage, which has been created by the user. The user has uploaded a set of documents to the S3 bucket.\n",
     "\n",
     "Check if\n",
     " - [Document Grounding is enabled](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-resource-group-for-ai-data-management?q=document%20grounding) and\n",