docs: Document Serverless network-volume warm cache (VolumeCache)

docs.json

@@ -128,6 +128,7 @@
                   "serverless/workers/deploy",
                   "serverless/workers/github-integration",
                   "serverless/storage/overview",
+                  "serverless/development/volume-cache",
                   "serverless/development/dual-mode-worker"
                 ]
               },

serverless/development/optimization.mdx

@@ -1,4 +1,4 @@
 ---
 title: "Optimize your endpoints"
 sidebarTitle: "Optimization guide"
 description: "Implement strategies to reduce latency and cost for your Serverless endpoints. Review configuration and operations guidance for Runpod Serverless."
@@ -12,9 +12,10 @@
 
 | Strategy | Impact | When to use |
 |----------|--------|-------------|
 | [Use cached models](/serverless/endpoints/model-caching) | ⬇️ Cold start (major) | Models on Hugging Face |
 | [Bake models into image](/serverless/workers/create-dockerfile#including-models-and-files) | ⬇️ Cold start | Private models |
+| [Cache files to a network volume](/serverless/development/volume-cache) | ⬇️ Cold start | Downloaded weights, attached volume |
 | [Set active workers > 0](/serverless/endpoints/endpoint-configurations#active-workers) | ⬇️ Cold start (eliminates) | Latency-sensitive apps |
 | [Select multiple GPU types](/serverless/endpoints/endpoint-configurations#gpu-configuration) | ⬆️ Availability | Production workloads |
 | [Increase max workers](/serverless/endpoints/endpoint-configurations#max-workers) | ⬆️ Throughput | High concurrency |
 | [Lower queue delay threshold](/serverless/endpoints/endpoint-configurations#auto-scaling-type) | ⬇️ Response time | Traffic spikes |
@@ -25,7 +26,7 @@
 
 | Metric | Description | Optimization |
 |--------|-------------|--------------|
 | **Delay time** | Waiting for a worker (includes cold start) | Model caching, active workers |
 | **Execution time** | GPU processing the request | Code optimization, GPU selection |
 
 **Delay time** breaks down into:
@@ -42,9 +43,9 @@
 
 ## Reduce cold starts
 
 ### Use cached models (recommended)
 
 For models on Hugging Face, [cached models](/serverless/endpoints/model-caching) provide the fastest cold starts and lowest cost.
 
 ### Bake models into images
 
@@ -60,24 +61,24 @@
 
 ## Improve availability
 
 ### Select multiple GPU types
 
 Specify multiple [GPU types](/references/gpu-types) in priority order. A single high-end GPU often outperforms multiple lower-tier cards for <InferenceTooltip />.
 
 For endpoints with five or more workers, Runpod [distributes workers across your GPU priorities](/serverless/endpoints/endpoint-configurations#gpu-priority-and-worker-distribution) to reduce throttling when your primary GPU type is constrained.
 
 ### Add headroom to max workers
 
 Set [max workers](/serverless/endpoints/endpoint-configurations#max-workers) ~20% above expected concurrency to handle load spikes without throttling.
 
 ### Tune auto-scaling
 
 Lower the [queue delay threshold](/serverless/endpoints/endpoint-configurations#auto-scaling-type) to 2-3 seconds (default: 4) for faster worker provisioning.
 
 ## Architecture considerations
 
 | Choice | Tradeoff |
 |--------|----------|
 | **Baked models** | Fastest loading, but larger images |
 | **Network volumes** | Flexible, but restricts to specific data centers |
 | **Multiple GPU types** | Higher availability, variable performance |

serverless/development/volume-cache.mdx

@@ -0,0 +1,80 @@
+---
+title: "Cache files to a network volume"
+sidebarTitle: "Volume cache"
+description: "Persist model weights and other files to an attached network volume to speed up Serverless worker cold starts."
+---
+
+import { ColdStartTooltip } from "/snippets/tooltips.jsx";
+
+The Runpod Python SDK includes `VolumeCache`, a helper that warms local directories across Serverless workers using an attached network volume. It keeps a browsable mirror of your cache directories on the volume: on cold start it restores previously cached files into place, and after your code runs it copies newly written files back. This turns a repeated multi-GB model download on every cold start into a one-time cost per endpoint, which reduces <ColdStartTooltip /> times.
+
+`VolumeCache` is best-effort and self-contained, and it never affects the outcome of a job. If any part of the cache fails, the worker falls back to a normal cold start. You add it explicitly by wrapping the code that populates your cache, so nothing runs until you opt in.
+
+## Requirements
+
+- A [network volume](/storage/network-volumes#network-volumes-for-serverless) attached to your endpoint, mounted at `/runpod-volume`. The mirror is stored on the volume, so every operation is a safe no-op when no volume is mounted.
+- The Runpod Python SDK installed in your worker image.
+- An endpoint namespace to scope the mirror. On Serverless this defaults to `RUNPOD_ENDPOINT_ID`, which Runpod sets automatically.
+
+## Usage
+
+`VolumeCache` is a context manager. Wrapping it around your model load hydrates the cache before the block runs and syncs any changes back afterward:
+
+```python title="handler.py"
+import runpod
+from runpod.serverless import VolumeCache
+
+def handler(job):
+    ...
+
+# Hydrate from the volume, download only what's missing, then sync back
+with VolumeCache(dirs=["/root/.cache/huggingface"]):
+    model = load_model()   # downloads land in the cached directory
+
+runpod.serverless.start({"handler": handler})
+```
+
+When you enter the block, `hydrate()` copies files that are missing or newer on the volume mirror into the container. When you exit, `sync()` copies files that are missing or newer in the container back onto the mirror. By default the sync runs on a background daemon thread and returns immediately, so the `with` block doesn't wait on it. A process-exit hook completes any outstanding syncs, so short-lived processes still finish syncing before they exit.
+
+You can also call the phases directly when they happen at different points in your worker's lifecycle:
+
+```python
+vc = VolumeCache(dirs=["/data/models"], namespace="my-model-cache")
+
+vc.hydrate()                 # restore cached files (for example, at startup)
+model = load_model()         # populate the cache
+vc.sync(background=False)     # persist new files back to the volume, inline
+```
+
+## Constructor
+
+`VolumeCache` accepts the following arguments:
+
+| Argument | Default | Description |
+|----------|---------|-------------|
+| `dirs` | Required | A list of local directories to cache. |
+| `namespace` | `RUNPOD_ENDPOINT_ID` | Isolation key for the on-volume mirror. Must be a single safe path component. |
+| `volume_path` | `/runpod-volume` | Mount point of the network volume. |
+| `best_effort` | `True` | When `True`, cache errors are logged and swallowed instead of raised. Set to `False` while debugging. |
+| `max_workers` | `min(32, (os.cpu_count() or 4) * 4)` | Thread count for the parallel copy of large files. The work is I/O-bound, so the default oversubscribes the CPU count. |
+
+## How it works
+
+`VolumeCache` adapts its transport to the size of your files, which keeps both many-small-file caches and multi-GB weight files fast.
+
+- **Size-bucketed mirror:** Cached files live at `{volume_path}/.cache/{namespace}` on the volume. Files below 256 KiB are packed into a single `small.tar` archive, which collapses the per-file metadata round-trips that make many small files slow on a network volume. Larger files are copied unpacked into a `big/` subdirectory, which preserves their original relative paths so the large-file subtree stays browsable. A versioned `manifest.json`, written last, records the size and modification time of every cached file and marks the mirror as complete. A mirror without a valid, current manifest is treated as absent, so a sync self-heals.
+- **Incremental large files, whole-archive small files:** Large-file transfers are diffed per file against the manifest, so unchanged files are skipped. The `small.tar` archive is repacked whole whenever any small file changes, since unpacking and re-diffing many tiny files individually is slower than the volume's per-file overhead.
+- **Parallel copy:** Large-file transfers run across a thread pool sized by `max_workers`. The work is I/O-bound, so the default oversubscribes the CPU count.
+- **Safety:** Symbolic links are never followed or copied. Every archive member and every large-file destination is checked to resolve inside one of your configured `dirs` before it is written, so a mirror entry can't write outside your cached directories.
+
+## Limitations
+
+- **Concurrent cold-start write amplification:** If several workers cold-start at the same time, each may miss the still-empty mirror, download the model, and sync a full copy back. There's no coordination between concurrent syncs, so the mirror reflects whichever worker synced most recently.
+- **Background sync on short-lived processes:** `sync()` schedules the copy on a daemon thread. If the process exits without a normal interpreter shutdown (for example, `os._exit` or `SIGKILL`), the exit hook never runs and the sync may not complete.
+- **Orphaned large files aren't pruned:** If you delete or rename a large file locally, its copy under `big/` stays on the volume. Hydration is manifest-driven and ignores it, but volume usage grows as you swap model versions.
+
+## Next steps
+
+- [Optimize your endpoints](/serverless/development/optimization): Combine the volume cache with other strategies to reduce cold starts.
+- [Cached models](/serverless/endpoints/model-caching): Use Runpod's platform-level model cache for Hugging Face models.
+- [Storage options](/serverless/storage/overview): Compare container disks, network volumes, and S3-compatible storage.