Add Kueue and Cluster Autoscaler on AKS example

[Li Chen (AaronLiChen)]

Summary

• Add an end-to-end example for running batch jobs on AKS with Kueue admission control and the cluster autoscaler
• Kueue gates jobs and, via a ProvisioningRequest AdmissionCheck, asks the cluster autoscaler to atomically provision capacity before the job is admitted (provision-first, avoids half-scheduled gangs)
• Three modules walk through infrastructure, queue + provisioning-gate configuration, and workload submission

What's included

Module	Content
1 — Infrastructure	AKS cluster with an autoscaling node pool + Kueue install (Helm)
2 — Kueue Queues	ResourceFlavor, ProvisioningRequestConfig, AdmissionCheck, ClusterQueue, LocalQueue
3 — Workload	Suspended Job that triggers an atomic scale-up via ProvisioningRequest

Follows the structure of the kueue-and-ray-on-aks example. A companion how-to in the AKS docs references this directory.

Test plan

• Manifests apply cleanly on a fresh cluster
• Kueue creates a Workload + ProvisioningRequest for the suspended Job
• Job admitted and runs once nodes are provisioned

[Copilot]

Pull request overview

Adds a new AKS example (examples/kueue-and-cas-on-aks) that demonstrates using Kueue admission control together with the AKS cluster autoscaler via the ProvisioningRequest flow, organized as a three-module walkthrough.

Changes:

• Added top-level documentation explaining the Kueue + CAS “provision-first” architecture and a quick start.
• Added Module 1/2/3 READMEs that walk through infrastructure provisioning, queue/gate setup, and workload submission/observation.
• Added Kubernetes manifests for the namespace, Kueue queueing resources (ResourceFlavor, AdmissionCheck, ProvisioningRequestConfig, ClusterQueue, LocalQueue), and a sample suspended Job.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
examples/kueue-and-cas-on-aks/README.md	Introduces the scenario, architecture diagram, modules, and end-to-end quick start/cleanup.
examples/kueue-and-cas-on-aks/1-infrastructure/README.md	Provides AKS + node pool provisioning steps and installs Kueue via Helm.
examples/kueue-and-cas-on-aks/2-kueue-queues/README.md	Explains and applies the queue + ProvisioningRequest gate configuration.
examples/kueue-and-cas-on-aks/2-kueue-queues/manifests/00-namespace.yaml	Creates the example namespace for queues/workload.
examples/kueue-and-cas-on-aks/2-kueue-queues/manifests/10-resource-flavor.yaml	Pins Kueue quota to the autoscaling node pool via agentpool=scalepool.
examples/kueue-and-cas-on-aks/2-kueue-queues/manifests/20-provisioning.yaml	Defines the AdmissionCheck + ProvisioningRequestConfig and connects them to the queues.
examples/kueue-and-cas-on-aks/3-workload/README.md	Walks through submitting and observing the workload/admission/scale-up flow.
examples/kueue-and-cas-on-aks/3-workload/manifests/job.yaml	Adds a suspended Job manifest intended to trigger scale-up via ProvisioningRequest.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

examples/kueue-and-ray-on-aks/3-workloads/cas-batch-job/manifests/job.yaml:45

• The Job uses the pause image, which never exits, so the Job will not reach Complete/COMPLETIONS 3/3 as described in the READMEs. Use an image/command that terminates successfully so the example can complete end-to-end.

          image: mcr.microsoft.com/oss/v2/kubernetes/pause:3.6
          resources:
            requests:
              cpu: "1800m"
              memory: "256Mi"

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (3)

examples/kueue-and-ray-on-aks/2-kueue-queues/manifests/40-autoscale-queue.yaml:84

• nominalQuota values don’t match the guidance in the comments (example says 5 × 4 = 20 cores) and exceed the max capacity implied by the docs for scalepool (Standard_D4s_v3, max-count 5). Oversizing these quotas can make Kueue accept workloads that CAS can never satisfy, leaving them stuck in provisioning.

              # Upper bound Kueue will admit before asking CAS to grow the pool.
              # Size to the pool's max-count × per-node cores (e.g. 5 × 4 = 20).
              nominalQuota: "100"
            - name: memory
              nominalQuota: 200Gi

examples/kueue-and-ray-on-aks/3-workloads/cas-batch-job/manifests/job.yaml:45

• The Job container uses the Kubernetes pause image, which doesn't terminate, so the Job will never reach Complete (it will run indefinitely after Kueue unsuspends it). Use an image + command that exits so the example can actually complete.

      containers:
        - name: worker
          image: mcr.microsoft.com/oss/v2/kubernetes/pause:3.6
          resources:
            requests:
              cpu: "1800m"
              memory: "256Mi"

examples/kueue-and-ray-on-aks/2-kueue-queues/README.md:83

• The "Choose one" warning still only mentions 20-single-queue.yaml and 30-team-queues.yaml, but this PR adds a third mutually exclusive option (40-autoscale-queue.yaml). This can lead readers to apply multiple queue configs or delete the wrong one when switching.

> **⚠️ Choose one.** `20-single-queue.yaml` and `30-team-queues.yaml` are
> independent configurations. To switch between them, delete the active one
> first:
> ```bash
> kubectl delete -f manifests/20-single-queue.yaml   # then apply 30-team-queues.yaml

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (5)

examples/kueue-and-ray-on-aks/2-kueue-queues/README.md:83

• The “Choose one” warning block still only references 20-single-queue.yaml and 30-team-queues.yaml, but this module now also adds 40-autoscale-queue.yaml as a mutually exclusive option.

> **⚠️ Choose one.** `20-single-queue.yaml` and `30-team-queues.yaml` are
> independent configurations. To switch between them, delete the active one
> first:
> ```bash
> kubectl delete -f manifests/20-single-queue.yaml   # then apply 30-team-queues.yaml

examples/kueue-and-ray-on-aks/2-kueue-queues/manifests/40-autoscale-queue.yaml:92

• nominalQuota is documented as “max-count × per-node cores (e.g. 5 × 4 = 20)” but is set to 100. If left oversized relative to the pool’s --max-count, Kueue may admit/trigger ProvisioningRequests that CAS can never satisfy.

              # Upper bound Kueue will admit before asking CAS to grow the pool.
              # Size to the pool's max-count × per-node cores (e.g. 5 × 4 = 20).
              nominalQuota: "100"

examples/kueue-and-ray-on-aks/3-workloads/README.md:121

• The comparison table entries for cas-batch-job don’t match the actual workload: the Job is labeled with kueue.x-k8s.io/queue-name: cas-local-queue (not autoscale), and it uses a BusyBox image (not a pause image).

| Queue label | `default` | `default` | `default` | — | `autoscale` |
| Deps | Aurora, torch, azure-storage-blob | LLaMA-Factory, azure-storage-blob | vLLM, azure-storage-blob | Aurora, azure-storage-blob | none (pause image) |

examples/kueue-and-ray-on-aks/3-workloads/cas-batch-job/manifests/job.yaml:49

• This Job sets resource requests but no limits. Other workloads in this example set both requests and limits, and limits help keep example behavior predictable (especially when demonstrating autoscaling/provisioning).

          resources:
            requests:
              cpu: "1800m"
              memory: "256Mi"

examples/kueue-and-ray-on-aks/2-kueue-queues/README.md:76

• Option C is described as a standalone autoscale demo, but the Apply steps above always create the ray namespace and Ray workload identity ServiceAccount, which aren’t needed for the cas-kueue-demo + plain Job flow. Adding a short note here would reduce confusion.

#   Option C — Autoscale queue (provision capacity on demand via CAS)
#   Requires an autoscaling `scalepool` pool — see the section below.
kubectl apply -f manifests/40-autoscale-queue.yaml

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (2)

examples/kueue-and-ray-on-aks/3-workloads/cas-batch-job/manifests/job.yaml:49

• This Job sets resource requests but no limits. The other Module 3 workload templates set both requests and limits; adding limits here prevents unbounded CPU/memory usage and keeps the example consistent.

          resources:
            requests:
              cpu: "1800m"
              memory: "256Mi"

examples/kueue-and-ray-on-aks/3-workloads/cas-batch-job/manifests/job.yaml:38

• Add ttlSecondsAfterFinished so this Job is cleaned up automatically, matching the Module 3 convention (ttlSecondsAfterFinished: 300) documented in 3-workloads/README.md. Without it, completed Jobs/pods can accumulate during repeated runs of the example.

spec:
  parallelism: 3
  completions: 3
  suspend: true
  template:

[Pengfei Ni (feiskyer)]

Great example for autoscaling, LGTM