From 11b09f9cfeff6f8fe3dfd293c572b4b12a9c0dab Mon Sep 17 00:00:00 2001 From: blackdragoon26 Date: Tue, 11 Aug 2026 18:35:06 +0530 Subject: [PATCH] docs(scheduler): document per-pod scoring weights Signed-off-by: blackdragoon26 --- docs/developers/scheduling.md | 65 ++++++++++++++++++++++++++++++----- 1 file changed, 57 insertions(+), 8 deletions(-) diff --git a/docs/developers/scheduling.md b/docs/developers/scheduling.md index 4597384e0..62982c106 100644 --- a/docs/developers/scheduling.md +++ b/docs/developers/scheduling.md @@ -132,46 +132,95 @@ In `Spread` policy, `Node2` is selected. ![HAMi GPU scheduler policy diagram, comparing Binpack and Spread scores on each card](/img/docs/common/developers/scheduling/gpu-scheduler-policy-demo.png) +#### Per-Pod device scoring weights + +By default, HAMi gives equal influence to predicted virtual-device slot, device-core, and device-memory utilization when it scores a physical device. To change that balance for one workload, add the `hami.io/device-scoring-weights` annotation to the Pod: + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: memory-weighted-gpu-pod + annotations: + hami.io/device-scoring-weights: "slot=1,core=1,memory=3" +spec: + containers: + - name: workload + image: ubuntu:22.04 + command: ["bash", "-c", "sleep 86400"] + resources: + limits: + nvidia.com/gpu: 1 + nvidia.com/gpumem-percentage: 40 +``` + +HAMi predicts each candidate device's utilization after placing the request, then calculates its device score as follows: + +```text +score = 10 * ( + slotWeight * predictedSlotUtilization + + coreWeight * predictedCoreUtilization + + memoryWeight * predictedMemoryUtilization +) +``` + +The annotation must contain the `slot`, `core`, and `memory` keys. Each value must be a non-negative integer, and at least one value must be greater than zero. Key order and surrounding whitespace do not matter. If the annotation is absent, HAMi uses `slot=1,core=1,memory=1`, which preserves the default scoring behavior. An invalid annotation prevents the Pod from being scheduled until the annotation is corrected. + +For example, consider two candidate GPUs after accounting for a Pod that requests one vGPU and 40% device memory: + +| Device | Predicted slot utilization | Predicted core utilization | Predicted memory utilization | +| ------ | -------------------------: | -------------------------: | ---------------------------: | +| GPU A | 0.2 | 0.9 | 0.5 | +| GPU B | 0.8 | 0.1 | 0.6 | + +With the default `1:1:1` weights, GPU A scores `16` and GPU B scores `15`, so `binpack` prefers GPU A. With `slot=1,core=1,memory=3`, GPU A scores `26` and GPU B scores `27`, so `binpack` prefers GPU B. Under `spread`, the lower score is preferred instead. + +The annotation changes only the utilization score used to order candidate devices. It does not bypass device fit or capacity checks, mutex rules, NUMA or topology constraints, or vendor-specific `Fit` behavior. These constraints keep their existing precedence; when topology candidates are otherwise tied, their utilization-score ordering can act as the tie-breaker. + #### Binpack -Binpack mainly focuses on the computing power and video memory usage of each card. The more it is used, the higher the score. +Binpack prefers the card with the higher device-utilization score. The following default-weight example assumes each card has ten virtual-device slots and no slot is currently in use: ```text -score: ((request.core + used.core) / allocatable.core + (request.mem + used.mem) / allocatable.mem)) * 10 +score: ((request.slot + used.slot) / allocatable.slot + + (request.core + used.core) / allocatable.core + + (request.mem + used.mem) / allocatable.mem) * 10 ``` 1. Binpack scoring information for GPU 1 is as follows ```text -GPU1 Score: ((20+10)/100 + (1000+2000)/8000)) * 10 = 6.75 +GPU1 Score: ((1+0)/10 + (20+10)/100 + (1000+2000)/8000) * 10 = 7.75 ``` 1. Binpack scoring information for GPU 2 is as follows ```text -GPU2 Score: ((20+70)/100 + (1000+6000)/8000)) * 10 = 17.75 +GPU2 Score: ((1+0)/10 + (20+70)/100 + (1000+6000)/8000) * 10 = 18.75 ``` In `Binpack` policy, `GPU2` is selected. #### Spread -Spread mainly focuses on the computing power and video memory usage of each card. The less it is used, the higher the score. +Spread prefers the card with the lower device-utilization score. Using the same default-weight example: ```text -score: ((request.core + used.core) / allocatable.core + (request.mem + used.mem) / allocatable.mem)) * 10 +score: ((request.slot + used.slot) / allocatable.slot + + (request.core + used.core) / allocatable.core + + (request.mem + used.mem) / allocatable.mem) * 10 ``` 1. Spread scoring information for GPU 1 is as follows ```text -GPU1 Score: ((20+10)/100 + (1000+2000)/8000)) * 10 = 6.75 +GPU1 Score: ((1+0)/10 + (20+10)/100 + (1000+2000)/8000) * 10 = 7.75 ``` 1. Spread scoring information for GPU 2 is as follows ```text -GPU2 Score: ((20+70)/100 + (1000+6000)/8000)) * 10 = 17.75 +GPU2 Score: ((1+0)/10 + (20+70)/100 + (1000+6000)/8000) * 10 = 18.75 ``` In `Spread` policy, `GPU1` is selected.