balloons: IRQ CPU affinity controls - #722
Conversation
Add pkg/irq for reading interrupts from /proc/interrupts and reading and writing CPU affinities through /proc/irq/NUMBER/smp_affinity_list. Wire the proc root to the plugin host root and mount the host proc filesystem into the balloons daemonset so that affinities of host IRQs can be updated (write access needed). Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
f80e862 to
a93ae5f
Compare
| p.BlockMeters() | ||
| defer p.UnblockMeters() | ||
| defer p.commitCpuClasses() | ||
| defer p.applyIrqAffinities() |
There was a problem hiding this comment.
Related to configuration and reconfiguration: is my understanding correct that an irqMode: sink, irqClaim: [ $IRQ_CLAIMS ] is not sensible/meaningful combination for any single balloon type. If it is, should we verify against it in pkg/apis/config/*/resmgr/policy/balloons.Config.Validate() ?
There was a problem hiding this comment.
Yes, that's the most obscure combination.
On the other hand, if almost all balloon types would require isolation from IRQs, declaring the few non-sensitive balloon types as IRQ sinks might be more convenient than declaring all latency-sensitive-tailored balloon types isolated from IRQs. Under these circumstances it could be that a non-sensitive balloon type would also want to claim a certain IRQ, yet still being ok to participate in balancing all other IRQs, too.
In other words, a sink+claim balloon among other sink balloons would make equally sense than (plain) claim balloon among dont-care balloons when many isolated balloons are present.
Therefore, I wouldn't add a verification against this configuration. I hope that sink/isolate/claim semantics would be clear enough (and documented clearly enough) for users to understand what happens in this case.
klihub
left a comment
klihub
left a comment
There was a problem hiding this comment.
I have a few minor questions and comments.
Add the balloon type options irqClaim and irqMode to the balloons policy configuration. Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
a93ae5f to
c63979b
Compare
Set CPU affinities of system IRQs according to the irqClaim and irqMode balloon type options. For each IRQ, the logic is: find balloons that want exactly this IRQ (irqClaim). If nobody, then find balloons that want all unclaimed IRQs (irqMode: sink). If no volunteers, then list balloons that do not want any IRQs (irqMode: isolate), and set IRQs affinity to all those who do not care. The default is nobody cares (no irq options present), meaning that the balloons policy will not touch IRQ CPU affinities at all, keeping the original behavior. Affinities are updated after balloon changes. Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
Add an end-to-end test that verifies irqClaim, irqMode sink and irqMode isolate by inspecting /proc/irq affinities on the test VM. Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
Describe the irqClaim and irqMode balloon type options and their common use cases. Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
c63979b to
67b143e
Compare
|
Thanks @askervin . This is a very useful addition and actually something that we have been thinking about, but more on the topology-aware plugin. I will take my time tomorrow and go through the changes. |
3 participants
Control IRQ CPU affinities using new
irqClaimandirqModeballoon type options.For each IRQ, the logic is:
Combining irqClaim with "irqMode: isolate" in the same balloon type creates balloons that handle only claimed IRQs and no others.