Skip to content

Report AIGuard worker health trend evidence #377

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hyeokjun32 merged 1 commit into from
Jun 22, 2026
Merged

Report AIGuard worker health trend evidence #377

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/ci/runtime_intelligence_gitlab_artifacts.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,7 @@ packaging runs. That marker set preserves `Runtime Intelligence Risk Summary`,
`AIGuard operation risk rollup evidence`,
`AIGuard operation timeline evidence`,
`AIGuard scheduler fairness evidence`,
`AIGuard worker health trend evidence`,
`AIGuard policy pressure evidence`,
`AIGuard runtime operation anomalies`,
`AIGuard remote dispatch event summary`,
Expand Down
2 changes: 1 addition & 1 deletion docs/portfolio/edgeenv_runtime_regression_lab_handoff.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ Expected Lab behavior:
- The same gate requires EdgeEnv-preserved Orchestrator producer markers to carry `source_repository=InferEdgeOrchestrator`, `artifact_role=orchestrator-supplemental-operation-context`, and `producer_contract=inferedge-orchestrator-edgeenv-runtime-telemetry-feed-v1`.
- When EdgeEnv preservation context is present, Lab renders `Lab EdgeEnv preservation context` with `lab_report_preservation_context_present=True`, `lab_preservation=present`, and `lab_context=present` so the Runtime Intelligence report gate and entrypoint evidence index use the same Lab-owned marker vocabulary.
- When an EdgeEnv handoff manifest is provided, the bundle gate requires EdgeEnv-produced file keys, external AIGuard file keys, source repository mapping, artifact roles, producer contracts, and boundary flags to match Lab's Runtime Intelligence bundle contract.
- The same manifest gate requires `expected_report_markers` to preserve these exact Lab-owned report markers: `Runtime Intelligence Risk Summary`, `Runtime replay duration scope`, `Orchestrator operation feed context`, `EdgeEnv fixture matrix coverage`, `Reviewer operation quick scan`, `Orchestrator task event rollup`, `Lab EdgeEnv preservation context`, `AIGuard operation risk rollup evidence`, `AIGuard task event rollup evidence`, `AIGuard operation timeline evidence`, `AIGuard scheduler fairness evidence`, `AIGuard policy pressure evidence`, `AIGuard runtime operation anomalies`, `AIGuard remote dispatch event summary`, `AIGuard remote event summary consistency`, `Remote fallback starter evidence`, `lab=Remote fallback starter evidence; evidence=remote_execution_recovered_by_fallback`, `AIGuard producer-lineage guard alignment`, and `Lab remains the final deployment decision owner.`.
- The same manifest gate requires `expected_report_markers` to preserve these exact Lab-owned report markers: `Runtime Intelligence Risk Summary`, `Runtime replay duration scope`, `Orchestrator operation feed context`, `EdgeEnv fixture matrix coverage`, `Reviewer operation quick scan`, `Orchestrator task event rollup`, `Lab EdgeEnv preservation context`, `AIGuard operation risk rollup evidence`, `AIGuard task event rollup evidence`, `AIGuard operation timeline evidence`, `AIGuard scheduler fairness evidence`, `AIGuard worker health trend evidence`, `AIGuard policy pressure evidence`, `AIGuard runtime operation anomalies`, `AIGuard remote dispatch event summary`, `AIGuard remote event summary consistency`, `Remote fallback starter evidence`, `lab=Remote fallback starter evidence; evidence=remote_execution_recovered_by_fallback`, `AIGuard producer-lineage guard alignment`, and `Lab remains the final deployment decision owner.`.
- Reviewer navigation can trace Orchestrator curated samples through AIGuard evidence into Lab report rows: `agent_scheduler_delay_sample.json` -> `scheduler_delay_pattern` -> `Reviewer operation quick scan` / `Orchestrator queue/deadline/fallback markers`, and `remote_fallback_recovery_sample.json` -> `remote_execution_recovered_by_fallback` -> `Remote fallback starter evidence`. Lab does not treat those curated samples as benchmark outputs, production retry proof, or deployment policy inputs.
- The external AIGuard artifact gate also requires `runtime_queue_overload`
raw context to preserve
Expand Down
1,744 changes: 1,552 additions & 192 deletions examples/runtime_intelligence_chain/aiguard_runtime_operation_guard_analysis.json
View file
Open in desktop

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions examples/runtime_intelligence_chain/bundle_manifest.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@
"AIGuard task event rollup evidence",
"AIGuard operation timeline evidence",
"AIGuard scheduler fairness evidence",
"AIGuard worker health trend evidence",
"AIGuard policy pressure evidence",
"AIGuard runtime operation anomalies",
"AIGuard remote dispatch event summary",
Expand Down
13 changes: 10 additions & 3 deletions examples/runtime_intelligence_chain/edgeenv_lab_handoff_manifest.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,8 @@
"runtime_history_seed_run_config_traceability",
"runtime_queue_overload",
"runtime_thermal_instability",
"remote_execution_recovered_by_fallback"
"remote_execution_recovered_by_fallback",
"edgeenv_orchestrator_worker_health_trend"
],
"optional_aiguard_evidence_types": [
"stale_frame_risk",
Expand Down Expand Up @@ -119,7 +120,8 @@
"Remote fallback starter evidence",
"lab=Remote fallback starter evidence; evidence=remote_execution_recovered_by_fallback",
"AIGuard producer-lineage guard alignment",
"Lab remains the final deployment decision owner."
"Lab remains the final deployment decision owner.",
"AIGuard worker health trend evidence"
],
"source_repositories": {
"runtime_result": "InferEdge-Runtime",
Expand Down Expand Up @@ -199,12 +201,17 @@
"orchestrator_operation_timeline_summary_present": true,
"orchestrator_operation_timeline_summary_run_ids": [
"edgeenv-smoke-candidate"
],
"orchestrator_worker_health_trend_present": true,
"orchestrator_worker_health_trend_run_ids": [
"edgeenv-smoke-candidate"
]
},
"notes": [
"This manifest is EdgeEnv producer-side handoff metadata.",
"AIGuard guard_analysis is intentionally not produced by EdgeEnv.",
"Lab remains the final deployment decision owner.",
"This is a local artifact contract, not production observability."
"This is a local artifact contract, not production observability.",
"EdgeEnv preserves Orchestrator worker_health_trend for AIGuard deterministic evidence and Lab report context."
]
}
68 changes: 61 additions & 7 deletions examples/runtime_intelligence_chain/edgeenv_regression_with_orchestrator_context.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -370,14 +370,64 @@
"degraded": [
"vision_agent"
],
"constrained": []
"constrained": [
"voice_command_agent"
]
},
"review_hints": [
"review_queue_pressure",
"review_scheduler_delay",
"review_deadline_miss",
"review_fallback_use"
]
"review_fallback_use",
"review_worker_health_trend"
],
"worker_health_trend": {
"schema_version": "inferedge-orchestrator-worker-health-trend-v1",
"operation_context_role": "supplemental",
"scheduler_owner": "orchestrator",
"decision_owner": "lab",
"not_a_deployment_decision": true,
"source": "worker_health_snapshot+runtime_event_summary",
"health_state_counts": {
"healthy": 1,
"degraded": 1,
"constrained": 1
},
"tasks_by_health_state": {
"healthy": [
"safety_monitor_agent"
],
"degraded": [
"vision_agent"
],
"constrained": [
"voice_command_agent"
]
},
"task_health_context": {
"vision_agent": {
"worker_id": "vision_agent",
"health_state": "degraded",
"primary_health_reason": "worker_health_degraded"
},
"voice_command_agent": {
"worker_id": "voice_command_agent",
"health_state": "constrained",
"primary_health_reason": "fallback_overuse"
}
},
"degraded_workers": [
"vision_agent"
],
"constrained_workers": [
"voice_command_agent"
],
"review_hints": [
"review_worker_health_degradation",
"review_worker_health_constraints"
],
"interpretation": "Worker health trend is supplemental operation evidence; Lab remains the final deployment decision owner."
}
}
},
"resource": {
Expand Down Expand Up @@ -425,15 +475,17 @@
],
"aiguard_evidence_candidates": [
"runtime_queue_overload",
"runtime_thermal_instability"
"runtime_thermal_instability",
"edgeenv_orchestrator_worker_health_trend"
]
},
"downstream_guard_alignment": {
"declared_by": "orchestrator",
"producer_lineage_evidence_type": "edgeenv_orchestrator_producer_lineage",
"operation_evidence_candidates": [
"runtime_queue_overload",
"runtime_thermal_instability"
"runtime_thermal_instability",
"edgeenv_orchestrator_worker_health_trend"
],
"validated_by": [
"edgeenv runs telemetry inspect-history",
Expand Down Expand Up @@ -714,15 +766,17 @@
],
"aiguard_evidence_candidates": [
"runtime_queue_overload",
"runtime_thermal_instability"
"runtime_thermal_instability",
"edgeenv_orchestrator_worker_health_trend"
]
},
"downstream_guard_alignment": {
"declared_by": "orchestrator",
"producer_lineage_evidence_type": "edgeenv_orchestrator_producer_lineage",
"operation_evidence_candidates": [
"runtime_queue_overload",
"runtime_thermal_instability"
"runtime_thermal_instability",
"edgeenv_orchestrator_worker_health_trend"
],
"validated_by": [
"edgeenv runs telemetry inspect-history",
Expand Down
68 changes: 61 additions & 7 deletions examples/runtime_intelligence_chain/runtime_telemetry_history.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -293,14 +293,64 @@
"degraded": [
"vision_agent"
],
"constrained": []
"constrained": [
"voice_command_agent"
]
},
"review_hints": [
"review_queue_pressure",
"review_scheduler_delay",
"review_deadline_miss",
"review_fallback_use"
]
"review_fallback_use",
"review_worker_health_trend"
],
"worker_health_trend": {
"schema_version": "inferedge-orchestrator-worker-health-trend-v1",
"operation_context_role": "supplemental",
"scheduler_owner": "orchestrator",
"decision_owner": "lab",
"not_a_deployment_decision": true,
"source": "worker_health_snapshot+runtime_event_summary",
"health_state_counts": {
"healthy": 1,
"degraded": 1,
"constrained": 1
},
"tasks_by_health_state": {
"healthy": [
"safety_monitor_agent"
],
"degraded": [
"vision_agent"
],
"constrained": [
"voice_command_agent"
]
},
"task_health_context": {
"vision_agent": {
"worker_id": "vision_agent",
"health_state": "degraded",
"primary_health_reason": "worker_health_degraded"
},
"voice_command_agent": {
"worker_id": "voice_command_agent",
"health_state": "constrained",
"primary_health_reason": "fallback_overuse"
}
},
"degraded_workers": [
"vision_agent"
],
"constrained_workers": [
"voice_command_agent"
],
"review_hints": [
"review_worker_health_degradation",
"review_worker_health_constraints"
],
"interpretation": "Worker health trend is supplemental operation evidence; Lab remains the final deployment decision owner."
}
}
},
"resource": {
Expand Down Expand Up @@ -343,15 +393,17 @@
],
"aiguard_evidence_candidates": [
"runtime_queue_overload",
"runtime_thermal_instability"
"runtime_thermal_instability",
"edgeenv_orchestrator_worker_health_trend"
]
},
"downstream_guard_alignment": {
"declared_by": "orchestrator",
"producer_lineage_evidence_type": "edgeenv_orchestrator_producer_lineage",
"operation_evidence_candidates": [
"runtime_queue_overload",
"runtime_thermal_instability"
"runtime_thermal_instability",
"edgeenv_orchestrator_worker_health_trend"
],
"validated_by": [
"edgeenv runs telemetry inspect-history",
Expand Down Expand Up @@ -438,15 +490,17 @@
],
"aiguard_evidence_candidates": [
"runtime_queue_overload",
"runtime_thermal_instability"
"runtime_thermal_instability",
"edgeenv_orchestrator_worker_health_trend"
]
},
"downstream_guard_alignment": {
"declared_by": "orchestrator",
"producer_lineage_evidence_type": "edgeenv_orchestrator_producer_lineage",
"operation_evidence_candidates": [
"runtime_queue_overload",
"runtime_thermal_instability"
"runtime_thermal_instability",
"edgeenv_orchestrator_worker_health_trend"
],
"validated_by": [
"edgeenv runs telemetry inspect-history",
Expand Down
59 changes: 59 additions & 0 deletions inferedgelab/report/runtime_intelligence.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@
"edgeenv_orchestrator_scheduler_fairness_summary"
)

ORCHESTRATOR_WORKER_HEALTH_TREND_EVIDENCE_TYPE = (
"edgeenv_orchestrator_worker_health_trend"
)

ORCHESTRATOR_POLICY_PRESSURE_EVIDENCE_TYPE = (
"edgeenv_orchestrator_policy_pressure_summary"
)
Expand Down Expand Up @@ -1568,6 +1572,16 @@ def _append_aiguard_runtime_operation_rows(
)
)

worker_health_trend_label = _aiguard_worker_health_trend_label(evidence_items)
if worker_health_trend_label:
rows.append(
(
"AIGuard worker health trend evidence",
worker_health_trend_label,
"AIGuard preserves degraded/constrained worker trend context as deterministic review evidence; Lab still owns the deployment decision.",
)
)

policy_pressure_label = _aiguard_policy_pressure_label(evidence_items)
if policy_pressure_label:
rows.append(
Expand Down Expand Up @@ -2051,6 +2065,51 @@ def _aiguard_scheduler_fairness_label(
return ", ".join(parts)


def _aiguard_worker_health_trend_label(
evidence_items: list[dict[str, Any]],
) -> str:
evidence = _find_evidence_item(
evidence_items,
ORCHESTRATOR_WORKER_HEALTH_TREND_EVIDENCE_TYPE,
)
if evidence is None:
return ""

parts: list[str] = []
status = evidence.get("status")
if status is not None:
parts.append(f"status={status}")
observed = evidence.get("observed_value")
if observed is not None:
parts.append(f"markers={_format_compact_value(observed)}")

raw_context = evidence.get("raw_context")
if isinstance(raw_context, dict):
context = raw_context.get("worker_health_trend")
if isinstance(context, dict):
degraded_workers = _string_list(context.get("degraded_workers"))
if degraded_workers:
parts.append("degraded=" + ",".join(degraded_workers))
constrained_workers = _string_list(context.get("constrained_workers"))
if constrained_workers:
parts.append("constrained=" + ",".join(constrained_workers))
health_state_counts = context.get("health_state_counts")
if isinstance(health_state_counts, dict):
state_labels = [
f"{state}:{_format_compact_value(count)}"
for state, count in health_state_counts.items()
if isinstance(state, str)
and state in {"healthy", "degraded", "constrained"}
and count is not None
]
if state_labels:
parts.append("health_states=" + ",".join(state_labels))
boundary_valid = context.get("boundary_markers_valid")
if boundary_valid is not None:
parts.append(f"boundary_valid={boundary_valid}")
return ", ".join(parts)


def _aiguard_policy_pressure_label(
evidence_items: list[dict[str, Any]],
) -> str:
Expand Down
15 changes: 14 additions & 1 deletion scripts/check_runtime_intelligence_artifact_bundle.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@
"aiguard_task_event_rollup_type": "edgeenv_orchestrator_task_event_rollup",
"aiguard_operation_timeline_evidence": (
"| AIGuard operation timeline evidence | "
"status=warning, markers=6"
"status=warning, markers=7"
),
"aiguard_operation_timeline_type": (
"edgeenv_orchestrator_operation_timeline_summary"
Expand All @@ -156,6 +156,13 @@
"aiguard_scheduler_fairness_type": (
"edgeenv_orchestrator_scheduler_fairness_summary"
),
"aiguard_worker_health_trend_evidence": (
"| AIGuard worker health trend evidence | "
"status=warning, markers=5"
),
"aiguard_worker_health_trend_type": (
"edgeenv_orchestrator_worker_health_trend"
),
"aiguard_policy_pressure_evidence": (
"| AIGuard policy pressure evidence | "
"status=warning, markers=5"
Expand Down Expand Up @@ -292,6 +299,12 @@
"aiguard_scheduler_fairness_type": (
"edgeenv_orchestrator_scheduler_fairness_summary"
),
"aiguard_worker_health_trend_evidence": (
"AIGuard worker health trend evidence"
),
"aiguard_worker_health_trend_type": (
"edgeenv_orchestrator_worker_health_trend"
),
"aiguard_policy_pressure_evidence": "AIGuard policy pressure evidence",
"aiguard_policy_pressure_type": (
"edgeenv_orchestrator_policy_pressure_summary"
Expand Down
Loading
Loading