Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/ci/runtime_intelligence_gitlab_artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,7 @@ downstream guard-alignment marker for `edgeenv_orchestrator_producer_lineage`
to remain present in EdgeEnv/AIGuard handoff context. It also requires AIGuard
`edgeenv_orchestrator_task_event_rollup`,
`edgeenv_orchestrator_operation_timeline_summary`,
`edgeenv_orchestrator_scheduler_fairness_summary`,
`runtime_history_seed_run_config_traceability`, and
`remote_execution_recovered_by_fallback` evidence so the smoke remains a
cross-repo handoff fixture rather than a Lab-only report sample. When an
Expand All @@ -183,6 +184,7 @@ packaging runs. That marker set preserves `Runtime Intelligence Risk Summary`,
`AIGuard task event rollup evidence`,
`AIGuard operation risk rollup evidence`,
`AIGuard operation timeline evidence`,
`AIGuard scheduler fairness evidence`,
`AIGuard runtime operation anomalies`,
`AIGuard remote dispatch event summary`,
`AIGuard remote event summary consistency`,
Expand All @@ -205,7 +207,7 @@ timeline summary run IDs against the preserved EdgeEnv regression context, so
compact AIGuard operation evidence remains traceable to producer-side
Orchestrator context.

The artifact gate is implemented by `scripts/check_runtime_intelligence_artifact_bundle.py`. It checks the generated Markdown / HTML report for the required Runtime Intelligence rows, including the short `Review Path` section, the `Review path` note, the `Reviewer Focus` quick-scan table, Lab ownership, EdgeEnv comparability, `EdgeEnv fixture matrix coverage`, telemetry coverage-gap markers, Runtime replay duration scope with `source=entrypoint_requested_frames` traceability, Orchestrator operation feed context, the Lab-owned `Reviewer operation quick scan` row, compact queue/deadline/fallback operation markers with `max_total_queue_depth`, AIGuard max queue raw-context traceability, Orchestrator task event rollup, Lab EdgeEnv preservation context, Jetson/device-local preservation identity and detail labels, Orchestrator `operation_risk_summary` navigation context, AIGuard runtime operation anomalies, AIGuard `edgeenv_orchestrator_operation_risk_summary` evidence, AIGuard `edgeenv_orchestrator_operation_risk_rollup` evidence, AIGuard `edgeenv_orchestrator_task_event_rollup` evidence, AIGuard `edgeenv_orchestrator_operation_timeline_summary` evidence, remote dispatch starter event summary, `Remote fallback starter evidence`, `edgeenv_orchestrator_producer_lineage`, `runtime_history_seed_run_config_traceability`, `remote_execution_recovered_by_fallback`, and triggered deployment review rules.
The artifact gate is implemented by `scripts/check_runtime_intelligence_artifact_bundle.py`. It checks the generated Markdown / HTML report for the required Runtime Intelligence rows, including the short `Review Path` section, the `Review path` note, the `Reviewer Focus` quick-scan table, Lab ownership, EdgeEnv comparability, `EdgeEnv fixture matrix coverage`, telemetry coverage-gap markers, Runtime replay duration scope with `source=entrypoint_requested_frames` traceability, Orchestrator operation feed context, the Lab-owned `Reviewer operation quick scan` row, compact queue/deadline/fallback operation markers with `max_total_queue_depth`, AIGuard max queue raw-context traceability, Orchestrator task event rollup, Lab EdgeEnv preservation context, Jetson/device-local preservation identity and detail labels, Orchestrator `operation_risk_summary` navigation context, AIGuard runtime operation anomalies, AIGuard `edgeenv_orchestrator_operation_risk_summary` evidence, AIGuard `edgeenv_orchestrator_operation_risk_rollup` evidence, AIGuard `edgeenv_orchestrator_task_event_rollup` evidence, AIGuard `edgeenv_orchestrator_operation_timeline_summary` evidence, AIGuard `edgeenv_orchestrator_scheduler_fairness_summary` evidence, remote dispatch starter event summary, `Remote fallback starter evidence`, `edgeenv_orchestrator_producer_lineage`, `runtime_history_seed_run_config_traceability`, `remote_execution_recovered_by_fallback`, and triggered deployment review rules.

The bundle manifest gate also checks the external AIGuard artifact before the
rendered report stage. In particular, `runtime_queue_overload` must preserve
Expand Down
15 changes: 12 additions & 3 deletions docs/portfolio/agent_runtime_reliability_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,12 @@ Newer AIGuard guard analysis can also include `stale_frame_risk` or preserved
count, stale-drop rate, affected tasks, reason counts, and reason classes in
the AIGuard Orchestrator Operation Evidence section as deployment review
context.
When AIGuard preserves
`edgeenv_orchestrator_scheduler_fairness_summary`, Lab surfaces protected
high-priority tasks, starvation-risk tasks, scheduler-delay tasks, degraded
tasks, and boundary markers as scheduler fairness review context. This remains
Lab-owned deployment review evidence and does not make AIGuard or Orchestrator
the final decision owner.

The report also preserves the Orchestrator operation-health fields added for
runtime operation review:
Expand Down Expand Up @@ -157,8 +163,9 @@ runtime operation review:
`worker_health_degradation` and `scheduler_delay_pattern` when Orchestrator
worker health or runtime event telemetry is analyzed by AIGuard.
Lab preserves health reasons, policy/drop/stale-drop reason counts, scheduler
delay counts, stale-drop affected tasks, and stale-drop boundary markers as
deployment context without making AIGuard the final decision owner.
delay counts, stale-drop affected tasks, scheduler fairness task groups, and
boundary markers as deployment context without making AIGuard the final
decision owner.

These fields make the report path explicit:

Expand Down Expand Up @@ -211,7 +218,9 @@ example, `worker_health_degradation` shows degraded/constrained worker reasons
such as fallback policy use or dropped frames, while `scheduler_delay_pattern`
shows scheduler delay counts and related policy/drop reasons. These evidence
shows scheduler delay counts and related policy/drop reasons.
`stale_frame_risk` shows which tasks had stale/backlog drops and why. These
`stale_frame_risk` shows which tasks had stale/backlog drops and why, and
`edgeenv_orchestrator_scheduler_fairness_summary` shows which tasks were
protected, delayed, starved, or degraded under scheduler pressure. These
evidence items contribute through AIGuard's overall guard verdict and remain
separate from Lab's final policy ownership.

Expand Down
4 changes: 2 additions & 2 deletions docs/portfolio/edgeenv_runtime_regression_lab_handoff.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ Expected Lab behavior:
- The same gate requires EdgeEnv-preserved Orchestrator producer markers to carry `source_repository=InferEdgeOrchestrator`, `artifact_role=orchestrator-supplemental-operation-context`, and `producer_contract=inferedge-orchestrator-edgeenv-runtime-telemetry-feed-v1`.
- When EdgeEnv preservation context is present, Lab renders `Lab EdgeEnv preservation context` with `lab_report_preservation_context_present=True`, `lab_preservation=present`, and `lab_context=present` so the Runtime Intelligence report gate and entrypoint evidence index use the same Lab-owned marker vocabulary.
- When an EdgeEnv handoff manifest is provided, the bundle gate requires EdgeEnv-produced file keys, external AIGuard file keys, source repository mapping, artifact roles, producer contracts, and boundary flags to match Lab's Runtime Intelligence bundle contract.
- The same manifest gate requires `expected_report_markers` to preserve these exact Lab-owned report markers: `Runtime Intelligence Risk Summary`, `Runtime replay duration scope`, `Orchestrator operation feed context`, `EdgeEnv fixture matrix coverage`, `Reviewer operation quick scan`, `Orchestrator task event rollup`, `Lab EdgeEnv preservation context`, `AIGuard operation risk rollup evidence`, `AIGuard task event rollup evidence`, `AIGuard operation timeline evidence`, `AIGuard runtime operation anomalies`, `AIGuard remote dispatch event summary`, `AIGuard remote event summary consistency`, `Remote fallback starter evidence`, `lab=Remote fallback starter evidence; evidence=remote_execution_recovered_by_fallback`, `AIGuard producer-lineage guard alignment`, and `Lab remains the final deployment decision owner.`.
- The same manifest gate requires `expected_report_markers` to preserve these exact Lab-owned report markers: `Runtime Intelligence Risk Summary`, `Runtime replay duration scope`, `Orchestrator operation feed context`, `EdgeEnv fixture matrix coverage`, `Reviewer operation quick scan`, `Orchestrator task event rollup`, `Lab EdgeEnv preservation context`, `AIGuard operation risk rollup evidence`, `AIGuard task event rollup evidence`, `AIGuard operation timeline evidence`, `AIGuard scheduler fairness evidence`, `AIGuard runtime operation anomalies`, `AIGuard remote dispatch event summary`, `AIGuard remote event summary consistency`, `Remote fallback starter evidence`, `lab=Remote fallback starter evidence; evidence=remote_execution_recovered_by_fallback`, `AIGuard producer-lineage guard alignment`, and `Lab remains the final deployment decision owner.`.
- Reviewer navigation can trace Orchestrator curated samples through AIGuard evidence into Lab report rows: `agent_scheduler_delay_sample.json` -> `scheduler_delay_pattern` -> `Reviewer operation quick scan` / `Orchestrator queue/deadline/fallback markers`, and `remote_fallback_recovery_sample.json` -> `remote_execution_recovered_by_fallback` -> `Remote fallback starter evidence`. Lab does not treat those curated samples as benchmark outputs, production retry proof, or deployment policy inputs.
- The external AIGuard artifact gate also requires `runtime_queue_overload`
raw context to preserve
Expand All @@ -208,7 +208,7 @@ Expected Lab behavior:
- The same handoff gate verifies `edgeenv_report_summary.fixture_matrix_*` fields against the EdgeEnv regression report's `fixture_matrix_context`, keeping fixture coverage as EdgeEnv-owned replay context rather than Lab decision policy.
- The same handoff gate verifies that missing telemetry entries remain evidence gaps while preserving Orchestrator producer markers, owner boundary flags, and EdgeEnv mapping hints when Orchestrator context is attached.
- The same handoff gate validates `edgeenv_report_summary.producer_lineage_guard_alignment_run_ids` against the preserved EdgeEnv regression context so the Orchestrator-declared `edgeenv_orchestrator_producer_lineage` marker cannot disappear between EdgeEnv producer output and Lab bundle ingestion.
- The same handoff gate validates `edgeenv_report_summary.orchestrator_operation_risk_rollup_run_ids` and `edgeenv_report_summary.orchestrator_operation_timeline_summary_run_ids` against preserved Orchestrator context, so AIGuard operation rollup/timeline evidence cannot drift away from EdgeEnv's producer-side context.
- The same handoff gate validates `edgeenv_report_summary.orchestrator_operation_risk_rollup_run_ids` and `edgeenv_report_summary.orchestrator_operation_timeline_summary_run_ids` against preserved Orchestrator context, so AIGuard operation rollup/timeline/fairness evidence cannot drift away from EdgeEnv's producer-side context.
- The committed smoke also carries AIGuard's EdgeEnv handoff alignment artifact, which confirms that EdgeEnv's producer summary and AIGuard's `edgeenv_orchestrator_producer_lineage` raw context agree on `edgeenv-smoke-candidate` and `edgeenv-smoke-missing` as the producer-lineage guard-alignment run IDs.
- The bundle gate also requires AIGuard coverage evidence raw context to preserve the same Orchestrator mapping hint and producer markers, proving that AIGuard kept EdgeEnv/Orchestrator ownership markers as diagnosis context rather than recomputing coverage or owning deployment policy.
- The same gate requires AIGuard `edgeenv_orchestrator_producer_lineage` evidence to preserve candidate and missing-telemetry device-local producer lineage as traceability evidence.
Expand Down
2 changes: 1 addition & 1 deletion docs/portfolio/final_validation_completion.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ InferEdge is complete for the current portfolio milestone when it can replay a l
| Normal demo case | done | `examples/validation_demo/subset/` |
| Problem demo cases | done | annotation missing, invalid structure, contract mismatch reports |
| Report formats | done | JSON, Markdown, HTML evaluation reports |
| Runtime Intelligence risk summary | done | Orchestrator `operation_risk_rollup` -> EdgeEnv handoff -> AIGuard deterministic evidence -> Lab deployment risk report |
| Runtime Intelligence risk summary | done | Orchestrator `operation_risk_rollup` / scheduler fairness context -> EdgeEnv handoff -> AIGuard deterministic evidence -> Lab deployment risk report |
| Tests | done | full `pytest` suite passing locally |

## Validated Numbers
Expand Down
4 changes: 2 additions & 2 deletions docs/portfolio/inferedge_pipeline_status.ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
- EdgeEnv runtime telemetry/regression context ingestion.
- AIGuard deterministic runtime warning evidence preservation.
- Orchestrator queue/deadline/fallback context를 supplemental operation evidence로 표시.
- Runtime Intelligence operation risk rollup chain: Orchestrator operation risk/timeline context -> EdgeEnv handoff -> AIGuard deterministic evidence -> Lab Runtime Intelligence Risk Summary.
- Runtime Intelligence operation risk rollup chain: Orchestrator operation risk/timeline/scheduler fairness context -> EdgeEnv handoff -> AIGuard deterministic evidence -> Lab Runtime Intelligence Risk Summary.

## 현재 evidence snapshot

Expand All @@ -23,7 +23,7 @@
| Jetson TensorRT FP16 25W demo | `10.066401 ms` mean, `15.548438 ms` p99, `99.340373 FPS` |
| Demo speedup | 약 `4.51x` |
| Jetson EdgeEnv preservation smoke | `device_local_starter`, `run-20260529-034704-fbf753f0`, `runtime_operation_summary` |
| Runtime Intelligence rollup chain | `operation_risk_rollup` -> EdgeEnv handoff -> AIGuard evidence -> Lab Risk Summary |
| Runtime Intelligence rollup chain | `operation_risk_rollup` / scheduler fairness -> EdgeEnv handoff -> AIGuard evidence -> Lab Risk Summary |

## 아직 구현하지 않았거나 명시적으로 제외한 것

Expand Down
2 changes: 1 addition & 1 deletion docs/portfolio/inferedge_pipeline_status.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ The current cross-repository loop is covered by documentation, fixtures, and smo
- YOLOv8 COCO subset evaluation report generated from 10 local images and 89 converted COCO-style person annotations, with metric backend `simplified`, mAP@50 0.1410, precision 0.2941, recall 0.1685, and structural validation passed
- Validation problem case fixtures for annotation-missing review, invalid detection structure blocking, and contract shape mismatch blocking
- Runtime Intelligence evidence chain from Orchestrator operation context and `operation_risk_rollup`, through EdgeEnv telemetry/regression handoff, optional AIGuard deterministic runtime evidence, and Lab's Runtime Intelligence Risk Summary / deployment risk report
- Runtime Intelligence bundle gates for reviewer quick-scan markers, operation timeline evidence, task event rollup evidence, remote fallback starter evidence, and explicit Lab final decision ownership without changing existing JSON contracts
- Runtime Intelligence bundle gates for reviewer quick-scan markers, operation timeline evidence, scheduler fairness evidence, task event rollup evidence, remote fallback starter evidence, and explicit Lab final decision ownership without changing existing JSON contracts

This means the current product boundary is testable without running the production worker infrastructure.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@
"diagnosis_owner": "aiguard",
"handoff_schema_version": "edgeenv.runtime-intelligence-lab-handoff.v1",
"guard_analysis_schema_version": "inferedge-aiguard-diagnosis-v1",
"required_evidence_type_count": 9,
"required_evidence_type_count": 10,
"optional_evidence_type_count": 2,
"guard_evidence_type_count": 10,
"lab_expected_report_marker_count": 17,
"guard_evidence_type_count": 11,
"lab_expected_report_marker_count": 18,
"lab_expected_report_markers": [
"Runtime Intelligence Risk Summary",
"Runtime replay duration scope",
Expand All @@ -21,6 +21,7 @@
"AIGuard operation risk rollup evidence",
"AIGuard task event rollup evidence",
"AIGuard operation timeline evidence",
"AIGuard scheduler fairness evidence",
"AIGuard runtime operation anomalies",
"AIGuard remote dispatch event summary",
"AIGuard remote event summary consistency",
Expand All @@ -44,6 +45,7 @@
"edgeenv_orchestrator_operation_risk_rollup",
"edgeenv_orchestrator_task_event_rollup",
"edgeenv_orchestrator_operation_timeline_summary",
"edgeenv_orchestrator_scheduler_fairness_summary",
"runtime_history_seed_run_config_traceability",
"runtime_queue_overload",
"runtime_thermal_instability",
Expand All @@ -60,6 +62,7 @@
"edgeenv_orchestrator_operation_risk_rollup",
"edgeenv_orchestrator_task_event_rollup",
"edgeenv_orchestrator_operation_timeline_summary",
"edgeenv_orchestrator_scheduler_fairness_summary",
"runtime_history_seed_run_config_traceability",
"runtime_thermal_instability",
"runtime_queue_overload",
Expand Down
Loading
Loading