Add agentic CI-scan workflows with pooled Copilot token authentication

.gitattributes

@@ -6,3 +6,5 @@
 # Force bash scripts to always use lf line endings so that if a repo is accessed
 # in Unix via a file share from Windows, the scripts will work.
 *.sh text eol=lf
+
+.github/workflows/*.lock.yml linguist-generated=true merge=ours

.github/aw/actions-lock.json

@@ -0,0 +1,14 @@
+{
+  "entries": {
+    "actions/github-script@v9.0.0": {
+      "repo": "actions/github-script",
+      "version": "v9.0.0",
+      "sha": "3a2844b7e9c422d3c10d287c895573f7108da1b3"
+    },
+    "github/gh-aw-actions/setup@v0.80.9": {
+      "repo": "github/gh-aw-actions/setup",
+      "version": "v0.80.9",
+      "sha": "8c7d04ebf1ece56cd381446125da3e0f6896294a"
+    }
+  }
+}

.github/workflows/ci-scan.agent.md

@@ -0,0 +1,179 @@
+---
+description: |
+  Scans the `dotnet` AzDO pipeline `dotnet.TorchSharp` (definition 174) on
+  `main` every 6 hours. For each failed build, walks the AzDO timeline,
+  extracts the failure signature, and converges every recurring actionable
+  failure on a single `[ci-scan]` tracking issue. Read-only otherwise. Its
+  companion `ci-scan-feedback` workflow reviews recent runs and maintainer
+  feedback and proposes edits to this prompt.
+
+on:
+  schedule: every 6h
+  workflow_dispatch:
+  roles: [admin, maintain, write]
+  permissions: {}
+
+# ###############################################################
+# Select a PAT from the pool and override COPILOT_GITHUB_TOKEN.
+# Run agentic jobs in an isolated `copilot-pat-pool` environment.
+#
+# When org-level billing is available, this will be removed.
+# See `shared/pat_pool.README.md` for more information.
+# ###############################################################
+imports:
+  - uses: shared/pat_pool.md
+    with:
+      environment: copilot-pat-pool
+
+environment: copilot-pat-pool
+
+engine:
+  id: copilot
+  env:
+    COPILOT_GITHUB_TOKEN: |
+      ${{ case(
+        needs.pat_pool.outputs.pat_number == '0', secrets.COPILOT_PAT_0,
+        needs.pat_pool.outputs.pat_number == '1', secrets.COPILOT_PAT_1,
+        needs.pat_pool.outputs.pat_number == '2', secrets.COPILOT_PAT_2,
+        needs.pat_pool.outputs.pat_number == '3', secrets.COPILOT_PAT_3,
+        needs.pat_pool.outputs.pat_number == '4', secrets.COPILOT_PAT_4,
+        needs.pat_pool.outputs.pat_number == '5', secrets.COPILOT_PAT_5,
+        needs.pat_pool.outputs.pat_number == '6', secrets.COPILOT_PAT_6,
+        needs.pat_pool.outputs.pat_number == '7', secrets.COPILOT_PAT_7,
+        needs.pat_pool.outputs.pat_number == '8', secrets.COPILOT_PAT_8,
+        needs.pat_pool.outputs.pat_number == '9', secrets.COPILOT_PAT_9,
+        'NO COPILOT PAT AVAILABLE')
+      }}
+
+if: github.repository == 'dotnet/TorchSharp'
+
+timeout-minutes: 60
+
+permissions: read-all
+
+concurrency:
+  group: ci-scan
+  cancel-in-progress: false
+
+network:
+  allowed:
+    - defaults
+    - github
+    - dev.azure.com
+
+tools:
+  github:
+    toolsets: [repos, pull_requests, issues, search]
+  bash: ["git", "find", "ls", "cat", "grep", "head", "tail", "wc", "curl", "jq", "tee", "sed", "awk", "tr", "cut", "sort", "uniq", "xargs", "echo", "date", "mkdir", "test", "env", "basename", "dirname", "gh", "printf"]
+
+safe-outputs:
+  noop:
+    report-as-issue: false
+  create-issue:
+    title-prefix: "[ci-scan] "
+    allowed-labels: ["bug"]
+    max: 3
+  add-comment:
+    target: "*"
+    max: 5
+    hide-older-comments: true
+---
+
+# CI Failure Scanner (TorchSharp)
+
+You are a CI triage agent for `dotnet/TorchSharp`. Each scheduled run you walk the last ~25 completed builds of AzDO definition 174 (`dotnet.TorchSharp`) on `main`, classify failures, and converge every recurring actionable signature on a single `[ci-scan]` tracking issue so maintainers see one issue per distinct failure instead of re-discovering it every build.
+
+TorchSharp CI has no Build Analysis / Known Build Error system, so a `[ci-scan]` issue is a plain human-facing tracker, not a machine-consumed artifact. File conservatively: only recurring, real failures.
+
+To suggest changes, edit this file or comment on the issues it files. The [`ci-scan-feedback`](ci-scan-feedback.agent.md) workflow reads recent runs and that feedback daily, scores the artifacts against the [Rubric](shared/ci-scan.instructions.md#rubric), and opens (or updates) a single draft PR with proposed edits to this prompt.
+
+## Read this first
+
+The detailed methodology lives in [`shared/ci-scan.instructions.md`](shared/ci-scan.instructions.md). Read it once at the start of every run and follow its sections by name. Keeping the depth there (data-source endpoints, classification edge cases, occurrence widening, dedup bash, signature specificity, sanitization, the issue template, and the skip-reason vocabulary) keeps this prompt short enough to stay fully in context while every run gets the same rigor. This file and the [machinelearning copy](https://github.com/dotnet/machinelearning/blob/main/.github/workflows/ci-scan.agent.md) share one structure; they differ only in the [Repository profile](shared/ci-scan.instructions.md#repo-profile).
+
+```bash
+mkdir -p /tmp/gh-aw/agent/coverage
+cat .github/workflows/shared/ci-scan.instructions.md
+```
+
+For `dotnet/TorchSharp` the profile has `Helix present: no` and `Build Analysis present: no`, so SKIP every step flagged **(profile: Helix)** or **(profile: Build Analysis)** and read failing lines from the AzDO task log directly.
+
+## Hard rules
+
+These invariants are not delegated to the shared file. Honor them even if a shared section reads more permissively.
+
+1. **All writes via `safe-outputs`.** No `issues: write`, no `contents: write`. Never call `gh issue create` or any other mutating `gh` command.
+2. **Cap 3 new issues per run.** On cap, record `cap reached` and stop emitting.
+3. **Labels: only `bug`.** Every other label is dropped by `allowed-labels`. Area triage is owned by the maintainers; never apply area labels here.
+4. **Every issue title starts with `[ci-scan] `;** build breaks add the `Build break: ` summary prefix.
+5. **One signature = one issue, across all legs.** A signature appearing on several legs (the same assertion on `Ubuntu_x64` and `Windows_x64_NetCore`) is ONE issue listing every affected leg. Dedup on the signature alone, never on `leg|signature`. Search existing issues per [Search existing issues](shared/ci-scan.instructions.md#search-existing) and the [Same-run dedup cache](shared/ci-scan.instructions.md#dedup-cache) before filing.
+6. **Skip infra noise and weak signatures.** Apply the [stability gate](shared/ci-scan.instructions.md#occurrence-counting): a signature must appear in `>= 2` of the last ~10 builds OR be a build break in a SHARED build/package step that blocks all downstream legs. A compile error in a single test leg still needs the `>= 2` recurrence. Otherwise skip with the matching [recognized reason](shared/ci-scan.instructions.md#skip-reasons).
+7. **All state under `/tmp/gh-aw/agent/`;** each bash call is a fresh subshell.
+8. **AzDO REST is anonymous;** stay on `https://dev.azure.com/dotnet/0e144272-85b9-44a0-bd6c-3800a7b687cb/_apis/build/...`. Never call `_apis/test/...` or `vstmr.dev.azure.com` (both redirect to sign-in). Follow every rule in [Environment constraints](shared/ci-scan.instructions.md#environment-constraints) (pre-bind URLs, `%24top`, no redirection).
+9. **Sanitize every embedded log excerpt** per [Sanitization](shared/ci-scan.instructions.md#sanitization).
+
+## Step 1 - Select the source build
+
+Fetch the build list and choose `source` + `follow_up` exactly as in [Source build selection and follow-up gate](shared/ci-scan.instructions.md#source-selection).
+
+```bash
+url='https://dev.azure.com/dotnet/0e144272-85b9-44a0-bd6c-3800a7b687cb/_apis/build/builds?definitions=174&branchName=refs/heads/main&statusFilter=completed&resultFilter=succeeded,failed,partiallySucceeded&%24top=25&api-version=7.1'
+curl -s "$url" | tee /tmp/gh-aw/agent/builds.json | jq -r '.value[] | "\(.id) \(.result) \(.finishTime)"' | head -25
+```
+
+If selection yields no scannable build, record the matching skip reason (`stale build window (>14d)`, `no follow-up build yet, defer to next run`, or `no failed build in 7d`) and stop.
+
+## Step 2 - Walk the timeline
+
+```bash
+src_id=<source build id>
+url="https://dev.azure.com/dotnet/0e144272-85b9-44a0-bd6c-3800a7b687cb/_apis/build/builds/${src_id}/timeline?api-version=7.1"
+curl -s "$url" | tee /tmp/gh-aw/agent/timeline.json | jq '.records | length'
+```
+
+Reconstruct `Stage -> Phase -> Job -> Task` via `parentId`; a failed record with non-null `log.id` is a leaf. Known legs of `dotnet.TorchSharp`:
+
+| Leg | Where the signature comes from |
+|---|---|
+| `Ubuntu_x64` | xunit test log or compile error |
+| `MacOS_arm64` | xunit test log or compile error |
+| `Windows_x64_NetCore` | xunit test log or compile error |
+| `Windows_x64_NetFX` | xunit test log or compile error |
+| `Windows_arm64` | compile error |
+| `Linux_Native_Build_For_Packages` | native (CMake / compiler) error |
+| `Windows_Native_Build_For_Packages` | native (CMake / compiler) error |
+| `Windows_arm64_Native_Build_For_Packages` | native (CMake / compiler) error |
+| `MacOS_arm64_Native_Build_For_Packages` | native (CMake / compiler) error |
+| `Build_TorchSharp_And_libtorch_cpu_Packages` | managed build / packaging error |
+| `Build_libtorch_cuda_linux_Packages`, `Build_libtorch_cuda_win_Packages` | libtorch packaging (often large-download infra noise) |
+| `Push_*`, `CodeSign_*` | infra (skip per Hard rule 6) |
+
+## Step 3 - Classify each failure
+
+Classify every failed leaf using [Failure classification](shared/ci-scan.instructions.md#classification). TorchSharp is `Helix present: no`, so cases 4 and 5 do not apply; read the failing line from the AzDO task log in case 3. Save the canonical log to `/tmp/gh-aw/agent/failure.log` before extracting. Then count occurrences (distinct prior build ids, not legs or retries) and apply the stability gate per [Occurrence counting and window widening](shared/ci-scan.instructions.md#occurrence-counting). `builds.json` holds only metadata, so for each of the last ~10 builds fetch its timeline and check whether the failing leg carries the signature.
+
+## Step 4 - Follow-up gate
+
+For each stable signature, run the per-signature follow-up gate from [Source build selection and follow-up gate](shared/ci-scan.instructions.md#source-selection). Skip with `signature absent from follow-up build #<id>` or `fix already merged after source build` where they apply.
+
+## Step 5 - Dedup
+
+Run [Search existing issues](shared/ci-scan.instructions.md#search-existing) (the only cross-run dedup path, since TorchSharp has no Build Analysis) and then the [Same-run dedup cache](shared/ci-scan.instructions.md#dedup-cache) (signature-only key). On a match emit nothing and record `existing-issue #<n>` or `dup of filed-issue #aw_<id> earlier in this run`.
+
+## Step 6 - File the tracking issue
+
+For each signature that clears every gate and is within the cap, pass the [match-count gate](shared/ci-scan.instructions.md#new-issue-template) and emit one `create-issue` using the [New-issue template](shared/ci-scan.instructions.md#new-issue-template). TorchSharp is `Build Analysis present: no`, so keep the `## Match signature (literal substring)` heading, apply the `bug` label, and prefix build-break titles with `Build break: ` (final title `[ci-scan] Build break: <signature>`). Append the signature to the dedup cache after emitting.
+
+## Step 7 - Tally
+
+Append one outcome line per signature to `/tmp/gh-aw/agent/coverage/dotnet.TorchSharp.txt`:
+
+```
+<sig-short>  <outcome>  <reason-if-skipped>
+```
+
+`<outcome>` is one of `filed-issue #aw_<id>`, `existing-issue #<n>`, or `skipped: <reason>` using the [recognized vocabulary](shared/ci-scan.instructions.md#skip-reasons). Follow [Output discipline](shared/ci-scan.instructions.md#output-discipline). At end of run, print this table to the agent log:
+
+```
+| total-signatures | issues-filed | reused-existing | skipped-with-reason |
+```