Skip to content

feat(metrics): bind the metrics program to sol at medium effort - #1128

Closed
gewenyu99 wants to merge 1 commit into
posthog/metrics-programfrom
posthog/metrics-sol-binding
Closed

feat(metrics): bind the metrics program to sol at medium effort#1128
gewenyu99 wants to merge 1 commit into
posthog/metrics-programfrom
posthog/metrics-sol-binding

Conversation

@gewenyu99

Copy link
Copy Markdown
Collaborator

Stacked on #1102: routes wizard metrics to gpt-5.6-sol on the pi harness at medium effort instead of the default anthropic binding.

🤖 Generated with Claude Code

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gewenyu99
gewenyu99 requested a review from a team as a code owner August 21, 2026 15:23
@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/groq
  • /wizard-ci ai-observability/manual-capture
Show more apps
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit

Results will be posted here when complete.

metrics: DEFAULT_BINDING,
metrics: {
sequence: Sequence.linear,
harness: Harness.pi,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: File access outside the project

Routing metrics to Pi exposes its uncontained file tools by default: an attacker-controlled repository can direct the agent to read or overwrite files outside session.installDir using absolute paths or .. traversal. Keep this program on the sandboxed harness until Pi rejects paths outside the project root for read, write, edit, grep, find, and list operations.

@veria-ai

veria-ai Bot commented Aug 21, 2026

Copy link
Copy Markdown

PR overview

This pull request updates the agent runner switchboard to bind the metrics program to Sol with a medium effort level.

A significant security issue remains open: an attacker-controlled repository can direct the metrics agent’s file tools to read or overwrite files outside the project directory using absolute paths or traversal sequences. No issues have yet been addressed, so the metrics program should remain on the sandboxed harness until all file operations enforce the project boundary.

Open issues (1)

Fixed/addressed: 0 · PR risk: 7/10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant