Skip to content

Add provider-neutral per-node model policy #175

Description

@eshulman2

Problem

Forge currently uses broadly global model configuration, although workflow stages have different quality, latency, context-window, tool, residency, and cost requirements. A PR-description task and a complex implementation task should not necessarily use the same model.

The earlier proposal in PR #24 used fixed fast, standard, and advanced tiers mapped to Claude models. That abstraction is too provider-specific and prevents projects from selecting exact models available through Anthropic, Vertex AI, Gemini API, or future backends.

Proposed model policy

Add a provider-neutral, free-form mapping from workflow node or skill to an exact model target. Do not define built-in semantic tiers or assume an ordering between models.

model_connections:
  vertex-production:
    backend: vertex-ai
    credential_ref: gcp-workload-identity
    project: forge-production
    location: global
  anthropic-production:
    backend: anthropic
    credential_ref: anthropic-gateway-session

model_policy:
  generate_prd:
    connection: vertex-production
    model: gemini-3.5-pro
  generate_pr_description:
    connection: vertex-production
    model: gemini-3.5-flash
  implement-task:
    connection: anthropic-production
    model: claude-sonnet-4-6

Model targets may also define validated provider-neutral options such as temperature, token/output limit, or required capabilities where supported. Unsupported options must fail validation rather than being silently ignored.

Resolution and precedence

Resolve model policy in this order:

  1. Project-specific node/skill override.
  2. Global node/skill mapping.
  3. Global default model target.

Project overrides may select only connections and models allowed by administrator policy. Missing mappings use the global default. Invalid connection, backend/model mismatch, unavailable model, or unmet capability requirement should fail configuration validation with an actionable error.

The resolved non-secret policy should be pinned when a workflow starts so configuration changes do not silently change the model used by an in-flight workflow. Retry behavior must define whether an unavailable pinned model fails closed or uses an explicitly configured fallback; implicit fallback is not allowed.

Credential boundary

Model selection and credential management are separate concerns:

  • Policy references a named connection and credential reference; it never contains a token, API key, service-account key, or raw credential value.
  • Jira project properties may choose from allowlisted model targets but must not store credential material.
  • Workflow state, checkpoints, prompts, task files, Langfuse metadata, and logs contain only non-secret connection/model identifiers.
  • Credentials are resolved by the worker or credential broker after policy evaluation and outside the model context.
  • Container execution receives only the minimum credential material required for the selected target, using short-lived/scoped credentials where supported.
  • Authorization must prevent a project override from selecting another project's connection or credential scope.
  • Errors and tracing must redact provider responses or configuration values that could expose secrets.

This feature should integrate with the short-lived credential direction in #82 rather than adding a new secret-distribution mechanism.

Runtime integration

  • Apply policy consistently to orchestrator-hosted agents and sandbox/container agents.
  • Resolve by stable node and skill identifiers, not display names.
  • Carry the selected model target through task execution without exposing credentials.
  • Record backend, connection identifier, exact model, node/skill, policy source, latency, tokens, and cost in observability metadata.
  • Provide a command or validation mode that prints the resolved non-secret policy for every known node/skill.

Acceptance criteria

  • Administrators can define provider-neutral named connections and exact model targets.
  • Global and project-specific mappings select models by stable node or skill identity.
  • No fixed fast/standard/advanced tiers or provider-specific hierarchy is required.
  • Resolution follows project override → global mapping → global default.
  • Project overrides are restricted to allowlisted connections/models.
  • Backend/model/capability/option compatibility is validated before execution.
  • Resolved policy is pinned for in-flight workflow consistency.
  • Host and container agent paths use the same resolution contract.
  • Credentials never enter Jira properties, policy payloads, workflow state, prompts, task artifacts, traces, or logs.
  • Credential delivery uses the selected connection's existing secure path and supports short-lived credentials.
  • Langfuse identifies the resolved backend, connection, model, and policy source for each invocation.
  • Tests cover precedence, unknown nodes, invalid models/options, unauthorized connections, pinned policy, host/container parity, redaction, and explicit fallback behavior.

Context

Converted from proposal PR #24 after review. This issue replaces Claude-shaped model tiers with exact provider-neutral node/skill policy and adds explicit credential isolation.

Related: #82 (short-lived credentials), PR #102 (provider-neutral backend configuration), and roadmap PR #157 (per-node model policy and credential boundaries).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:agentsAgents, prompts, models, skills, and generated artifactsarea:integrationsJira and other external service integrationsenhancementNew feature or requestmajorLarge or cross-cutting effort requiring coordinated design and implementationperformancePerformance, latency, cost, or resource-efficiency improvementssecurity

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions