test(model-onboarding): four payloads per path, Studio Web coded-copy agents#1009
Open
denispetre wants to merge 1 commit into
Open
test(model-onboarding): four payloads per path, Studio Web coded-copy agents#1009denispetre wants to merge 1 commit into
denispetre wants to merge 1 commit into
Conversation
… agents Every code path now runs four capability payloads, rolled into the single success boolean: - simple: plain ainvoke, non-empty completion - tools: full tool-calling round trip (get_weather; executes the tool and asserts the final answer uses the result) - files/<name>: the coded file-processing agent — Studio Web's "Clone as Coded Agent" of the low-code FileProcessingAgent — run through the production ReAct stack (create_agent + Analyze Files internal tool) with the test file uploaded as a real platform attachment - is_tools: call-only Activity-tool check — bind the coded copy's real IS tools (Slack Send Message to Channel, Outlook 365 Send Email) and assert the model produces a well-formed tool call (known bound name + required args). Tools are never executed; nothing is sent; no connections needed. Agents live under src/agents/, one subfolder per coded agent. Each agent.py is the Studio Web generated main.py with exactly one structural change: the hardcoded llm becomes build_graph(llm, ...)/build_tools(...) so the model built from model_spec is injected — model + API flavor stay configurable. The vertex path pins vendor_type + api_flavor=GENERATE_CONTENT explicitly. utils.py is the generated interpolation helper, verbatim. Verified: graph compiles, input.json validates, both agents construct, the call-only assertion covers well-formed/missing-args/unknown-tool/no-call, and `uipath init` loads the graph. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The model-onboarding testcase now runs four capability payloads for every
get_chat_modelcode path, all rolled into the singlesuccessboolean:simpleainvoke→ non-empty completiontoolsget_weather): the tool is executed and the final answer must use its resultfiles/<name>create_agent+ Analyze Files internal tool), test file uploaded as a real platform attachmentis_toolschannel+messageToSend/message.toRecipients). Tools are never executed, nothing is sent, no connections neededDesign
src/agents/, one subfolder per coded agent, documented insrc/agents/README.md.agent.pyis the Studio Web generatedmain.py(downloaded viauip agent file get) with exactly one structural change: the hardcodedllm = get_chat_model('gpt-5.4', ...)becomesbuild_graph(llm, ...)/build_tools(...)so the testcase injects the model built frommodel_spec— model + API flavor stay configurable, and therun.sh/input.json/ dispatch-workflow contract is unchanged.llm_gateway_automated_testing— FileProcessingAgent (solution1823d2d6-…, codede7b8598c-…), IsToolsAgent (solution925fc5db-…, coded0959f23b-…).vertexpath now pinsvendor_type+api_flavor=GENERATE_CONTENTexplicitly (was autodetect-only).Verification
Graph compiles,
input.jsonvalidates, both agents construct (integration tools under stub auth —create_integration_toolbuildsUiPath()at tool-creation time), the call-only assertion was exercised across all outcomes (well-formed ✓ with tool names, missing-args ✗, unknown-tool ✗, no-call ✗), anduipath initloads the graph. Live LLM cells run in CI / the on-demandmodel_onboarding.ymldispatch.Supersedes #1007 (same end state, without the intermediate churn and provenance-copy files).
🤖 Generated with Claude Code