Skip to content

RTECO-1651 - Update agent plugins tests - #3618

Open
udaykb2 wants to merge 13 commits into
masterfrom
RTECO-1651-update-plugins-e2e-tests
Open

RTECO-1651 - Update agent plugins tests#3618
udaykb2 wants to merge 13 commits into
masterfrom
RTECO-1651-update-plugins-e2e-tests

Conversation

@udaykb2

@udaykb2 udaykb2 commented Jul 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor
  • All tests have passed. If this feature is not already covered by the tests, new tests have been added.
  • The pull request is targeting the master branch.
  • The code has been validated to compile successfully by running go vet ./....
  • The code has been formatted properly using go fmt ./....

Summary

Expands and hardens jf agent plugins e2e coverage in agent_plugins_test.go (85 → 99 tests), and fixes the agent-plugins CI workflow so tests run reliably on Linux/Windows.

What the suite covers

Area Focus
Publish Basic publish, version override, collisions (CI), invalid semver/slug, wrong/missing repo, prebuilt zip, build-info/module/env vars, signing key, checksum/agentpackages.digest, multi-harness marketplace indexing
Install Latest / specific version, marketplace (with --harness), global/path scopes, multi-harness (claude/codex/cursor), plugin-info.json, evidence gate, JSON format, flag mutual exclusion, agent-config override, custom my-agent lifecycle
Update Per-slug, --all, dry-run, force, non-interactive, JSON format, project-scope rejection, flag matrix
List Local/remote, latest-only, empty local, --check-updates statuses, limit (harness/repo), flag validation
Search Exact/substring, latest-only, no matches, empty/blank query, repo from env
Delete Delete, dry-run (incl. multi-version / not found), version-specific, missing args, repo from env
Repo / server / proxy Env vs --repo, no-repo error, --server-id, unreachable Artifactory, proxy / no-proxy, insecure TLS
Round-trips / CI Publish→install→update→delete flows, CI pipeline path

Notable test improvements

  • Stronger error assertions (assertErrorContainsAll instead of loose OR matches)
  • Checksum / Artifactory property verification
  • Marketplace install prefers --harness and retries until marketplace indexes
  • Custom agent install/update/list/delete lifecycle e2e
  • Shared harness matrix for claude / codex / cursor (and multi-harness)

CI / workflow

  • agentPluginsTests.yml: use -test.agentPlugins=true (Windows-safe), force bash shell
  • Stop pinning Artifactory VERSION (avoids pruned release breakage)
  • Gosec: suppress G703 on test stub binary write under t.TempDir

@udaykb2
udaykb2 force-pushed the RTECO-1651-update-plugins-e2e-tests branch from 08b7669 to 3d2cedd Compare July 27, 2026 08:52
@udaykb2
udaykb2 force-pushed the RTECO-1651-update-plugins-e2e-tests branch from 3d2cedd to 3cce295 Compare July 27, 2026 09:23
@udaykb2
udaykb2 force-pushed the RTECO-1651-update-plugins-e2e-tests branch from 3cce295 to 84a71f9 Compare July 29, 2026 11:36
@udaykb2
udaykb2 force-pushed the RTECO-1651-update-plugins-e2e-tests branch from 84a71f9 to c7096cd Compare July 29, 2026 12:11
@udaykb2
udaykb2 force-pushed the RTECO-1651-update-plugins-e2e-tests branch from 9ed0455 to ae14475 Compare July 29, 2026 18:21
@udaykb2
udaykb2 force-pushed the RTECO-1651-update-plugins-e2e-tests branch from ae14475 to 7a367be Compare July 29, 2026 18:30
@udaykb2 @cursoragent
Fix agent plugins test workflow flag format
e43d891 
Changed -test.agentPlugins (incorrect format) to -test.agentPlugins=true
(correct format). Go test flags use single dash prefix with =value format.

The test was being skipped before because the flag was not properly parsed,
causing the exit code 1 failure in the workflow.

Co-authored-by: Cursor <cursoragent@cursor.com>
@udaykb2
udaykb2 force-pushed the RTECO-1651-update-plugins-e2e-tests branch from 7a367be to e43d891 Compare July 29, 2026 18:32
udaykb2 and others added 2 commits July 30, 2026 11:20
@udaykb2 @cursoragent
Fix agent plugins tests: JSON parsing and repo config errors
27b1d1c 
1. Fix TestAgentPluginsInstallSpecificVersion JSON parsing error:
   - Changed manifest type from map[string]string to map[string]any
   - This allows parsing the 'skills' array field added to plugin.json

2. Fix TestAgentPluginsNoRepoConfigured repo auto-discovery:
   - Test was expecting failure when no repo configured
   - But repo auto-discovery now finds matching agentplugins repos
   - Updated test to verify error handling for non-existent repos instead

Co-authored-by: Cursor <cursoragent@cursor.com>
@udaykb2 @cursoragent
Fix TestAgentPluginsInstallAgentConfigOverride - test default global …
9990dab 
…scope

The install command defaults to --global scope when neither --global nor
--project-dir is specified. Update the test to verify that explicit
--project-dir flag works correctly instead of assuming a different default.

This way we test production code as-is rather than changing production code
to match incorrect test expectations.

Co-authored-by: Cursor <cursoragent@cursor.com>
@udaykb2
udaykb2 force-pushed the RTECO-1651-update-plugins-e2e-tests branch from f3ec26a to 9990dab Compare July 30, 2026 05:53
@udaykb2 @cursoragent
Fix Windows test flag parsing in agent plugins workflow
11c0d87 
Explicitly use bash shell for the test command to prevent PowerShell from
interpreting the -test flag prefix. This fixes the 'flag provided but not
defined: -test' error on Windows runners.

Co-authored-by: Cursor <cursoragent@cursor.com>
udaykb2 and others added 3 commits July 30, 2026 16:55
@udaykb2 @cursoragent
Tighten agent plugins e2e error assertions and checksum coverage
17ba92f 
Replace loose OR-based error string checks with assertErrorContainsAll
matched to production messages. Rewrite NoRepoConfigured to delete the
agentplugins repo and assert ResolveRepo discovery failure. Require md5,
sha1, and sha256 on checksum tests. Document marketplace install without
marketplace slug syntax and keep multi-harness paths for cursor, claude,
and codex.

Co-authored-by: Cursor <cursoragent@cursor.com>
@udaykb2 @cursoragent
Strengthen agent plugins search and delete e2e coverage
99e7d89 
Search now captures JSON stdout and asserts name/version/repo, substring
matching, latest-version-only rows, blank-query rejection, env-repo
resolution, and the no-match log message. Delete asserts exact production
errors, covers missing slug/version/env-repo paths, and verifies a missing
version of an existing plugin fails without deleting the kept version.

Co-authored-by: Cursor <cursoragent@cursor.com>
@udaykb2 @cursoragent
Strengthen agent plugins list e2e coverage
dbb371a 
Assert list JSON rows for remote/local modes, limits, and exact flag/usage
errors instead of smoke-only success checks.

Co-authored-by: Cursor <cursoragent@cursor.com>
@udaykb2 @cursoragent
Tighten agent plugins e2e coverage after production review
3e7a6e2 
Remove duplicate install/list/search smoke tests, assert JSON summaries
and manifest fields, cover project-scope rejection for list/update, env
repo resolution for install/update, and marketplace slug-not-listed errors.

Co-authored-by: Cursor <cursoragent@cursor.com>
@udaykb2
udaykb2 force-pushed the RTECO-1651-update-plugins-e2e-tests branch from f3d7e64 to c03e8c0 Compare July 30, 2026 12:01
@udaykb2 @cursoragent
Add custom my-agent lifecycle e2e for plugins
6857fff 
Cover install, list, check-updates, and update for an agent-config.json
custom agent as a fifth harness case beyond the built-in four.

Also fix three CI assertion mismatches: search no-match log buffer,
update of an uninstalled plugin error text, and ValidateSemver
rejection messages (now table-driven across patch/major/segment cases).

Co-authored-by: Cursor <cursoragent@cursor.com>
@udaykb2
udaykb2 force-pushed the RTECO-1651-update-plugins-e2e-tests branch from c03e8c0 to 6857fff Compare July 30, 2026 13:23
@udaykb2
udaykb2 force-pushed the RTECO-1651-update-plugins-e2e-tests branch from 7784be1 to 1e729b2 Compare July 30, 2026 15:22
@udaykb2 @cursoragent
Prefer marketplace install with --harness
1769983 
Omit --version on harness installs unless a specific older version must be
pinned for update/check-updates scenarios. Custom agents still pass
--version because they are not indexed into marketplace.json.

Custom agent lifecycle covers install, list, and update end-to-end.
Also bump jfrog-cli-artifactory to 659839816e66.

Co-authored-by: Cursor <cursoragent@cursor.com>
@udaykb2
udaykb2 force-pushed the RTECO-1651-update-plugins-e2e-tests branch from 1e729b2 to 1769983 Compare July 30, 2026 15:29
@udaykb2
udaykb2 marked this pull request as ready for review July 31, 2026 04:37
@udaykb2
udaykb2 requested review from a team, agrasth, bhanurp, fluxxBot, itsmeleela and naveenku-jfrog and removed request for a team July 31, 2026 04:37
@udaykb2 @cursoragent
Suppress gosec G703 on test stub binary write
9e5d37c 
Co-authored-by: Cursor <cursoragent@cursor.com>
@udaykb2
udaykb2 deployed to build-gate July 31, 2026 04:40 — with GitHub Actions Active
@github-actions

github-actions Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

👍 Frogbot scanned this pull request and did not find any new security issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@itsmeleela itsmeleela Awaiting requested review from itsmeleela itsmeleela was automatically assigned from jfrog/ecomatrix
@bhanurp bhanurp Awaiting requested review from bhanurp bhanurp was automatically assigned from jfrog/ecomatrix
@agrasth agrasth Awaiting requested review from agrasth agrasth was automatically assigned from jfrog/ecomatrix
@fluxxBot fluxxBot Awaiting requested review from fluxxBot fluxxBot was automatically assigned from jfrog/ecomatrix
@naveenku-jfrog naveenku-jfrog Awaiting requested review from naveenku-jfrog naveenku-jfrog was automatically assigned from jfrog/ecomatrix

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@udaykb2