UN-3479 [FEAT] CLI restructure with user-group cloning + share-state replication

[chandrasekharan-zipstack]

Summary

• CLI restructure: unstract Click group with clone as first subcommand; click/rich moved from optional extra to core (fixes broken-by-default ImportError)
• User-group cloning: new group phase (runs first) clones org user groups by name with reuse-on-match; optional --clone-group-members flag (email-matched, missing users skipped and reported)
• Share-state replication: after each resource create, its sharing state (users by email / groups by name / org-wide flag) is applied via the share endpoint; non-fatal warnings channel on report
• Service account detection: via backend's is_service_account field with email-suffix fallback for older backends

Test Plan

• Unit tests: uv run pytest → 184 passed (167 pre-existing + 17 new)
• E2E on GKE dev namespace: full clone of seeded org (3 groups incl. empty, memberships, group/user/org shares) verified via API on target
• Dry-run: full plan incl. would-skip warnings with zero writes
• Rerun: adopts instead of duplicates

Dependencies

• Companion backend PR: Zipstack/unstract feat/org-migration-platform-api-gaps (service-account group management)
• Older backends: group phase fails cleanly (403s), everything else clones

README

• Added concise "Unstract CLI" section describing unstract clone usage and optional --clone-group-members flag

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

This PR restructures the unstract CLI into a top-level Click group with clone as its first subcommand, promotes click/rich to core dependencies, and adds two new capabilities: user-group cloning (GroupPhase) and post-create share-state replication (sharing.py).

• GroupPhase (runs first) clones org groups by name with idempotent adopt-on-match; optional --clone-group-members email-matches members to target users and reports misses as warnings.
• Share-state replication is wired into every shareable phase via a new apply_share base helper: users are mapped by email, groups through the group remap table, and the org-wide flag is copied as-is; failures are non-fatal and surfaced in a new warnings section of the report.
• _cached in sharing.py correctly prefers a real result over a previously committed _FETCH_FAILED sentinel when two threads race on the same key (addresses prior review thread).

Confidence Score: 4/5

Safe to merge with one fix recommended: a group that is successfully created but whose immediate re-listing throws will have its remap left unrecorded, silently dropping group-share entries for any resources that reference it.

The create/re-list try block in _create_group spans two independent operations. A transient failure on the list_groups() call after a successful create_group() POST causes the group remap to be omitted; share-state replication for every resource that names this group then silently skips its group-share axis. The rest of the changes — _cached race fix, share-payload construction, member cloning, CLI restructure — are well-tested and correct.

src/unstract/clone/phases/group.py — _create_group wraps the POST and the re-listing call in a single try/except.

Important Files Changed

Filename	Overview
src/unstract/clone/phases/group.py	New phase cloning org groups by name with email-based member migration; a try/except that spans both the POST and the re-list can mask a successful create and leave the remap unrecorded.
src/unstract/clone/sharing.py	New module replicating share state post-create; race condition in _cached addressed, payload construction and dry-run handling look correct.
src/unstract/clone/client.py	New org users/groups/share API methods; list_group_members still lacks the or {} null guard present in list_users (prior thread).
src/unstract/clone/phases/base.py	Adds share_path_template class attribute and apply_share helper that delegates to sharing.apply_share_state; clean delegation pattern.
src/unstract/clone/phases/custom_tool.py	Adds share-state replication and moves source registry check before the export gate to skip republishing empty/never-exported tools; logic is sound.
src/unstract/clone/orchestrator.py	Registers GroupPhase as the first phase so group remaps are available for share-state replication in all later phases.
src/unstract/clone/report.py	Adds warnings list to PhaseResult and a _render_warnings_summary section to the rich/plain/JSON report renderers.
src/unstract/clone/context.py	Adds clone_group_members option, share_cache dict, and share_cache_lock to CloneContext; thread-safe per-run memoisation of user/group listings.
src/unstract/cli.py	New top-level unstract Click group that registers clone as a subcommand; keeps python -m unstract.clone working via the existing local group.
pyproject.toml	Promotes click and rich from optional [clone] extra to core dependencies and renames the console script from unstract-clone to unstract.
tests/clone/test_sharing.py	Comprehensive tests covering payload building, cache races, dry-run, group-phase exclusion, and POST failures; good coverage.
tests/clone/test_group_phase.py	Tests for group creation, idempotent adoption, dry-run, member cloning, and service-account skipping; all key paths covered.

Sequence Diagram

sequenceDiagram
    participant CLI as unstract clone
    participant GP as GroupPhase
    participant AP as Resource Phase
    participant SH as sharing.py
    participant SRC as Source API
    participant TGT as Target API

    CLI->>GP: run()
    GP->>SRC: list_groups()
    GP->>TGT: list_groups()
    loop each source group
        alt group exists on target
            GP->>GP: adopt + record remap
        else
            GP->>TGT: create_group()
            GP->>TGT: list_groups() re-list for id
            GP->>GP: record remap(src_id to tgt_id)
        end
        opt --clone-group-members
            GP->>SRC: list_group_members()
            GP->>SH: target_user_id_by_email()
            SH->>TGT: list_users() cached
            GP->>TGT: add_group_members()
        end
    end
    CLI->>AP: run()
    loop each resource
        AP->>TGT: create resource
        AP->>SH: apply_share_state()
        opt share axes missing
            SH->>SRC: get_detail() src_detail_fn
        end
        SH->>SRC: list_users() cached
        SH->>SH: remap.resolve group ids
        SH->>TGT: share_resource(path, payload)
    end

Loading

_{Reviews (2): Last reviewed commit: "UN-3479 [FIX] Address review: prefer suc..." | Re-trigger Greptile}