Skip to content

feat: support cloud filesystem model paths - #657

Open
goutamadwant wants to merge 1 commit into
sigstore:mainfrom
goutamadwant:support-cloud-filesystems-148
Open

feat: support cloud filesystem model paths#657
goutamadwant wants to merge 1 commit into
sigstore:mainfrom
goutamadwant:support-cloud-filesystems-148

Conversation

@goutamadwant

Copy link
Copy Markdown

Summary

Closes #148.

This change allows model hashing, signing, verification, and digest operations to read model artifacts from GCS and S3 URIs without staging the complete model on local disk.

The implementation:

  • introduces a small URI-aware filesystem boundary using etils.epath;
  • preserves existing local pathlib behavior;
  • traverses remote directories without recursive glob assumptions;
  • streams normal and BLAKE3 hashing in bounded chunks;
  • uses seek/range reads for sharded hashing;
  • adds optional gcs and s3 provider extras;
  • keeps keys, certificates, trust configuration, and signature bundles local;
  • documents provider setup and the tested support boundary.

The cloud path is covered with an isolated in-memory gs:// backend, including local/remote manifest equivalence, bounded reads, relative ignore paths, signing and verification, tamper detection, digest CLI behavior, BLAKE3, and sharded hashing.

Validation completed:

  • Python 3.10-3.14: 208 tests passed per version, 1,040 total
  • cloud-specific tests: 8 passed
  • Ruff formatting and lint: passed
  • Pytype: 54 sources checked with no errors
  • wheel and source distribution builds: passed
  • package metadata and clean archive validation: passed
  • local CLI key/certificate and historical fixture tests: passed
  • git diff --check: passed

Live cloud credentials were not exercised. External Fulcio signing was unavailable in the test environment because of a TLS WRONG_VERSION_NUMBER failure, and PKCS#11 integration requires SoftHSM, which was not installed.

Checklist
  • All commits are signed-off, using DCO
  • All new code has docstrings and type annotations
  • New cloud filesystem behavior is covered by focused tests
  • Public-facing changes are paired with documentation changes
  • A release note has been added to CHANGELOG.md

Signed-off-by: Goutam Adwant <8672451+goutamadwant@users.noreply.github.com>
@goutamadwant
goutamadwant requested review from a team as code owners August 18, 2026 03:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support cloud filesystems

1 participant