Skip to content

Support duplicate handling for Appwrite imports#205

Open
premtsd-code wants to merge 11 commits into
mainfrom
skip-override-migration
Open

Support duplicate handling for Appwrite imports#205
premtsd-code wants to merge 11 commits into
mainfrom
skip-override-migration

Conversation

@premtsd-code

@premtsd-code premtsd-code commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add Appwrite duplicate handling modes for skip and overwrite across imported resources
  • carry createdAt and updatedAt through Appwrite source resources so overwrite only updates when the source is newer
  • preserve fail-mode create behavior as the default path
  • overwrite SDK-backed resources in place where supported, match natural-key resources such as memberships/platforms/rules, and replace existing files by deleting only the matching destination file ID before re-upload
  • keep deployment overwrite conservative: existing function/site deployments are skipped instead of deleted and recreated

Validation

  • composer format
  • composer lint
  • composer check
  • live source-to-destination fail-mode migration
  • live skip-mode migration with source-only resources and existing destination conflicts
  • live overwrite-mode migration with matching resources, source-only resources, and destination-only sentinels
  • live file overwrite check confirmed matching file content is replaced, source-only file is created, and unrelated destination file is preserved

Review Notes

  • overwrite only applies when the source updatedAt is newer than destination updatedAt; otherwise the resource is skipped
  • file overwrite deletes only the matching file ID in the matching bucket, then reuses the existing upload path
  • deployment overwrite remains skipped because safely replacing deployment artifacts needs a separate delete-and-reupload recovery path

@greptile-apps

greptile-apps Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds skip and overwrite duplicate-handling modes to the Appwrite-to-Appwrite migration path. It propagates createdAt/updatedAt timestamps through all resource models so that overwrite can be gated on "source is newer than destination," and adds per-resource SDK or DB lookups before each create operation.

  • Duplicate handling: a new handleDuplicate helper centralises the skip/overwrite/create routing; every resource type (buckets, files, users, teams, memberships, functions, env vars, deployments, sites, messaging providers/topics/subscribers/messages, webhooks, platforms, API keys, and proxy rules) now has a corresponding natural-key lookup and conditional update or skip path.
  • Source timestamp propagation: createdAt/updatedAt are wired through all 11 resource model constructors and carried from the Appwrite source API responses; the Range header capitalisation and the last-chunk content-range end-byte are also fixed as part of the same pass.
  • Chunked upload continuity: a skippedChunkedUploads map guards subsequent chunks of a skipped file or deployment from attempting a partial upload, and a probe request is added to determine deployment size when the server omits a content-length header.

Confidence Score: 4/5

The logic for skip and overwrite is sound across most resource types; existing open threads on the file/deployment delete-then-upload window and the file overwrite timestamp check are the main remaining concerns before merging.

The duplicate-handling plumbing is well-structured and the live-migration validation described in the PR gives reasonable confidence. Two open issues from the previous review round have not been closed: file overwrite always replaces regardless of whether the source is actually newer, and the delete-before-upload window leaves a destination file absent if the upload fails mid-stream.

src/Migration/Destinations/Appwrite.php: the importFile overwrite path and importDeploymentWithDuplicateCheck both warrant a second look before merge.

Important Files Changed

Filename Overview
src/Migration/Destinations/Appwrite.php Core of the PR: adds handleDuplicate/getSdkResourceOrNull helpers plus per-resource skip/overwrite logic. Several pre-existing issues remain open in review threads (non-atomic file delete, file overwrite skipping timestamp check); the new provider/topic/subscriber/message duplicate handling is logically correct with timestamps already populated from source.
src/Migration/Sources/Appwrite.php Adds createdAt/updatedAt propagation for all migrated resource types; fixes lowercase 'range' to 'Range' header; introduces firstChunkData probe for deployments missing content-length. The probe's open-ended range could over-fetch for large deployments.
src/Migration/Resources/Auth/User.php Adds optional createdAt/updatedAt constructor params and round-trips them through fromArray/jsonSerialize; no logic changes.
src/Migration/Resources/Functions/Deployment.php Adds optional createdAt/updatedAt constructor params; no logic changes.
src/Migration/Resources/Storage/File.php Adds optional createdAt/updatedAt constructor params; no logic changes.

Reviews (11): Last reviewed commit: "Respect timestamps for binary overwrites" | Re-trigger Greptile

Comment thread src/Migration/Destinations/Appwrite.php Outdated
Comment thread src/Migration/Destinations/Appwrite.php
Comment thread src/Migration/Resources/Backups/Policy.php
Comment thread src/Migration/Destinations/Appwrite.php
Comment thread src/Migration/Destinations/Appwrite.php Outdated
Comment thread src/Migration/Destinations/Appwrite.php
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant