fix: Mitigate render-drift for archive-overlays#257
Draft
Tonisal-byte wants to merge 13 commits into
Draft
Conversation
added 13 commits
June 26, 2026 17:07
Add three new overlay types (tarball-file-remove, tarball-search-replace, tarball-patch) that modify files inside source tarballs during source preparation. Operations are performed in pure Go on the host. Includes: - internal/utils/tarball: reusable deterministic tar extract/repack library - Overlay type registration, validation, and fingerprinting - Source prep integration with sources file hash rehashing - User documentation and TOML examples
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces first-class support for archive-scoped overlays (overlays that mutate files inside a source tarball) and adds a mechanism to stabilize render --check-only by allowing configs to record the post-overlay archive hash via a new origin.type = "overlay" source-files entry.
Changes:
- Add archive overlay processing (extract → apply overlays → deterministic repack) and refresh affected
sourcesentries by rehashing repacked archives. - Add
origin.type = "overlay"tosource-filesto record expected post-overlay hashes and avoid render drift when sources aren’t downloaded. - Harden archive extraction: sniff real compression from magic bytes, ignore PAX headers, and optionally fail on unsupported tar entry types to prevent silent data loss on repack.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| schemas/azldev.schema.json | Extends schema enum to allow "overlay" as an origin type. |
| internal/utils/archive/archive.go | Adds compression sniffing, extraction options, and convenience helpers for archive detection/extraction. |
| internal/utils/archive/archive_test.go | Adds coverage for magic-byte sniffing, mislabeled compression, and unsupported tar entry handling. |
| internal/providers/sourceproviders/sourcemanager.go | Skips downloading origin.type=overlay entries and guards against accidental download attempts. |
| internal/projectconfig/overlay.go | Adds archive-scoped overlay path detection/validation and separates “archive” vs “loose file” overlay behavior. |
| internal/projectconfig/overlay_test.go | Tests archive-scoped path validation and Modifies* classification. |
| internal/projectconfig/fingerprint_test.go | Improves fingerprint tag validation by parsing name,options and constraining allowed options. |
| internal/projectconfig/configfile.go | Validates overlay-origin source-files invariants (hash/hash-type required, replace-upstream enforced, uri forbidden). |
| internal/projectconfig/component.go | Introduces OriginTypeOverlay and updates jsonschema tags accordingly. |
| internal/fingerprint/fingerprint_test.go | Adds regression test ensuring archive scoping impacts component fingerprinting. |
| internal/app/azldev/core/sources/sourceprep.go | Adds archive overlay application phase and rehashes sources entries for repacked archives; validates overlay hash expectations. |
| internal/app/azldev/core/sources/sourceprep_test.go | Adds end-to-end tests for repack+rehash behavior and missing sources-entry erroring. |
| internal/app/azldev/core/sources/overlays.go | Updates overlay application to use the new “loose files” predicate. |
| internal/app/azldev/core/sources/archiveoverlays.go | Implements archive overlay grouping, extraction-root resolution, and atomic repack preserving real compression. |
| internal/app/azldev/core/sources/archiveoverlays_test.go | Adds exported-path integration tests for archive overlay semantics (glob removal, search/replace, skip-sources behavior). |
| internal/app/azldev/core/sources/archiveoverlays_internal_test.go | Adds internal unit/regression tests for grouping, extract-root resolution, strict extraction, and repack correctness. |
| internal/app/azldev/cmds/component/preparesources.go | Refactors conditional preparer option wiring into a helper. |
| docs/user/reference/config/overlays.md | Documents archive overlays (path scoping, extraction root, supported entry types, batching). |
| docs/user/reference/config/components.md | Documents origin.type="overlay" and recommended workflow for recording post-overlay hashes. |
Comment on lines
+14
to
+17
| // Only regular files, directories, and symlinks are extracted. Other entry | ||
| // types (hardlinks, devices, FIFOs, etc.) are skipped by default, or cause a | ||
| // failure when [WithErrorOnUnsupportedEntry] is set — callers that repack the | ||
| // tree must set it so such entries aren't silently dropped. |
Comment on lines
+370
to
+373
| 1. Add the archive overlay(s) in the component's `[[overlays]]` array. | ||
| 2. Run `prep-sources` once — this repacks the archive and prints the computed hash in the error message. | ||
| 3. Paste the computed `hash` and `hash-type` into the `source-files` entry above. | ||
| 4. Run `prep-sources` again to confirm the hash matches, then commit. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We need a way to track the resulting hash from the sum of archive overlays applied to a source. This is a blocking issue for adding archive-overlays to upstream; otherwise the render checks fail from having a stale
sourcesfile pre-archive-overlay.Example of new origin-stub: