Synchronizer: Add local docker-compose harness for manual sync testing - #257
Open
saidixith002 wants to merge 1 commit into
Open
Synchronizer: Add local docker-compose harness for manual sync testing#257saidixith002 wants to merge 1 commit into
saidixith002 wants to merge 1 commit into
Conversation
Brings up a source and target Apache Polaris instance with shared FILE storage so sync-polaris (including --skip-iceberg-content from PR apache#256) can be exercised end-to-end without a real Polaris deployment. Documents the flag and links to the harness from the main README.
dimas-b
reviewed
Jul 23, 2026
dimas-b
left a comment
dimas-b
left a comment
There was a problem hiding this comment.
Thanks for continuing your contributions, @saidixith002 !
| - polaris-net | ||
|
|
||
| polaris-target: | ||
| image: apache/polaris:1.1.0-incubating |
| cd polaris-synchronizer/manual-test | ||
| docker compose up --build -d | ||
| docker compose ps # wait for polaris-init to show "Exited (0)" | ||
| cat seed/credentials.json |
There was a problem hiding this comment.
Why advise users to dump credentials to STDOUT?
|
|
||
| # ── SOURCE: principal role ──────────────────────────────────────────────────── | ||
|
|
||
| mgmt_post "$SOURCE" "$SRC_TOKEN" "/principal-roles" '{"principalRole": {"name": "analyst-role"}}' |
There was a problem hiding this comment.
How about using the setup feature of Polaris CLI?
https://polaris.apache.org/blog/2026/03/29/introducing-the-setup-command-in-apache-polaris/
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
polaris-synchronizer/manual-test/: a self-contained docker-compose setup that brings up a source and target Apache Polaris instance (in-memory metastore, noPostgres/S3 needed) to manually exercise
sync-polarisend-to-end.sync and a
--skip-iceberg-contentsync be run back-to-back in one session and compared.--skip-iceberg-contentin the mainpolaris-synchronizer/README.md(previously undocumented there) and links to the new harness under a "Testing Locally"section.
seed/credentials.json(bind-mounted volume, persists acrossdocker compose down).Related
Depends on / references #256. The manual-test walkthrough documents and exercises
--skip-iceberg-content, which is only implemented on that branch — until#256 merges, following the harness's
--skip-iceberg-contentsteps against a jar built frommainwon't work, since the flag doesn't exist yet onmain. Everything else(docker-compose harness, seeding, full-sync steps) works independently of #256.
Test plan
docker compose up --build -dfrompolaris-synchronizer/manual-test/— source, target, andpolaris-initall come up healthy.create-omnipotent-principalagainst source (read-only) and target (--write-access).sync-polariswith--skip-iceberg-content— catalog/catalog-role/grant sync succeeds, target namespaces empty.sync-polariswithout the flag — namespace/table sync succeeds (shared storage), target ends up withtest_ns/test_table.curlagainst target's management/catalog APIs at each step.