feat(ddx-datafusion): make the adapter publishable - #62
Merged
Conversation
Everything crates.io needs, which was more than dropping `publish = false`. - **The workspace `ddx-core` dependency had no version.** cargo uses the path for workspace builds and the version for what it writes into a published dependent's manifest, and refuses to publish a crate whose path dependency lacks one. This is why the adapter could never have been published by flipping a flag. - **A real version.** `version.workspace = true` resolved to the 0.0.0 placeholder. Now 0.1.0, versioned independently of ddx-core: the two publish on their own cadence, and an adapter forced to move whenever the engine did would mean releases for changes that do not touch it. - **README, keywords, categories.** The crate had no README at all — the page crates.io and docs.rs show would have been empty. It covers the two routes, when to reach for each, where they disagree about what may be the `wrt`, and the sqlparser-unification constraint that decides which datafusion versions can work. - **A changelog** with a hand-written first entry, matching how ddx-core 0.1.0 was documented. - **A publish dry-run in CI**, so packaging breakage surfaces on a pull request rather than when a release first tries to upload. Verified by `cargo publish --dry-run`: packages 19 files, then downloads ddx-core from crates.io and compiles the adapter against the *published* crate rather than the local path — which is what a consumer actually gets. Not yet ready to release: that dry-run resolved ddx-core 0.2.0, which returns a zero gradient for a NULL row under `grad(abs(u), x)`. The floor must be tightened to 0.2.1 once that release lands, so a consumer pinned to 0.2.0 gets a resolution error instead of a silently wrong gradient. 153 tests, clippy and fmt clean.
The dry-run previously resolved 0.2.0, which returns a zero gradient for a NULL row under grad(abs(u), x). With the floor at 0.2.1 a consumer pinned to 0.2.0 gets a resolution error rather than that gradient. Verified: cargo publish --dry-run now downloads ddx-core v0.2.1 and compiles the adapter against it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes
ddx-datafusionpublishable to crates.io. Part of the M2 release, alongsideddxdb→ PyPI.This was more than dropping
publish = falseThe blocker was one line up, in the workspace manifest:
cargo uses the
pathfor workspace builds and theversionfor what it writes into a published dependent's manifest — and it refuses to publish any crate whose path dependency has none. So the adapter could never have been released by flipping its own flag; the fix belongs at the workspace level, where it also unblocksddx-adlater without rediscovering the same thing.The rest of what crates.io needs and the crate lacked:
version.workspace = trueresolved to the0.0.0placeholder. Now0.1.0, versioned independently ofddx-core— the two publish on their own cadence, and an adapter forced to move whenever the engine did would mean releases for changes that never touched it.wrt, and thesqlparser-unification constraint that decides whichdatafusionversions can possibly work.keywords/categories/readmemetadata, matchingddx-core.ddx-core0.1.0 used.Verified against the published crate, not the local path
That resolution is the point of the exercise.
cargo publish --dry-runbuilds the packaged crate against what a consumer would actually download, so this proves the manifest works outside the workspace — the thing acargo testrun can never tell you.The dependency floor is
0.2.1, deliberatelyAn earlier dry-run on this branch resolved
ddx-core0.2.0, which returns a zero gradient for a NULL row undergrad(abs(u), x)— a silently wrong result. Requiring0.2.1means a consumer pinned to 0.2.0 gets a resolution error rather than that gradient."0.2"would have compiled and been quietly worse.Not included
ddx-adstayspublish = false. It is still an empty v2 scaffold, and publishing an empty crate to hold the name is a cost with no reader.Tests
153 Rust, clippy and fmt clean, plus the new dry-run gate.
🤖 Generated with Claude Code