Skip to content

feat(ddx-datafusion): make the adapter publishable - #62

Merged
alxmrs merged 2 commits into
mainfrom
release-ddx-datafusion
Aug 9, 2026
Merged

feat(ddx-datafusion): make the adapter publishable#62
alxmrs merged 2 commits into
mainfrom
release-ddx-datafusion

Conversation

@alxmrs

@alxmrs alxmrs commented Aug 9, 2026

Copy link
Copy Markdown
Member

Makes ddx-datafusion publishable to crates.io. Part of the M2 release, alongside ddxdb → PyPI.

This was more than dropping publish = false

The blocker was one line up, in the workspace manifest:

ddx-core = { path = "crates/ddx-core" }     # no version

cargo uses the path for workspace builds and the version for what it writes into a published dependent's manifest — and it refuses to publish any crate whose path dependency has none. So the adapter could never have been released by flipping its own flag; the fix belongs at the workspace level, where it also unblocks ddx-ad later without rediscovering the same thing.

The rest of what crates.io needs and the crate lacked:

  • A real version. version.workspace = true resolved to the 0.0.0 placeholder. Now 0.1.0, versioned independently of ddx-core — the two publish on their own cadence, and an adapter forced to move whenever the engine did would mean releases for changes that never touched it.
  • A README. There wasn't one, so the page crates.io and docs.rs render would have been blank. It covers the two routes, when to reach for each, where they disagree about what may be the wrt, and the sqlparser-unification constraint that decides which datafusion versions can possibly work.
  • keywords / categories / readme metadata, matching ddx-core.
  • A changelog, with a hand-written first entry in the shape ddx-core 0.1.0 used.
  • A publish dry-run in CI, so packaging breakage surfaces on a pull request instead of when a release first tries to upload.

Verified against the published crate, not the local path

Packaged 19 files, 216.3KiB (60.3KiB compressed)
Downloaded ddx-core v0.2.1
Compiling ddx-core v0.2.1
Compiling ddx-datafusion v0.1.0
Finished

That resolution is the point of the exercise. cargo publish --dry-run builds the packaged crate against what a consumer would actually download, so this proves the manifest works outside the workspace — the thing a cargo test run can never tell you.

The dependency floor is 0.2.1, deliberately

An earlier dry-run on this branch resolved ddx-core 0.2.0, which returns a zero gradient for a NULL row under grad(abs(u), x) — a silently wrong result. Requiring 0.2.1 means a consumer pinned to 0.2.0 gets a resolution error rather than that gradient. "0.2" would have compiled and been quietly worse.

Not included

ddx-ad stays publish = false. It is still an empty v2 scaffold, and publishing an empty crate to hold the name is a cost with no reader.

Tests

153 Rust, clippy and fmt clean, plus the new dry-run gate.

🤖 Generated with Claude Code

alxmrs and others added 2 commits August 9, 2026 13:20
Everything crates.io needs, which was more than dropping `publish = false`.

- **The workspace `ddx-core` dependency had no version.** cargo uses the
  path for workspace builds and the version for what it writes into a
  published dependent's manifest, and refuses to publish a crate whose
  path dependency lacks one. This is why the adapter could never have
  been published by flipping a flag.
- **A real version.** `version.workspace = true` resolved to the 0.0.0
  placeholder. Now 0.1.0, versioned independently of ddx-core: the two
  publish on their own cadence, and an adapter forced to move whenever
  the engine did would mean releases for changes that do not touch it.
- **README, keywords, categories.** The crate had no README at all —
  the page crates.io and docs.rs show would have been empty. It covers
  the two routes, when to reach for each, where they disagree about what
  may be the `wrt`, and the sqlparser-unification constraint that decides
  which datafusion versions can work.
- **A changelog** with a hand-written first entry, matching how ddx-core
  0.1.0 was documented.
- **A publish dry-run in CI**, so packaging breakage surfaces on a pull
  request rather than when a release first tries to upload.

Verified by `cargo publish --dry-run`: packages 19 files, then downloads
ddx-core from crates.io and compiles the adapter against the *published*
crate rather than the local path — which is what a consumer actually gets.

Not yet ready to release: that dry-run resolved ddx-core 0.2.0, which
returns a zero gradient for a NULL row under `grad(abs(u), x)`. The floor
must be tightened to 0.2.1 once that release lands, so a consumer pinned
to 0.2.0 gets a resolution error instead of a silently wrong gradient.

153 tests, clippy and fmt clean.
The dry-run previously resolved 0.2.0, which returns a zero gradient for
a NULL row under grad(abs(u), x). With the floor at 0.2.1 a consumer
pinned to 0.2.0 gets a resolution error rather than that gradient.

Verified: cargo publish --dry-run now downloads ddx-core v0.2.1 and
compiles the adapter against it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@alxmrs alxmrs left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alxmrs
alxmrs merged commit 4eb4802 into main Aug 9, 2026
9 checks passed
@alxmrs
alxmrs deleted the release-ddx-datafusion branch August 9, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant