Skip to content

Error publishing datafusion-spark in datafusion 55.0.0 release "cannot find module or crate datafusion in this scope" #24474

Description

@alamb

Describe the bug

To Reproduce

Run

(cd datafusion/spark && cargo publish)

This errored like

   Compiling datafusion-spark v55.0.0 (/Users/andrewlamb/Downloads/apache-datafusion-55.0.0/target/package/datafusion-spark-55.0.0)
error[E0433]: cannot find module or crate `datafusion` in this scope
  --> src/function/string/quote.rs:20:5
   |
20 | use datafusion::logical_expr::{Coercion, ColumnarValue, Signature, TypeSignatureClass};
   |     ^^^^^^^^^^ use of unresolved module or unlinked crate `datafusion`
   |
   = help: if you wanted to use a crate named `datafusion`, use `cargo add datafusion` to add it to your `Cargo.toml`

For more information about this error, try `rustc --explain E0433`.
error: could not compile `datafusion-spark` (lib) due to 1 previous error
error: failed to verify package tarball

I had to change it to refer to datafusion_expr to publish it:

- use datafusion::logical_expr::{Coercion, ColumnarValue, Signature, TypeSignatureClass};
+ use datafusion_expr::{Coercion, ColumnarValue, Signature, TypeSignatureClass};

Expected behavior

Should publish cleanly

Additional context

I think the issue is that datafusion is an optional dependency for datafusion-spark which is activated in the workspace

We can probably catch this error in CI by doing clippy check on datafusion-spark without default features

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions