Skip to content

Fix Flight LargeList schema encoding#10298

Draft
yinli-systems wants to merge 1 commit into
apache:mainfrom
yinli-systems:kevin/fix-flight-large-list-schema-10291
Draft

Fix Flight LargeList schema encoding#10298
yinli-systems wants to merge 1 commit into
apache:mainfrom
yinli-systems:kevin/fix-flight-large-list-schema-10291

Conversation

@yinli-systems

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

prepare_field_for_flight currently rebuilds DataType::LargeList fields with Field::new_list, which changes the schema to a regular List. This silently loses the original large-list offset type in the Flight encoder's prepared schema.

The existing ListView / LargeListView path below this code already preserves the large variant, so this appears to be the same preservation rule missing for LargeList.

What changes are included in this PR?

  • Use Field::new_large_list when preparing DataType::LargeList fields for Flight.
  • Add a regression test that verifies the prepared Flight schema preserves a LargeList field, including field metadata and the nested item field.

Are these changes tested?

Yes, locally with Rust 1.96.0 on macOS because the repository-pinned Rust 1.96.1 toolchain download was not completing cleanly in this environment.

  • RUSTUP_TOOLCHAIN=1.96.0-aarch64-apple-darwin cargo fmt --check
  • RUSTUP_TOOLCHAIN=1.96.0-aarch64-apple-darwin cargo test -p arrow-flight test_large_list_schema_encoded
  • RUSTUP_TOOLCHAIN=1.96.0-aarch64-apple-darwin cargo test -p arrow-flight encode::tests
  • git diff --check

I used AI assistance to help inspect the nearby Flight schema-preparation paths and generate the focused regression test, then validated the change with the commands above.

Are there any user-facing changes?

Yes. Flight-encoded schemas now preserve LargeList instead of exposing it as List after schema preparation.

@github-actions github-actions Bot added arrow Changes to the arrow crate arrow-flight Changes to the arrow-flight crate labels Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate arrow-flight Changes to the arrow-flight crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Schema for LargeList in Flight data encoder

1 participant