Skip to content

Autoharness: support BoundedArbitrary argument types - #4693

Merged
feliperodri merged 3 commits into
model-checking:mainfrom
tautschnig:autoharness-bounded-containers
Aug 23, 2026
Merged

Autoharness: support BoundedArbitrary argument types#4693
feliperodri merged 3 commits into
model-checking:mainfrom
tautschnig:autoharness-bounded-containers

Conversation

@tautschnig

@tautschnig tautschnig commented Jul 29, 2026

Copy link
Copy Markdown
Member

Description

Stacked on #4691 (only the last commit is new; review that one).

Autoharness can now generate harnesses for arguments whose types implement BoundedArbitrary rather than Arbitrary — reusing Kani's existing machinery (the trait, its Vec<T>/String/Box<[T]> implementations, and #[derive(BoundedArbitrary)]) directly: the generated harness calls kani::bounded_any::<T, 4>(). Detection uses the same Instance-resolution approach as implements_arbitrary, enabled by a new fn_marker on kani::bounded_any.

Like the slice/string support in #4691, this is gated behind --bounded-arguments and reported: without the flag, eligible functions are skipped with Requires --bounded-arguments for argument(s) ...; with it, harnesses are marked (bounded) in the summary table and the post-table note spells out that results only hold up to the bound.

The bound of 4 is measured, not guessed: BoundedArbitrary values are heap allocated, and String's implementation reasons about UTF-8 validity — at bound 8, even a trivial String-consuming harness exceeds the default 60s harness timeout.

Follow-up work is planned to explore unbounded generation for container types.

Testing

New script-based test cargo_autoharness_bounded exercising both modes: the skip-reason hint without the flag; with the flag, Vec<u8>/String consumption (passes), OOB indexing on possibly-empty values (fails, correctly), a user struct deriving BoundedArbitrary with a #[bounded] field (passes), a cover check proving max-length vectors with specific contents are generated (SATISFIED), and graceful skipping of Vec<Vec<u8>>.

Full autoharness suite, verify_std_cmd/std_codegen, and compiler/driver/metadata unit tests pass.

Towards #3832

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

@tautschnig
tautschnig requested a review from a team as a code owner July 29, 2026 08:41
Copilot AI review requested due to automatic review settings July 29, 2026 08:41
@github-actions github-actions Bot added Z-EndToEndBenchCI Tag a PR to run benchmark CI Z-CompilerBenchCI Tag a PR to run benchmark CI labels Jul 29, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends Kani’s autoharness generation to support argument types that implement BoundedArbitrary (e.g., Vec<T>, String, and user types deriving BoundedArbitrary) in addition to existing Arbitrary/compiler-derived Arbitrary support. It does so by adding model detection for kani::bounded_any (via a new fn_marker) and updating autoharness eligibility + MIR generation to call kani::bounded_any::<T, 4>() when appropriate, alongside the already-bounded slice/&str support.

Changes:

  • Add BoundedAnyModel marker and compiler-side support to detect and generate harness arguments via kani::bounded_any::<T, 4>() when T: BoundedArbitrary.
  • Introduce/extend Kani model functions for bounded slice and string references (AnySliceRefModel, AnyStrRefModel) and wire them into autoharness MIR generation.
  • Add script-based regression tests for bounded arguments and for slice/&str arguments; update autoharness filter expectations/docs accordingly.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/script-based-pre/cargo_autoharness_slices/src/lib.rs New test crate exercising autoharness support for &[T]/&mut [T] and &str, including cover checks and expected failures.
tests/script-based-pre/cargo_autoharness_slices/slices.sh Runs the new slices autoharness test.
tests/script-based-pre/cargo_autoharness_slices/slices.expected Expected output for the slices autoharness run (incl. skip + cover statuses).
tests/script-based-pre/cargo_autoharness_slices/config.yml Script-based test harness config for the slices test.
tests/script-based-pre/cargo_autoharness_slices/Cargo.toml Cargo manifest for the slices test crate.
tests/script-based-pre/cargo_autoharness_filter/src/lib.rs Updates the filter test to include a slice argument case as eligible.
tests/script-based-pre/cargo_autoharness_filter/filter.expected Updates expected output for the filter test (now including slice eligibility).
tests/script-based-pre/cargo_autoharness_bounded/src/lib.rs New test crate exercising autoharness support for BoundedArbitrary arguments (Vec, String, derived user type), plus skip for nested Vec<Vec<u8>>.
tests/script-based-pre/cargo_autoharness_bounded/config.yml Script-based test harness config for the bounded test.
tests/script-based-pre/cargo_autoharness_bounded/Cargo.toml Cargo manifest for the bounded test crate.
tests/script-based-pre/cargo_autoharness_bounded/bounded.sh Runs the new bounded autoharness test.
tests/script-based-pre/cargo_autoharness_bounded/bounded.expected Expected output for the bounded autoharness run (incl. skip + cover status).
library/kani_core/src/lib.rs Adds fn_marker for kani::bounded_any so the compiler can detect BoundedArbitrary support.
library/kani_core/src/arbitrary.rs Adds model helpers and markers for slice and &str generation (any_slice_ref, any_str_ref).
kani-compiler/src/kani_middle/transform/automatic.rs Extends autoharness MIR generation to use slice/str models and fall back to bounded generation for T: BoundedArbitrary.
kani-compiler/src/kani_middle/mod.rs Adds implements_bounded_arbitrary and extends argument-type eligibility to include BoundedArbitrary.
kani-compiler/src/kani_middle/kani_functions.rs Registers new KaniModel entries: AnySliceRef, AnyStrRef, BoundedAny.
kani-compiler/src/kani_middle/codegen_units.rs Updates autoharness partitioning logic to use the new unified “supported arg type” check and avoid cache poisoning.
docs/src/reference/experimental/autoharness.md Documents bounded slice/str support and bounded BoundedArbitrary argument generation + caveats.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/src/reference/experimental/autoharness.md Outdated
Comment thread kani-compiler/src/kani_middle/transform/automatic.rs Outdated
Comment thread tests/script-based-pre/cargo_autoharness_filter/filter.expected
@tautschnig
tautschnig force-pushed the autoharness-bounded-containers branch from 8e11ec6 to 11e22fb Compare July 29, 2026 09:24
@feliperodri feliperodri added the Z-Autoharness Issue related to autoharness subcommand label Jul 29, 2026
@tautschnig
tautschnig force-pushed the autoharness-bounded-containers branch from 11e22fb to ea60fdf Compare July 30, 2026 03:51
MavenRain pushed a commit to MavenRain/kani that referenced this pull request Jul 31, 2026
### Description

`Box<T>` implements `Arbitrary`, but `Rc<T>` and `Arc<T>` did not, so
functions taking reference-counted arguments could not be verified
against nondeterministic inputs — and were skipped by `kani autoharness`
with "Missing Arbitrary implementation" (smart-pointer receivers are
among the largest skip classes in the top-100 crates.io evaluation).
This PR adds the analogous implementations.

Note that unlike slice/container arguments (model-checking#4691/model-checking#4693), these need no
bound and no opt-in flag: a smart pointer to `T` covers exactly the
values of `T`, so the generated values retain Kani's usual full-coverage
guarantee.

A follow-up will extend autoharness to smart pointers around types that
only *can-derive* `Arbitrary` (compiler-synthesized implementations);
that requires compiler-side models that depend on `alloc` and hence some
optional-model plumbing for the `no_core` flow.

### Testing

New test `tests/kani/Arbitrary/rc_arc.rs` with cover checks proving
extreme values, specific values, and nested smart pointers
(`Rc<Arc<u8>>`) are all generated (all SATISFIED). The `Arbitrary` suite
and `kani` library unit/doc tests pass; verified via autoharness that
`Rc<T>`/`Arc<T>`-taking functions are now selected and verified.

Towards model-checking#3832

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 and MIT licenses.

Co-authored-by: Kiro <kiro-agent@users.noreply.github.com>
@tautschnig
tautschnig force-pushed the autoharness-bounded-containers branch from ea60fdf to 9c78470 Compare August 5, 2026 09:41
@feliperodri feliperodri added this to the Autoharness milestone Aug 18, 2026
@feliperodri
feliperodri force-pushed the autoharness-bounded-containers branch from 9c78470 to 4cc7be8 Compare August 23, 2026 02:03
tautschnig and others added 2 commits August 23, 2026 02:39
Functions taking owned container arguments like Vec<T> or String were
skipped with 'Missing Arbitrary implementation', even though Kani already
ships BoundedArbitrary implementations for exactly these types (and a
derive macro for user types).

Reuse that machinery directly: when an argument type does not implement
(and cannot derive) Arbitrary, but does implement BoundedArbitrary --
detected by the same Instance-resolution approach as implements_arbitrary,
via a new fn_marker on kani::bounded_any -- the generated harness calls
kani::bounded_any::<T, 4>() instead of kani::any(). This covers Vec<T>,
String, Box<[T]>, and user types deriving BoundedArbitrary.

The bound of 4 reflects measured verification cost: BoundedArbitrary values
are heap allocated, and String's implementation reasons about UTF-8
validity; a bound of 8 already makes simple String harnesses exceed Kani's
default 60s harness timeout. As with slices and strings, the
only-valid-up-to-the-bound caveat is documented in the autoharness
reference.

The eligibility loop no longer inserts top-level argument verdicts into the
shared Arbitrary cache at all: argument-position support (slice references,
BoundedArbitrary containers) differs from field-position support, and
implements_arbitrary already memoizes its own recursion, so the top-level
memoization was both redundant and a poisoning hazard.

Towards model-checking#3832

Co-authored-by: Kiro <kiro-agent@users.noreply.github.com>
…mitting bounded_any

Probing implements_bounded_arbitrary with an unsized type instantiates the
generic model with that type, which can crash constant evaluation during
body retrieval (found by re-running the top-100 crates.io evaluation, e.g.
on bytes). Reject unsized types up front.

Also assert that the type implements BoundedArbitrary before emitting a
bounded_any call in harness generation: Instance::resolve does not check
trait bounds, so an eligibility/transform mismatch would otherwise only
surface as an ICE during reachability.

Co-authored-by: Kiro <kiro-agent@users.noreply.github.com>
@feliperodri
feliperodri force-pushed the autoharness-bounded-containers branch from 4cc7be8 to 11ac25a Compare August 23, 2026 02:40
@feliperodri
feliperodri requested a balanced review from Copilot August 23, 2026 02:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Comment thread docs/src/reference/experimental/autoharness.md
Comment thread docs/src/reference/experimental/autoharness.md Outdated
- Skip derived `kani::BoundedArbitrary` impls (their generated `bounded_any`
  method) from automatic harness selection, consistent with how `kani::Arbitrary`
  and `kani::Invariant` impls are already skipped. Without this, autoharness
  picked up e.g. `<Packet as kani::BoundedArbitrary>::bounded_any::<2>` as an
  extra harness, which is Kani-internal machinery rather than user code.
- Update cargo_autoharness_filter expected output: Vec<u8> arguments now
  implement BoundedArbitrary, so they are skipped with 'Requires
  --bounded-arguments' rather than 'Missing Arbitrary implementation'
  (the run does not pass --bounded-arguments).
- Harden the cargo_autoharness_bounded test: validate the 'cargo kani --list'
  status instead of masking it behind grep, assert the cover description with
  its SATISFIED status, and raise --harness-timeout so the UTF-8 String
  harnesses do not spuriously time out on slow CI runners. Match the skip
  reasons by their (unique) reason substring rather than the full rendered
  table row, so the assertions do not depend on column widths that shift with
  the set of selected functions.
- Docs: fix the BoundedArbitrary link (was a 404 to a nonexistent
  experimental/ page) and reconcile the 'Arguments Implementing Arbitrary'
  limitations section with the BoundedArbitrary/slice/string bounded support.
@feliperodri
feliperodri force-pushed the autoharness-bounded-containers branch from 11ac25a to 7392fd0 Compare August 23, 2026 03:10
@feliperodri
feliperodri requested a balanced review from Copilot August 23, 2026 03:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

@feliperodri
feliperodri enabled auto-merge August 23, 2026 03:39
@feliperodri
feliperodri added this pull request to the merge queue Aug 23, 2026
Merged via the queue into model-checking:main with commit 8185456 Aug 23, 2026
33 of 34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Z-Autoharness Issue related to autoharness subcommand Z-CompilerBenchCI Tag a PR to run benchmark CI Z-EndToEndBenchCI Tag a PR to run benchmark CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants