WIP Lots of improvements#898
Draft
snazy wants to merge 62 commits into
Draft
Conversation
snazy
force-pushed
the
broad-review-fixes
branch
7 times, most recently
from
July 17, 2026 16:35
259b2f7 to
2872887
Compare
Jackson property type inference only recognized List as a CEL list-like type and treated Optional as an object-like value. Runtime adaptation had the same List-only limitation and returned Optional values as ordinary objects. Generalize runtime list adaptation to Collection, unwrap Optional values to either their contained CEL value or null, and teach the Jackson 2 and Jackson 3 type descriptors to infer Collection<T> as list<T> and Optional<T> as T. Add regression coverage for Collection, Set, Optional.of, Optional.empty, and nested Optional values.
The standalone artifact had no consumer-style JVM coverage for the supported Jackson and generated-protobuf combinations, and the native smoke only covered the base ScriptHost path. Add Gradle JVM test suites that compile and execute against the produced standalone jar with Jackson 2 or 3 and either generated protobuf artifact. Extend the standalone Quarkus smoke to exercise the Jackson 2 registry in both JVM and native runs.
Avoid eager list, primitive-array, and repeated unsigned protobuf materialization on value adaptation and field access. Return interpreter-built CEL maps directly and fast-path already adapted CEL maps to skip redundant wrapping work. Add a lazy protobuf map field view so indexed lookup no longer builds a full Java map for every field read. | Area | Before | After | Result | | --- | ---: | ---: | --- | | `nativeToValue(ArrayList)` at 1k entries | 4048 B/op, 0.998 us/op | 32 B/op, 0.008 us/op | Fixed via list-backed `ListT`. | | `nativeToValue(ListValue)` at 1k entries | 4048 B/op, 1.024 us/op | 32 B/op, 0.013 us/op | Fixed via list-backed `ListT`. | | `nativeToValue(int[])` at 1k entries | 28200 B/op, 5.331 us/op | 32 B/op, 0.008 us/op | Fixed via primitive-array-backed `ListT`. | | `nativeToValue(long[])` at 1k entries | 28200 B/op, 5.559 us/op | 32 B/op, 0.007 us/op | Fixed via primitive-array-backed `ListT`. | | `nativeToValue(double[])` at 1k entries | 28248 B/op, 6.268 us/op | 32 B/op, 0.008 us/op | Fixed via primitive-array-backed `ListT`. | | `nativeToValue(Map<Val, Val>)` at 1k entries | 95304 B/op, 31.331 us/op | 24 B/op, 2.189 us/op | Fixed via already-CEL map fast path. | | Dynamic map literal | 1144 B/op, 0.331 us/op | 664 B/op, 0.166 us/op | Fixed via direct wrapped-map construction. | | Java `ArrayList.exists()` early match at 1k entries | 4320 B/op, 0.893 us/op | 304 B/op, 0.129 us/op | Fixed by avoiding eager input copy. | | Java `int[].exists()` early match at 1k entries | 28472 B/op, 6.359 us/op | 304 B/op, 0.109 us/op | Fixed by avoiding eager primitive boxing. | | Java map lookup at 1k entries | 104 B/op, 0.048 us/op | 104 B/op, 0.063 us/op | No generic lazy-map rewrite; normal lookup was already allocation-flat. | | Protobuf map lookup at 1k entries | 79488 B/op, 29.683 us/op | 31320 B/op, 33.275 us/op | Allocation reduced by lazy protobuf map view; CPU roughly neutral/slower in this short run because lookup scans entries. | | Protobuf repeated unsigned `exists()` early match at 1k entries | 43592 B/op, 11.079 us/op | 640 B/op, 0.238 us/op | Fixed via lazy unsigned list. | | Protobuf repeated unsigned `exists()` late match at 1k entries | 339218 B/op, 134.378 us/op | 296202 B/op, 108.904 us/op | Improved, but still dominated by per-element wrapping during full scan. |
Cache protobuf field metadata lookups per registry and avoid duplicate allocation under concurrent access. Reduce map activation lookups for non-null bindings and use checked overload IDs for custom function dispatch.
Protect first-use Jackson type and enum discovery when registries are shared across evaluations. Use concurrent maps plus synchronized discovery to avoid unsafe cache mutation without relying on recursive computeIfAbsent.
Return CEL values from terminal checked field qualifiers while leaving intermediate field traversal native. This avoids the final generic adaptation step for checked field access without adding wrapper churn to nested field chains.
Pass checked expression metadata directly into the interpreter planner during program creation. This avoids constructing a temporary CheckedExpr on the hot compile-to-program path while preserving the existing checked-expression API.
Keep protobuf map fields scan-based for the first lookup, then build a CEL-value index for repeated lookups. This avoids repeated O(n) scans for repeated map access while preserving missing-key and unsigned-key behavior.
Return the evaluated Val directly when callers request Val.class from Script execution. This gives aggregate-returning scripts an explicit zero-conversion result path while keeping Object.class native conversion behavior unchanged.
Recognize map and filter macro fold shapes during planning and build result lists with a mutable evaluation-local buffer. This avoids repeated accumulator list copying while preserving generic fold fallback for non-standard accumulator references.
Filter checker overload candidates by call style and arity before building function types and running assignability checks. This avoids repeated substitution and mapping copies for overloads that cannot match while preserving the existing resolution path for viable candidates.
Plan undecorated checked top-level variables as direct activation lookups instead of absolute attributes. Select roots and decorated programs keep the attribute shape so qualifier planning, partial unknowns, and custom decorators retain their existing behavior.
Route unresolved varargs calls through a receiver-specific evaluator that avoids building the full argument array before receiver dispatch. Also pass the complete receiver tail to dynamic receivers and cover unchecked receiver varargs calls with multiple arguments.
Includes some Java 17 specific improvements, not cosmetic fluff.
This change adds the necessary base work for the CEL-Spec bump.
The original Bazel-driven build in CEL-Java to run conformance-tests was leveraging the `simple_test` convenience binary to run tests. That binary no longer exists for CEL-Spec 0.25.2 and was replaced with a Java client in the CEL-Spec version bump PR. This change goes further and removes the Bazel build entirely and replaces the client/server split, which was needed for the `simple_test` tool, with JUnit test-factory.
Update string-to-bool conversion to accept the CEL-Spec spellings while rejecting unsupported mixed-case values. Compare type values by CEL type name so well-known object type values match their built-in timestamp and duration type values. Allow null assignment to wrapper fields and leave wrapper, message, timestamp, and duration fields unset during protobuf construction. Support backtick-quoted field names in select expressions and message field initializers, then remove the now-passing conformance skips.
Handle CEL null values at the protobuf construction boundary for repeated and map fields whose value type is timestamp, duration, or a protobuf wrapper. Convert map message values to the actual target protobuf message type instead of always using google.protobuf.Value. Avoid hasField() for repeated wrapper fields when reading protobuf values, since repeated fields do not support presence checks.
The map Any null-retention conformance cases now pass after the protobuf map conversion changes. Remove the proto2 and proto3 skip entries so the JUnit conformance suite covers those cases.
Protobuf message construction could let bad field conversions escape as Java runtime exceptions, which broke disabled-check conformance cases expecting evaluation errors. Wrap field conversion and assignment failures from newValue() as CEL Err values, add provider coverage for invalid null assignments, and enable the remaining proto2/proto3 set_null conformance cases.
Include the CEL-Spec type_deduction suite in the JUnit conformance runner to cover checker result types for already-supported language features. Keep explicit skips for the remaining unsupported checker cases around optionals, flexible type parameter generality, wrapper promotion, and legacy nullable generic candidates.
List literal element type deduction was order-sensitive for matching primitive and wrapper types. When a wrapper appeared before its primitive, the common type was reduced to the primitive, while the reverse order preserved the wrapper type. Prefer the matching wrapper type as the common type for primitive/wrapper unification, add checker coverage for both element orders, and enable the wrapper promotion conformance case.
List literal type deduction selected null as the common type when nullable concrete values were joined with null. This caused message, duration, and timestamp candidates to lose their concrete element type. Prefer the nullable concrete type when it is unified with null, add checker coverage for both element orders, and enable the corresponding legacy nullable conformance cases.
The checker only freshened overload type parameters listed explicitly on the overload declaration. Conformance declarations can also use type_param entries directly in the parameter or result types, which left raw type parameters in resolved result types and broke later overload matching. Classify protobuf abstract types, derive overload type parameters from the full overload signature, and expose current substitutions while checking nested expressions so comprehension results keep refined accumulator types. Enable the matching type-deduction conformance cases.
Namespace resolution preferred container-qualified identifiers before lexical fold variables, so comprehension-local names could resolve to outer bindings. Parsed-only ASTs had the same issue through maybe-attribute resolution, and leading-dot absolute references could not bypass local fold variables. Make checker lookup prefer current lexical identifiers only in nested scopes, keep absolute references marked for evaluation, and make unchecked attributes prefer current fold variables before namespaced candidates. Add regression coverage and enable the namespace shadowing conformance cases.
Proto extension descriptors were not indexed as fields, so checked expressions could not resolve extension selects and parsed-only evaluation treated extension fields as absent. Any payloads with extension fields also lost those values when decoded without an extension registry. Index file-level and message-scoped extension descriptors, resolve them for matching containing message types, and use descriptor-backed extension registries when unpacking Any values. Add regression coverage and enable the proto2 extension has/get conformance cases.
The CEL proto extension conformance suite uses proto.hasExt/proto.getExt and references protobuf extension identifiers as expression values. Those identifiers were not available through checker/provider resolution, and the helper functions were not exposed as an explicit extension library. Expose registered protobuf extension names as string constants, add the opt-in proto extension library functions, and run the proto2 extension conformance file through that library.
The CEL string extension conformance file covers APIs that are already implemented by StringsLib, but the harness did not include that file or enable the library for those expressions. Add the string extension conformance file with explicit skips for unsupported quote, format, and reverse sections. Tighten multi-arity string extension dispatch so calls with too many arguments return no-such-overload instead of evaluating the expected argument prefix.
The string extension conformance file still skipped quote, format, format error, and reverse coverage. StringsLib also lacked the corresponding declarations and runtime implementations. Add strings.quote, reverse, and CEL string formatting support for the covered clauses, including recursive list and map rendering. Fix escaped string literal unescaping for raw non-BMP Unicode code points, and remove the remaining string extension conformance skips.
The CEL-Spec type-deduction suite uses optional.none and optional.of in check-only cases. Those expressions could not type-check because the optional helper functions were not declared. Add an opt-in optional library with compile-time declarations for the optional helper functions, and use it in the conformance harness for optional expressions. This enables optional type deduction without adding runtime optional semantics.
The strong enum sections contain a mix of unsupported typed-enum semantics and cases that already work with the current int-backed enum behavior. Replace the broad section skips with targeted skips for the remaining typed-enum gaps, and convert conformance enum_value bindings to integer values so disabled-check conversion cases can run.
The nested protobuf message literal case now passes under the current conformance runner and evaluator. Remove the stale skip so the parse nesting coverage runs with the rest of the CEL-Spec simple suite.
The CEL-Spec encoder extension suite covers Base64 encode and decode helpers, including decode input without padding. Add an opt-in encoder extension library with base64.encode and base64.decode, wire it into the conformance harness for base64 expressions, and include direct core coverage for successful and invalid decode inputs.
The CEL-Spec bindings extension uses cel.bind to introduce a local variable in the result expression. CEL-Java parsed those expressions as ordinary calls, so the checker reported undeclared local variables. Add cel.bind as a standard macro, expanding it to a single-iteration comprehension with a dynamic accumulator initializer. Add direct macro coverage and include bindings_ext.textproto in the conformance suite.
The CEL-Spec math extension suite exercises numeric helpers such as min/max, rounding, finite checks, and bitwise operations. CEL-Java did not expose those extension functions, so the suite could not be enabled. Add an opt-in math extension library, wire it into the conformance harness for math expressions, and include direct coverage for representative numeric, rounding, and bitwise cases.
The CEL-Spec network extension suite requires net.IP and net.CIDR values with strict literal parsing, canonical formatting, receiver methods, and type-literal support. CEL-Java did not expose those extension functions or custom network types. Add an opt-in network extension library, support custom object type values with explicit traits, wire the library into the conformance harness, and cover representative IP and CIDR behavior with direct tests.
The CEL-Spec block suite uses test-only macros to model sequential local bindings and generated comprehension variable names. CEL-Java did not provide those macros, so the suite could not be parsed or executed in the pure Java conformance harness. Add opt-in parser macros for the block conformance cases, keep them out of the default macro set, wire them into the conformance parser only when needed, and enable block_ext with explicit skips for the optional-dependent cases.
The updated CEL-Spec includes macros that expose both key/index and value variables during comprehension evaluation. CEL-Java only modeled one iteration variable, which prevented exists/all/existsOne and transform macros from covering list indexes and map values. Add parser, checker, and interpreter support for the second comprehension variable, add transformList and transformMap macro expansion, preserve iter_var2 in AST pruning and exhaustive evaluation, and enable the macros2 conformance file with direct unit coverage.
Move CEL grammar generation into cel-core using CongoCC and remove the generated-antlr module, shadow jar dependency, and ANTLR runtime wiring. The generated parser now builds internal AST nodes with typed CEL expression conversion hooks, keeping parser construction close to the grammar while preserving the existing parser API and source-location handling. This simplifies dependency management for downstream consumers, removes the relocated ANTLR artifact from the build graph, and makes future grammar changes easier to review in the core module.
Add runtime optional values and wire optional constructors, receiver methods, lazy optMap/optFlatMap macros, and optional access operators. Parse optional select, index, list, map, and message syntax with CongoCC, then preserve the protobuf optional-entry AST fields through checking and interpretation. Enable optionals conformance coverage and remove the optional-dependent block-extension skips.
Struct-pruning was effectively doing nothing.
snazy
force-pushed
the
broad-review-fixes
branch
from
July 17, 2026 19:17
1994983 to
577c9b5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.