Advance the facade to the ecosystem-pin-2026-06-22a MEOS surface#28
Closed
estebanzimanyi wants to merge 23 commits into
Closed
Advance the facade to the ecosystem-pin-2026-06-22a MEOS surface#28estebanzimanyi wants to merge 23 commits into
estebanzimanyi wants to merge 23 commits into
Conversation
estebanzimanyi
force-pushed
the
feat/facade-surface-22a
branch
from
June 22, 2026 23:56
dffe697 to
8320ac3
Compare
… surface Bump codegen/input/meos-idl.json to the MEOS-API IDL and regenerate functions.GeneratedFunctions over the full consolidated superset: mul_* (incl. tbigint); minDistance; the circular-buffer and network-point MF-JSON readers; the ever- and always-covers families (ecovers_*/acovers_*); trgeo_*; the H3 / th3index family (ever_eq_h3indexset_th3index, h3index_in/out, H3Index lowered to long); PostgreSQL type I/O; tgeogpoint_great_circle_distance; meos_initialize_noexit_error_handler. 2916 functions.
Add the codegen exec step to jmeos-core so functions/GeneratedFunctions.java is produced from codegen/input/meos-idl.json during the build, and drop the committed copy.
Run meos_initialize() from the binding's load path so the allocator and collation are installed before any call. MEOS splits startup into granular steps; without the collation step, text comparisons route into varstr_cmp with an uninitialized collation and crash.
Call the generated functions class from TText and TextSet instead of the hand-maintained facade, so the text conversions use the canonical master names (cstring_to_text, text_to_cstring).
…ld flags The functions.GeneratedFunctions facade is generated at build time from the MEOS IDL with the optional type families selected by the same flag names and ON|OFF (also 1|0) values as the MobilityDB/MEOS build: -DCBUFFER, -DNPOINT, -DPOSE, -DRGEO, -DH3. Every family is included by default; passing -DCBUFFER=OFF (or =0) drops that family's functions from the generated binding so a subset jar ships without it (RGEO needs POSE). FunctionsGenerator maps each function's source header to its family and omits excluded families; jmeos-core runs the generator at generate-sources (so the flag flows through mvn) and compiles the generated functions.GeneratedFunctions.
Hoists the tier-aware MeosOps* facade (62 classes) into JMEOS so every JVM binding inherits the one canonical Java idiom from the shared jar instead of duplicating it per engine. The facade forwards to functions.GeneratedFunctions under a package-private MeosOpsRuntime probe gated by the canonical -Dmeos.enabled property; javadoc is engine-neutral. Relocates the maintained generator (regen_facade_from_jar + the gap / sql / tbigint / h3 emitters + parity_audit + meos-ref) under jmeos-core/tools so the facade stays regenerated, not hand-edited; regeneration is idempotent against the pin jar.
…surface Apply the 77-symbol drift from pin 588768d7ab to master: 20 trgeo→trgeometry renames (A); 12 pg_*/cstring2text/text2cstring/pose_normalise/temporal_instants_p/ temporal_hausdorff_distance renames (B); 21 removed scalar helpers dropped with no callers (C: add_date_int, add_interval_interval, add_timestamptz_interval, date_to_timestamp, date_to_timestamptz, float_exp, float_ln, float_log10, float_round, int32_cmp, int64_cmp, interval_make, mul_interval_double, timestamp_to_date, timestamptz_shift, timestamptz_to_date, minus_date_date, minus_date_int, minus_timestamptz_interval, minus_timestamptz_timestamptz, meos_initialize_pointcloud); 23 signature updates (D): value_at_timestamptz gains a result out-param and returns bool; value_split/value_time_split/ temporal_time_split gain bins/count out-params; set_spans/spanset_spans/ spanset_spanarr drop the count out-param; super_union_span_span drops the boolean merge flag; text_cmp gains a collation pointer; gbox_make gains hasm and geodetic booleans and min/max range pairs; geom_min_bounding_radius gains a radius out-param; pose_make_2d/3d gain a geodetic boolean; trgeoinst_make is renamed trgeometryinst_make. All xxx_hash_extended functions change their seed and return type from long to int (bigint_to_span/spanset likewise). 1735 tests pass, BUILD SUCCESS.
MeosSetSetJoin exposes the MEOS *_tgeoarr_tgeoarr family as eDwithinPairs / tDwithinPairs / aDisjointPairs over two arrays of temporal-geometry handles: it marshals the native pointer arrays the kernel prunes in C, keeps them reachable across the call with reachabilityFence, and reads back the flattened 0-based index pairs (and, for tDwithin, the per-pair tstzspanset of in-range times). Both JVM engines call it from the shared org.mobilitydb.meos layer, so the NxN spatial-join surface derives once. Verified against libmeos.
The IDL and bundled libmeos carry the 54a9d4bc54 public surface: the per-thread PROJ context, the box3d_in/gbox_in parsers, and tpose_to_tpoint. The parity-gap forwarders bind the value-at-timestamptz wrappers through their result-returning form and drop the pointcloud initializer absent from the surface.
Add extract_named_surface.py, which produces meos-named-surface.json from the two canonical sources already in the MobilityDB tree: the SQL CREATE FUNCTION catalog (named functions, overloads, per-argument DEFAULTs -> valid call arities) and the doxygen chain (@sqlfn on the PG wrapper, @csqlfn on the MEOS function) linking each SQL name to its PG and MEOS C functions. This is the layer above the C-FFI IDL from which a binding's named surface and its Spark Connect registrar are generated, rather than hand-maintained. 1284 named functions, asMFJSON resolving to temporal_as_mfjson with minArity 1 / maxArity 4.
…tter extract_spark_impls.py scans the MobilitySpark UDFs (register name + field + body GeneratedFunctions call) and joins on the named surface's SQL->MEOS C linkage to recover canonical name -> Spark impl mechanically, so the emitter needs no hand-written remap. The join classifies each function for emission: single-impl (identity name over one impl), multi-impl (identity name with a WKB-type-tag dispatch builder), and join gaps to close.
…face generate_spark_registrar.py joins the canonical named surface with the Spark impl scan and emits MobilitySparkConnectExtensionsGen.scala: a SparkSessionExtensions that injects each canonical function under its identity name (asMFJSON, not temporalAsMfjson), no hand-written remap. Shipped ScalaUDF closures live in a companion object so they capture only the serializable UDF; the builder null-pads the impl's optional args to the call-site arity. The 81 single-impl functions are generated, compiled, and serve live over Spark Connect under their identity names; the 139 multi-impl names are listed for the per-row meos_typeof_hexwkb dispatch.
…trar A multi-impl canonical name (one SQL name over several type-specific Spark impls) whose first argument differs in MEOS type is emitted as a single ScalaUDF that peeks meostype_name(meos_typeof_hexwkb(arg0)) per row and routes to the impl whose receiver type matches, with the Temporal-receiver impl as the catch-all default. The receiver category is read from the impl's primary MEOS function (the last non-marshaling GeneratedFunctions call in the UDF body) first C-parameter type. The registrar serves the /items-collection OGC function set under identity names: asMFJSON, stbox, the Xmin/Ymin/Xmax/Ymax/Tmin/Tmax accessors, numSequences, sequenceN, trajectory. Functions that differ only on a later argument (atTime on its time argument) are listed for the arg-N dispatch extension.
…th the SQL default Several MobilitySpark UDFs register one MEOS operation under both a bare name and a camelCase name (asText/tpointAsText, getTime/time, cumulativeLength/...). When a canonical name's impls all share one primary MEOS function, bind the identity name to a single impl rather than treating it as a type dispatch. Capture each optional argument's SQL DEFAULT literal in the named surface and fill an omitted optional argument with it, but only when the impl exposes a full overload's worth of arguments (impl arity equals that overload's maxArity, so the positions align); otherwise null-pad and let the impl's own default hold. This serves asText/asEWKT (maxdecimaldigits default 15) while keeping asMFJSON at full coordinate precision (its impl exposes a non-leading argument subset).
Generalize the multi-impl dispatch from arg0 to the first argument position at which the type-specific impls differ, peeking that argument's MEOS type tag: atTime routes on its time argument (tstzspan/tstzset/tstzspanset), duration routes on its span argument. The concrete type for a generic Span/Set receiver is taken from the MEOS type-name embedded in the impl's primary function name. Each dispatch route carries its impl's arity and SQL-default fills, so an omitted optional argument of the chosen impl is filled with the canonical default rather than a null pad (duration on a tstzspanset supplies boundspan=FALSE); boolean SQL defaults are emitted alongside integer ones. The named surface is regenerated from the pin, so speed(tgeompoint) resolves through the dedicated Tpoint_speed wrapper to tpoint_speed and binds to the speed impl.
Functions that return a struct larger than 16 bytes by value (the seven *Split returns plus MvtGeom) use the SysV/AArch64 sret calling convention: the caller allocates the struct and passes a hidden pointer as an implicit first argument; the callee fills it and returns it. The emitter previously collapsed such a return to a bare Pointer, so jnr-ffi mis-read the return register and the struct fields (notably count) came back as garbage. Parse the IDL "structs" section, compute each struct's size, and for a by-value return larger than 16 bytes emit a hidden leading Pointer _sret parameter in the interface, allocate it in the wrapper, and return the filled buffer. Register-returned structs (<=16B) are logged, not silently mis-bound. Regenerates GeneratedFunctions with the new bindings.
Repoint the last three main-code classes from the hand-rolled functions.functions facade onto the generated GeneratedFunctions surface, completing the main-code side of the dual-facade wipe. No main-code class imports functions.functions now. Renames and reshapes applied (verified against the generated signatures): - pg_timestamptz_in/pg_interval_in -> timestamptz_in/interval_in (identical sigs). - the temporal spatial-relationship calls (tcontains/tdisjoint/tdwithin/ tintersects/ttouches) drop the trailing restr,atvalue booleans the current MEOS signatures no longer take (dwithin keeps its distance argument). - value_at_timestamp uses the generated bool+out-param wrapper, which returns the GSERIALIZED* directly, instead of reading the out buffer at the wrong offset. Defects surfaced and fixed while migrating (all confirmed via smoke): - count out-parameters were read at offset 4 (getInt(Integer.BYTES)) instead of 0 in values/make_simple/stboxes, yielding out-of-bounds garbage counts. - Memory.allocate(Runtime.getRuntime(runtime), n) threw ClassCastException (Runtime.getRuntime expects a library proxy, not a Runtime); pass runtime. Smoke: values=3, make_simple=1, stboxes=2, value_at=POINT(5 5). Type suites TGeomPoint/TGeogPoint/TInt/TFloat/TBool all green (624 tests, 0 fail/0 err); the only residual is the pre-existing varstr_cmp ttext_in crash in TTextTest.
Repoint value_split / value_time_split / time_split / space_split / space_time_split (TNumber, Temporal, TPoint) onto the generated GeneratedFunctions split wrappers, which now return the *Split struct via the sret convention. The methods read fragments at offset 0 and count at the struct's count offset (16 for the 3-field splits, 24 for the 4-field time splits) instead of the stale pre-735f out-parameter signature. Also fixes defects surfaced while migrating: - duration/start were parsed in an inverted branch so the duration was dropped (or null-dereferenced) on the default-start path; parse the duration unconditionally and default only the start. - timedelta_to_interval passed cumulative units (toHours/toMinutes/ toSeconds give the whole span in each unit) to interval_make, which then re-added them on top of the days. It now decomposes per field and parses a textual interval with interval_in, sidestepping a jnr-ffi quirk that mis-passes interval_make's trailing double after its six int arguments. Verified end to end on both the Duration and the String duration paths: all five split methods return the correct fragment counts through the OO API.
… public) Wipe step 10: STBox onto the generated facade. gserialized_in -> geom_in (identical sig); geo_expand_spatial(gs, d) -> stbox_expand_space(geo_to_stbox(gs), d) (the public composition). 77 STBox tests green.
Wipe step 9: tstzspan (adjacent_period_timestamp -> adjacent_span_timestamptz, pg_timestamptz_in -> timestamptz_in) and tstzset (timestampset_out -> tstzset_out; tstzset_values gained a count out-param) onto the generated facade.
…+ value_at) Wipe step 8: TText onto the generated facade. cstring2text/text2cstring -> cstring_to_text/text_to_cstring (now public in 11g). value_at reshaped: the generated ttext_value_at_timestamptz returns the text* directly (or null), so read it via text_to_cstring; null-safe (throws on no-value, was the offset-8 garbage read). Verified "hello" via smoke.
Repoint every test from functions.functions to the generated GeneratedFunctions surface (the calls are same-name same-signature, so this is a mechanical repoint) and remove the hand-rolled functions.java. The dual-facade irregularity is gone: the whole library — main code and tests — now uses the single generated facade. The other functions-package classes (GeneratedFunctions, the Meos* error types, error_handler/error_handler_fn) are unaffected. Full suite after deletion: 1735 tests, 0 failures, 0 errors, 0 native crashes.
estebanzimanyi
force-pushed
the
feat/facade-surface-22a
branch
from
July 2, 2026 12:21
8320ac3 to
d7dd7b4
Compare
Member
Author
|
Superseded by #30, which regenerates at the current MobilityDB master (4529 functions), newer than the 2026-06-22a pin. The shared facades are consumer projections generated in the consumers. Closing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Regenerates
codegen/input/meos-idl.jsonand the generated facade from the MEOS-API catalog atecosystem-pin-2026-06-22a(4463 functions, 89 structs, 20 enums), and absorbs the two MEOS surface changes since the pin-15c regen.Generator: managed wrapper for the
valueout-param familyThe
*_value_at_timestamptzaccessors renamed their out-paramresult→value.FunctionsGeneratornow recognisesvalueas an output-result parameter, so it emits the managed three-argument wrapper (allocate the buffer, call the boolean native, return the typed value) uniformly fortbool,tint,tfloat,tbigint,ttext,tcbuffer,tjsonb, andtnpoint. Detection is gated on a pointer C type so a by-valueDatuminput namedvalue(e.g.contains_set_value) is never mistaken for an out-param, and the forwarded native argument is decoupled from the parameter name.OO call sites adapted to the catalog
spanset_spansno longer takes thecountout-param; the.spans()accessors drop the stale argument.Temporal **fragment array directly and report the bins and the fragment count through out-params;value_split,value_time_split, andtime_splitallocate those buffers and read the count.Tests
Adds runtime coverage for the value accessors and the split operations (previously uncovered). Full
jmeos-coresuite: 1739 tests, 0 failures, 0 errors against an all-familieslibmeosbuilt at the pin.Stacks on the open facade-regen chain (#27
pin-12l) and the pushedpin-14l/pin-15cregens, which do not yet have their own PRs — their commits appear in this PR's cumulative diff againstmain.