An open-source geospatial trajectory data streaming platform based on Apache Kafka.
MobilityKafka explores the advantages of MobilityDB datatypes and functions in the Kafka environment, using the JMEOS library as middleware.
The MobilityDB project is developed by the Computer & Decision Engineering Department of the Université libre de Bruxelles (ULB) under the direction of Prof. Esteban Zimányi. ULB is an OGC Associate Member and member of the OGC Moving Feature Standard Working Group (MF-SWG).
More information about MobilityDB, including publications, presentations, etc., can be found in the MobilityDB website.
The streaming-side parity matrix runs all nine BerlinMOD reference queries (Q1..Q9) in three streaming forms each on this runtime: continuous (always-on, per-event emission), windowed (tumbling 10-second aggregation), and snapshot (5-second tick — the parity-oracle form whose output at watermark T equals the batch BerlinMOD-Q result on data up to T).
| Q | Topic | Continuous | Windowed | Snapshot |
|---|---|---|---|---|
| Q1 | "which vehicles have appeared in the stream?" | ✓ | ✓ | ✓ |
| Q2 | "where is vehicle X at time T?" | ✓ | ✓ | ✓ |
| Q3 | "vehicles within d of P at time T?" | ✓ | ✓ | ✓ |
| Q4 | "vehicles entered region R, and when?" | ✓ | ✓ | ✓ |
| Q5 | "pairs of vehicles meeting near P" | ✓ | ✓ | ✓ |
| Q6 | "cumulative distance per vehicle" | ✓ | ✓ | ✓ |
| Q7 | "first passage of vehicles through POIs" | ✓ | ✓ | ✓ |
| Q8 | "vehicles close to a road segment" | ✓ | ✓ | ✓ |
| Q9 | "distance between vehicles X and Y at time T" | ✓ | ✓ | ✓ |
27 / 27 cells = the full MobilityKafka parity-matrix row. Each cell has a dedicated Q<N>{Continuous,Windowed,Snapshot}Processor class in kafka-streams-app/src/main/java/berlinmod/ and is locally verified via BerlinMODQ1LocalTest running on the Kafka-Streams TopologyTestDriver (no real broker required).
kafka-streams-app/ is a Maven project (Java 21, Kafka Streams 3.6.0) holding:
- 27 per-cell
Q<N>{Continuous,Windowed,Snapshot}Processorclasses BerlinMODTopology— unified topology fanning input topicberlinmodto per-Q-form output topicsBerlinMODTrip+BerlinMODTripSerde— shared data class + JSON Serde (byte-shape equivalent to MobilityFlink'sBerlinMODTrip)Haversine+SegmentDistance+PointOfInterest— pure-Java geometry utilities used by the spatial-predicate cellsBerlinMODQ1LocalTest— TopologyTestDriver-based local end-to-end driver
The streaming snapshot form converges to the batch BerlinMOD result on the same scale-factor corpus, anchored against the cross-platform outputs in MobilityDB-BerlinMOD.
Spatial predicates today use pure-Java great-circle (Haversine) and planar segment-distance (SegmentDistance) utilities; each call site is marked TODO(meos) for JMEOS-bridge migration after JMEOS#15 (the MEOS 1.4 regen) settles.
The spatial predicates route through MEOS via the JMEOS
bridge, so the build needs the JMEOS jar and the native libmeos.so. Neither is
committed to this repository — generate them from source with the helper script,
which clones MobilityDB and JMEOS at pinned, immutable refs, builds libmeos.so,
builds the jar, installs the jar into the local Maven repository, and stages
libmeos.so for the runtime:
./build-jmeos.sh
Run it once (re-run it only to bump the pinned MobilityDB/JMEOS refs at the top of the script). After it succeeds, JMEOS resolves as an ordinary Maven dependency.
cd kafka-streams-app
mvn -q clean package -DskipTests
java --add-opens java.base/java.lang=ALL-UNNAMED \
--add-opens java.base/java.util=ALL-UNNAMED \
--add-opens java.base/java.lang.reflect=ALL-UNNAMED \
-cp target/mobility-kafka-streams-1.0-SNAPSHOT.jar \
berlinmod.BerlinMODQ1LocalTest
The driver pipes a 21-event sorted-event-time corpus plus two sentinel records at t = T0+15001 and t = T0+20001 (to step the STREAM_TIME punctuator through the desired tick boundaries) and reads every per-Q-form output topic with the appropriate deserializer. Expected per-Q-form counts are in the PR body for the open scaffold PR.
- MobilityFlink#3 — the same 27-cell row on Flink
- MobilityNebula#15 — 15 of 27 cells on NebulaStream (Q1, Q2, Q3, Q4, Q7-via-POI-fanout)
- MobilityDB-BerlinMOD#29 — the batch BerlinMOD-9 cross-platform timings (the snapshot form's gold-answer source)
- MobilityDB/.github#10 — the ecosystem-profile description of the stream-layers tier