Skip to content

Example traffic app#181

Open
xiyuoh wants to merge 16 commits into
mainfrom
xiyu/example_app
Open

Example traffic app#181
xiyuoh wants to merge 16 commits into
mainfrom
xiyu/example_app

Conversation

@xiyuoh

@xiyuoh xiyuoh commented Mar 13, 2026

Copy link
Copy Markdown
Member

New feature implementation

sample.mp4

Added a traffic app to our existing pool of Crossflow examples. The app allows users to interact with a simple traffic simulator via Crossflow diagrams. Users can modify their diagrams, make different node connections and observe how the vehicle behavior changes accordingly. This is meant to be a resource to help new Crossflow integrators to understand how they can make use of the library and diagram editor tool better.

The app is currently still very rudimentary and does not showcase all the operations Crossflow has to offer. Some operations like Spread and Collect are also relatively new and not available on the diagram editor yet. In the example JSONs I tried to include as many operations that make sense, and will be looking to add on further in the coming weeks.

Opening this draft PR first to make it available for testing and to gather some feedback.

GenAI Use

We follow OSRA's policy on GenAI tools

  • I used a GenAI tool in this PR.
  • I did not use GenAI

Used Gemini 3.0 to generate simple buttons to toggle traffic signal

Try it out!

From crossflow/examples/diagram/traffic_app/, run:

cargo run -- serve

You should (1) see a Bevy traffic simulator window, and (2) open the Diagram Editor on http://localhost:3000. You may choose to load any of the existing sample JSONs from traffic_app/diagrams/, or create your own diagram from scratch.

With a diagram ready on the editor,

  • Click on the Run Workflow button on the editor
  • Input a desired distance (e.g. 300) for the vehicle
  • Click Run

You should see the vehicle start to move forward on the road. Depending on which diagram/workflow you're using, it will react differently to its surrounding environment, e.g. respond to traffic signals, avoid pedestrians (tiny squares moving horizontally), or both! Feel free to use the user panel in the app to toggle some settings.

TODO

  • Documentation
  • Cleanup/add example JSONs
  • Add integration for "approach intersection" similar to handbook
  • Add SpeedLimit integration + create a node that processes Join operation for TrafficState
  • Improve lane change logic
  • Explore how else to incorporate other Crossflow operations
  • Address some TODOs sprinkled across the code
  • Move some toggle-able settings in the simulator into the ops catalog

Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
@xiyuoh

xiyuoh commented Mar 13, 2026

Copy link
Copy Markdown
Member Author

I also had to modify basic_executor a little to accommodate launching a Bevy app alongside the diagram editor - we were doing app.run() from a separate thread in serve() before, and Bevy isn't happy about that when rendering the app, it needs to be done on the main thread. I flipped it around and tried running the app on the main thread, while doing axum::serve() in a thread instead.

xiyuoh added 2 commits March 13, 2026 16:59
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
@mxgrey mxgrey moved this from Inbox to In Progress in PMC Board Mar 24, 2026
xiyuoh added 7 commits March 27, 2026 04:07
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
@xiyuoh xiyuoh force-pushed the xiyu/example_app branch from b6a4e95 to ab64c7c Compare April 6, 2026 07:40
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
@xiyuoh

xiyuoh commented Apr 6, 2026

Copy link
Copy Markdown
Member Author
Screencast.From.2026-04-06.17-12-16.mp4

Some changes from the draft:

  • Added SpeedLimit and ApproachingIntersection logic
  • Improved workflow design - including modifying reached_destination node to wait_for_destination node, moving away from an event loop approach
  • Configurable thresholds for obstacle/arriving nodes
  • Documentation + added some sprites!

@xiyuoh xiyuoh marked this pull request as ready for review April 6, 2026 09:15
@xiyuoh xiyuoh requested a review from mxgrey April 6, 2026 09:15
xiyuoh added 3 commits April 7, 2026 05:32
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
@ArizmendiWan

Copy link
Copy Markdown
Contributor

This is very interesting! I think a game like setting is great for users to learn how to use crossflow. I am also looking into this idea and also this PR. Maybe some level system that gradually introduce new operations can help, like many other puzzle games.

@mxgrey mxgrey left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've spent a lot of time playing with this PR now, and I think there's a great concept emerging here, but we need to bring the role of the traffic app closer to being a kinematic simulation with inputs and outputs that are continuous values representing the state of the vehicle (e.g. speed, distance, steering) instead of discrete events and discrete actions.

I understand the initial design was trying its best to accommodate some limitations of visual programming by embedding as many modes, events, and actions as possible into the compiled node builders so users can just connect these events and modes together. The problem I saw emerging is that there aren't many logical choices for how to connect these modes, events, and actions together beyond the particular diagram structures that you had in mind when implementing them.

I think the way to get around this is to lean heavily on the new Python scripting feature, which is now available in this PR after merging main into it. Now the compiled node builders can be minimalist representations of the inputs/outputs/states of the vehicle, and the actual logic (especially math) for reacting to the state of the simulation can be written into the diagram instead of compiled into node builders.

I've made a branch grey/example_app which begins the work of going in this direction. It's a bit of a hack job, done very quickly and experimentally, but it should be a strong push in the right direction. Broadly speaking, these are things that I think should be addressed for this example app to reach its full potential, much of which the grey/example_app branch begins to address:

  • Inputs should resemble vehicle controls (e.g. throttle and steering)
  • Outputs should resemble sensor data (e.g. obstacle locations, odometer reading)
  • The application logic should resemble a kinematic simulator and not be too specific to a preplanned set of actions. I.e. we should be using numerical integration to drive the simulated world.
  • We should use a consistent set of metric units within the simulation, not arbitrary pixel coordinates. An exception can be made for the speed we show to the user, which may be km/h instead of using meters and seconds.
  • The traffic app should provide a button in the UI that has a node tell the current active workflow that the user has requested a stop. Think of this like a "user cancellation sensor".
  • Assets should be moved to gz-fuel and fetched, perhaps by a build script, instead of stored in the git history.

To get a sense of this new direction, I recommend looking through the diagrams in my branch in the following order:

  • drive.json
  • donuts.json
  • stoplight.json
  • stoplight_and_obstacles.json

See the descriptions of each workflow and their input examples.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

3 participants