Example traffic app#181
Conversation
|
I also had to modify |
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Screencast.From.2026-04-06.17-12-16.mp4Some changes from the draft:
|
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
Signed-off-by: Xiyu Oh <xiyu@openrobotics.org>
|
This is very interesting! I think a game like setting is great for users to learn how to use crossflow. I am also looking into this idea and also this PR. Maybe some level system that gradually introduce new operations can help, like many other puzzle games. |
There was a problem hiding this comment.
I've spent a lot of time playing with this PR now, and I think there's a great concept emerging here, but we need to bring the role of the traffic app closer to being a kinematic simulation with inputs and outputs that are continuous values representing the state of the vehicle (e.g. speed, distance, steering) instead of discrete events and discrete actions.
I understand the initial design was trying its best to accommodate some limitations of visual programming by embedding as many modes, events, and actions as possible into the compiled node builders so users can just connect these events and modes together. The problem I saw emerging is that there aren't many logical choices for how to connect these modes, events, and actions together beyond the particular diagram structures that you had in mind when implementing them.
I think the way to get around this is to lean heavily on the new Python scripting feature, which is now available in this PR after merging main into it. Now the compiled node builders can be minimalist representations of the inputs/outputs/states of the vehicle, and the actual logic (especially math) for reacting to the state of the simulation can be written into the diagram instead of compiled into node builders.
I've made a branch grey/example_app which begins the work of going in this direction. It's a bit of a hack job, done very quickly and experimentally, but it should be a strong push in the right direction. Broadly speaking, these are things that I think should be addressed for this example app to reach its full potential, much of which the grey/example_app branch begins to address:
- Inputs should resemble vehicle controls (e.g. throttle and steering)
- Outputs should resemble sensor data (e.g. obstacle locations, odometer reading)
- The application logic should resemble a kinematic simulator and not be too specific to a preplanned set of actions. I.e. we should be using numerical integration to drive the simulated world.
- We should use a consistent set of metric units within the simulation, not arbitrary pixel coordinates. An exception can be made for the speed we show to the user, which may be km/h instead of using meters and seconds.
- The traffic app should provide a button in the UI that has a node tell the current active workflow that the user has requested a stop. Think of this like a "user cancellation sensor".
- Assets should be moved to gz-fuel and fetched, perhaps by a build script, instead of stored in the git history.
To get a sense of this new direction, I recommend looking through the diagrams in my branch in the following order:
drive.jsondonuts.jsonstoplight.jsonstoplight_and_obstacles.json
See the descriptions of each workflow and their input examples.
New feature implementation
sample.mp4
Added a traffic app to our existing pool of Crossflow examples. The app allows users to interact with a simple traffic simulator via Crossflow diagrams. Users can modify their diagrams, make different node connections and observe how the vehicle behavior changes accordingly. This is meant to be a resource to help new Crossflow integrators to understand how they can make use of the library and diagram editor tool better.
The app is currently still very rudimentary and does not showcase all the operations Crossflow has to offer. Some operations like
SpreadandCollectare also relatively new and not available on the diagram editor yet. In the example JSONs I tried to include as many operations that make sense, and will be looking to add on further in the coming weeks.Opening this draft PR first to make it available for testing and to gather some feedback.
GenAI Use
We follow OSRA's policy on GenAI tools
Used Gemini 3.0 to generate simple buttons to toggle traffic signal
Try it out!
From
crossflow/examples/diagram/traffic_app/, run:You should (1) see a Bevy traffic simulator window, and (2) open the Diagram Editor on
http://localhost:3000. You may choose to load any of the existing sample JSONs fromtraffic_app/diagrams/, or create your own diagram from scratch.With a diagram ready on the editor,
Run Workflowbutton on the editor300) for the vehicleRunYou should see the vehicle start to move forward on the road. Depending on which diagram/workflow you're using, it will react differently to its surrounding environment, e.g. respond to traffic signals, avoid pedestrians (tiny squares moving horizontally), or both! Feel free to use the user panel in the app to toggle some settings.
TODO
SpeedLimitintegration + create a node that processesJoinoperation forTrafficState