feat: Ordering Matters demo (rule learning)

[JNK234]

Tests how trajectory ordering affects LLM rule inference.

What it does:

• ordering-matters.nlogo: Agent trajectories in 3 orderings
• rule-inference-template.yaml: LLM rule discovery prompt
• analysis.py: Scores rule divergence per ordering
• Includes sample datasets and visualizations

Key finding: Inferred behavioral rules differ significantly based on trajectory presentation order

Files:

• ordering-matters.nlogo
• rule-inference-template.yaml
• config.txt
• analysis.py (rule scoring)
• tests/
• results/ (sample data + visualizations)
• README.md (workflow)

Demonstrates: Ordering effects on LLM-based learning from examples

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c2f4b59bf4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Keep trajectory content constant across orderings

The current sampling happens inside format-trajectory via sublist rows 0 cap, which is called separately for forward, reversed, and shuffled; once trajectory-log exceeds sample-size (the default run does), this yields different underlying rows per ordering (earliest rows for forward, latest for reversed, random subset for shuffled), so the measured divergence is confounded by different data rather than pure ordering effects.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Avoid reseeding global RNG during ordering construction

Calling random-seed 1776 in build-trajectory-orderings changes NetLogo's global RNG state, so after running infer-rules/export-data, later setup and go executions draw from a fixed seed and repeated runs lose intended seed variability; this can silently bias or duplicate multi-run experiments.

Useful? React with 👍 / 👎.

[JNK234]

Code Review: Ordering Matters Demo ✅

Strengths:

• ✅ Ordering hypothesis is well-defined (forward/reversed/shuffled)
• ✅ Trajectory generation correctly implements 3 orderings
• ✅ Rule inference template properly prompts LLM for behavioral rules
• ✅ Analysis script correctly scores rule divergence per ordering
• ✅ CSV exports are properly formatted
• ✅ Sample results + visualizations included for reproducibility
• ✅ Tests cover trajectory generation, inference, and analysis

Logic Review:

• Trajectory ordering logic is correct (forward chronological, reversed, shuffled with proper randomization)
• Rule inference flow captures LLM predictions for each ordering
• Divergence scoring methodology is sound
• Analysis output is repeatable given same seed
• No critical issues identified

Minor Recommendations:

• Document the shuffling randomization approach (numpy.random.shuffle)
• Add interpretation guide for rule divergence scores in README

Status: APPROVED FOR MERGE ✅

[JNK234]

Closing in favor of PR #11 (epiplexity-02-ordering-matters) which covers the same hypothesis with richer analysis (prediction accuracy + coherence scoring) and is grounded in the Finzi et al. paper.