Add confidence intervals to IRV charts

[fsargent]

Addresses #47.

Computes 95% normal-approximation confidence intervals from election-level IRV VSE observations, writes interval fields to the reproducible CSV, and renders interval bars for IRV/RCV points in the static charts.

The companion #54 adds the reproducible full-method CSV path needed to extend intervals across every chart series.

Validation:

• uv run python scripts/recalculate_irv_pages.py --elections 20 --workers 1 --output /tmp/irv-ci-smoke.csv
• uv run python -m pytest -q
• uv run --group dev python scripts/regenerate_pages_images.py --workers 10

Summary by Sourcery

Add calculation and export of 95% confidence intervals for IRV VSE results and render them in static charts.

New Features:

• Compute election-level 95% normal-approximation confidence intervals for IRV VSE metrics.
• Include confidence interval statistics in the IRV reproducible CSV output.
• Display confidence interval error bars for IRV/RCV points in static VSE charts.

Enhancements:

• Extend IRV recalculation pipeline to track squared VSE sums needed for variance estimation.
• Document the confidence interval methodology and its use in IRV simulation charts.

Tests:

• Update regression smoke test for IRV recalculation to cover confidence interval outputs.

[sourcery-ai]

Sorry @fsargent, your pull request is larger than the review limit of 150000 diff characters

[coderabbitai]

Warning

Review limit reached

@fsargent, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 38 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 708ecebe-e4dd-42ba-a235-c566134bf00f

📥 Commits

Reviewing files that changed from the base of the PR and between 605e6c4 and 3851689.

⛔ Files ignored due to path filters (4)

• docs/5vse.png is excluded by !**/*.png
• docs/5vse_small.png is excluded by !**/*.png
• docs/vse.png is excluded by !**/*.png
• docs/vsestrat.png is excluded by !**/*.png

📒 Files selected for processing (4)

• docs/irv-simulation.md
• scripts/recalculate_irv_pages.py
• scripts/regenerate_pages_images.py
• tests/test_regressions.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/p1-confidence-intervals

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[sourcery-ai]

Reviewer's Guide

Adds computation and propagation of 95% normal-approximation confidence intervals for IRV/RCV VSE results, writes them to the IRV reproducible CSV, and renders them as error bars on static VSE charts while updating tests and docs accordingly.

Sequence diagram for propagating IRV/RCV confidence intervals into static VSE charts

sequenceDiagram
    participant main
    participant recalculate
    participant render_vse
    participant matplotlib_axis

    main->>recalculate: recalculate(elections, seed, workers)
    recalculate-->>main: results, outcomes, scenario_results, scenario_outcomes, intervals

    main->>render_vse: render_vse(data, vse_png_path, size, intervals)
    render_vse->>matplotlib_axis: axis.scatter(value, positions[method])
    render_vse->>matplotlib_axis: axis.errorbar(value, positions[method], xerr=intervals[IRV_CHOOSERS[strategy]])

Loading

File-Level Changes

Change	Details	Files
Accumulate second moments during IRV VSE recalculation to compute 95% confidence intervals and return them from the pipeline.	Extend summary and scenario_summary accumulators to track count, sum, and sum of squares for VSE observations. Merge per-chunk sum-of-squares into the overall aggregates when recombining worker results. Compute mean VSE per chooser and derive a normal-approximation variance and 95% confidence-interval half-width per chooser using 1.96 * sqrt(variance / n). Add a confidence_intervals mapping to the recalculate return tuple and propagate it to callers.	scripts/recalculate_irv_pages.py
Write confidence-interval values into the IRV reproducible CSV output for downstream use.	Add ci95_half_width and low/high percent VSE bounds to each output CSV row for each chooser. Update CSV header to include the new confidence-interval fields. Ensure CI values default correctly for choosers with fewer than two observations.	scripts/recalculate_irv_pages.py
Render IRV/RCV confidence intervals as horizontal error bars in static VSE charts.	Update render_vse signature to accept an intervals mapping from chooser to CI half-width. Draw matplotlib errorbar overlays for IRV/RCV methods using strategy-specific colors and IRV_CHOOSERS mapping to pick the correct interval. Thread intervals from recalculate through main into all VSE chart render calls, including selected-method variants.	scripts/regenerate_pages_images.py
Document the presence and provenance of IRV/RCV confidence intervals in the IRV simulation docs.	Add a short explanation that static VSE charts now show 95% normal-approximation CIs for IRV/RCV. Clarify that CIs are computed from election-level VSE values from the same seeded run and that the CSV includes the CI half-width.	docs/irv-simulation.md
Keep regression coverage in sync with the new confidence-interval output from IRV recalculation.	Adjust test_irv_recalculation_smoke to unpack the new intervals value returned from recalculate. Assert that the intervals mapping includes honBallot and that its CI is non-negative, maintaining a basic sanity check.	tests/test_regressions.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've left some high level feedback:

• The use of [count, total, total_squared] lists in summary/scenario_summary with positional indexing is getting a bit brittle; consider wrapping these in a small dataclass/namedtuple or helper functions to make the statistics fields self-documenting and less error-prone to extend.
• In render_vse, intervals[IRV_CHOOSERS[strategy]] assumes every plotted IRV/RCV strategy has a corresponding interval; it may be safer to use .get (or gate on membership) so chart rendering degrades gracefully if a mapping or interval entry is missing.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The use of `[count, total, total_squared]` lists in `summary`/`scenario_summary` with positional indexing is getting a bit brittle; consider wrapping these in a small dataclass/namedtuple or helper functions to make the statistics fields self-documenting and less error-prone to extend.
- In `render_vse`, `intervals[IRV_CHOOSERS[strategy]]` assumes every plotted IRV/RCV strategy has a corresponding interval; it may be safer to use `.get` (or gate on membership) so chart rendering degrades gracefully if a mapping or interval entry is missing.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}