Skip to content

Benchmark PRs on PGLib IEEE 300 - #70

Merged
samtalki merged 1 commit into
mainfrom
ck/issue-61-larger-pr-benchmarks
Aug 11, 2026
Merged

Benchmark PRs on PGLib IEEE 300#70
samtalki merged 1 commit into
mainfrom
ck/issue-61-larger-pr-benchmarks

Conversation

@cameronkhanpour

Copy link
Copy Markdown
Collaborator

Summary

  • Run PR benchmarks on the PGLib IEEE 300 case instead of PowerModels' IEEE 30 fixture.
  • Resolve the pinned case through PowerDiff's existing PGLib artifact.
  • Remove the now-unused PowerModels benchmark dependency and fallback loader.

Why

The previous 30-bus case left several KKT benchmarks in the sub-microsecond range, which made PR-to-main comparisons noisy and less informative. IEEE 300 is large enough to expose meaningful runtime and allocation changes while remaining practical for routine pull request runs.

PEGASE 1354 was considered, but its dense AC switching Jacobian would occupy roughly 510 MiB per evaluation. IEEE 300 keeps that matrix near 22 MiB.

Impact

Benchmark results will report the 300-bus case across parser, DC OPF, and AC OPF groups. The workflow no longer installs PowerModels solely to find a test fixture.

Validation

  • Loaded the benchmark script in a fresh Julia environment without PowerModels.
  • Ran every benchmark once with samples=1, evals=1.
  • julia --project=. -e "using Pkg; Pkg.test()" (2,916 assertions passed).
  • Parsed .github/workflows/Benchmark.yml successfully.

Closes #61

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results (Julia v1)

Time benchmarks
main f626352... main / f626352...
ac_opf/kkt_jacobian/pglib_opf_case300_ieee.m 29.8 ± 1.9 ms 29.7 ± 0.95 ms 1.01 ± 0.07
ac_opf/kkt_param/pglib_opf_case300_ieee.m/switching 3.13 ± 0.75 ms 3.12 ± 0.52 ms 1 ± 0.29
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/cost_linear 1.11 ± 0.29 μs 1.21 ± 0.33 μs 0.92 ± 0.35
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/cost_quadratic 0.558 ± 0.18 μs 0.549 ± 0.18 μs 1.02 ± 0.47
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/demand 3.12 ± 0.44 μs 2.99 ± 0.47 μs 1.05 ± 0.22
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/flowlimit 7.7 ± 3.5 μs 7.5 ± 4 μs 1.03 ± 0.72
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/full 0.089 ± 0.021 ms 0.0848 ± 0.023 ms 1.05 ± 0.37
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/susceptance 6.25 ± 0.27 ms 5.82 ± 0.32 ms 1.07 ± 0.075
parser/pglib_opf_case300_ieee.m 0.694 ± 0.013 ms 0.694 ± 0.015 ms 1 ± 0.029
time_to_load 1.98 ± 0.02 s 1.96 ± 0.018 s 1.01 ± 0.013
Memory benchmarks
main f626352... main / f626352...
ac_opf/kkt_jacobian/pglib_opf_case300_ieee.m 0.373 M allocs: 12.3 MB 0.373 M allocs: 12.3 MB 1
ac_opf/kkt_param/pglib_opf_case300_ieee.m/switching 14.8 k allocs: 26.2 MB 14.8 k allocs: 26.2 MB 1
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/cost_linear 6 allocs: 1.83 kB 6 allocs: 1.83 kB 1
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/cost_quadratic 6 allocs: 1.83 kB 6 allocs: 1.83 kB 1
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/demand 9 allocs: 12 kB 9 allocs: 12 kB 1
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/flowlimit 9 allocs: 16.3 kB 9 allocs: 16.3 kB 1
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/full 0.108 k allocs: 0.751 MB 0.108 k allocs: 0.751 MB 1
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/susceptance 27.4 k allocs: 21.5 MB 27.4 k allocs: 21.5 MB 1
parser/pglib_opf_case300_ieee.m 0.082 k allocs: 4.7 kB 0.082 k allocs: 4.7 kB 1
time_to_load 0.149 k allocs: 11.1 kB 0.149 k allocs: 11.1 kB 1

@cameronkhanpour
cameronkhanpour marked this pull request as ready for review August 11, 2026 14:11
@samtalki
samtalki merged commit 016ba65 into main Aug 11, 2026
5 checks passed
@samtalki
samtalki deleted the ck/issue-61-larger-pr-benchmarks branch August 11, 2026 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consider larger case(s) for PR benchmarks

2 participants