FEAT: allow use of different numbers of samples per event - #142
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #142 +/- ##
==========================================
+ Coverage 95.15% 95.21% +0.06%
==========================================
Files 14 14
Lines 907 940 +33
==========================================
+ Hits 863 895 +32
- Misses 44 45 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR adds an opt-in/opt-out mechanism to support different numbers of posterior samples per event in HyperparameterLikelihood, while retaining a backward-compatible mode that enforces equal sample counts per event.
Changes:
- Add
require_equal_samplesflag and implement per-event sample-count handling via flattened storage + eventtransitions. - Refactor expectation/variance calculations to work with either equal-sample (2D) or variable-sample (flattened) representations.
- Update tests to explicitly request backward-compatible behavior and to parametrize the new flag in the example test.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 9 comments.
| File | Description |
|---|---|
gwpopulation/hyperpe.py |
Adds require_equal_samples flag, implements variable-sample resampling, and centralizes expectation logic. |
gwpopulation/experimental/numpyro.py |
Switches expectation computation to reuse likelihood._weight_expectation for consistency with new storage modes. |
test/likelihood_test.py |
Updates existing tests to enforce backward-compatible equal-sample behavior explicitly. |
test/example_test.py |
Parametrizes tests over require_equal_samples (and JIT) to exercise both modes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
I'll take a look. But I'll note that I've run into issues with this approach in practice, which manifests differently depending on the sampler:
I'm still not sure why this happens. A couple of ideas are that I'm using way too many samples overall, or that summing in probability space instead of log space over all those samples is just not stable. I need to do some more systematic tests though, because it does work in some cases 🤷 |
|
Thanks for the comments! |
|
I've also seen that issue with the equal samples version. I compared the evaluation of the likelihood and variance with my implementation (on GWTC-5 BBHs) - the variance matched but only to 1 d.p. (even in 64 bit, maybe expected given such a large cumulative sum?), and the log likelihoods differed but by a fixed constant (again to around 1 d.p., I'm not sure where the difference in normalization lies but it doesn't matter anyway) |
This uses an approach suggested by @mdmould to allow different numbers of samples per event.
@mdmould, do you think you could take a look at this?