Bugfixes 2.2.1#152
Conversation
…umnValue
ColumnByColumnEqualsColumnValueGenerator emitted ({a} * {b}) <> {r} with no
tolerance. The CSV loader infers numeric columns as Float64/DOUBLE, so the
product is computed in IEEE-754 and compared bit-exactly. Rows whose
exact-decimal product equals the result but whose float64 product differs by a
rounding bit (e.g. 0.000015 * 20 -> 0.00030000000000000003 vs 0.0003) were
wrongly flagged non-conformant.
Replace the exact comparison in both the violation and predicate SQL with a
relative epsilon via a shared _tolerance_expr helper so the two can't drift:
ABS(({a} * {b}) - {r}) <= 1e-9 * GREATEST(ABS({r}), 1). The tolerance absorbs
only float representation error, staying faithful to FOCUS exact-equality intent.
Resolves false failures for CAU-ListCost-C-011-C, CAU-ListUnitPrice-C-014-C,
CAU-ContractedCost-C-011-C, and CAU-ContractedUnitPrice-C-013-C.
Add end-to-end DuckDB regression tests and update the composite-calculation
generator test to assert the new tolerance SQL.
Signed-off-by: Mike Fuller <mike@finops.org>
Signed-off-by: Mike Fuller <mike@finops.org>
Signed-off-by: Mike Fuller <mike@finops.org>
Matt-Cowsert
left a comment
Matt-Cowsert
left a comment
There was a problem hiding this comment.
Thanks Mike, the fixes all check out. I ran the #144 repro and the full test suite against this branch locally (429 passing), since CI can't run here until the merge conflict is cleared.
Two housekeeping items:
The diff is mostly the already-merged 2.2.0 work. The real change vs main is about 120 lines: the float tolerance fix, the publish workflow fix, the version bump, and the new tests. A rebase onto main will collapse the diff to just that, clear the conflict, and let CI run. It will also drop the two commits missing sign-offs that have DCO stuck (one is the suggestion of mine you accepted; those don't get signed automatically).
Suggest dropping #149 from the fix list. That cycle comes from the 1.4 model's dependency declarations, and the actual fix is the FOCUS_Spec model change you filed (FinOps-Open-Cost-and-Usage-Spec/FOCUS_Spec#2536), not anything in this PR. Dropping it here keeps the release notes accurate.
One inline suggestion on the new tag patterns; everything else looks good.
Co-authored-by: Matt Cowsert <matthew@finops.org>
Matt-Cowsert
left a comment
Matt-Cowsert
left a comment
There was a problem hiding this comment.
thanks for incorporating my suggestion MIke, approving now
2 participants
Bug fix list from raised issues: