docs: clarify decimal negative scale behavior#10304
Conversation
|
@jayzhan211 @alamb @wjones127 Please accept my apologies. We agreed the last time while closing #5793 that we should fix the docs, and I was supposed to do it. Life happened in the meantime and I totally forgot about it. 😅 This morning I found it in my old TODOs and decided to do it. Lemme know what y'all think of the changes. |
| /// For example, with precision 2 and scale -3, the value 12345 is | ||
| /// rounded to 12000 and represented as the stored integer 12. |
There was a problem hiding this comment.
i find it a little confusing that we talk about how 12345 is converted to a negative scale decimal here, instead of in cast kernels 🤔
imo at this level all we care about is explaining how values already stored as decimals are represented and what they actually represent, aka we can say a value of 12 but with -3 scale is equivalent to 12000
(i skimmed the linked discussion so maybe this was already discussed 😅)
There was a problem hiding this comment.
I updated it. TBH I find explaining this somewhat confusing by nature. 😬
But please see if the newer one is more clearer.
|
thanks @ByteBaker |
Which issue does this PR close?
Rationale for this change
The issue was discussed in the PR and issue mentioned above and it was agreed here that we should update the docs. We just never got to actually doing it.
What changes are included in this PR?
Updates the
Decimal32,Decimal64,Decimal128, andDecimal256docs to explain that negative scale rounds values to the left of the decimal point, and that precision is the maximum number of non-rounded digits.Also adds an example showing how a value is rounded and represented internally.
Are these changes tested?
Ran fmt. Doc only changes, no other test required.
No runtime tests were added because this is a documentation-only change.
Are there any user-facing changes?
Documentation only. No API or behavior changes.