[libcudacxx][fp] add fptool: fp_custom and fpmp2_stat + unit tests by akolesov-nvidia · Pull Request #10830 · NVIDIA/cccl

akolesov-nvidia · 2026-08-14T20:34:53Z

Description

Adds fptool, the analysis half of the FP feature set, behind one new umbrella
<cuda/fptool>. fp_custom is a drop-in double whose exponent and mantissa sizes are
template parameters, so an application swaps one typedef, reruns, and sees where
reduced precision moves its result; the sizes can also be left dynamic and set at
run time. fpmp2_stat is a drop-in fpmp2 that records what the arithmetic did - the
operation counts, the exponent range and limb gap of every operand and result, and
the degenerate events such as cancellation, overflow and inverted limb pairs - while
reproducing the wrapped type's results bit for bit, so measuring a run costs one
typedef as well. Both live in cuda::experimental and rest on the already-promoted
fpemu (#9777) and fpmp (#10517, #10697).
Nine files, of which four are unit tests, all run on host and device under C++17 and
C++20 and checked through NVRTC. Nothing existing changes behavior; the only edit
outside the new files is a four-line fix in fpmp_impl.h, whose host fallbacks called
fma, sqrt and round unqualified from inside cuda::experimental, which made including
<cuda/fptool> before <cuda/fpmp_math> fail to compile.

closes

Checklist

New or existing tests cover these changes.
The documentation is up to date with these changes.

The host branches of the double-precision primitives called fma, sqrt and round unqualified from inside cuda::experimental, relying on the name resolving to the global namespace. Any header that declares an overload of one of those names in cuda::experimental first stops that lookup, so including <cuda/fptool> before <cuda/fpmp_math> failed to compile. Spell them ::cuda::std::<fn> instead, which is the portable form used elsewhere in the file, and ::rsqrt for the CUDA builtin in the device branch.

A drop-in `double` whose exponent and mantissa sizes are template parameters, for algorithm sensitivity analysis: narrow the format, rerun, see where the result moves. Every arithmetic operation rounds its result to the requested format, so an application changes one typedef and nothing else. The sizes can also be left dynamic and set at run time, which lets one build sweep a range of formats. Sits in <cuda/fptool> next to fpemu and fpmp, and needs neither of them.

A drop-in fpmp2 that records what the arithmetic did: how many operations of each kind, the exponent range and limb gap of every operand and result, and the degenerate events - cancellation, underflow, overflow, subnormals, overlapping and inverted limb pairs. Numerically it is the wrapped type exactly, so results are bit-identical and an application swaps one typedef to measure a run. Counting happens on the device through atomics on one program-wide record; the host side is a transparent pass-through. The math functions come with the type in <cuda/fptool>, since these are analysis types that few translation units carry.

main moved to clang-format 22.1.5 in NVIDIA#10729, which aligns continuation lines differently from the 20.1.7 these files were written under.

copy-pr-bot · 2026-08-14T20:34:57Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

akolesov-nvidia · 2026-08-14T21:07:51Z

/ok to test 32ffc15

akolesov-nvidia · 2026-08-14T22:39:34Z

/ok to test b91307e

akolesov-nvidia · 2026-08-15T04:25:24Z

/ok to test 11571d8

akolesov-nvidia · 2026-08-17T16:09:05Z

/ok to test 11571d8

akolesov-nvidia · 2026-08-17T16:51:14Z

/ok to test f3e5484

github-actions · 2026-08-17T17:30:01Z

⏱️ CCCL compile-time benchmark comparison: Public headers compile-time bench

Result: 0 regression row(s), 3 improvement row(s) above threshold.

Run	Value
Config	`public-headers-gcc13`
Baseline	`origin/main`
Preset	`all-dev`
Targets	`cub.headers.base, thrust.cpp.cuda.headers.base, libcudacxx.test.public_headers`
GPU / launch args	`rtx2080` / `--cuda 13.3 --host gcc13`

Artifacts: reports and traces

Direct file processing

-f file-processing exclusive --sort total

🟢 Direct file processing — Improvements

Rank	Improvement impact	Selected Δ	Baseline	Current	Event	Matched traces
1	`0.680044`	`-0.680044`	`5.857029`	`5.176985`	Processing Header File: `libcudacxx/include/cuda/std/__cccl/prologue.h`	552
2	`0.237293`	`-0.237293`	`10.850170`	`10.612877`	Processing Header File: `libcudacxx/include/cuda/__device/physical_device.h`	93
3	`0.206977`	`-0.206977`	`1.807475`	`1.600498`	Processing Header File: `libcudacxx/include/cuda/std/__cccl/epilogue.h`	552

Review follow-up. The host side of the fp_custom runtime sizes and of the fpmp2_stat record went through cudaMemcpyToSymbol, which picks the current device, synchronizes with the default stream and returns a status that the callers dropped. Both now take a cuda::stream_ref: the stream names the device the state belongs to and orders the copy against the kernels that read it, and a failure throws cuda::cuda_error. fpmp2_stat_read_device_data returns the record by value. The device-code accessors stay, for a JIT program with no host side to set a size from, and are now device-only rather than host-device. Conversion out of fp_custom to float is implicit where binary32 represents the requested format and explicit anywhere the mantissa bits or the exponent range would be lost, instead of always explicit.

The record's fields carried a paragraph each while others had a line, so the reference was spread over two structs and the file comment. The file documentation now has a Metrics section covering the operation counters, the numerical events, the per-slot statistics and how to read the limb gap, with a short table naming the fields and prose where a metric needs it; the fields keep a one-line description each. Nothing is dropped: the field paragraphs move up, and the prose from the fpdev metrics note comes with them, including what each metric is good for and the paragraph the inf/NaN counters never had.

The low level built a two-word reciprocal - a Newton step, a squared reciprocal, a Dekker product against the divisor and a low_mul by the numerator - which came to more arithmetic than the mid-accuracy quotient it was meant to undercut, and in a dependent chain it divided slower than mid. It now runs the same Nagai iteration as mid, stopped after the first residual correction and without the closing renormalization, leaving the pair unnormalized as this level does throughout. That is a reciprocal plus five operations, nine SASS instructions against twenty-four, and on an L40S it divides 2.2x faster than mid for one bit less. The residual is exact: the quotient estimate is within an ulp or two of a_hi/b_hi, so the subtraction inside the fma cancels without rounding. Measured with ts/fpmp on fp32mp2: the division holds 44 of 48 bits where it used to range over 43 to 45, and the eight math functions that divide at this level gain 3 to 11 percent with no accuracy class changing. tan and atan2 ask for the mid division explicitly and are untouched.

akolesov-nvidia · 2026-08-18T21:29:36Z

/ok to test de34b2d

The unpacked form's mantissa field carries 9 guard bits below the binary64 significand (EXTRA_BITS, placing the significand at bits 61..9), so a chain that stays unpacked computes on 62 significand bits and rounds to the storage format once, at the pack. That is why it is more accurate than double or packed fpemu in a chain - and why its chained results are not bit-identical to double even at fpemu_accuracy::high, whose individual operations are. Both properties are intentional, and neither was written down where a user would look for it.

…VRTC <cuda/stream> cannot be included under NVRTC: stream_ref.h defines stream_ref only where the CUDA runtime is available, while get_stream.h declares an operator() returning it by value regardless, so the umbrella header fails with an incomplete type. The stream-based runtime-size cases in this test were already guarded for NVRTC; the include was not, which failed the C++17 and C++20 NVRTC jobs. Guarding only the include keeps NVRTC coverage of the stream-independent fp_custom tests. Verified by compiling the test with NVRTC directly: the incomplete-type errors from get_stream.h are gone.

akolesov-nvidia · 2026-08-19T01:03:53Z

/ok to test daadb15

Answers the review on the promotion PR. Drop the <cuda_runtime.h> includes from fptool_custom.h and fptool_stat.h. A CUDA compiler pre-includes that header, and the host-only path already gets cuda_runtime_api.h from the CCCL config when a CTK is present, so naming it bought nothing. Take a value into fp_custom implicitly only where the requested format holds the source exactly, which is the rank rule CCCL applies to its floating-point types, with integers counting as double. A narrower or dynamic format takes one by cast instead. The float constructor is constrained to a deduced float rather than naming one, so that a double cannot reach a narrow format through it, which would round it twice and through the narrower range of the two. What the cast reports is the format the value is entering, not a loss in the constructor: fp_custom stores the value unreduced, and the sizes are applied by the first arithmetic operation. Add CCCL_FP_CUSTOM_EXPLICIT_CASTS alongside, defaulting to 1 and mirroring CCCL_FPMP_EXPLICIT_CASTS. Setting it to 0 leaves the narrowing side implicit, for moving a codebase written against double onto the type, which is what fp_custom is for. custom.pass.cpp asserts the conversion matrix at both settings.

akolesov-nvidia · 2026-08-19T17:37:07Z

/ok to test a1489b8

github-actions · 2026-08-19T19:15:26Z

🥳 CI Workflow Results

🟩 Finished in 1h 34m: Pass: 100%/117 | Total: 2d 04h | Max: 1h 34m | Hits: 71%/481930

See results here.

miscco · 2026-08-20T08:32:48Z

+  static constexpr uint16_t __mant_size = 52;
+};
+
+#if __STDCPP_FLOAT64_T__ == 1 && !_CCCL_CUDA_COMPILER(NVCC)


This should be a global _CCCL_HAS_FLOAT64() define

miscco · 2026-08-20T08:37:03Z

+#if _CCCL_CUDA_COMPILATION()
+    NV_IF_ELSE_TARGET(NV_IS_DEVICE,
+                      (return __fp_custom_device_mantissa_size<_FpType>;),
+                      (return __fp_custom_host_mantissa_size<_FpType>;))
+#else // ^^^ _CCCL_CUDA_COMPILATION() ^^^ / vvv !_CCCL_CUDA_COMPILATION() vvv
+    return __fp_custom_host_mantissa_size<_FpType>;
+#endif // !_CCCL_CUDA_COMPILATION()


This can just be

Suggested change

#if _CCCL_CUDA_COMPILATION()

NV_IF_ELSE_TARGET(NV_IS_DEVICE,

(return __fp_custom_device_mantissa_size<_FpType>;),

(return __fp_custom_host_mantissa_size<_FpType>;))

#else // ^^^ _CCCL_CUDA_COMPILATION() ^^^ / vvv !_CCCL_CUDA_COMPILATION() vvv

return __fp_custom_host_mantissa_size<_FpType>;

#endif // !_CCCL_CUDA_COMPILATION()

NV_IF_ELSE_TARGET(NV_IS_DEVICE,

(return __fp_custom_device_mantissa_size<_FpType>;),

(return __fp_custom_host_mantissa_size<_FpType>;))

miscco · 2026-08-20T08:37:50Z

+    int64_t __unbiased_exp = static_cast<int64_t>(__exp_bits) - __original_bias;
+    int64_t __new_exp_bits = __unbiased_exp + __new_bias;


Nitpick: could be const

akolesov-nvidia added 4 commits August 14, 2026 11:25

fptool: format with the clang-format main pins

1a9410e

main moved to clang-format 22.1.5 in NVIDIA#10729, which aligns continuation lines differently from the 20.1.7 these files were written under.

akolesov-nvidia requested a review from davebayer August 14, 2026 20:34

github-project-automation Bot added this to CCCL Aug 14, 2026

github-project-automation Bot moved this to Todo in CCCL Aug 14, 2026

cccl-authenticator-app Bot moved this from Todo to In Progress in CCCL Aug 14, 2026

akolesov-nvidia added 2 commits August 14, 2026 13:55

fptool: fix end-of-file in <cuda/fptool>

aaac255

Merge branch 'main' into dev/akolesov/fptool_draft

32ffc15