[Question] Clarification on FP8 Micro-block Scaling and FP4 Support Timeline

[ultism]

Hi cuTile team,

I have two specific questions regarding the support for Blackwell-specific hardware features:

1. Automatic Micro-block Scaling for FP8
   When using fp8 with ct.matmul, how is the Micro-block Scaling (1x16) handled?

Automation: Does the tileiras compiler automatically handle the scaling logic and hardware invocation (5th-gen Tensor Core) under the hood?

Explicit Scaling: If it is not fully automatic, how should we provide the scale-factor tiles to the ct.matmul operator? Currently, the ct.matmul(A, B) signature seems to only accept data tiles. Is there a plan for a signature like ct.matmul(A, B, A_scale, B_scale)?

2. NVFP4 (FP4) Support Roadmap
   The current documentation and samples focus on fp8 and bf16. Since Blackwell's throughput peak is tied to NVFP4:
   When can we expect the support for 4-bit narrow-precision tiles in cuTile Python?

Thanks for this great library!

[haijieg]

Hi @ultranationalism

The current TileIR release (CTK 13.1) only provides Tensorcore enabled FP8 e4m3fn and e5m2 matmul without micro-scaling. Micro scaling (MXFP8) and NVFP4 support is being actively worked on and will be provided in the near future CTK releases. As for API in cutile, we haven't started designing it yet. Something like ct.scaled_matmul(A, B, A_scale, B_scale) is a reasonable starting point.

[Edward-lyz]

Hi   嗨@ultranationalism

The current TileIR release (CTK 13.1) only provides Tensorcore enabled FP8 e4m3fn and e5m2 matmul without micro-scaling. Micro scaling (MXFP8) and NVFP4 support is being actively worked on and will be provided in the near future CTK releases. As for API in cutile, we haven't started designing it yet. Something like ct.scaled_matmul(A, B, A_scale, B_scale) is a reasonable starting point.当前的 TileIR 发布版本（CTK 13.1）仅提供支持 Tensorcore 的 FP8 e4m3fn 和 e5m2 矩阵乘法，没有微缩放功能。微缩放（MXFP8）和 NVFP4 支持正在积极开发中，将在不久的将来提供的 CTK 版本中实现。至于 cutile 中的 API，我们还没有开始设计它。类似 ct.scaled_matmul(A, B, A_scale, B_scale) 可以作为合理的起点。

That's great! This is a really important feature for developers.

[odelame]

Hey, is there an update on this? An expected timeline? Is it still being worked on?

[haijieg]

Hi @odelame, yes it's being worked on and will be ready soon.

[ultism]

Hi @odelame, yes it's being worked on and will be ready soon.

What a pity. I've already written my nvfp4 operator using the CUTE DSL and tuned it to production-ready level. I'll just have to wait for another chance to try out this framework.

https://github.com/ultranationalism/svdquant-kernels :)

[haijieg]

mma_scaled, mxfp8 and nvfp4 is supported by v1.4.0.