Skip to content

Expose RoPE theta via config API; cache fp32 cos/sin tables - #320

Open
KakaruHayate wants to merge 3 commits into
openvpi:mainfrom
KakaruHayate:feat/expose-rope-config
Open

Expose RoPE theta via config API; cache fp32 cos/sin tables#320
KakaruHayate wants to merge 3 commits into
openvpi:mainfrom
KakaruHayate:feat/expose-rope-config

Conversation

@KakaruHayate

@KakaruHayate KakaruHayate commented Aug 8, 2026

Copy link
Copy Markdown

Summary

Two changes in one PR:

  1. Expose RoPE theta via config API — forward rope_theta (plus existing rope_interleaved) through the encoder, so RoPE theta is config-tunable instead of hardcoded to 10000.
  2. Refactor RoPE to cache fp32 cos/sin tables (modules/commons/rotary_embedding_torch.py), per review comments:
    • cache fp32 cos/sin at init instead of raw angles, so trig runs once instead of every forward;
    • tables stay fp32, only cast to the activation dtype at use — removes the error of recomputing trig on fp16/bf16 angles during 16-mixed training;
    • the interleaved path no longer needs einops, simplifying exported ONNX graphs (no Cos/Sin/Einsum ops).

Compatibility

fp32 path is bit-identical to the previous implementation; only the fp16/bf16 column improves in precision. Buffers remain persistent=False, so no checkpoint or config changes — existing models resume seamlessly. No tests included, per repo convention (verified via normal community testing).

@KakaruHayate
KakaruHayate force-pushed the feat/expose-rope-config branch 2 times, most recently from ffba20b to 3956360 Compare August 8, 2026 03:42
@KakaruHayate KakaruHayate changed the title feat: expose RoPE theta/dim via config API feat: expose RoPE theta via config API Aug 8, 2026
@KakaruHayate
KakaruHayate force-pushed the feat/expose-rope-config branch from 3956360 to f5ba7b6 Compare August 8, 2026 04:00
@KakaruHayate KakaruHayate changed the title feat: expose RoPE theta via config API Expose RoPE theta via config API; cache fp32 cos/sin tables Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant