fix: address Ideogram review feedback

[aegioscy]

Summary

• Register and apply optional Ideogram weight_scale tensors for linear layers so FP8-with-scale checkpoints are not silently run without their scales.
• Use graph-reachable parameter staging for Ideogram non-segmented compute so cond/uncond passes only offload the active transformer weights.
• Leaves the requested CFG/uncond-path enforcement comment out of scope per follow-up direction.

Test plan

• git diff --check
• cmake -S . -B build-ideogram-review -DSD_BUILD_EXAMPLES=OFF -DSD_BUILD_SHARED_LIBS=OFF -DGGML_NATIVE=OFF
• cmake --build build-ideogram-review --config Release -j 8

Made with Cursor

[gianni-cor]

This still applies weight_scale in the wrong order for biased linear layers. Linear::forward() has already added the bias, so this returns (xW + b) * weight_scale. For FP8 scaled weights the scale belongs to the weight product, not the bias; the expected computation is xW * weight_scale + b.

This matters for Ideogram because many of these linears have bias=true (input_proj, llm_cond_proj, timestep/adalan projections, final linear, etc.), so official FP8-with-scale checkpoints will still produce incorrect activations even though the scale tensor is now loaded. Please either teach Linear to handle weight_scale before bias, or have Ideogram4Linear call the underlying matmul without bias, multiply by weight_scale, then add bias afterward.