Skip to content

fix(tensor): relu/leaky_relu/softmax/log_softmax accept a scalar (#632)#664

Merged
InauguralPhysicist merged 1 commit into
mainfrom
fix/tensor-scalar-632
Jul 18, 2026
Merged

fix(tensor): relu/leaky_relu/softmax/log_softmax accept a scalar (#632)#664
InauguralPhysicist merged 1 commit into
mainfrom
fix/tensor-scalar-632

Conversation

@InauguralPhysicist

Copy link
Copy Markdown
Collaborator

Closes #632

Four tensor functions returned a silent null — not a value, not an error — when handed a scalar, while every sibling in the same docs/BUILTINS.md tables (sqrt, exp, log, negative, add) is scalar-clean.

tensor_to_flat returns NULL for a 0-dim VAL_NUM, and each function did if (!flat) return make_null().

Why the silent null is the real problem

It propagates until it hits an unrelated operator, so the failure surfaces far from the cause:

define layer(x) as: return relu of x
layer of ([-3, 4])     # [0, 4]
layer of -3            # null      <- silently poisons
(relu of -3) + 1       # Error line N: cannot apply '+' to null and num

A generic layer that works on a batch quietly poisons on a single sample. Same shape as the #490#512 silent-tolerance train.

Fix

A scalar fast path at the top of each — the degenerate element-wise case, matching sqrt/exp/log:

function scalar result
relu of x max(0, x)
leaky_relu of x x < 0 ? 0.01x : x
softmax of x 1.0 (one-element normalization)
log_softmax of x 0.0 (log(1))

The list/buffer/2-D paths are untouched — relu of [-3, 4] still gives [0, 4].

Verification

All from the issue, confirmed on main:

relu of -3        -> 0      leaky_relu of -10 -> -0.1
relu of 5         -> 5      softmax of 1      -> 1
relu of [-3, 4]   -> [0,4]  log_softmax of 1  -> 0
(relu of -3) + 1  -> 1      (no more null poison)

Six regression asserts added to test_flat_buffer_tensor.eigs, each failing on the pre-fix null (including the downstream-poison case). BUILTINS.md documents the scalar behavior for all four.

Gates

  • Release suite 3078/3078
  • ASan + UBSan detect_leaks=1 3082/3082, leak tally 0
  • doc-drift clean

🤖 Generated with Claude Code

The four returned a silent null — not a value, not an error — when handed a
scalar, while every sibling in the same BUILTINS.md tables (sqrt, exp, log,
negative, add) is scalar-clean. tensor_to_flat returns NULL for a 0-dim
VAL_NUM, and the functions did `if (!flat) return make_null()`.

The silent null is the real defect: it propagates until it hits an unrelated
operator, so the error surfaces far from the cause —

    define layer(x) as: return relu of x
    layer of ([-3, 4])        # [0, 4]
    layer of -3               # null   <- silently poisons
    (relu of -3) + 1          # Error line N: cannot apply '+' to null and num

A generic layer that works on a batch quietly poisons on a single sample.
Same shape as the #490-#512 silent-tolerance train.

Fix: a scalar fast path at the top of each, the degenerate element-wise case,
matching sqrt/exp/log:
  relu(x)        = max(0, x)
  leaky_relu(x)  = x < 0 ? 0.01x : x
  softmax(x)     = 1.0        (one-element normalization)
  log_softmax(x) = 0.0        (log(1))

Tests: six asserts in test_flat_buffer_tensor.eigs (each fails on the pre-fix
null), including that (relu of -3) + 1 no longer poisons downstream. BUILTINS.md
notes the scalar behavior for all four.

Suite 3078/3078 release, 3082/3082 ASan+UBSan (detect_leaks=1, tally 0).
@InauguralPhysicist
InauguralPhysicist merged commit bb44746 into main Jul 18, 2026
18 checks passed
@InauguralPhysicist
InauguralPhysicist deleted the fix/tensor-scalar-632 branch July 18, 2026 03:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

relu / leaky_relu / softmax / log_softmax silently return null on a scalar

1 participant