Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,23 @@ All notable changes to EigenScript are documented here.

## [Unreleased]

### Added
- **gfx audio capture: `audio_capture_open` / `audio_capture_read` /
`audio_capture_close` (#579).** The gfx extension's audio surface was
output-only; a DAW transport (DeslanStudio F-DS-17) could never
record a real microphone. Pull-model recording via the already
dlopen'd SDL2: `SDL_OpenAudioDevice(iscapture=1)` in queue mode +
`SDL_DequeueAudio` — no callback, no C→eigs re-entry, fits the
existing polling clients. `audio_capture_read` drains new samples as
a **buffer** (the #578 fast type) of floats in `[-1, 1]`, capped at
2048 samples per call so each trace record stays under the 64 KiB
budget (i.e. replayable); loop until empty to drain fully. Tape-first:
open and read are TAKE/RECORD builtins — under `EIGS_REPLAY` no real
device is opened or read, the recorded id and sample buffers are
served off the tape (docs/TRACE.md). `SDL_AUDIODRIVER=dummy` provides
a silent capture device, so the gfx-suite tests run without hardware;
graceful `0`/`null` without SDL or a device.

### Fixed
- **gfx audio: `audio_play` / `audio_play_loop` accept `VAL_BUFFER`
samples (#578).** `audio_convert_samples` was list-only, so handing a
Expand Down
6 changes: 5 additions & 1 deletion docs/BUILTINS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ audio (`audio_open`, `audio_close`, `audio_pause`, `audio_play`,
`audio_play_loop`, `audio_volume`, `audio_stop`, `audio_queue_size`, `audio_clear`, `audio_sine`,
`audio_saw`, `audio_sweep`,
`audio_square`, `audio_noise`, `audio_mix`, `audio_gain`,
`audio_envelope`), and `free_val` for memory management.
`audio_envelope`, `audio_capture_open`, `audio_capture_read`,
`audio_capture_close`), and `free_val` for memory management.

## Core Language

Expand Down Expand Up @@ -615,6 +616,9 @@ receiver.
| `audio_play_loop` | `audio_play_loop of [samples, loops]` | Play `samples` `loops` times on one mixer channel; `loops == -1` loops forever (the mixer rewinds — no memory multiplication). Returns the channel id, or `0` on bad args / closed device. |
| `audio_volume` | `audio_volume of [channel, vol]` | Live per-channel volume, `0.0`–`4.0`. Returns `1`, or `0` on a bad/inactive channel. |
| `audio_stop` | `audio_stop of channel` | Stop one mixer channel. Returns `1`, or `0` on a bad/inactive channel. |
| `audio_capture_open` | `audio_capture_open of [freq, channels]` | Open the recording (microphone) device and start capturing (#579). Defaults `[44100, 1]`; SDL converts to exactly the requested format. Returns the device id, or `0` when SDL/capture is unavailable. Re-opening closes the previous capture device. Trace-recorded — under `EIGS_REPLAY` no real device is opened. |
| `audio_capture_read` | `audio_capture_read of null` | Drain samples accumulated since the last read as a **buffer** of floats in `[-1, 1]` (interleaved when `channels > 1`). At most 2048 samples per call — loop until the returned buffer is empty to drain fully (keeps each trace record replayable). Empty buffer = nothing new yet; `null` = no capture device open. Trace-recorded — replay serves the recorded samples, never a live microphone. |
| `audio_capture_close` | `audio_capture_close of null` | Stop and close the recording device, dropping undrained samples. Safe to call twice or with no device open. |

## Internal (sanitizer builds only)

Expand Down
18 changes: 18 additions & 0 deletions docs/TRACE.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,24 @@ perspective lands on the tape as an `N` record:
the live argv; #471)
- **HTTP extension:** `http_post` (success and all error paths),
`http_request_body`, `http_session_id`, `http_request_headers`
- **Audio capture (gfx extension, #579):** `audio_capture_open`,
`audio_capture_read`. Captured audio is a device input, so the whole
capture chain is TAKE/RECORD-wrapped: under `EIGS_REPLAY` the tape is
taken *before any SDL call* — replay never opens or reads a real
microphone; the recorded device id and sample buffers (the `b[…]`
encoding) are served instead. `audio_capture_read` returns at most
2048 samples per call precisely so every `N` record fits the 64 KiB
record budget — an over-budget record would be `…<truncated>` and
replay would silently fall back to the live microphone. Drain loops
("read until empty") replay faithfully: one record per call, empties
included. (`audio_capture_close` is deterministic — always `null` —
and untraced; under replay it is a no-op because no device was
opened.) The audio *output* device (`audio_open`) is deliberately
untraced: playback is a side effect that replay re-performs live,
like `print`. The residual gap — `audio_open`'s
environment-dependent return (`0` on a machine with no audio) can
steer a branch differently on replay — is accepted for now; closing
it would change how existing tapes replay.

The hook is the `TRACE_NONDET_RET` macro in `src/trace.h`, used at
every nondet return site — adding a new nondet builtin means wrapping
Expand Down
116 changes: 116 additions & 0 deletions src/ext_gfx.c
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

#include "eigenscript.h"
#include "ext_names.h"
#include "trace.h" /* audio capture is a nondet input source (#579) */

#if EIGENSCRIPT_EXT_GFX

Expand Down Expand Up @@ -119,11 +120,13 @@ static void (*p_SDL_UnlockAudioDevice)(int);
static void (*p_SDL_PauseAudioDevice)(int, int);
static Uint32 (*p_SDL_GetQueuedAudioSize)(int);
static void (*p_SDL_ClearQueuedAudio)(int);
static Uint32 (*p_SDL_DequeueAudio)(int, void*, Uint32);

/* Audio state */
static int g_audio_device = 0;
static int g_audio_freq = 44100;
static int g_audio_channels = 1;
static int g_capture_device = 0; /* #579: microphone/input device */

static int load_sdl2(void) {
if (g_sdl_lib) return 1;
Expand Down Expand Up @@ -160,6 +163,7 @@ static int load_sdl2(void) {
p_SDL_PauseAudioDevice = dlsym(g_sdl_lib, "SDL_PauseAudioDevice");
p_SDL_GetQueuedAudioSize = dlsym(g_sdl_lib, "SDL_GetQueuedAudioSize");
p_SDL_ClearQueuedAudio = dlsym(g_sdl_lib, "SDL_ClearQueuedAudio");
p_SDL_DequeueAudio = dlsym(g_sdl_lib, "SDL_DequeueAudio");
if (!ok) {
dlclose(g_sdl_lib);
g_sdl_lib = NULL;
Expand Down Expand Up @@ -289,6 +293,7 @@ Value* builtin_gfx_close(Value *arg) {
(void)arg;
if (g_fb_texture && p_SDL_DestroyTexture) { p_SDL_DestroyTexture(g_fb_texture); g_fb_texture = NULL; g_fb_w = 0; g_fb_h = 0; }
if (g_audio_device) { p_SDL_CloseAudioDevice(g_audio_device); g_audio_device = 0; }
if (g_capture_device) { p_SDL_CloseAudioDevice(g_capture_device); g_capture_device = 0; }
if (g_renderer) { p_SDL_DestroyRenderer(g_renderer); g_renderer = NULL; }
if (g_window) { p_SDL_DestroyWindow(g_window); g_window = NULL; }
if (g_sdl_lib) {
Expand Down Expand Up @@ -860,6 +865,117 @@ Value* builtin_audio_pause(Value *arg) {
return make_null();
}

/* ================================================================
* AUDIO CAPTURE (#579 — DeslanStudio F-DS-17, live recording input)
* ================================================================
* Pull-model recording: SDL_OpenAudioDevice(iscapture=1) in queue mode
* (callback == NULL) accumulates samples in SDL's internal queue;
* audio_capture_read drains them with SDL_DequeueAudio. No callback
* means no C->eigs re-entry and no new threading surface — the shape
* matches the existing polling clients (gfx_poll loops,
* DeslanStudio's transport_tick_recording).
*
* Tape-first: captured audio is the textbook nondeterministic input,
* so audio_capture_open and audio_capture_read are TAKE/RECORD
* builtins. Under EIGS_REPLAY the TAKE short-circuits before any SDL
* call — replay never opens or reads a real microphone; the recorded
* device id and sample buffers are served off the tape. The b[…]
* buffer encoding round-trips through N records natively.
*
* AUDIO_CAPTURE_READ_MAX bounds one read at 2048 samples so a single
* N record always fits the tape's TRACE_NONDET_MAX (64 KiB) budget:
* worst-case %.17g is ~24 bytes/sample + ", " → ~53 KiB. An over-
* budget record would be …<truncated> and replay would silently fall
* back to the LIVE microphone — the exact divergence the tape exists
* to prevent. Callers drain by looping until the read comes back
* empty; the loop replays faithfully (one N record per call). */
#define AUDIO_CAPTURE_READ_MAX 2048

/* Build a VAL_BUFFER of `n` float samples in [-1, 1] from int16 PCM.
* 32767 → 1.0 exactly (mirror of audio_convert_samples' s*32767). */
static Value* capture_samples_to_buffer(const int16_t *pcm, int n) {
Value *v = xcalloc(1, sizeof(Value));
v->type = VAL_BUFFER;
v->refcount = 1;
v->data.buffer.count = n;
v->data.buffer.data = xcalloc(n > 0 ? (size_t)n : 1, sizeof(double));
for (int i = 0; i < n; i++) {
double s = (double)pcm[i] / 32767.0;
if (s < -1.0) s = -1.0; /* only -32768 exceeds */
v->data.buffer.data[i] = s;
}
return v;
}

/* audio_capture_open of [freq, channels] — open the recording device
* (queue mode, iscapture=1). Defaults 44100/1 like audio_open;
* allowed_changes=0 so SDL converts to exactly the requested format.
* Returns the device id (>= 2), or 0 when SDL/capture is unavailable.
* Re-opening closes the previous capture device first. */
Value* builtin_audio_capture_open(Value *arg) {
TRACE_NONDET_TAKE("audio_capture_open");
if (!g_sdl_lib) {
if (!load_sdl2()) TRACE_NONDET_RECORD("audio_capture_open", make_num(0));
}
if (!p_SDL_OpenAudioDevice || !p_SDL_DequeueAudio) {
fprintf(stderr, "audio_capture_open: SDL2 capture symbols not available\n");
TRACE_NONDET_RECORD("audio_capture_open", make_num(0));
}
p_SDL_Init(MY_SDL_INIT_AUDIO);

SDL_AudioSpec want = {0}, have = {0};
want.freq = 44100;
want.format = MY_SDL_AUDIO_S16LSB;
want.channels = 1;
want.samples = 1024;
want.callback = NULL; /* queue mode: drain via SDL_DequeueAudio */

if (arg && arg->type == VAL_LIST && arg->data.list.count >= 2) {
want.freq = (int)arg->data.list.items[0]->data.num;
want.channels = (int)arg->data.list.items[1]->data.num;
}

if (g_capture_device) {
p_SDL_CloseAudioDevice(g_capture_device);
g_capture_device = 0;
}
g_capture_device = p_SDL_OpenAudioDevice(NULL, 1, &want, &have, 0);
if (g_capture_device < 2) {
g_capture_device = 0;
TRACE_NONDET_RECORD("audio_capture_open", make_num(0));
}
p_SDL_PauseAudioDevice(g_capture_device, 0); /* start recording */
TRACE_NONDET_RECORD("audio_capture_open", make_num(g_capture_device));
}

/* audio_capture_read of null — drain accumulated samples since the
* last read as a buffer of floats in [-1, 1] (interleaved when the
* device was opened with channels > 1). At most
* AUDIO_CAPTURE_READ_MAX samples per call — loop until the returned
* buffer is empty to drain fully. Empty buffer = nothing new yet;
* null = no capture device open. */
Value* builtin_audio_capture_read(Value *arg) {
(void)arg;
TRACE_NONDET_TAKE("audio_capture_read");
if (!g_capture_device)
TRACE_NONDET_RECORD("audio_capture_read", make_null());
int16_t pcm[AUDIO_CAPTURE_READ_MAX];
Uint32 got = p_SDL_DequeueAudio(g_capture_device, pcm, sizeof pcm);
TRACE_NONDET_RECORD("audio_capture_read",
capture_samples_to_buffer(pcm, (int)(got / 2)));
}

/* audio_capture_close of null — stop and close the recording device.
* Undrained samples are dropped. Safe to call twice / with no device. */
Value* builtin_audio_capture_close(Value *arg) {
(void)arg;
if (g_capture_device) {
p_SDL_CloseAudioDevice(g_capture_device);
g_capture_device = 0;
}
return make_null();
}

/* audio_music_play of [path, loops] — stream a music file (mp3/ogg/wav) via
* SDL_mixer. loops: -1 = loop forever, 0 = play once, N = N+1 plays. Replaces
* any current track. Returns 1 on success, 0 on failure (missing mixer lib,
Expand Down
3 changes: 3 additions & 0 deletions src/ext_names.h
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,9 @@
X(ppu_render_frame, builtin_ppu_render_frame) \
X(audio_open, builtin_audio_open) \
X(audio_close, builtin_audio_close) \
X(audio_capture_open, builtin_audio_capture_open) \
X(audio_capture_read, builtin_audio_capture_read) \
X(audio_capture_close, builtin_audio_capture_close) \
X(audio_volume, builtin_audio_volume) \
X(audio_stop, builtin_audio_stop) \
X(audio_pause, builtin_audio_pause) \
Expand Down
12 changes: 6 additions & 6 deletions tests/run_all_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2084,15 +2084,15 @@ AUDIO_PROBE_OUT=$(./eigenscript "$AUDIO_PROBE_FILE" 2>&1)
rm -f "$AUDIO_PROBE_FILE"

if ! echo "$AUDIO_PROBE_OUT" | grep -q "undefined variable"; then
echo "[62] Audio Synthesis (24 checks)"
echo "[62] Audio Synthesis (38 checks)"
AU_OUTPUT=$(./eigenscript ../tests/test_audio.eigs 2>&1); AU_OUTPUT_RC=$?
if rc_ok "$AU_OUTPUT_RC" "$AU_OUTPUT" && echo "$AU_OUTPUT" | grep -q "All tests passed"; then
TOTAL=$((TOTAL + 24))
PASS=$((PASS + 24))
echo " PASS: all 24 audio checks"
TOTAL=$((TOTAL + 38))
PASS=$((PASS + 38))
echo " PASS: all 38 audio checks"
else
TOTAL=$((TOTAL + 24))
FAIL=$((FAIL + 24))
TOTAL=$((TOTAL + 38))
FAIL=$((FAIL + 38))
echo " FAIL: audio tests"
echo "$AU_OUTPUT" | grep -iE "assert|error|FAIL" | head -5
fi
Expand Down
65 changes: 65 additions & 0 deletions tests/test_audio.eigs
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,71 @@ else:
assert_eq of [1, 1, "audio_play_loop skipped (no device)"]
assert_eq of [1, 1, "audio_play_loop skipped (no device)"]

# ---- #579: audio capture (recording input) — device-gated like the
# mixer tests. SDL_AUDIODRIVER=dummy provides a silent capture device;
# a real driver captures whatever the mic hears. Both satisfy the
# shape/cap/range/accumulation checks below; with no SDL/device the
# checks skip with the count held constant.
assert_eq of [audio_capture_read of null, null, "capture read with no device is null (#579)"]
capdev is audio_capture_open of [44100, 1]
if capdev != 0:
assert_true of [capdev >= 2, "capture open returns an SDL device id"]
chunk is audio_capture_read of null
assert_eq of [type of chunk, "buffer", "capture read returns a buffer"]
assert_true of [(len of chunk) <= 2048, "per-call cap respected on first read"]
# Bounded poll until samples accumulate (the dummy driver fills in
# ~23ms chunks at 44100; real drivers vary).
tries is 0
loop while (len of chunk) == 0 and tries < 80:
gfx_delay of 25
chunk is audio_capture_read of null
tries is tries + 1
assert_true of [(len of chunk) > 0, "capture accumulates samples"]
assert_true of [(len of chunk) <= 2048, "per-call cap respected on a full chunk"]
inrange is 1
ci is 0
loop while ci < (len of chunk):
if chunk[ci] > 1 or chunk[ci] < (0 - 1):
inrange is 0
ci is ci + 1
assert_eq of [inrange, 1, "captured samples are floats in [-1, 1]"]
# Drain until empty terminates (each call returns at most 2048).
drains is 0
more is audio_capture_read of null
loop while (len of more) > 0 and drains < 200:
more is audio_capture_read of null
drains is drains + 1
assert_eq of [len of more, 0, "drain-until-empty terminates"]
# Monotonic accumulation: new samples appear again after a drain.
again is audio_capture_read of null
tries2 is 0
loop while (len of again) == 0 and tries2 < 80:
gfx_delay of 25
again is audio_capture_read of null
tries2 is tries2 + 1
assert_true of [(len of again) > 0, "capture keeps accumulating after a drain"]
audio_capture_close of null
assert_eq of [audio_capture_read of null, null, "read after close is null"]
assert_eq of [audio_capture_close of null, null, "double close is safe"]
capdev2 is audio_capture_open of null
assert_true of [capdev2 >= 2, "re-open after close works (default args)"]
r2 is audio_capture_read of null
assert_eq of [type of r2, "buffer", "re-opened device reads buffers"]
audio_capture_close of null
else:
assert_eq of [1, 1, "audio capture skipped (no device)"]
assert_eq of [1, 1, "audio capture skipped (no device)"]
assert_eq of [1, 1, "audio capture skipped (no device)"]
assert_eq of [1, 1, "audio capture skipped (no device)"]
assert_eq of [1, 1, "audio capture skipped (no device)"]
assert_eq of [1, 1, "audio capture skipped (no device)"]
assert_eq of [1, 1, "audio capture skipped (no device)"]
assert_eq of [1, 1, "audio capture skipped (no device)"]
assert_eq of [1, 1, "audio capture skipped (no device)"]
assert_eq of [1, 1, "audio capture skipped (no device)"]
assert_eq of [1, 1, "audio capture skipped (no device)"]
assert_eq of [1, 1, "audio capture skipped (no device)"]

test_summary of null

# ---- Mixer channels (GAP-002/GAP-003): device-gated — SDL_AUDIODRIVER=dummy
Expand Down
47 changes: 47 additions & 0 deletions tests/test_replay.sh
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,53 @@ else
fail "is_dir replay" "rec='$REC_ID' rep='$REP_ID'"
fi

# ---- #579: audio capture is a taped nondeterminism source ----
# Gated: needs a gfx build AND a working capture device (the dummy SDL
# driver provides a silent one; the CI dev image has no libSDL2, so this
# skips there and runs wherever SDL exists). Record a capture session,
# then replay it with the SDL driver deliberately broken — the recorded
# device id and sample buffers must be served off the tape (replay must
# never open or read a real device).
cat > "$TMPDIR/p_cap_probe.eigs" <<'EOF'
print of (audio_capture_open of [44100, 1])
EOF
CAP_PROBE=$(SDL_AUDIODRIVER=dummy "$EIGS" "$TMPDIR/p_cap_probe.eigs" 2>&1)

if ! echo "$CAP_PROBE" | grep -q "undefined variable" \
&& [ "$(echo "$CAP_PROBE" | tail -1)" != "0" ]; then
cat > "$TMPDIR/p_cap.eigs" <<'EOF'
dev is audio_capture_open of [44100, 1]
print of (dev != 0)
chunk is audio_capture_read of null
tries is 0
loop while (len of chunk) == 0 and tries < 80:
gfx_delay of 25
chunk is audio_capture_read of null
tries is tries + 1
print of (len of chunk)
print of chunk[0]
audio_capture_close of null
print of (audio_capture_read of null)
EOF

TAPE_C="$TMPDIR/cap.tape"
REC_C=$(SDL_AUDIODRIVER=dummy EIGS_TRACE="$TAPE_C" "$EIGS" "$TMPDIR/p_cap.eigs" 2>&1)
# Replay with a nonexistent SDL audio driver: a live open would fail
# (and a live read would return null), so matching output proves the
# values came from the tape, not the device.
REP_C=$(SDL_AUDIODRIVER=doesnotexist EIGS_REPLAY="$TAPE_C" "$EIGS" "$TMPDIR/p_cap.eigs" 2>&1)
REC_LEN=$(echo "$REC_C" | sed -n '2p')

if [ "$REC_C" = "$REP_C" ] && [ "$(echo "$REC_C" | sed -n '1p')" = "1" ] \
&& [ "${REC_LEN:-0}" -gt 0 ] && grep -q '^N audio_capture_read=b\[' "$TAPE_C"; then
ok "capture replay: recorded mic session replays without a device (#579)"
else
fail "capture replay" "rec='$REC_C' rep='$REP_C'"
fi
else
echo " SKIP: capture replay (no gfx build / no capture device)"
fi

echo
echo "REPLAY: $PASS passed, $FAIL failed"
[ "$FAIL" = "0" ] && exit 0 || exit 1
Loading