Skip to content

fix(mpeg): pace sceMpegGetPicture to the stream's frame rate#175

Draft
smmathews wants to merge 1 commit into
ran-j:mainfrom
smmathews:feature/14-scempeg-frame-rate-pacing
Draft

fix(mpeg): pace sceMpegGetPicture to the stream's frame rate#175
smmathews wants to merge 1 commit into
ran-j:mainfrom
smmathews:feature/14-scempeg-frame-rate-pacing

Conversation

@smmathews

Copy link
Copy Markdown
Contributor

Problem

With the FFmpeg-backed MpegFfmpegDecoder, sceMpegGetPicture pops and returns the next
decoded frame every time the guest calls it. Decode is driven synchronously by the guest's
sceMpegDemuxPss/sceMpegAddBs calls, and host file I/O is instant, so a guest that feeds the
elementary stream in a burst decodes far ahead of real time. The standard PS2 FMV pump — one
sceMpegGetPicture call per vertical blank, a vsync-gated read/decode/present loop — then gets
frames handed back near-instantly: the movie runs at ~2x or the elementary stream drains across
a handful of calls, reaching end-of-stream before the present loop has shown more than the first
frames.

Root cause

sceMpegGetPicture tracks frame order but has no notion of the movie's timeline. On real
hardware the delivery rate is bounded by the disc/stream read feeding the demux ring buffer; the
IPU decodes faster than real time and the disc-fed buffer is the throttle. A host FFmpeg decoder
fed by instant I/O has no such throttle, so playback outruns the encoded rate.

Fix

Meter delivery at the sceMpegGetPicture seam, without throttling the decoder (it stays as fast
as possible — good for prefetch):

  • Per mpeg handle, the first time a real decoded frame is served, capture a base vsync tick
    and latch the stream rate from AVCodecContext::framerate (via a new
    MpegFfmpegDecoder::getFrameRate()) on the first valid, in-range read — re-reading on
    later frames until it is known, falling back to 30000/1001 meanwhile, and rejecting rates
    outside 10–120 fps as garbage. (That clamp is a plausibility gate, not a correctness one: an
    interlaced field rate like 59.94 would pass and double-pace, but PS2 FMVs are overwhelmingly
    29.97/25 fps.)
  • Frame N is due at base + floor(N * 60 * fpsDen / fpsNum), computed fresh from N every call
    (not accumulated), so integer rounding never drifts across a long movie.
  • While GetCurrentVSyncTick() is before the due tick, park on the existing
    WaitForNextVSyncTick(rdram, runtime) primitive (the same one sceGsSyncV uses).

The meter is ahead-only and bounded. due(N) is measured from a fixed base, so the park is
a no-op whenever the guest calls at or below the encoded rate — a normal or slow consumer is
never throttled; only a consumer running ahead of the movie's own timeline is metered. A single
park is hard-capped at a few vblanks, so even a bogus stream rate can never stall one
GetPicture call.

The 60 here is the emulator's fixed vsync-tick-worker rate (kVblankPeriod), not a
display-refresh figure, so a park's real duration is N * fpsDen / fpsNum seconds regardless of
the guest's NTSC/PAL display mode — PAL content (e.g. 25fps) is paced correctly by its own frame
rate. kVblankPeriod is exported from Interrupt.h and a static_assert in MPEG.cpp fails the
build if this rate ever drifts from it, so retuning the worker cannot silently miscalibrate pacing.
A second static_assert couples the single-park hard cap to the sanity floor —
kMpegPacingMaxParkVsyncTicks * kMpegMinSaneFps >= kMpegPacingVsyncHz (8 × 10 ≥ 60) — so the cap
can never be lowered below the widest per-frame spacing the slowest in-range stream
(kMpegMinSaneFps = 10 fps) can demand, which would otherwise clip a legitimately slow stream's
park short.

The park holds no lock: the slot is reserved under g_mpeg_stub_mutex and the mutex is
released before the wait, so a multi-vblank park never blocks stream-feed callbacks or another
handle's GetPicture. The loop re-checks runtime->isStopRequested() each iteration and breaks
out, so it cannot spin (or wedge teardown) during shutdown once the tick counter stops advancing.

Scope

Pacing models a property of the sceMpeg boundary itself (host decode speed vs. the delivery rate
real hardware bounds by disc reads), not any one title's content. It is per-handle and only
arms when a real decoded frame is served — the duplicate-last-frame and blank/no-frame paths, and
any title that never uses the sceMpeg FMV path, are unaffected.

The cadence is a single constant frame rate latched once per handle, so a stream whose per-frame
duration is not constant — variable-frame-rate content, or 2:3 pulldown where the field cadence
alternates — is paced to its nominal rate rather than frame-exact (the 59.94-field case above is
one instance of this). PS2 FMVs are almost universally constant-rate 29.97/25, so this is a
theoretical rather than practical limitation.

The base vsync tick is captured once, on the first served frame, and never re-anchored: if a
consumer falls behind the movie's timeline mid-FMV (a long stall, say), every remaining due tick
is already in the past and pacing no-ops for the rest of that handle — delivery then runs at full
decode speed to catch up. That is the intended ahead-only, wall-clock-honest behavior (the movie
runs late rather than resyncing), not a defect. The concrete trigger is a mid-FMV pause or seek
that outlasts the buffered content: because the base tick never moves, resume finds every due tick
already past and hands the backlog back in a burst to catch up, where disc-throttled hardware would
instead re-feed at the encoded rate. The practical impact is narrow — the PS2 intro FMVs this
targets are overwhelmingly unpausable and unskippable, so that timeline break rarely arises.

Audio is delivered on its own, separate real-time clock; by advancing video at the stream's
encoded rate instead of as fast as the guest drains frames, this keeps video from outrunning that
independent audio timeline — an unpaced sceMpegGetPicture hands frames back near-instantly and
races ahead of it. This is delivery pacing, not an A/V resynchronizer: it removes the runaway, it
does not correct drift between the two paths.

Files touched

  • ps2xRuntime/src/lib/Kernel/Stubs/MPEG.cppMpegFfmpegDecoder::getFrameRate(), per-handle
    pacing state on MpegPlaybackState, mpegFrameDueTick() / mpegFrameRateIsSane() /
    mpegParkUntilDueTick(), and the reservation + park in sceMpegGetPicture.
  • ps2xRuntime/src/lib/Kernel/Stubs/MPEG_internal.h — internal declarations for the pacing
    helpers (kept off the guest syscall surface in MPEG.h) and a test-seam frame-rate accessor.
  • ps2xRuntime/src/lib/Kernel/Syscalls/Interrupt.h (+ Interrupt.cpp) — moves the authoritative
    kVblankPeriod from the .cpp into the (already-declared) interrupt_state namespace in the
    header, so pacing can static_assert its 60Hz tick rate against a single source of truth. No
    functional change to the interrupt worker.
  • ps2xTest/src/ps2_mpeg_pacing_tests.cpp (+ mpeg_pacing_fixture_m2v.h, main.cpp,
    CMakeLists.txt) — regression suite and its embedded MPEG-2 test vector.

Testing

New PS2MpegPacing suite:

  • Due-tick sequence equals the exact drift-free rational (several N, monotonic over 10 000
    frames, div-by-zero-safe fallback).
  • Frame-rate sanity clamp accepts real rates (24/25/30/29.97/60) and rejects garbage/out-of-range.
  • The park blocks the expected vblanks, releases promptly on runtime stop, and is hard-capped
    against a pathological due tick.
  • Concurrent stub calls that take g_mpeg_stub_mutex make progress while a park runs (the park
    holds no global mutex).
  • End-to-end: a real embedded MPEG-2 elementary stream fed through sceMpegAddBs and drained via
    sceMpegGetPicture asserts two things. First, the handle latches the fixture's true 25fps
    directly from the decoder (not the 30000/1001 fallback) — a jitter-free pin on the
    getFrameRate path, since the 25fps and fallback delivery-span floors (12 vs 10 vsync ticks)
    differ by less than vsync-worker jitter and span alone cannot separate them. Second, delivery is
    metered to the encoded rate (the inter-frame vsync-tick span meets the encoded rate's drift-free
    floor), which catches a wholesale unpaced regression. The duplicate-last-frame path (re-served
    after the bounded no-frame wait) and the no-frame path are both verified to add no pacing park.

This adds the repository's first test fixture generated from a real encoded binary, mpeg_pacing_fixture_m2v.h: a ~2.4 KB
fully synthetic MPEG-2 elementary stream (16×16, 25 fps (PAL), all-intra ffmpeg/lavfi testsrc
output — no third-party content), embedded so the getFrameRate latch is exercised through the
real decode path with no external fixture files or network. The exact regeneration command is in
the header.

Full runtime build clean; full ps2x_tests suite green.

Known follow-up (not in this PR)

  • This PR gates delivery; it does not bound decoder read-ahead buffering (a separate concern
    for very long FMVs).

With the FFmpeg-backed MpegFfmpegDecoder, sceMpegGetPicture pops and returns
the next decoded frame as fast as the guest asks for it. Decode is driven
synchronously by the guest's demux/AddBs calls and host file I/O is instant,
so a guest that feeds the elementary stream in a burst decodes far ahead of
real time: the standard vsync-gated PS2 FMV pump (one GetPicture ~ one field
of movie time) instead gets frames handed back near-instantly, so the movie
runs at 2x+ or drains to a premature end-of-stream before its own present loop
has shown more than the first frames. On hardware the rate is bounded by the
disc-fed stream buffer; that throttle has no analogue here unless delivery is
metered at the library boundary.

Meter delivery at the sceMpegGetPicture seam without throttling the decoder
(which stays as fast as possible, good for prefetch): the first time a handle
serves a real decoded frame, capture a base vsync tick; latch the stream rate
from AVCodecContext::framerate on the first valid, in-range read (re-reading on
later frames until it is known, falling back to 30000/1001 meanwhile, and
rejecting rates outside 10..120 fps as garbage (a plausibility gate, not a
correctness one: an interlaced 59.94 field rate would pass, but PS2 FMVs are
overwhelmingly 29.97/25)). Frame N is then due at
base + floor(N * 60 * fpsDen / fpsNum), computed fresh from N every call so
integer rounding never drifts across a long movie. While the current tick is
before the due tick, park on the existing WaitForNextVSyncTick primitive.

The meter is ahead-only and bounded: the park is a no-op unless the guest is
running ahead of the movie's own timeline, so a normal or slow consumer is
never throttled; and a single park is hard-capped at a few vblanks, so a
pathological rate can never stall one GetPicture call. 60 is the emulator's
fixed vsync-tick-worker rate (kVblankPeriod), not a display-refresh number, so
a park's real duration is N * fpsDen / fpsNum seconds regardless of NTSC/PAL
display mode -- PAL content is paced correctly by its own frame rate.
kVblankPeriod is exported from Interrupt.h and a static_assert in MPEG.cpp
pins this rate to it, so retuning the vsync worker can never silently
miscalibrate pacing.

The park holds no lock: the pacing slot is reserved under g_mpeg_stub_mutex and
the mutex is released before the wait, so a multi-vblank park never blocks
stream-feed callbacks or another handle's GetPicture. The park loop also
re-checks runtime->isStopRequested() each iteration and breaks out, so it
cannot spin during shutdown once the tick counter stops advancing. Pacing is
per-handle and inert for any path that never serves a real decoded frame -- the
duplicate-last-frame and blank/no-frame paths and any title that does not use
the sceMpeg FMV path see no behavioral change.

Pacing math is split into mpegFrameDueTick()/mpegFrameRateIsSane() and the park
into mpegParkUntilDueTick() (internal header, not the guest syscall surface),
pinned by a new regression suite: the due-tick sequence equals the exact
drift-free rational; the rate clamp accepts real rates and rejects garbage; the
park blocks the expected vblanks, releases promptly on stop, and is hard-capped
against a pathological due tick; concurrent stub calls make progress during a
park (the park holds no global mutex); and a real embedded MPEG-2 stream fed
through sceMpegAddBs and drained via sceMpegGetPicture is delivered at the
stream's encoded rate, with the duplicate-last-frame and no-frame paths both
left unpaced.

Full build clean; ps2x_tests 340/340.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant