feat(ai-video): video generation for custom providers on the job system#16208
Draft
DeJeune wants to merge 3 commits into
Draft
feat(ai-video): video generation for custom providers on the job system#16208DeJeune wants to merge 3 commits into
DeJeune wants to merge 3 commits into
Conversation
Add end-to-end video generation for aggregator providers (DMXAPI, PPIO, AiHubMix) running on the async job system, mirroring the custom-provider image-generation pipeline. - aiCore runtime: new generateVideo execution path (executor, plugin engine, runtime types/errors/index, provider extension wiring) - Submit/poll video transports resolved by videoTransportRegistry: DMXAPI (HappyHorse/Vidu via /v1/responses, Hailuo via 3-step REST), PPIO (unified), AiHubMix (Sora-compatible) - Job system: videoGenerationJobHandler, videoGenerationModel contract, videoOptions per-provider param mapping - Wiring: AiService, new IPC channel, preload bridge - provider-registry: video-generation capability, new providers (FAL, Replicate, Kling AI), video model entries - Tests: per-transport, generateVideo runtime, videoOptions, AiService Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: suyao <sy20010504@gmail.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: suyao <sy20010504@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Before this PR:
generateVideoruntime path.After this PR:
generateVideoexecution path (executor, plugin engine, runtime types/errors/index, provider extension wiring).videoTransportRegistrykeyed on the resolved SDK provider id:/v1/responses, Hailuo via the 3-step REST flow.videoGenerationJobHandlerruns transports on the job system and rebuilds the exact transport after a restart from the persisted model id;videoGenerationModeldefines the provider-agnostic transport contract;videoOptionsmaps scalar params (resolution / duration / ratio / seed / watermark / …) per provider.AiServiceintegration, new IPC channel, and preload bridge.video-generationcapability (enum + schema), new providers (FAL, Replicate, Kling AI), and video model entries.generateVideoruntime test,videoOptionstest, andAiServicetest additions.Fixes #
Why we need it and why it was done in this way
The following tradeoffs were made:
VideoModelV3wrapper; native providers (Veo / Grok / Luma / Kling / Seedance / Wan) bypass this and resolve throughprovider.videoModel().The following alternatives were considered:
Links to places where the discussion took place: N/A
Breaking changes
None — additive (new capability, providers, models, and runtime path).
Special notes for your reviewer
video-generation-research). DMXAPI Hailuo transport and provider/model coverage are still being expanded.resolveImageTransport/imageGenerationModel.Checklist
This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR.
Approvers are expected to review this list.
mainfor active development,v1for v1 maintenance fixes/gh-pr-review,gh pr diff, or GitHub UI) before requesting review from othersRelease note