Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,8 @@ The Playwright/VHS/demo-function/per-function/discover-tests/catalog surface are
Commands registered on the **`docgen`** CLI include:

- **`init`** — scaffold bundle layout and `docgen.yaml`.
- **`wizard`** — local web UI for narration/bootstrap workflows (focus files, **in-place narration revise**, per-segment **asset freshness** + **rebuild-from-here**, **Tool** tab to pip-upgrade docgen and pin `requirements-docgen.txt`).
- **`wizard`** — local web UI for narration/bootstrap workflows (focus files, **in-place narration revise**, per-segment **asset freshness** + **rebuild-from-here**, Vue **Benchmark** view, **Tool** tab to pip-upgrade docgen and pin `requirements-docgen.txt`).
- **`gui`** — desktop window over the same Vue/Flask UI (`pip install 'docgen[gui]'` for pywebview). PyInstaller spec: ``packaging/docgen-gui.spec`` (`pip install 'docgen[packaging]'`). Frozen apps resolve templates/static/benchmark JSON via ``docgen.resources``.
- **`tts`** — text-to-speech for segment files.
- **`timestamps`** — word/segment timing (`timing.json`). Default engine **`local`** aligns the known narration text against the mp3 offline (ffmpeg silencedetect, no API); **`--engine whisper`** keeps OpenAI whisper-1 transcription. Both emit the same Whisper-shaped blocks.
- **`image-generate`** — render scene-spec **image elements** (`image:` + `prompt:` boxes) via the OpenAI Images API into the bundle (also runs for missing assets inside `generate-all`).
Expand Down Expand Up @@ -84,7 +85,7 @@ String assertions on compiled ``scenes.py`` and ``simulate_reveal_timeline`` are

## Testing (downstream relevance)

Tests should cover **CLI-visible behavior** and contracts that adopters rely on: **`yaml-generate`**, **`scene-spec-generate`**, **`scene-compile`**, **`validate`**, **`compose`**, **`generate-all`**, **`pages`**, **`init`**, **`benchmark`**, **config** loading (`repo_root`, `env_file`), and package exports. Use small in-tree fixtures; this library does not ship a dogfood bundle. Clock / compile changes must keep ``docgen benchmark`` at or above ``benchmark_data/baseline.json`` — string assertions on ``scenes.py`` are not enough.
Tests should cover **CLI-visible behavior** and contracts that adopters rely on: **`yaml-generate`**, **`scene-spec-generate`**, **`scene-compile`**, **`validate`**, **`compose`**, **`generate-all`**, **`pages`**, **`init`**, **`benchmark`**, **`gui`** / freeze-safe ``docgen.resources`` paths, **config** loading (`repo_root`, `env_file`), and package exports. Use small in-tree fixtures; this library does not ship a dogfood bundle. Clock / compile changes must keep ``docgen benchmark`` at or above ``benchmark_data/baseline.json`` — string assertions on ``scenes.py`` are not enough. Do not run a full PyInstaller freeze in routine pytest.

## Cursor Cloud specific instructions

Expand All @@ -94,4 +95,4 @@ Tests should cover **CLI-visible behavior** and contracts that adopters rely on:
- **OpenAI-gated vs offline commands:** `tts`, `timestamps --engine whisper`, `image-generate`, `narration-generate`, `scene-spec-generate`, and `yaml-generate --llm` call OpenAI and need `OPENAI_API_KEY` (integration tests auto-skip without it). Fully offline: `init`, `scene-compile`, `manim`, `compose`, `validate`, `lint`, `pages`, `concat`, `yaml-generate` (no `--llm`), `timestamps` (default `local` engine), and `benchmark`.
- **`scene-compile` gotcha:** paced specs (`wait_word`) need a `timing.json` entry for that stem (`docgen timestamps` after TTS). Prefer `scene-compile --retime` after fresh timestamps; for a fully offline smoke render, author rows without wait indices only if you accept unpaced reveals.
- **No in-repo dogfood bundle:** exercise the pipeline against a scratch bundle (`docgen init /tmp/<name> --defaults` in a throwaway git dir). Do not hand-edit consumer generated assets (see `.cursor/rules/no-asset-edits.mdc`).
- **Wizard:** `docgen wizard --port 8501` is an optional local Flask app (long-lived); run it from a bundle directory that contains `docgen.yaml`.
- **Wizard / desktop GUI:** `docgen wizard --port 8501` is the Flask UI (bundle optional for the Benchmark view). `docgen gui` / `docgen benchmark --gui` open the Vue benchmark view in a desktop window when `pywebview` is installed. Do not run a full PyInstaller freeze in routine CI; keep `packaging/docgen-gui.spec` and `docgen.gui.packaging` in sync instead.
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,10 @@ pip install 'docgen @ git+https://github.com/jmjava/documentation-generator.git'
# Prefer a SHA pin in CI:
# pip install 'docgen @ git+https://github.com/jmjava/documentation-generator.git@<sha>'

# Optional Manim extra
# Optional extras
pip install 'docgen[manim] @ git+https://github.com/jmjava/documentation-generator.git'
pip install 'docgen[gui] @ git+https://github.com/jmjava/documentation-generator.git' # pywebview desktop window
# Freeze the Vue GUI only (not the full Manim CLI): pip install 'docgen[packaging]' && pyinstaller packaging/docgen-gui.spec

# Isolated global CLI (no project venv)
pipx install 'docgen @ git+https://github.com/jmjava/documentation-generator.git'
Expand All @@ -100,6 +102,8 @@ cd documentation-generator
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
# Optional: pip install -e ".[gui]" # pywebview window for `docgen gui`
# Optional: pip install -e ".[packaging]" # PyInstaller (see packaging/README.md)
pytest
docgen benchmark # required: scene-timing corpus vs committed baseline
```
Expand All @@ -125,7 +129,8 @@ docgen validate --pre-push
|---------|-------------|
| `docgen --version` | Show installed version + recommended `pip install` line (external tool) |
| `docgen init [TARGET_DIR] [--defaults] [--segments-file FILE]` | Scaffold a bundle: `docgen.yaml`, `requirements-docgen.txt`, wrapper scripts, directories |
| `docgen wizard [--port 8501]` | Local web GUI: focus files, **revise narration**, asset freshness / rebuild-from-here, and a **Tool** tab to upgrade the installed `docgen` package (pip) + rewrite `requirements-docgen.txt` |
| `docgen wizard [--port 8501]` | Local web GUI: focus files, **revise narration**, asset freshness / rebuild-from-here, Vue **Benchmark** view, and a **Tool** tab to upgrade the installed `docgen` package (pip) + rewrite `requirements-docgen.txt` |
| `docgen gui [--view benchmark] [--browser]` | Desktop GUI (Vue + Flask). Install `docgen[gui]` for a pywebview window; `--browser` uses the system browser. Freeze with `pyinstaller packaging/docgen-gui.spec` |
| `docgen tts [--segment 01] [--dry-run]` | Generate TTS audio |
| `docgen timestamps [--engine local\|whisper]` | Extract word/segment timestamps from TTS audio → `timing.json` (default `local`: offline narration-text alignment; `whisper`: OpenAI transcription) |
| `docgen image-generate [--segment 01 \| --all \| --spec PATH] [--force] [--dry-run] [--model …] [--size …]` | Generate scene-spec image assets (`image:` + `prompt:` boxes) via the OpenAI Images API into the bundle |
Expand All @@ -142,7 +147,7 @@ docgen validate --pre-push
| `docgen yaml-generate [--merge-defaults] [--llm] [--dry-run] [--list-gaps]` | Merge defaults into `docgen.yaml`; optional OpenAI refresh of `tts.instructions` / `wizard.system_prompt` (rewrites the file — review in Git) |
| `docgen scene-compile [SPEC.scene.yaml \| --all] [--retime] [--dry-run]` | Compile declarative scene YAML into `animations/scenes.py`. **`--all --retime`** re-derives `wait_word` from current `timing.json` with no OpenAI; unmatched labels fail closed (or set `pace: none`) |
| `docgen scene-spec-generate [--segment 01 \| --all] [--compile] [--print-only] [--output PATH] [--hint …] [--model …]` | Call OpenAI to emit YAML only (same schema as `scene-compile`); rejects frame-budget overflow and **subject-beat coverage** failures (hold board on same topic; cover topic shifts; no invented labels — not a blind count); auto-paginate + word-alignment; optionally writes `animations/specs/<stem>.scene.yaml` and `--compile`s into `scenes.py` |
| `docgen benchmark [--case ID] [--format text\|json] [--update-baseline]` | Score the **standard scene-timing corpus** (execute compiled `construct()` on the real `_TimedScene` clock, no Manim). Diffs a committed baseline so clock changes are measured, not guessed |
| `docgen benchmark [--case ID] [--format text\|json] [--update-baseline] [--gui]` | Score the **standard scene-timing corpus** (execute compiled `construct()` on the real `_TimedScene` clock, no Manim). Diffs a committed baseline so clock changes are measured, not guessed. `--gui` opens the Vue view |

## Configuration

Expand Down
26 changes: 26 additions & 0 deletions packaging/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Packaging the desktop GUI

The freeze target is **`docgen-gui`**: a Vue window over the local Flask
wizard (benchmark view first). It is **not** a frozen copy of the full
`docgen` CLI (Manim / ffmpeg / OpenAI stay on the pip tool).

```bash
pip install -e '.[packaging]'
pyinstaller packaging/docgen-gui.spec
# dist/docgen-gui/docgen-gui
```

From a source checkout you can run the same entry without freezing:

```bash
pip install -e '.[gui]'
docgen gui # pywebview window if the extra is installed
docgen gui --browser # system browser fallback
docgen gui --config path/to/docgen.yaml
docgen benchmark --gui
python -m docgen.gui
```

Asset paths use ``docgen.resources.package_root()`` so templates, static
Vue files, and ``benchmark_data/`` resolve both in an editable install and
under ``sys._MEIPASS`` after PyInstaller.
50 changes: 50 additions & 0 deletions packaging/docgen-gui.spec
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# -*- mode: python ; coding: utf-8 -*-
"""PyInstaller spec for the Vue desktop GUI (not the full Manim CLI).

From the repo root, after ``pip install -e '.[packaging]'``:

pyinstaller packaging/docgen-gui.spec
"""

import sys
from pathlib import Path

SPECDIR = Path(SPECPATH).resolve().parent
ROOT = SPECDIR.parent
sys.path.insert(0, str(ROOT / "src"))

from docgen.gui.packaging import pyinstaller_datas, pyinstaller_hiddenimports # noqa: E402

a = Analysis(
[str(ROOT / "src" / "docgen" / "gui" / "__main__.py")],
pathex=[str(ROOT / "src")],
binaries=[],
datas=pyinstaller_datas(),
hiddenimports=pyinstaller_hiddenimports(),
hookspath=[],
hooksconfig={},
runtime_hooks=[],
excludes=["manim", "cv2", "torch", "IPython"],
noarchive=False,
)
pyz = PYZ(a.pure)
exe = EXE(
pyz,
a.scripts,
[],
exclude_binaries=True,
name="docgen-gui",
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=False,
console=False,
)
coll = COLLECT(
exe,
a.binaries,
a.datas,
strip=False,
upx=False,
name="docgen-gui",
)
14 changes: 13 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,11 @@ dependencies = [

[project.optional-dependencies]
manim = ["manim>=0.18"]
gui = ["pywebview>=5.0"]
packaging = [
"pyinstaller>=6.0",
"pywebview>=5.0",
]
dev = [
"pytest>=7.0",
"pytest-cov",
Expand All @@ -32,12 +37,19 @@ dev = [

[project.scripts]
docgen = "docgen.cli:main"
docgen-gui = "docgen.gui.__main__:main"

[tool.setuptools.packages.find]
where = ["src"]

[tool.setuptools.package-data]
docgen = ["templates/*.html", "static/*.css", "static/*.js", "benchmark_data/*"]
docgen = [
"templates/*.html",
"static/*.css",
"static/*.js",
"static/vendor/*",
"benchmark_data/*",
]

[tool.pytest.ini_options]
testpaths = ["tests"]
Expand Down
50 changes: 50 additions & 0 deletions src/docgen/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,38 @@ def init(
print_summary(plan, created)


@main.command("gui")
@click.option("--port", default=0, help="Bind port (0 = ephemeral).")
@click.option(
"--view",
default="benchmark",
show_default=True,
help="Initial view: benchmark, setup, production, or tool.",
)
@click.option(
"--browser",
is_flag=True,
help="Open the system browser instead of a pywebview window.",
)
@click.pass_context
def gui(ctx: click.Context, port: int, view: str, browser: bool) -> None:
"""Desktop GUI (Vue). Prefer `pip install 'docgen[gui]'` for a native window.

This is the entry PyInstaller freezes (see packaging/docgen-gui.spec).
"""
from docgen.gui.__main__ import main as gui_main

args: list[str] = ["--view", view]
if port:
args.extend(["--port", str(port)])
if browser:
args.append("--browser")
cfg = ctx.obj.get("config") if ctx.obj else None
if cfg is not None and getattr(cfg, "yaml_path", None) and Path(cfg.yaml_path).is_file():
args.extend(["--config", str(cfg.yaml_path)])
gui_main(args)


@main.command()
@click.option("--port", default=8501, help="Port for the wizard web server.")
@click.pass_context
Expand Down Expand Up @@ -1174,12 +1206,20 @@ def rebuild_after_audio(ctx: click.Context, regen_scene_specs: bool) -> None:
type=click.Path(dir_okay=False, path_type=Path),
help="Write the full JSON report to this path.",
)
@click.option(
"--gui",
is_flag=True,
help="Open the Vue benchmark view (desktop window if pywebview is installed).",
)
@click.pass_context
def benchmark(
ctx: click.Context,
case_id: str | None,
fmt: str,
update_baseline: bool,
baseline_path: Path | None,
output_path: Path | None,
gui: bool,
) -> None:
"""Score the standard scene-timing corpus (no bundle, no Manim, no OpenAI).

Expand All @@ -1198,6 +1238,16 @@ def benchmark(
write_baseline,
)

if gui:
from docgen.gui.desktop import launch_desktop

path = "/?view=benchmark"
if case_id:
path += "&case=" + case_id
cfg = ctx.obj.get("config") if ctx.obj else None
launch_desktop(cfg, path=path)
return

try:
scores = run_benchmark(case_id=case_id)
except ValueError as exc:
Expand Down
7 changes: 7 additions & 0 deletions src/docgen/gui/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
"""Desktop GUI entry (Vue + Flask, packable with PyInstaller)."""

from __future__ import annotations

from docgen.gui.desktop import launch_desktop, serve_url

__all__ = ["launch_desktop", "serve_url"]
53 changes: 53 additions & 0 deletions src/docgen/gui/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
"""PyInstaller / ``python -m docgen.gui`` entry point."""

from __future__ import annotations

import argparse


def _load_config(config_path: str | None):
if not config_path:
return None
from docgen.config import Config

return Config.from_yaml(config_path)


def main(argv: list[str] | None = None) -> None:
parser = argparse.ArgumentParser(description="docgen desktop GUI (Vue + Flask).")
parser.add_argument("--port", type=int, default=0, help="Bind port (0 = ephemeral).")
parser.add_argument(
"--view",
default="benchmark",
help="Initial wizard view (benchmark, setup, production, tool).",
)
parser.add_argument(
"--browser",
action="store_true",
help="Force the system browser instead of a pywebview window.",
)
parser.add_argument(
"--config",
default=None,
help="Optional path to a consumer docgen.yaml (Setup / Production).",
)
args = parser.parse_args(argv)
path = f"/?view={args.view}"
port = args.port or None
config = _load_config(args.config)
if args.browser:
import webbrowser

from docgen.gui.desktop import _wait_until_interrupt, serve_url

url, httpd = serve_url(config, port=port, path=path)
webbrowser.open(url)
_wait_until_interrupt(httpd)
return
from docgen.gui.desktop import launch_desktop

launch_desktop(config, port=port, path=path)


if __name__ == "__main__":
main()
76 changes: 76 additions & 0 deletions src/docgen/gui/desktop.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
"""Launch the Vue wizard in a desktop window (or a local browser fallback)."""

from __future__ import annotations

import socket
import threading
import time
import webbrowser
from typing import Any
from wsgiref.simple_server import make_server


def _free_port() -> int:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.bind(("127.0.0.1", 0))
return int(sock.getsockname()[1])


def serve_url(
config: Any | None = None,
*,
host: str = "127.0.0.1",
port: int | None = None,
path: str = "/?view=benchmark",
) -> tuple[str, Any]:
"""Start Flask in a daemon thread. Returns ``(url, httpd)``."""
from docgen.wizard import create_app

app = create_app(config)
bind_port = int(port) if port else _free_port()
httpd = make_server(host, bind_port, app)
thread = threading.Thread(target=httpd.serve_forever, name="docgen-gui", daemon=True)
thread.start()
time.sleep(0.05)
url = f"http://{host}:{bind_port}{path}"
return url, httpd


def _wait_until_interrupt(httpd: Any) -> None:
try:
while True:
time.sleep(0.4)
except KeyboardInterrupt:
httpd.shutdown()


def launch_desktop(
config: Any | None = None,
*,
port: int | None = None,
path: str = "/?view=benchmark",
width: int = 1100,
height: int = 760,
) -> str:
"""Open the GUI. Prefer ``pywebview``; fall back to the system browser.

Returns the URL that was opened.
"""
url, httpd = serve_url(config, port=port, path=path)
try:
import webview
except ImportError:
webbrowser.open(url)
_wait_until_interrupt(httpd)
return url

webview.create_window(
"docgen",
url,
width=width,
height=height,
min_size=(800, 560),
)
webview.start()
httpd.shutdown()
return url
Loading