feat: merge multiple recording files into one session (multi-mcap, mixed formats)

.github/workflows/ci.yml

@@ -22,6 +22,8 @@ jobs:
 
   e2e:
     runs-on: ubuntu-latest
+    env:
+      MCAP_CLI_VERSION: 'v0.2.0'
     steps:
       - uses: actions/checkout@v4
       - uses: actions/setup-node@v4
@@ -30,6 +32,17 @@ jobs:
           cache: 'npm'
       - run: npm ci
       - run: npx playwright install chromium --with-deps
+      - name: Install mcap CLI
+        # Used by scripts/gen-test-mcap-filtered.mjs to derive a fixture via
+        # the real `mcap filter` command (not @mcap/core). That script
+        # self-skips if this step is ever removed, so it's not load-bearing
+        # for the rest of CI, but keeping it here exercises the multi-source
+        # merge feature's "mcap filter"-derived fixture on every run.
+        run: |
+          curl -sSL -o /usr/local/bin/mcap \
+            "https://github.com/foxglove/mcap/releases/download/releases/mcap-cli/${MCAP_CLI_VERSION}/mcap-linux-amd64"
+          chmod +x /usr/local/bin/mcap
+          mcap --version
       - name: Generate E2E fixtures
         run: npm run gen:e2e:fixtures
       - run: npm run test:e2e

.gitignore

@@ -38,4 +38,8 @@ yarn.lock
 /public/examples/
 /test-results/
 /playwright-report/
-.tmp-urdf-debug-*/
+.tmp-urdf-debug-*/
+
+# Python (fixture generator scripts, e.g. gen-test-hdf5.py / gen-test-bag.py)
+__pycache__/
+*.pyc

docs/DEVELOPMENT.md

@@ -29,9 +29,20 @@ This runs automatically as `pretest:e2e` before `npm run test:e2e`.
 | `test_h264.mcap` | H.264 CompressedImage decode |
 | `test_minimal.hdf5` | ALOHA-schema HDF5 (~7 KB) |
 | `test_minimal.bvh` | Minimal BVH skeleton |
+| `test_multi_base.mcap` | Multi-source merge: "base recording" (camera + `/joint_states`, 0-5s) |
+| `test_multi_incremental.mcap` | Multi-source merge: separately-authored file adding one topic, 3-7s |
+| `test_multi_filtered.mcap` | Multi-source merge: derived from `test_multi_base.mcap` via the real `mcap filter` CLI — **optional**, see below |
+| `test_multi.bag` | Multi-source merge: ROS1 bag for mcap+bag mixed-format tests (copied from the committed `test-fixtures/media/minimal-multi.bag`) |
 
 Vitest layout round-trip tests import JSON directly from `test-fixtures/layouts/`.
 
+### Optional external tools
+
+Two fixtures depend on a tool that isn't an npm package. Their generator scripts detect absence and skip (exit 0, no output file) rather than failing the whole `gen:e2e:fixtures` run; Playwright specs that need one of these files check for it and `test.skip()` themselves when it's missing.
+
+- **`mcap` CLI** (Rust rewrite, `mcap --version` ≥ 0.1) — needed only to (re)generate `test_multi_filtered.mcap` via `mcap filter`. Install with `brew install mcap` or download a binary from <https://github.com/foxglove/mcap/releases?q=mcap-cli>. CI installs it explicitly (see `.github/workflows/ci.yml`), so this fixture and its tests always run there.
+- **`rosbags`** Python package — needed only to *regenerate* the committed `test-fixtures/media/minimal-multi.bag` source (e.g. after changing its schema in `scripts/gen-test-bag.py`); `npm run gen:e2e:fixtures` just copies that committed file into `public/examples/test_multi.bag`, so CI and most contributors never need to install it. If you do: `pip install rosbags`.
+
 For sample deep links (`?url=sample://…`), set `VITE_SAMPLE_DATASETS_MANIFEST_URL` in `.env` to a reachable JSON manifest (see `src/services/sampleDatasets.ts`).
 
 For remote lists in the browser during dev, prefer **same-origin** URLs, e.g.  
@@ -81,3 +92,15 @@ npm run test:e2e
 4. Open `/`, upload or drag a local `.mcap`; confirm load succeeds.
 
 Full E2E coverage requires `npm run gen:e2e:fixtures` so `public/examples/` is populated; no files outside the repo are needed.
+
+## Acceptance (multi-source merge)
+
+Loading multiple recording files together (any mix of `.mcap`/`.bag`/`.db3`/`.hdf5`/`.bvh`) merges them into one session: topics from every file are concatenated in file order, and the playback range becomes the union of each file's time range. See `tests/multi-sources.spec.ts` for the automated coverage (multi-mcap merge, mcap+bag mixed formats, merging into an already-active session, Data tab grouping).
+
+**Manual checks**
+
+1. `npm run gen:e2e:fixtures`, then `npm run dev`.
+2. Drag `public/examples/test_multi_base.mcap` onto the welcome screen; confirm it loads with 2 topics (`/camera/front/image_raw/compressed`, `/joint_states`) and a 0-5s range.
+3. Drag `public/examples/test_multi_incremental.mcap` in on top of the already-loaded session; confirm the topic list grows to 3 topics and the playback range extends to 0-7s (the union), **without** switching away to a separate "Data" entry.
+4. Open the new topic's "more" menu (`⋯` in the sidebar Topics tab); confirm it shows a "Source: …" line naming both files once a topic name happens to be shared, or the single owning file otherwise.
+5. Optional, with the real `mcap` CLI installed and a large real-world recording on hand: split it into two files by topic/time range with `mcap filter in.mcap -o a.mcap -y /some/topic` (and a complementary `-n` regex for the rest), then load both together and confirm the merged topic count/time range match `mcap info` on the original file.