Add "Annotations" support (DataLab-desktop-style)

[PierreRaybaut]

Warning

This implementation plan is an initial draft and must not be taken as an absolute blueprint. A thorough analysis of constraints, dependencies, and architectural impacts is strictly required before starting the implementation.

Summary

Bring structured, DataLab-desktop-style annotations to DataLab-Web. Today the app only persists a flat, untyped Plotly { shapes, annotations } payload captured opportunistically from the modebar (_dlw_plotly_annotations metadata key). DataLab desktop instead offers a dedicated Edit annotations mode with a fixed palette of typed annotation items (labels, cursors, segments, rectangles, circles, ellipses, points) whose geometry and computed measurements are stored in a portable, round-trippable format. This issue plans the work to reach feature and data parity, so that annotations created in either application survive a round-trip through the other via the HDF5 workspace.

Motivation

• Feature parity with DataLab desktop: users expect to annotate signals/images with labelled shapes and cursors, not just free-draw modebar shapes.
• Interoperability: an HDF5 workspace saved by the desktop app already carries obj.annotations (a JSON string). DataLab-Web should display and edit those annotations rather than ignoring them, and write them back in the same format.
• Discoverability & UX: a dedicated Edit annotations action with a constrained toolbar is far clearer than relying on raw Plotly modebar draw tools that today double as the ROI editor.
• Persistence correctness: annotations should be a first-class, typed part of the object model, not an opaque blob that silently breaks across processing operations.

Background: how DataLab desktop does it

Data model (Sigima)

• BaseObj.annotations is a JSON string field on every SignalObj / ImageObj — see sigima/objects/base.py (annotations: str = "", ~L212).
• obj.get_annotations() -> list[dict] parses the JSON; obj.set_annotations(list[dict]) serialises it back (sigima/objects/base.py, ~L597 / ~L628).
• The string is stored verbatim in the HDF5 workspace as a standard object attribute — no special I/O path is needed.

Bridge to plot items (DataLab desktop)

• datalab/adapters_plotpy/annotations.py — PlotPyAnnotationAdapter converts between the Sigima list[dict] and live PlotPy AnnotatedShape items.
• Each annotation dict carries a plotpy_json field: the result of PlotPy's save_items() (via guidata.io.JSONWriter) for one item. Deserialisation uses plotpy.io.load_items() / JSONReader.
• An AnnotatedShape serialises three blocks: annotationparam (style + label visibility), shape (coordinates), and label (anchor, font, colors).

Annotation palette (desktop)

Defined per panel as ANNOTATION_TOOLS:

• Signals (datalab/gui/panel/signal.py, ~L53): LabelTool, VCursorTool, HCursorTool, XCursorTool, SegmentTool, RectangleTool, HRangeTool.
• Images (datalab/gui/panel/image.py, ~L54): AnnotatedCircleTool, AnnotatedSegmentTool, AnnotatedRectangleTool, AnnotatedPointTool, AnnotatedEllipseTool, LabelTool.

Interaction (desktop)

• Edit annotations calls panel.open_separate_view(edit_annotations=True) (datalab/gui/panel/base.py, ~L2750), which opens a separate plot view with a dedicated annotation toolbar and sets the data items to selectable=False.
• On OK, __separate_view_finished collects the serialisable items and writes them back into obj.annotations.

Current state in DataLab-Web

What already exists (the foundation to build on):

• Persistence layer: _dlw_plotly_annotations metadata key with get_plotly_annotations(oid) / set_plotly_annotations(oid, payload) in src/runtime/bootstrap.py (~L3277). Key is registered in _HIDDEN_METADATA_KEYS (survives processing) and exported.
• Runtime bridge: PlotlyAnnotations type ({ shapes: unknown[]; annotations: unknown[] }) and getPlotlyAnnotations / setPlotlyAnnotations in src/runtime/runtime.ts (~L295).
• App wiring: src/App.tsx holds an annotations state, loads it on selection change, and persists via handleAnnotationsChange.
• Plot rendering / editing: src/components/SignalPlot.tsx and src/components/ImagePlot.tsx keep localShapes / localAnnotations, inject them into Plotly layout.shapes / layout.annotations, and intercept plotly_relayout (debounced) to persist edits. Plotly editable: true + modebar draw tools (drawline, drawrect, drawcircle, eraseshape) are already enabled and currently shared with ROI editing.
• Overlay precedent: analysis-result overlays (buildGeometryOverlays, resultBox.ts) and ROI overlays (signalRoi.ts, imageRoi.ts) show how to convert structured geometry into Plotly shapes/annotations/traces.

Gaps to close

1. No typed annotation model — annotations are an opaque Plotly blob, not a discriminated union of label/cursor/segment/rectangle/circle/ellipse/point.
2. No dedicated Edit annotations mode — drawing is only available through the generic modebar, which overlaps with ROI editing and offers no constrained palette.
3. No editing dialog — users cannot rename an annotation, set exact coordinates, or toggle the computed-measurement label (cf. the existing RoiDialog.tsx).
4. No desktop interoperability — DataLab-Web ignores obj.annotations (the plotpy_json format) on import and never writes it; desktop ignores _dlw_plotly_annotations.
5. No computed-measurement labels — desktop labels show live geometry (length, area, center, radius…); DataLab-Web has no equivalent for user annotations.

Goals / Non-goals

Goals

• A typed, persisted annotation model shared by signals and images.
• A dedicated Edit annotations mode with a constrained, DataLab-like palette per object kind.
• An Annotations management dialog to list, rename, edit coordinates, toggle the info label, and delete items.
• Computed-measurement labels (length, area, center, radius, angle…) attached to shapes, mirroring desktop semantics.
• Round-trip interoperability with DataLab desktop via the Sigima obj.annotations (plotpy_json) format through the HDF5 workspace.

Non-goals (for this issue)

• Reproducing every PlotPy style option pixel-for-pixel.
• Polygon annotations (desktop image AnnotatedPolygon exists but is lower priority — track separately).
• Annotation-driven computations (annotations remain descriptive overlays, like desktop).

Proposed design

1. Typed annotation model (shared)

Introduce a discriminated union in TypeScript and a mirrored Python representation, stored in object metadata (not the flat Plotly blob).

Signal annotation kinds: label, vcursor, hcursor, xcursor, segment, rectangle, hrange.
Image annotation kinds: label, point, segment, rectangle, circle, ellipse.

Each item carries:

• id (stable), kind, title (user-editable text),
• geometry in physical coordinates (consistent with SignalRoiSegment / ImageRoiSegment conventions in src/runtime/runtime.ts),
• showInfo: boolean (whether the computed-measurement label is shown),
• optional minimal style (color, line width) with sane defaults.

Persist as a new structured payload (e.g. metadata key _dlw_annotations, kept distinct from the legacy _dlw_plotly_annotations during migration). Keep it in _HIDDEN_METADATA_KEYS so it survives processing.

2. Desktop interoperability layer (Python, in Pyodide)

• On import / read: if obj.annotations (the Sigima JSON string with plotpy_json items) is non-empty, translate the supported AnnotatedShape subtypes into the typed model. PlotPy is not available in Pyodide, so parse the plotpy_json structure directly (read shape coordinates + annotationparam/label) rather than via load_items().
• On save / write: serialise the typed model back into the Sigima obj.annotations plotpy_json shape so a desktop user reopening the workspace sees native PlotPy annotations.
• Keep this translation isolated in a dedicated module (e.g. src/runtime/annotations.py pushed into Pyodide, alongside bootstrap.py), with the JSON schema documented and unit-tested.

3. Runtime bridge (TypeScript)

• Replace/augment PlotlyAnnotations with the typed model: getAnnotations(oid): Annotation[] and setAnnotations(oid, items) in src/runtime/runtime.ts, delegating to the new Python helpers.
• Keep get/setPlotlyAnnotations temporarily for migration, then remove once the typed path lands.

4. Rendering

• Add src/components/annotationsOverlay.ts (analogous to signalRoi.ts / imageRoi.ts / buildGeometryOverlays) that converts typed annotations into Plotly shapes, annotations, and helper scatter traces (points/cursors), plus the computed-measurement label text.
• Wire it into SignalPlot.tsx and ImagePlot.tsx so annotations render in view mode (read-only) and in edit mode (interactive).

5. Edit mode + palette

• Add an annotationEditMode state (sibling of roiEditMode) so annotation drawing does not collide with ROI editing.
• In edit mode, expose a constrained palette mapped to Plotly draw tools (drawline → segment/cursor, drawrect → rectangle, drawcircle → circle/ellipse, plus a click-to-place for label/point). Map each newly drawn Plotly shape to the correct annotation kind based on the active palette tool, in handleRelayout.
• Reuse the debounced relayout → persist loop already present in the plot components.

6. Management dialog

• Add src/components/AnnotationDialog.tsx modeled on src/components/RoiDialog.tsx: list items, edit title, edit exact coordinates, toggle showInfo, reorder/delete.

7. Actions / menus

• Register annotations.edit_graphical (toggle edit mode) and annotations.edit_dialog (open dialog) in src/actions/registry.ts, placed in the Edit (or View) menu, next to the ROI actions.
• All user-facing strings wrapped in t("…"); run the i18n extract/check tasks and fill src/locales/fr.json.

Data model sketch

// src/runtime/annotations.ts (types)
export type AnnotationStyle = { color?: string; width?: number };

export type SignalAnnotation =
  | { id: string; kind: "label"; title: string; x: number; y: number; style?: AnnotationStyle }
  | { id: string; kind: "vcursor"; title?: string; x: number; showInfo?: boolean; style?: AnnotationStyle }
  | { id: string; kind: "hcursor"; title?: string; y: number; showInfo?: boolean; style?: AnnotationStyle }
  | { id: string; kind: "xcursor"; title?: string; x: number; y: number; showInfo?: boolean; style?: AnnotationStyle }
  | { id: string; kind: "segment"; title?: string; x0: number; y0: number; x1: number; y1: number; showInfo?: boolean; style?: AnnotationStyle }
  | { id: string; kind: "rectangle"; title?: string; x0: number; y0: number; dx: number; dy: number; showInfo?: boolean; style?: AnnotationStyle }
  | { id: string; kind: "hrange"; title?: string; xmin: number; xmax: number; showInfo?: boolean; style?: AnnotationStyle };

export type ImageAnnotation =
  | { id: string; kind: "label"; title: string; x: number; y: number; style?: AnnotationStyle }
  | { id: string; kind: "point"; title?: string; x: number; y: number; showInfo?: boolean; style?: AnnotationStyle }
  | { id: string; kind: "segment"; title?: string; x0: number; y0: number; x1: number; y1: number; showInfo?: boolean; style?: AnnotationStyle }
  | { id: string; kind: "rectangle"; title?: string; x0: number; y0: number; dx: number; dy: number; showInfo?: boolean; style?: AnnotationStyle }
  | { id: string; kind: "circle"; title?: string; xc: number; yc: number; r: number; showInfo?: boolean; style?: AnnotationStyle }
  | { id: string; kind: "ellipse"; title?: string; xc: number; yc: number; rx: number; ry: number; angle?: number; showInfo?: boolean; style?: AnnotationStyle };

The Python side stores the equivalent dict list under _dlw_annotations, and the interop layer maps to/from the Sigima plotpy_json format on workspace read/write.

Tasks

• Schema — define the typed annotation model (TS + Python dict) and document the _dlw_annotations metadata payload.
• Python model — add typed annotation getters/setters in bootstrap.py (or a new annotations.py); register the key in _HIDDEN_METADATA_KEYS; export helpers.
• Desktop interop (read) — parse Sigima obj.annotations (plotpy_json) into the typed model for supported shape kinds (no PlotPy dependency in Pyodide).
• Desktop interop (write) — serialise the typed model back into obj.annotations so desktop reopens native annotations; add round-trip pytest in tests/python.
• Runtime bridge — add getAnnotations / setAnnotations to src/runtime/runtime.ts with typed interfaces.
• Overlay builder — src/components/annotationsOverlay.ts converting annotations → Plotly shapes/annotations/traces + computed-measurement labels.
• Signal rendering — wire overlay into SignalPlot.tsx (view + edit), including cursors and h-range.
• Image rendering — wire overlay into ImagePlot.tsx (view + edit), including ellipse/circle/point.
• Edit mode — add annotationEditMode, constrained palette → Plotly draw tools, and kind-aware handleRelayout mapping; ensure no collision with roiEditMode.
• Management dialog — src/components/AnnotationDialog.tsx (rename, exact coords, toggle info label, delete, reorder).
• Actions / menus — register annotations.edit_graphical and annotations.edit_dialog in src/actions/registry.ts; update the action-registry length test.
• Computed labels — implement geometry measurements (length, area, center, radius, angle) for the info labels.
• i18n — wrap all strings in t(...), run the extract/check tasks, fill src/locales/fr.json.
• Migration — read legacy _dlw_plotly_annotations into the new model once, then deprecate/remove the old path.
• Tests — Vitest for the overlay builder and runtime bridge; pytest for the Python interop round-trip; a Playwright spec exercising draw → persist → reload → edit.
• Docs — add an Annotations section to doc/userguide/ and update doc/architecture.md if a new subsystem/module is introduced.

Testing strategy

• Python (pytest, tests/python): typed-model round-trip, and desktop plotpy_json ↔ typed-model interop for each supported kind.
• Vitest + RTL: annotationsOverlay.ts produces the expected Plotly shapes/annotations/traces; AnnotationDialog.tsx edits update state; always run 🟢 Vitest (TS) (catches the action-registry length assertion).
• Playwright (E2E): draw an annotation of each kind, confirm it persists across reload, edit it via the dialog, and verify the rendered overlay (assert a real DOM node / trace, not just a window.runtime.* call).
• i18n: run 🌍 i18n: Check catalog (step 2/2) so CI does not fail on missing keys.

Risks & open questions

• No PlotPy in Pyodide — the plotpy_json interop must be implemented by hand from the documented JSON structure; brittle if PlotPy changes its serialization. Mitigate with focused round-trip tests and, if needed, a registry-tracked shim.
• Coordinate conventions — ensure annotation coordinates use the same physical-coordinate basis as ROIs (x0/y0/dx/dy for images) to avoid drift between view and edit.
• Image image-trace constraints — annotations must respect the explicit xaxis.range / autorange: false rules and the manual hover wiring already documented for image plots.
• Edit-mode arbitration — annotation editing and ROI editing both use the Plotly modebar; the UI must clearly switch between the two and never persist one as the other.
• Polygon annotations and full style fidelity are deferred — confirm scope before starting.

Key references

DataLab desktop / Sigima / PlotPy:

• sigima/objects/base.py — annotations field, get_annotations / set_annotations.
• datalab/adapters_plotpy/annotations.py — PlotPyAnnotationAdapter (plotpy_json format).
• datalab/gui/panel/base.py — open_separate_view(edit_annotations=True), __separate_view_finished.
• datalab/gui/panel/signal.py / datalab/gui/panel/image.py — ANNOTATION_TOOLS palettes.
• plotpy/items/annotation.py, plotpy/tools/annotation.py, plotpy/items/shape/ — item classes and serialization.

DataLab-Web:

• src/runtime/bootstrap.py — _dlw_plotly_annotations, get_plotly_annotations / set_plotly_annotations, _HIDDEN_METADATA_KEYS.
• src/runtime/runtime.ts — PlotlyAnnotations, getPlotlyAnnotations / setPlotlyAnnotations, SignalRoiSegment / ImageRoiSegment.
• src/components/SignalPlot.tsx, src/components/ImagePlot.tsx — localShapes / localAnnotations, handleRelayout, editable shapes.
• src/components/signalRoi.ts, src/components/imageRoi.ts, src/components/resultBox.ts, buildGeometryOverlays — overlay precedents.
• src/components/RoiDialog.tsx — dialog precedent for AnnotationDialog.tsx.
• src/actions/registry.ts — action registration (ROI actions precedent).