Build collections of the things you love. Murch turns your phone camera into a cataloging tool: point it at your stuff, say what you're collecting, and Murch identifies each item, traces it with a glowing outline, and builds a researched catalog entry — name, category, condition, provenance fields, and links — automatically.
It's aimed at collectors, resellers, estate organizers, and anyone with a shelf of interesting things they'd like to inventory without typing it all in by hand.
🌐 Live: murch.ai
You open the camera, tell Murch what you're after — a specific query like "Pokémon cards" or the catch‑all "all interesting things" — and capture one or more frames. From there Murch:
- Detects every matching item in your photos and locates it with a bounding box.
- Segments each item and paints a mint‑green spotlight outline around it.
- Catalogs it to your collection with two image views (the spotlighted shot and the raw frame).
- Enriches it in the background — assigning a category, condition, colors, and web research links.
- Details it with category‑specific metadata (e.g. a watch gets
movement_type,case_material,reference_number; a book getsauthor,publisher,edition). - De‑duplicates the collection when you finalize, merging the same physical item seen across multiple frames.
Everything streams back to the UI live as it happens.
| Stage | Engine | What happens |
|---|---|---|
| Detection | Gemini gemini-3-flash-preview + Google Search |
Finds matching items in a frame, returns each with a box_2d bounding box and a researched, specific name |
| Segmentation | Moondream /v1/segment |
Returns a precise object outline (SVG path), streamed over SSE; the browser composites the glow overlay onto the frame |
| Persistence | Cloudflare R2 + Supabase | Stores the spotlight + original images in R2 and the item record in Supabase |
| Enrichment | Gemini + Google Search | Verifies the item, assigns one of 58 categories, sets condition/colors, and captures grounded research links |
| Detailing | Gemini + Google Search | Populates category‑specific metadata fields from a per‑category schema |
| Deduplication | Gemini (LLM) + Jaccard | On finalize, merges duplicate detections of the same physical item |
Enrichment and detailing run asynchronously on a Cloudflare Queue, so capture stays instant while research happens in the background.
Murch runs as a single Cloudflare Worker that serves the web app, the API, media, and background jobs — no separate servers to manage.
┌──────────────────────────────────────────────┐
Browser (React SPA) │ Cloudflare Worker (Hono) │
───────────────────► │ │
capture / results │ • Static Assets → serves the React SPA │
│ • /api/* → JSON API (collections, │
│ live capture, finalize) │
│ • /frames,/storage → media from R2 │
│ • queue consumer → background enrichment │
└───────┬───────────────┬───────────┬──────────┘
│ │ │
┌────────────▼───┐ ┌───────▼─────┐ ┌───▼──────────┐
│ Gemini │ │ Moondream │ │ Supabase │
│ (detect/enrich/│ │ (segment) │ │ (Postgres) │
│ detail/dedup) │ └─────────────┘ └──────────────┘
└────────────────┘
┌────────────────┐ ┌─────────────┐
│ R2 (media) │ │ Queues │
│ murch-storage │ │ murch-enrich│
└────────────────┘ └─────────────┘
Why one Worker for everything: the SPA and the API are same‑origin (no CORS), media is served straight from R2 by native binding (no signing or sync), and durable background work is a first‑class Queue consumer in the same deployment. Supabase and R2 hold all persistent state, so the Worker itself is stateless and scales freely.
| Layer | Technology |
|---|---|
| Runtime | Cloudflare Workers |
| API framework | Hono (TypeScript) |
| Web app | React 18 + Vite + Tailwind CSS + React Router |
| Database | Supabase (Postgres) via @supabase/supabase-js |
| Media storage | Cloudflare R2 (native binding) |
| Background jobs | Cloudflare Queues |
| AI — vision & research | Google Gemini gemini-3-flash-preview via @google/genai |
| AI — segmentation | Moondream |
| Hosting | Cloudflare Static Assets (SPA) + custom domain |
worker/ Cloudflare Worker — API, SPA hosting, queue consumer
wrangler.jsonc Config: bindings (Assets, R2, Queue, rate limit), routes
src/
index.ts Hono app, middleware wiring, queue consumer handler
env.ts Typed bindings, vars, and secrets
routes/
collections.ts Collection + item CRUD, finalize (SSE dedup)
live.ts analyze-frame, segment (Moondream SSE), persist-item
media.ts Serve images from R2 (/frames, /screenshots, /storage)
lib/
gemini.ts Detection, enrichment, detailing, dedup (Gemini)
enrich.ts Background enrichment job (queue consumer logic)
supabase.ts Data access (collections, sessions, items, links)
dedup.ts Jaccard similarity + Union-Find clustering
category-schemas.ts 58 categories × metadata fields (source of truth)
middleware/
security.ts Security headers/CSP, CORS, API gate, rate limiting
.dev.vars.example Template for local secrets
frontend/ React + Vite single-page app
src/
App.jsx Routes
components/
HomePage.jsx Landing
CollectionCapture.jsx Live camera capture + segmentation overlay
CollectionResults.jsx Collection view
ItemGrid.jsx Item grid with filtering
ItemDetailView.jsx Full item detail + editing
ImageCarousel.jsx Spotlight / original image switcher
services/api.js API client
context/ThemeContext.jsx Theme
public/ Icons, manifest, _headers
.env.production Same-origin build config
docs/ Design notes
generate_favicons.py Favicon generation utility
State lives in Supabase (Postgres); media lives in R2.
| Table | Purpose |
|---|---|
collections |
A named collection of items (e.g. "Pokémon cards") |
sessions |
A capture session within a collection |
items |
Catalogued items with all metadata |
item_research_links |
Grounded web links found during enrichment |
item_voice_notes |
Voice notes attached to an item |
item_documents |
Supporting documents (receipts, COAs, appraisals) |
An item carries:
- Core — name, category, colors, condition, observations, confidence, frame images
- Pipeline — verdict, spotlight/original image paths,
detailing_completed, research links - Category metadata — a
category_metadataJSON object whose shape is defined per‑category incategory-schemas.ts(58 categories, ~1,065 possible fields) - User‑editable — acquisition, valuation, provenance, location, supporting documents, and flags (
is_favorite,for_sale,wishlist_item,user_notes)
All endpoints are served by the Worker. Health endpoints are public; /api/* is gated by an origin allowlist plus rate limiting (a broad per-IP cap, with tighter per-endpoint caps on the expensive AI/storage routes — see Security & abuse controls).
| Method | Path | Description |
|---|---|---|
GET |
/health, /api/health |
Health check |
POST |
/api/collection/start |
Create a collection + session |
GET |
/api/collection/:id/items |
List a collection's items |
GET |
/api/collection/:id/items/:itemId |
Get a single item |
PUT |
/api/collection/:id/items/:itemId |
Update an item |
DELETE |
/api/collection/:id/items/:itemId |
Delete an item |
POST |
/api/collection/:id/finalize |
Stabilize + LLM de‑dup (SSE progress) |
POST |
/api/live/analyze-frame |
Detect items in a frame → [{ item_name, box_2d }] |
POST |
/api/live/segment |
Moondream segmentation, streamed as SSE |
POST |
/api/live/persist-item |
Store images to R2, insert item, enqueue enrichment |
GET |
/frames/*, /screenshots/*, /storage/* |
Serve media from R2 |
| Binding | Type | Resource |
|---|---|---|
ASSETS |
Static Assets | Built SPA (frontend/dist, SPA fallback) |
BUCKET |
R2 | murch-storage (48h object lifecycle — see below) |
ENRICH_QUEUE |
Queue | murch-enrich (producer + consumer) |
RATE_LIMITER |
Rate limit | Broad /api/* cap, 60 req / 60s per IP |
RL_ANALYZE |
Rate limit | analyze-frame, 20 req / 60s per IP |
RL_SEGMENT |
Rate limit | segment, 40 req / 60s per IP |
RL_PERSIST |
Rate limit | persist-item, 15 req / 60s per IP |
IS_PRODUCTION, CONFIDENCE_THRESHOLD (default 0.85), DEDUPE_MODE (default hybrid), DEFAULT_COLLECTION_NAME, ALLOWED_ORIGINS.
Set locally in worker/.dev.vars and in production with wrangler secret put:
| Secret | Purpose |
|---|---|
GEMINI_API_KEY |
Google Gemini |
MOONDREAM_API_KEY |
Moondream segmentation |
SUPABASE_URL |
Supabase project URL |
SUPABASE_SECRET_KEY |
Supabase service key |
DEFAULT_USER_ID |
Owner user id (single‑user mode) |
MURCH_API_KEY |
(optional) require X-API-Key for non‑trusted origins |
R2 access is handled by the native binding — no access keys needed.
Murch is a public, no-signup demonstrator — anyone can open the camera and try it. There are no user accounts; the Worker runs in single-user mode (DEFAULT_USER_ID), so the design goal is not per-user data isolation but keeping the open service from being abused — chiefly to bound the cost of the paid AI/storage calls. The controls in place:
- Per-endpoint rate limiting (Cloudflare native, keyed on the Cloudflare-set client IP). A broad
/api/*cap plus much tighter caps on the expensive routes —analyze-frame,segment,persist-item— which drive Gemini, Moondream, and R2 usage. Those three are governed only by their dedicated caps (excluded from the broad counter) so a heavy-but-legitimate capture session isn't false-throttled. Limits are listed in Bindings. - Input allowlisting — item updates (
PUT …/items/:itemId) accept only a fixed set of user-editable fields; ownership/identity/pipeline columns andresearch_linksare dropped server-side, so the API can't be used to reassign or corrupt records (research_linksis therefore written only by the enrichment pipeline). - Safe link rendering — research-link URLs (AI/externally sourced) are rendered as clickable links only when they are
http(s); other schemes are inert. - Hardened responses —
Content-Security-Policy,Strict-Transport-Security,X-Content-Type-Options: nosniff,X-Frame-Options: DENY,frame-ancestors 'none'; API error responses are generic (details stay in logs). - Media retention — objects in
murch-storagecarry a 48-hour expiry lifecycle rule, so captured images are purged two days after upload. Database rows persist, so items older than the window intentionally have no image — appropriate for a demonstrator, and it keeps storage from growing unbounded.
Rate limits reduce the likelihood of abuse; the cost ceiling is enforced upstream via provider budgets/quotas (Gemini, Moondream) and Cloudflare billing alerts — account-level settings, not part of this repo.
Prerequisites: Node 18+, a Cloudflare account (Workers Paid plan for Queues), and access to the project's Supabase, Gemini, and Moondream keys.
cd worker
npm install
# add your secrets
cp .dev.vars.example .dev.vars # then fill in the values
# build the SPA so Static Assets has something to serve
npm run build:frontend
# run the Worker locally (serves SPA + API)
npm run devThe frontend builds same‑origin (frontend/.env.production sets an empty VITE_API_BASE_URL), so the SPA calls the Worker it's served from.
cd worker
npm run deploy # builds the SPA, then `wrangler deploy`First‑time production setup:
# set each secret once
wrangler secret put GEMINI_API_KEY
wrangler secret put MOONDREAM_API_KEY
wrangler secret put SUPABASE_URL
wrangler secret put SUPABASE_SECRET_KEY
wrangler secret put DEFAULT_USER_IDThe Worker is bound to murch.ai and www.murch.ai as custom domains (configured under routes in wrangler.jsonc) and also reachable at its *.workers.dev URL.
worker/src/lib/category-schemas.ts defines the metadata fields the detailing stage tries to populate for each of the 58 supported categories (Art, Books, Cameras, Coins & Paper Money, Comics, Watches & Clocks, Video Games, Wine & Spirits, …). It's the source of truth for category metadata — add or adjust a category's field list there.
Murch — capture, catalog, and curate the things you love.