Skip to content

Autocrop finds the subject in a background-removed image - #50

Merged
JamesmarkeyUK merged 2 commits into
mainfrom
claude/auto-crop-recognition-bug-sgquq0
Aug 14, 2026
Merged

Autocrop finds the subject in a background-removed image#50
JamesmarkeyUK merged 2 commits into
mainfrom
claude/auto-crop-recognition-bug-sgquq0

Conversation

@JamesmarkeyUK

Copy link
Copy Markdown
Contributor

Autocrop on a cut-out could hand back exactly the crop that was already there, looking like it had failed to see the picture at all. Reproduced in a headless browser, matching the reported rectangle byte for byte: a 4480×2520 cut-out with a 2128×2128 crop at x=1064, y=0 came back unchanged.

Why

Two faults in the content scan, both triggered by background removal:

  • Invisible alpha residue. Any pixel with alpha ≥ 16 counted as content. A matting model leaves a wash of near-zero alpha where it was unsure — invisible against the checkerboard, but enough to stretch the detected box to the edges of whatever was scanned. The box then filled its region, so 1:1 sized the square to the region and returned it. In the repro, alpha-20 haze turned the true subject box {1542, 681, 1786×1431} into {201, 60, 3899×2302}.
  • A single speck pinned the box. Bounds came from a raw min/max over every marked pixel, so one mote of dust or one stray pixel in a corner blew out the crop — four isolated pixels widened a 1786px-wide subject to 3604px.

Scanning was also confined to the current crop, so a crop drawn through the subject could hide part of it from the search.

What changed

  • Alpha decides a cut-out. Colour sampling is actively wrong on a transparent image: pixels under transparency read as black, so a dark subject gets called "background". The alpha channel is the answer already, with the bar at 64 — above the residue, below the subject's anti-aliased rim.
  • Specks are grouped and ignored. Marked pixels are grouped into blobs (run-based connected components, ~65 ms on a 40 MP source), and a blob is dropped only when it is tiny on both axes and holds under 0.5% of the content found. A hairline rule or a wire has real extent, so it is never at risk; if everything looks like a speck, everything is kept.
  • Background colour is the median of the border, not the average of four corners, so a subject touching an edge no longer poisons the sample.
  • The scan is always full-frame and replaces whatever crop is set. computeContentBounds loses its region parameter along with the last caller that passed one.
  • Autocrop never silently no-ops. The finished rectangle is compared with the crop you already have (or the whole image, when you have none); when they match, the crop is left alone and the panel says why instead of looking like a broken button.

Verified

Scenario Before After
Cut-out + stale 2128² crop, 1:1 unchanged 2128² {1542, 504, 1786×1786} — whole subject
Same, Max unchanged 2128² {1542, 681, 1786×1431} — exact subject
Crop drawn through the subject no-op finds the full subject
Full-bleed photo, 1:1 centred square centred square (unchanged)
Full-bleed photo, Max / Keep ratio whole-image crop "Nothing to trim"
Flat image arbitrary centred square crop untouched + note

Seven opaque-image regressions still pass — logo on white, photo on a black card, subject touching an edge, scan with a dust speck, flat colour, thin tall feature, black subject on transparency. npm run typecheck and npm run build are clean.


Generated by Claude Code

claude added 2 commits August 14, 2026 15:44
Autocrop on a background-removed image could hand back exactly the crop
that was already there, looking like it had failed to see the picture at
all.

Two causes, both in the content scan:

- Any pixel with alpha >= 16 counted as content. A matting model leaves a
  wash of near-zero alpha where it was unsure — invisible against the
  checkerboard, but enough to stretch the bounding box to the edges of
  whatever was scanned. The box then filled its region, so `square` sized
  itself to the region and returned it unchanged. Alpha now decides a
  cut-out on its own (colour sampling reads transparent pixels as black
  and drops a dark subject), with the bar at 64.

- One speck of dust or one stray pixel pinned the box to the border,
  because bounds came from a raw min/max over every marked pixel. Marked
  pixels are now grouped into blobs — run-based connected components, a
  few ms on a 40MP source — and a blob is ignored when it is tiny on both
  axes AND holds under 0.5% of the content found. A hairline rule or a
  wire has real extent, so it is never at risk; if everything looks like
  a speck, everything is kept.

Background colour, when it is still needed, is the median of the scanned
border rather than the average of four corners, so a subject touching an
edge no longer poisons the sample.

Autocrop also no longer silently returns the crop it was given: if
trimming inside the crop finds nothing (or nothing to trim), it re-scans
the whole image, and when there is genuinely no border to trim it leaves
the crop alone and says so in the panel.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DLjvwdNig18447roDzKQLs
Trimming inside the current crop meant a crop drawn through the subject
could hide part of it from the search: Autocrop would clamp its answer to
that crop and leave the subject's far edge outside. Scanning is now
always full-frame, and the result replaces whatever crop was there.

`computeContentBounds` loses its region parameter with the last caller
that passed one. In place of the old "did the box fill the region it was
scanned in" test, the finished rectangle is compared with the crop the
user already has (or the whole image, when they have none) — so `1:1` on
a full-bleed photo still gives a centred square, while `max` on the same
photo correctly reports that there is no border to trim.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DLjvwdNig18447roDzKQLs
@JamesmarkeyUK
JamesmarkeyUK merged commit 0586d3a into main Aug 14, 2026
2 checks passed
@JamesmarkeyUK
JamesmarkeyUK deleted the claude/auto-crop-recognition-bug-sgquq0 branch August 14, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants