Skip to content

Backward scans from strlen(s) - 1 walk off the front of the buffer - #815

Merged
xroche merged 10 commits into
masterfrom
fix/rtrim-underflow
Jul 27, 2026
Merged

Backward scans from strlen(s) - 1 walk off the front of the buffer#815
xroche merged 10 commits into
masterfrom
fix/rtrim-underflow

Conversation

@xroche

@xroche xroche commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Two loops start at the last character of a string and decrement with no lower bound, so both run past the front when the string is nothing but separators.

optinclude_file() right-trims every config line with a = line + strlen(line) - 1; while (is_realspace(*a)) *(a--) = '\0';. A line of nothing but spaces in a httrackrc walks below the 256-byte stack array, reading each byte on the way down and writing a NUL over it whenever that byte happens to be whitespace. ASan reports the underflow of line directly.

url_savename()'s collision rename does the same over digits: b = tempo + strlen(tempo) - 1; while (isdigit(*b)) b--;. That one needs no empty string, only an all-digit save name, and a crawled page supplies it. Under -N "%n.%t" depth is unrestricted, so a single starting URL is enough: a page linking /a/2024.html and /b/2024.html collides both onto 2024.html, the second goes through the -<n> rename, and the scan then has nothing but digits left to walk over. Reproduced under ASan at htsname.c:1703 by way of htsparse.c and httpmirror, both from one URL and from -g with the two spelled out.

Both now go through hts_rtrimlen(), which counts down from the end and stops at the start of the string. The config-file trim also moved ahead of the emptiness test, so a blank line is skipped instead of reaching the option parser as an empty name.

The Content-Disposition trim in treathead() was a third copy of the same loop and shares the helper now, but that one is a deduplication and not a fix: a is always preceded either by the = of filename= or by a / left behind by the RFC 2616 skip loop, and neither character is in is_space, so the scan could not leave tmp.

htsname.c:1680 sits inside the block being edited here and is left alone. It belongs to the x + strlen(x) - 1 class of #781, which the stacked PR converts.

-#test=rtrim poisons the byte under the string, which is where the difference shows: neither ASan nor _FORTIFY_SOURCE sees an overrun that lands inside the same stack frame. Test 116 drives the collision rename end to end by both routes and pins both call sites at the source, because on a plain build the byte below the buffer is usually not a digit, so a reverted scan stops anyway and produces the same names. It is added to the Windows job by name, since that job iterates a fixed glob that 116_engine-* does not match and the rename is savename logic, which behaves differently there.

Closes #814

optinclude_file()'s config-line right trim and url_savename()'s collision
rename both start at the last character and decrement with no lower bound.
An all-space httrackrc line and an all-digit save name walk below their
stack buffers; the second is reachable from a crawl.

Both go through hts_rtrimlen(), which counts down from the end. The
Content-Disposition trim was a third copy and now shares it.

Closes #814

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
xroche and others added 9 commits July 27, 2026 14:24
116_engine-rtrim ran nowhere on Windows: that job iterates a fixed glob
that 116_engine-* does not match, and the rename is savename logic, which
behaves differently there. Add it by name.

The test only drove the rename through -g, which pins depth to 0 and so
needs both colliding URLs on the command line. Under -N "%n.%t" depth is
unrestricted, so one starting URL is enough and the crawled page picks
both names itself. Also strip CR before the empty-option check, which a
CRLF log would otherwise pass vacuously.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Resolve the tests/Makefile.am TESTS-tail conflict: keep both 111 and
116 entries in numeric order.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Resolve the tests/Makefile.am TESTS-tail conflict: keep 110, 111 and
116 in numeric order.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Reorder the TESTS-tail union merge: 114 before 116, numeric order.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Register 116_engine-rtrim.test in tests-list.mk after the #845
TESTS-list restructure moved it out of tests/Makefile.am.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* x + strlen(x) - 1 points before the buffer on an empty string

The pointer spelling of #770. All 27 occurrences go through
hts_lastcharptr(), which clamps to the terminating NUL, and the two
hand-written ternary guards from #729 and #767 fold into it.

Closes #781

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

* Drop the historical narration from hts_lastcharptr's contract

The declaration is the API surface, so it keeps the double-evaluation
warning a caller cannot derive from the signature; what the macro
replaced is git's job. Its grep exclusion in the test goes with it,
since that comment was the only htssafe.h line the scan matched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>

---------

Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
@xroche
xroche enabled auto-merge (squash) July 27, 2026 18:07
@xroche
xroche merged commit 1b53b43 into master Jul 27, 2026
22 checks passed
@xroche
xroche deleted the fix/rtrim-underflow branch July 27, 2026 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Backward scans from strlen(s) - 1 walk off the front of the buffer

1 participant