Reading an .ndx cache walks the parser past the end of its buffer#811
Open
xroche wants to merge 4 commits into
Open
Reading an .ndx cache walks the parser past the end of its buffer#811xroche wants to merge 4 commits into
xroche wants to merge 4 commits into
Conversation
binput() returned count + 1 unconditionally, assuming it had consumed a separator. When it stopped on the buffer's terminating NUL, or because the destination filled, that byte was not a separator and the caller's cursor moved one past it. In PT_LoadCache__Old the cursor then leaves the heap buffer holding the .ndx and the next binput() call reads out of bounds; the loop's a < (use + ndxSize) guard cannot see it, since the drift happens after the check. Step over the byte only when it really is a newline. cache_brstr() carried a caller-side patch for the same drift and had to follow, or its adr[off-1] probe would read before the buffer once binput() can return 0. Closes #793 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
The trigger is the walk reaching the buffer's terminating NUL, which any .ndx not ending in a newline does; full-length fields are one route to it, not the only one. Add a field that stops short of every bound, and a pair straddling cache_brstr's own 256-byte field, the second destination the diff touches. items() returned 0 whenever the item-count line was missing, so a crawl that printed nothing passed the zero-key assertion. Signals now clean up on their own trap line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
Windows resolves NUL.<anything> to the null device, so nul.ndx never existed there and proxytrack reported an unloadable index. The suite's Win32 leg caught it once items() stopped treating a missing count as zero. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
binput()returnscount + 1, assuming it consumed a separator. It did not when it stopped on the buffer's terminating NUL, or because the destination filled: that byte is data, and the caller's cursor ends up one past it. InPT_LoadCache__Oldthe cursor then sits outside the heap buffer holding the.ndx, and the nextbinput()call reads out of bounds. It is a read and not a write. The trigger is the walk reachinguse[ndxSize], which any index that does not end in a newline reaches, so full-length URL fields are one route into it and not the only one; a field a byte under capacity that ends at EOF gets there too. Only the legacy.ndxreader is affected, and everything HTTrack writes ends in a newline, so it takes a hand-made or corrupt index. The loop'sa < (use + ndxSize)guard cannot catch any of this, because the drift happens after the check.Step over the byte only when it really is a newline.
cache_brstr()had to follow, since it carried a caller-side patch for the same drift and itsadr[off - 1]probe would read before the buffer now thatbinput()can return 0. Test 92 covers the neighbouring write overflow from #744 and appends a trailing entry on purpose, so the drifted cursor still lands inside the malloc'd buffer and 92 passes on the unfixed tree. Test 117 drives all three field bounds (1024, whatever the first URL half leaves, 200),cache_brstr's own 256, and a field that fills none of them. Two of its cases have teeth outside the sanitizer build: two URLs differing only on the 1024 bound must stay two cache keys, and a header one byte past 256 must parse like a header two bytes past. On master each loses an entry.Closes #793