Skip to content

Bounds-check HuffmanEncoderV2 tree loading against corrupted input - #134

Open
alexey-milovidov wants to merge 1 commit into
szcompressor:masterfrom
ClickHouse:ch-bounds-check-huffmanv2
Open

Bounds-check HuffmanEncoderV2 tree loading against corrupted input#134
alexey-milovidov wants to merge 1 commit into
szcompressor:masterfrom
ClickHouse:ch-bounds-check-huffmanv2

Conversation

@alexey-milovidov

Copy link
Copy Markdown

Problem

HuffmanEncoderV2::loadAsDFSOrder reconstructs the Huffman tree from the compressed data without validating any of it against the available bytes. The remaining_length parameter is accepted but never used. On corrupted or adversarial input (e.g. when fuzzing the decompressor) this leads to:

  • reading the fixed-size tree header past the end of the buffer;
  • tree.ht.reserve(tree.n << 1) with an untrusted tree.n — an unbounded allocation, and tree.n << 1 can overflow;
  • the DFS reconstruction loop calling readBit(bytes, i++) with no upper bound on i, reading past the end of the buffer.

Changes

Bounds checks only; valid data is unaffected (the checks fire only on input that would otherwise read out of bounds or allocate an untrusted amount):

  • require the fixed-size header (usemp/mbft, offset, n, maxval) to fit in remaining_length;
  • reject a node count larger than the available bits — each node consumes at least one bit of the DFS bitstream — which also avoids the tree.n << 1 overflow in reserve;
  • bound the DFS bit index against the available bytes; and
  • decrement remaining_length by the bytes actually consumed (the original code advanced the pointer but never updated the length).

Notes

This hardens the load path (which has a length). decode() does not currently receive a buffer length, so fully bounding its bit-reading loops would require threading a length through the EncoderInterface::decode signature; that is left out of this change.

Context

Found while integrating SZ3 as an experimental compression codec in ClickHouse and fuzzing the decompressor: ClickHouse/ClickHouse#108788

loadAsDFSOrder read the tree header and the DFS bitstream from the compressed
data without validating them against the available bytes (the remaining_length
parameter was accepted but never used), so corrupted input could read past the
end of the buffer and reserve an untrusted amount:
 - require the fixed-size header to fit in remaining_length;
 - reject a node count larger than the available bits (each node consumes at
   least one bit), which also avoids the tree.n << 1 overflow in reserve;
 - bound the DFS bit index against the available bytes; and
 - decrement remaining_length by the bytes actually consumed.

Found while integrating SZ3 into ClickHouse and fuzzing the decompressor:
ClickHouse/ClickHouse#108788
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant