Fix written binarycookies being rejected as corrupted by Safari/CFNetwork - #65
Open
dan1elt0m wants to merge 2 commits into
Open
Fix written binarycookies being rejected as corrupted by Safari/CFNetwork#65dan1elt0m wants to merge 2 commits into
dan1elt0m wants to merge 2 commits into
Conversation
Files written by dump/dumps were rejected as corrupted by Safari and CFNetwork because the serializer deviated from Apple's on-disk format in three ways: - The page header was written as 00 01 00 00 instead of 00 00 01 00. - The file ended with an 8-byte little-endian checksum. Apple expects a 4-byte big-endian checksum followed by the 8-byte footer magic 07 17 20 05 00 00 00 4B and a binary plist containing NSHTTPCookieAcceptPolicy. - Cookie records used a 60-byte header with commentOffset pointing at an empty comment string. Apple writes commentOffset=0 (and a zero commentURLOffset at bytes 36-39) when there is no comment, with strings starting at byte 56. The writer now also groups cookies into one page per domain, matching how Safari organizes pages, and treats naive datetimes as UTC instead of raising a TypeError. Reading is unaffected: the deserializer follows explicit offsets and ignores the file tail, so it still reads both old and new files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sx6xRSL4siqqspQD8rEbsz
- read_string decoded one byte at a time, so any multi-byte UTF-8 character in a real cookie file raised UnicodeDecodeError. It now reads the null-terminated slice and decodes it once. - Flags are now interpreted as a bitfield (0x1 Secure, 0x4 HttpOnly) instead of an exact-match table, and the raw bitfield is preserved on the new Cookie.raw_flags field so unknown bits survive a read-modify-write round trip. Serialization writes raw_flags when present and derives it from the flag enum otherwise. - Cookie comments are no longer dropped: the reader returns them on the new optional Cookie.comment field and the writer stores them before the domain string with the correct commentOffset. - Dict inputs to dump/dumps accept "domain" as an alias for "url", and Cookie exposes a read-only domain property. - Added a golden-file test with a fixture verified byte-by-byte against Apple's layout; it pins both parsing results and byte-identical re-serialization. - Aligned requires-python with the README and CI matrix (>=3.9). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sx6xRSL4siqqspQD8rEbsz
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Files written by
dump/dumpswere rejected as corrupted by Apple's cookie parser. This PR fixes the serializer to match Apple's on-disk format exactly, and adds full-fidelity round-trip support so real cookie files can be read, modified, and rewritten without losing data.Corruption fix
The writer deviated from Apple's
Cookies.binarycookiesformat in three ways, each enough for CFNetwork to reject the file:00 01 00 00; Apple expects00 00 01 00. (The library's own reader never validates this marker, which is why round-trip tests passed while Safari refused the files.)07 17 20 05 00 00 00 4B, and a trailing binary plist containingNSHTTPCookieAcceptPolicy.commentOffsetpointing at an empty comment string. Apple writescommentOffset = 0when there is no comment, with strings starting at byte 56.The writer now also groups cookies into one page per domain, matching how Safari organizes its files.
Fidelity improvements
read_stringdecoded one byte at a time and crashed withUnicodeDecodeErroron any multi-byte UTF-8 character; it now decodes the null-terminated slice in one step.0x1Secure,0x4HttpOnly) instead of an exact-match table, and the raw bitfield is preserved on the newCookie.raw_flagsfield, so unknown bits (e.g.0x4000005) survive a read-modify-write round trip.Cookie.commentfield and written back with the correctcommentOffset, instead of being silently dropped.domainas an alias forurl, andCookieexposes a read-onlydomainproperty.requires-pythonaligned with the README and CI matrix (>=3.9).Tests
tests/fixtures/golden.binarycookies, verified byte-by-byte against Apple's documented layout) pinning both parsing results and byte-identical re-serialization.Testing on a Mac
This container can't run Safari, so a quick end-to-end check on macOS is recommended before release:
example.comentry appears. With the old writer, Safari treated the store as corrupt and silently recreated it, so nothing showed up.cp ~/Cookies.binarycookies.bak "$CK".🤖 Generated with Claude Code
https://claude.ai/code/session_01Sx6xRSL4siqqspQD8rEbsz
Generated by Claude Code