Skip to content

Fix written binarycookies being rejected as corrupted by Safari/CFNetwork - #65

Open
dan1elt0m wants to merge 2 commits into
mainfrom
claude/binary-cookies-corruption-ltmz6y
Open

Fix written binarycookies being rejected as corrupted by Safari/CFNetwork#65
dan1elt0m wants to merge 2 commits into
mainfrom
claude/binary-cookies-corruption-ltmz6y

Conversation

@dan1elt0m

Copy link
Copy Markdown
Owner

Summary

Files written by dump/dumps were rejected as corrupted by Apple's cookie parser. This PR fixes the serializer to match Apple's on-disk format exactly, and adds full-fidelity round-trip support so real cookie files can be read, modified, and rewritten without losing data.

Corruption fix

The writer deviated from Apple's Cookies.binarycookies format in three ways, each enough for CFNetwork to reject the file:

  1. Page header — wrote 00 01 00 00; Apple expects 00 00 01 00. (The library's own reader never validates this marker, which is why round-trip tests passed while Safari refused the files.)
  2. File tail — wrote an 8-byte little-endian checksum and stopped. Apple expects a 4-byte big-endian checksum, the 8-byte footer magic 07 17 20 05 00 00 00 4B, and a trailing binary plist containing NSHTTPCookieAcceptPolicy.
  3. Cookie record — used a 60-byte header with commentOffset pointing at an empty comment string. Apple writes commentOffset = 0 when there is no comment, with strings starting at byte 56.

The writer now also groups cookies into one page per domain, matching how Safari organizes its files.

Fidelity improvements

  • read_string decoded one byte at a time and crashed with UnicodeDecodeError on any multi-byte UTF-8 character; it now decodes the null-terminated slice in one step.
  • Flags are interpreted as a bitfield (0x1 Secure, 0x4 HttpOnly) instead of an exact-match table, and the raw bitfield is preserved on the new Cookie.raw_flags field, so unknown bits (e.g. 0x4000005) survive a read-modify-write round trip.
  • Cookie comments are read into the new optional Cookie.comment field and written back with the correct commentOffset, instead of being silently dropped.
  • Dict inputs accept domain as an alias for url, and Cookie exposes a read-only domain property.
  • requires-python aligned with the README and CI matrix (>=3.9).

Tests

  • Byte-level conformance tests for the file header, page layout, cookie record offsets, checksum, footer magic, and trailing plist.
  • A golden fixture (tests/fixtures/golden.binarycookies, verified byte-by-byte against Apple's documented layout) pinning both parsing results and byte-identical re-serialization.
  • Round-trip tests for UTF-8 strings, comments, raw flags, naive datetimes, and per-domain paging.
  • Full suite passes under pydantic 1.10 and 2.x, matching the CI matrix.

Testing on a Mac

This container can't run Safari, so a quick end-to-end check on macOS is recommended before release:

  1. Install this branch and generate a test store:
    pip install git+https://github.com/dan1elt0m/binarycookies@claude/binary-cookies-corruption-ltmz6y
    python3 -c '
    import binarycookies
    cookie = {
        "name": "bc_test", "value": "hello", "url": "example.com", "path": "/",
        "flag": "Secure", "create_datetime": "2026-01-01T00:00:00Z",
        "expiry_datetime": "2036-01-01T00:00:00Z",
    }
    with open("/tmp/Cookies.binarycookies", "wb") as f:
        binarycookies.dump(cookie, f)
    '
  2. Quit Safari completely (⌘Q), then swap in the generated file (Terminal needs Full Disk Access in System Settings → Privacy & Security):
    CK=~/Library/Containers/com.apple.Safari/Data/Library/Cookies/Cookies.binarycookies
    cp "$CK" ~/Cookies.binarycookies.bak   # backup
    cp /tmp/Cookies.binarycookies "$CK"
  3. Open Safari → Settings → Privacy → Manage Website Data… and search for "example". If the file parses, the injected example.com entry appears. With the old writer, Safari treated the store as corrupt and silently recreated it, so nothing showed up.
  4. Restore your real cookies: quit Safari again and run cp ~/Cookies.binarycookies.bak "$CK".

🤖 Generated with Claude Code

https://claude.ai/code/session_01Sx6xRSL4siqqspQD8rEbsz


Generated by Claude Code

claude added 2 commits July 10, 2026 14:44
Files written by dump/dumps were rejected as corrupted by Safari and
CFNetwork because the serializer deviated from Apple's on-disk format in
three ways:

- The page header was written as 00 01 00 00 instead of 00 00 01 00.
- The file ended with an 8-byte little-endian checksum. Apple expects a
  4-byte big-endian checksum followed by the 8-byte footer magic
  07 17 20 05 00 00 00 4B and a binary plist containing
  NSHTTPCookieAcceptPolicy.
- Cookie records used a 60-byte header with commentOffset pointing at an
  empty comment string. Apple writes commentOffset=0 (and a zero
  commentURLOffset at bytes 36-39) when there is no comment, with
  strings starting at byte 56.

The writer now also groups cookies into one page per domain, matching
how Safari organizes pages, and treats naive datetimes as UTC instead
of raising a TypeError.

Reading is unaffected: the deserializer follows explicit offsets and
ignores the file tail, so it still reads both old and new files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sx6xRSL4siqqspQD8rEbsz
- read_string decoded one byte at a time, so any multi-byte UTF-8
  character in a real cookie file raised UnicodeDecodeError. It now
  reads the null-terminated slice and decodes it once.
- Flags are now interpreted as a bitfield (0x1 Secure, 0x4 HttpOnly)
  instead of an exact-match table, and the raw bitfield is preserved on
  the new Cookie.raw_flags field so unknown bits survive a
  read-modify-write round trip. Serialization writes raw_flags when
  present and derives it from the flag enum otherwise.
- Cookie comments are no longer dropped: the reader returns them on the
  new optional Cookie.comment field and the writer stores them before
  the domain string with the correct commentOffset.
- Dict inputs to dump/dumps accept "domain" as an alias for "url", and
  Cookie exposes a read-only domain property.
- Added a golden-file test with a fixture verified byte-by-byte against
  Apple's layout; it pins both parsing results and byte-identical
  re-serialization.
- Aligned requires-python with the README and CI matrix (>=3.9).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sx6xRSL4siqqspQD8rEbsz
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants