Skip to content

Ship type information (PEP 561) - #71

Merged
wbarnha merged 1 commit into
masterfrom
claude/ship-type-hints
Aug 7, 2026
Merged

Ship type information (PEP 561)#71
wbarnha merged 1 commit into
masterfrom
claude/ship-type-hints

Conversation

@wbarnha

@wbarnha wbarnha commented Aug 5, 2026

Copy link
Copy Markdown
Member

Independent of the #55 stack (#68 / #69 / #70) — this branches from master and touches no shared lines, so it can merge in any order.

Why

faust-cchardet is a drop-in replacement for chardet, and chardet ships py.typed with a fully annotated detect(). So switching to this package currently downgrades a user's type coverage:

error: Skipping analyzing "cchardet": module is installed, but missing
library stubs or py.typed marker  [import-untyped]

Every call becomes Any. All four of these pass silently today, and all four are caught after this change:

cchardet.detect(raw)["encoding"].upper()   # None-deref at runtime
cchardet.detect(raw)["confidence"]         # float used as int
cchardet.detect(raw)["encodng"]            # typo -> KeyError
cchardet.detect("a str, not bytes")        # TypeError at runtime

The typo even gets note: Did you mean "encoding"?.

What's in it

  • py.typed marker, and a DetectionResult TypedDict describing what detect() and UniversalDetector.result actually return — including that both members are None when nothing is detected, which is the case callers most often forget.
  • Annotations across the public API and the CLI.
  • _cchardet.pyi — without a stub the compiled extension is opaque and the package degrades to Any even with py.typed present.
  • Both files go through py.install_sources, and src/tests/test_typing.py guards that, because dropping them from that list fails silently — no error anywhere, downstream just quietly loses typing again.

What's deliberately not in it

No mypy CI gate. The shipped Python surface is 148 lines; mypy finds zero real bugs in it and --strict finds only missing-annotation boilerplate. All the substantive logic is Cython and C++, which mypy cannot see. Adding a gate would also mean adding a second unenforced linter next to ruff, which is already in requirements-dev.lock, isn't wired to CI, and reports 349 baseline errors. The value here is entirely outward-facing, so that's where the change is.

Two documentation bugs fixed in passing

Both surfaced purely by writing the types down:

  • detect()'s docstring claimed msg: str, but passing a str raises TypeError: expected bytes. It takes bytes.
  • detect() guarded isinstance(msg, (bytes, bytearray)) when picking the BOM prefix, implying bytearray support. The extension signature is bytes msg, so bytearray and memoryview both raise TypeError before that line is reached — the fallback branch was unreachable. If accepting the buffer protocol is wanted, that's a separate feature.

Verification

On Python 3.10 (the floor, since DetectionResult uses PEP 604 unions):

check result
mypy src/cchardet/ Success: no issues found in 6 source files
mypy --strict src/cchardet/ Success: no issues found in 6 source files
suite 134 passed, 1 skipped
built wheel contents contains both cchardet/py.typed and cchardet/_cchardet.pyi
CLI unchanged behaviour

__enter__ uses a bound TypeVar rather than typing.Self, which is 3.11+. Ruff's PYI019 would prefer Self; that's the one lint code this branch adds over the master baseline for this directory, and it's not applicable at a 3.10 floor. The __all__ addition incidentally fixes a pre-existing F401.


Generated by Claude Code

faust-cchardet is a drop-in replacement for chardet, and chardet ships
py.typed with a fully annotated detect(). So switching to this package
currently *downgrades* a user's type coverage: mypy reports

    error: Skipping analyzing "cchardet": module is installed, but
    missing library stubs or py.typed marker  [import-untyped]

and every call becomes Any. All of these pass silently today:

    cchardet.detect(raw)["encoding"].upper()   # None-deref at runtime
    cchardet.detect(raw)["confidence"]         # float used as int
    cchardet.detect(raw)["encodng"]            # typo -> KeyError
    cchardet.detect("a str, not bytes")        # TypeError at runtime

With this change mypy catches all four, the typo with a "Did you mean
encoding?" suggestion.

Adds a py.typed marker, a DetectionResult TypedDict describing what
detect() and UniversalDetector.result actually return, annotations across
the public API and the CLI, and _cchardet.pyi -- without a stub the
compiled extension is opaque and the package degrades to Any even with
py.typed present. Both files are installed via py.install_sources, and
src/tests/test_typing.py guards that, since dropping them from that list
fails silently.

Deliberately NOT adding a mypy CI gate. The shipped Python surface is 148
lines; mypy finds zero real bugs in it and --strict finds only
missing-annotation boilerplate. All the substantive logic is Cython and
C++, which mypy cannot see. The value here is entirely outward-facing.

Two documentation bugs fixed in passing, both surfaced by writing the
types down:

- detect()'s docstring claimed `msg: str`, but passing a str raises
  TypeError: expected bytes. It takes bytes.
- detect() guarded `isinstance(msg, (bytes, bytearray))` when picking the
  BOM prefix, implying bytearray support. The extension signature is
  `bytes msg`, so bytearray and memoryview both raise TypeError before
  that line is reached -- the fallback branch was unreachable.

__enter__ uses a bound TypeVar rather than typing.Self, which is 3.11+;
this package supports 3.10. Verified on 3.10: mypy clean in default and
--strict mode, 134 passed, and the built wheel contains both py.typed and
_cchardet.pyi.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TWqKLkfwd8fPxhU4KjXUVB
@wbarnha
wbarnha merged commit 986ef5d into master Aug 7, 2026
17 checks passed
@wbarnha
wbarnha deleted the claude/ship-type-hints branch August 7, 2026 18:46
wbarnha pushed a commit that referenced this pull request Aug 7, 2026
#71 (PEP 561 type information) merged to master while this branch was
open. It landed without a changelog entry, deliberately: this branch is
what renames the stale "3.1.0 (unreleased)" header and opens the 3.2.0
section, so adding one there beforehand would have guaranteed a conflict.
Adding it here now that the two are merged, so the 3.2.0 release notes
cover everything that will actually be in 3.2.0.

Verified the two changes coexist: free-threaded 3.13t build keeps the GIL
disabled, reports version 3.2.0, and the combined suite is 138 passed / 1
skipped (up from 134 -- the four typing tests came with the merge). mypy
is clean on the merged tree, so the critical_section decorators this
branch adds do not invalidate the _cchardet.pyi stub from #71.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TWqKLkfwd8fPxhU4KjXUVB
wbarnha pushed a commit that referenced this pull request Aug 8, 2026
#71 (PEP 561 type information) merged to master while this branch was
open. It landed without a changelog entry, deliberately: this branch is
what renames the stale "3.1.0 (unreleased)" header and opens the 3.2.0
section, so adding one there beforehand would have guaranteed a conflict.
Adding it here now that the two are merged, so the 3.2.0 release notes
cover everything that will actually be in 3.2.0.

Verified the two changes coexist: free-threaded 3.13t build keeps the GIL
disabled, reports version 3.2.0, and the combined suite is 138 passed / 1
skipped (up from 134 -- the four typing tests came with the merge). mypy
is clean on the merged tree, so the critical_section decorators this
branch adds do not invalidate the _cchardet.pyi stub from #71.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TWqKLkfwd8fPxhU4KjXUVB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants