Skip to content

🐛 fix(windows): stop leaking memory on repeated folder lookups#507

Merged
gaborbernat merged 2 commits into
tox-dev:mainfrom
gaborbernat:fix/ctypes-pointer-cache-leak
Jul 15, 2026
Merged

🐛 fix(windows): stop leaking memory on repeated folder lookups#507
gaborbernat merged 2 commits into
tox-dev:mainfrom
gaborbernat:fix/ctypes-pointer-cache-leak

Conversation

@gaborbernat

@gaborbernat gaborbernat commented Jul 15, 2026

Copy link
Copy Markdown
Member

Looking up a Windows folder path in a loop leaked memory. A service that builds a cache path with platformdirs.user_data_path about once a second grew by gigabytes of retained RAM over a few hours, as issue #501 reports. 🐛 Each call to get_win_folder_via_ctypes defined a new _GUID Structure subclass, and every POINTER(_GUID) registered an entry in ctypes._pointer_type_cache that nothing released.

The change builds the _GUID class and the three DLL bindings once behind functools.cache and runs a resolver closure over them on each call. get_win_folder_via_ctypes keeps its signature and delegates to the cached builder, so the fallback selector and every caller stay the same. A second commit closes the registry-fallback key with a context manager rather than leaving the OS handle for the garbage collector to reclaim.

I confirmed the fix by exercising the real function 10,000 times, stubbing only WinDLL and sys.platform so POINTER and Structure still run. On Python 3.13 the cache and the retained _GUID classes stop growing, and memray reports a large drop in peak memory. ✨

Python 3.13, 10,000 calls ctypes._pointer_type_cache live _GUID classes memray peak
before 3 → 10,021 0 → 10,000 66.8 MiB
after 3 → 22 0 → 1 1.9 MiB

Python 3.14 already reworked _pointer_type_cache upstream, which hides the cache symptom on that version, while the retained-object growth it fixes is version-independent. Returned paths are unchanged.

@gaborbernat gaborbernat requested a review from ofek as a code owner July 15, 2026 13:48
get_win_folder_via_ctypes defined a fresh _GUID Structure on every call.
Each POINTER(_GUID) registered a new entry in ctypes._pointer_type_cache
that was never released, so repeated calls leaked memory unboundedly.

Build the _GUID class and DLL bindings once via functools.cache and reuse
the resolver across calls.

Fixes tox-dev#501
get_win_folder_from_registry left the opened key handle to be reclaimed by
the garbage collector. Under frequent calls that relies on GC timing to
release OS handles; use a context manager so the handle closes deterministically.
@gaborbernat gaborbernat force-pushed the fix/ctypes-pointer-cache-leak branch from cf51027 to 5b52e66 Compare July 15, 2026 13:56
@gaborbernat gaborbernat added the bug Something isn't working label Jul 15, 2026
@gaborbernat gaborbernat changed the title fix(windows): stop leaking ctypes pointer types on repeated calls 🐛 fix(windows): stop leaking memory on repeated folder lookups Jul 15, 2026
@gaborbernat gaborbernat enabled auto-merge (squash) July 15, 2026 13:59
@gaborbernat gaborbernat merged commit d2e5756 into tox-dev:main Jul 15, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant