merge: cascade 8.4 into master - #261
Merged
Merged
Conversation
…shared memory RefreshWorkflowTest covers the patch -> refresh -> run loop under opcache.file_cache_only=1, where no shared-memory copy exists and writing the binary is the whole story. This adds SharedMemoryRefreshTest, whose workers all run with SHM ACTIVE (opcache.enable_cli=1 + opcache.file_cache=<dir>, NOT file_cache_only), exercising exactly what the file_cache_only legs never touch: - a warm worker's single include populates BOTH shared memory and the .bin, and after an API patch + refresh() a FRESH worker (empty SHM, like a pool worker after restart) executes the patched body - loaded through opcache's own consistency-checked file-cache-into-SHM path and resident in shared memory afterwards (opcache_is_script_cached); - the negative control that motivates refresh(): within ONE worker process, patch + save() WITHOUT invalidation leaves the SHM-resident original in service on re-include, while a fresh worker proves the patched binary really is on disk - a SHM-resident script is not re-read until invalidated; - within ONE worker process, refresh() evicts the SHM-resident copy. Each CLI process owns a private SHM segment, so the class docblock spells out what every leg can honestly prove. Deliberately not asserted (found while building this): in the invalidating process itself a re-include after refresh() does not pick the patched binary up, because opcache_invalidate() with opcache.file_cache set unlinks the .bin that save() has just written (zend_file_cache_invalidate); enshrining that in an assertion would freeze a bug, so it stays follow-up work on refresh(). Workers use opcache.file_update_protection=0 and opcache.validate_timestamps=0 for determinism, exit code 2 marks "shared-memory shape not exercised" and skips loudly (never silent), and the tests carry the opcache + opcache-relocator groups so the --fail-on-skipped gates cover them on NTS and the ZTS gate excludes them alongside the other relocator tests. Fixes #125 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
In a process running opcache shared memory WITH opcache.file_cache, opcache_invalidate() does more than evict the SHM entry: it also unlinks the script's cache binary (zend_file_cache_invalidate). refresh() used to save() first and invalidate second, so the invalidation deleted the patched binary refresh() had just written - the next load recompiled the original source and the patch was silently lost. refresh() now invalidates first and writes second, so the unlink hits the STALE binary. The ordering also picks the right failure direction: if save() throws after the invalidation, the worst case is a cache miss and a recompile of the original source - never a lost patch presented as a successful refresh. Under opcache.file_cache_only (and in processes without active opcache) opcache_invalidate() is a no-op, so the file-cache-only semantics are unchanged - RefreshWorkflowTest stays green. Same-process pickup is documented rather than changed: after an in-process invalidation, opcache's default key lookup finds the invalidated hash entry without resolving the script path and never consults the file cache again, so a re-include in the same process picks the patched binary up only under opcache.revalidate_path=1; a fresh worker needs no such setting. SharedMemoryRefreshTest now asserts the fixed contract instead of documenting the gap: the refresh leg proves the binary survives the invalidation (through a stat-cache-clearing check, so a regression cannot hide behind PHP's stat cache) and that a re-include in the SAME worker executes the patched body, loaded from the file cache back into shared memory (the leg runs with opcache.revalidate_path=1, and the save-only negative control runs with it too, proving its staleness is genuine SHM shielding rather than the default-lookup quirk). Fixes #252 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
…ch limit Un-sharing a method body for a builtin ZEND_RECV patch used to copy only the opcode array and rebase every IS_CONST operand back onto the source literals. An IS_CONST operand is a signed 32-bit opline-relative byte offset, so that only worked while the literals stayed within 2GB of the relocated opcodes - which an opcache-shared body never does, making slot substitution of builtin parameter types refuse every shared-memory method body. The copy now reproduces the engine's own pass_two() layout in one request- memory block: opcodes at the start, the literal zvals memcpy'd to the same 16-aligned offset right behind them, and every IS_CONST operand rebased onto the copied literal at the index its source operand addressed. With both halves in one block the rebased offset is bounded by the block size and always fits the 32-bit field, so shared-memory sources are fully supported and the refusal is gone. The zend.assertions verifier now proves each operand lands zval-aligned on the same literal index inside the copied table. Ownership follows the engine's one-shared-body model. The literal zvals are copied shallowly: releases happen only when the shared body refcount reaches zero, so exactly one dtor pass ever runs over exactly one of the sibling zval arrays, and with relative const addressing destroy_op_array() frees literals and opcodes as one allocation through the opcodes pointer (never a separate efree of literals once ZEND_ACC_DONE_PASS_TWO is set - the layout this block is built for). An opcache-shared source never reaches that pass at all: its refcount pointer is NULL and the immortal SHM payloads (interned strings, immutable arrays) are never refcounted. One block per patched method, request allocator reclaims whichever sibling the engine does not free. Enforcement surfaced a second shared-memory gap the old refusal was masking: opcache's optimizer assigns a mixed parameter's RECV (cached mask exactly MAY_BE_ANY) the RECV_NOTYPE handler variant, which never reads the cached mask, so patching the mask alone was silently unenforced. Such oplines are now rebound to the engine's generic mask-checking handler, transplanted from a donor opline whose int parameter can never be NOTYPE-specialized - exactly the handler the compiler assigns when a builtin parameter type is written in source. The two tests parked in the opcache-incompatible group return to the opcache-active runner job, leaving the group empty (the mechanism stays for future use), and docs/class-specialization.md documents the new copy model in place of the 2GB limitation. Fixes #131 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
Add the two ZEND_API extern function pointers from Zend/zend_inheritance.h to the variables manifest and regenerate the linux targets. Opcache installs its shared-memory lookup/publication callbacks into these globals; exporting them lets the runtime intercept the *_add pointer and decline publication of classes that received address-keyed handlers during lazy linking (#241). The darwin/windows artifacts cannot be generated on this machine and are refreshed by their native generate workflows, which trigger on pull requests touching tools/generator/**. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
…classes Under opcache, a class declared in a cached script links on a temporary mutable copy (zend_lazy_class_load); when linking completes, opcache's zend_inheritance_cache_add persists the linked result into shared memory, the class-table bucket is repointed at the published entry and the temporary dies - together with every z-engine handler keyed to its address. That silently lost handlers installed from an interface_gets_implemented hook (#238), and the interim fix was a loud SharedMemoryException from every installer. The real fix: Core::init() saves the opcache callback and installs an FFI interceptor over the zend_inheritance_cache_add global. Handler installation on a lazy-linking copy now records the temporary's address in a decline set instead of throwing; when that class finishes linking, the interceptor answers NULL - the engine's ordinary "not cached" outcome (opcache itself returns it when SHM is full or a restart is pending) - so the temporary stays in the class table as a process-local, request-lifetime class. The handlers keep firing, the class simply pays re-linking per process instead of cache reuse, and unhooked classes delegate to opcache unchanged. Decline records are consumed on interception and dropped at shutdown(), so the set stays bounded. FPM hazard removal: publishing handlers through the class entry (ce->default_object_handlers) was rejected because it would place a per-process trampoline address into shared memory, where sibling workers would dereference garbage. Declining publication inverts that: nothing belonging to the hooked class ever enters SHM. The interceptor can fire during compile-time early binding, where CG(in_compilation) promotes every thrown exception to an immediate fatal error - its hot path is therefore fully throw-free, using the new Core::pointerAddressOf() (addressOf() minus the throwing array-decay probe) for pointer identity. The SharedMemoryException guard remains only as fallback for platforms whose generated engine definitions predate the exported symbol (darwin/ windows until their generate workflows refresh them). The regression child now proves the #238 semantics end to end: the hook observes the lazy copy, setCreateObjectHandler/setWritePropertyHandler succeed, a property write on a new instance fires the installed handler, the surviving class entry is the very address the handlers were installed on (process-local, not immutable), and an untouched sibling class linking against a hook-free cached interface is still published into the inheritance cache (observable as its entry becoming immutable). Fixes #241 Fixes #238 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
…h-test test(opcache): prove BinaryCacheFile::refresh() semantics under real shared memory
…t SHM-donor swaps Root cause of the "first redefine() does not take effect" failure (the plateau child with opcache active): plateau_function()'s body was a single `return 'original';`, and when opcache caches a script its optimizer INLINES every same-file call to such a trivial constant-returning function (zend_try_inline_call, Zend/Optimizer/optimize_func_calls.c, optimizer pass 4 of the default opcache.optimization_level) - the dispatch call sites were replaced by the literal at cache time, so they did not exist at runtime and no redefine could ever reach them. The separate-file matrix leg stayed green because the optimizer cannot resolve a callee outside the script it compiles. This is a compile-time transformation, not a resolution path the copy-out could repoint: it is now a documented copy-out caveat in docs/hot-swap.md, together with its warm-cache twin - a caller whose run-time cache already resolved the shared-memory entry before the copy-out keeps dispatching it, because copy-out redirects name resolution only (the rule the class copy-out has always documented; the method leg additionally tripped over the "instances created before the copy-out keep the shared class entry" caveat). The library bug the same run exposed: destroyPreviousBody() returned early for a previous body without a refcount (a body shared with opcache SHM, e.g. any donor closure declared in a cached file), leaking the swap-minted HEAP_RT_CACHE run-time cache and a statics defaults duplicate on every swap - 16 bytes/cycle in the fixed-donor plateau series. destroy_op_array frees exactly those per-entry resources BEFORE its refcount check and returns without touching the shared arrays, so the destroy path now runs it for both lifetime classes instead of bailing out. Tests make the original failure shape a permanent regression test: - redefine-plateau.php returns a runtime-defined constant (no call site can be inlined at cache time) and instantiates PlateauClass inside the method dispatch (created after the first redefine's class copy-out); - RedefineLeakPlateauTest pins the child to opcache ON (jit off, file_update_protection=0), so every suite exercises the same-file first-redefine copy-out path; measured overheads stay 0 and the fixed-donor series is flat again (was +16000 bytes per 1000 cycles); - the opcache support matrix gains a same-file-redefine leg (a cold same-file call site must observe the writable copy through the repointed bucket) and an inlined-call-site-limitation leg pinning the documented pass-4 behavior, with the child's optimization_level pinned to the default pipeline. Fixes #242 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
…dering fix(opcache): invalidate before writing in BinaryCacheFile::refresh()
feat(reflection): copy literals alongside opcodes — lifts the 2GB IS_CONST reach limit
…heritance-cache feat(core): decline inheritance-cache publication for handler-hooked classes
…he payload Port the ZEND_TYPE_HAS_LIST branch of zend_file_cache_serialize_type / zend_file_cache_unserialize_type (ext/opcache/zend_file_cache.c, PHP-8.4.19): the zend_type_list pointer is relocated (SERIALIZE_PTR keeps walking through the still-real address, exactly like the C serialize/unserialize pair) and the walk recurses into every zend_type entry, so DNF sub-lists like (A&B)|C unfold naturally. The unsupportedPayload refusal for type lists is gone in both directions. New fixture tests/OpCache/fixtures/type-lists.php exercises a union parameter, a union return type, an intersection parameter and return type, and union/DNF property types (the DNF one nests an intersection list inside a union list). Acceptance evidence: - TypeListRelocationTest::testTypeListPayloadRoundTripsByteIdentical - the compiled fixture relocates and derelocates byte-for-byte - testUnmodifiedResaveStillExecutes / testPatchedTypeListFixtureExecutesFromCache - the re-serialized (and string-literal-patched) binary is executed by a fresh worker straight from the cache - full default suite, --group opcache --fail-on-skipped, phpstan level max and php-cs-fixer all green on PHP 8.4.19 NTS Fixes #112 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
Add Zend/zend_vm.h to the generator's preprocess unit and the function to the manifest, and regenerate the linux targets (native pre-check against the committed manifest was clean; the zts artifacts come from the docker pipeline). The opcache file cache stores every opline handler as an index (zend_serialize_opcode_handler); this ZEND_API counterpart restores the callable handler pointer and is what lets the cache-image bridge make relocated image bodies executable in-process (issue #122). The darwin/windows artifacts cannot be generated on this machine and are refreshed by their native generate workflows, which trigger on pull requests touching tools/generator/**. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
Wire the file-cache binary-patch pipeline to the runtime hot-swap API (CacheImageSync): a patched ReflectionOpcacheFile image is diffed against the live executor tables and every changed compiled BODY is swapped into the ALREADY-LOADED functions and methods in place, through the existing FunctionBodySwap machinery - no re-include, entry pointers preserved. Until now a patched binary only affected the next include (refresh()). - prepare() is a read-only diff; the equality basis (ImageFunctionDonor) compares body metrics, canonicalized opcodes (IS_CONST operands by literal index across the two storage forms, handlers and the uninitialized op1.num of implicit-$this receivers ignored), CV names, literals and static defaults by value - conservative where equality cannot be proven (array/AST literals re-apply, like ReflectionMethod). - Donor bodies are materialized per entry: opcodes+literals co-allocated into one process block, IS_CONST operands rewritten to the runtime relative form and handlers restored with the engine's own zend_deserialize_opcode_handler - the exact normalization zend_file_cache_unserialize performs. The image buffer is never written, so save()/refresh() stay valid after an apply. - apply() validates refusals first, copies opcache-shared targets out of SHM through the documented paths (redefine()'s function copy-out, extracted as FunctionLikeTrait::copyEntryOutOfSharedMemory(), and ReflectionClass::copyOutOfSharedMemory() for classes), then stages all swaps - functions before classes, alphabetical - and commits only when every swap staged; failures roll all staged bodies back. - Throw-or-work: changed enum/interface/trait methods, internal-name collisions and every SHM copy-out refusal throw; image-only entries are reported as not loaded in the explicit CacheImageSyncReport. - Lifetime: swapped-in bodies are refcount-less (engine never destroys them); the sync pins the materialized blocks and the image view now retains the relocated buffer's owner. - Seam for #121: prepare() is application-agnostic, an SHM publisher consumes the same prepared diff and replaces only the apply() target. The receiver-opcode constant stays untyped on purpose: a typed array constant holding a constant expression trips the debug-build assertion zend_update_class_constant:!EG(exception) under opcache.preload. Fixes #122 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
…swap-wiring # Conflicts: # docs/opcache-binary.md
…ype-lists feat(opcache): relocate intersection/union type lists in cache binaries
… payloads Port the num_traits branch of zend_file_cache_serialize_class / zend_file_cache_unserialize_class (ext/opcache/zend_file_cache.c, PHP-8.4.19): trait_names walks through the existing zend_class_name helper, and the NULL-terminated trait_aliases / trait_precedences pointer arrays are relocated entry by entry - each entry's trait_method.method_name / trait_method.class_name (NULL-able), the alias name, and every exclude_class_names slot of an insteadof precedence, in both directions. New raw-slot primitives (unPtrAt/serPtrAt, unStrAt/serStrAt) port (UN)SERIALIZE_PTR/STR for slots that have no owning struct field. New fixture tests/OpCache/fixtures/traits.php uses two traits with an insteadof precedence (exclude list), an alias with an explicit trait name and an alias without one that also changes visibility. Acceptance evidence: - TraitRelocationTest::testTraitPayloadRoundTripsByteIdentical - byte-for-byte round trip of the compiled fixture - testUnmodifiedResaveStillExecutes / testPatchedTraitFixtureExecutesFromCache - the re-serialized (and patched) binary executes from the cache with the flattened trait behavior intact - full default suite, --group opcache --fail-on-skipped (host and z-engine-php:debug84 container), phpstan level max and php-cs-fixer all green Fixes #114 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
The plain and refusal legs of CacheImageSyncTest pair an optimizer-OFF cache image (compiled by BinaryCacheFile::compile with opcache.optimization_level=0) with an unoptimized live side loaded from source, then assert an untouched image diffs as empty. That only holds when both are compiled at the SAME optimization level - the bridge's documented contract. The opcache-runner CI job sets opcache.enable_cli=1 in php.ini, which leaked into these children and ran the optimizer over their live-side require. The live entry then had a genuinely different compiled body (literal folding collapsed the 3-opcode source body to 1), so bodiesEqual() correctly reported a change and the untouched-diff assertion failed. Not a diff-basis gap: an optimizer-transformed body IS different machine code, and the diff is not meant to canonicalize across optimizer passes. Pin opcache.enable_cli=0 in the base child command so the plain leg is deterministic whatever the runner's php.ini says; the shared-memory leg re-enables it through $extraOptions, which come last and win. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
…raits feat(opcache): relocate trait-using classes in cache binaries
…fns)
Port the num_dynamic_func_defs branch of zend_file_cache_serialize_op_array /
zend_file_cache_unserialize_op_array (ext/opcache/zend_file_cache.c,
PHP-8.4.19): the zend_op_array* array is relocated, each def slot is
converted like the C SERIALIZE_PTR/UNSERIALIZE_PTR pair (offsets stored,
walking continues through the still-real address) and the walk recurses into
every nested op_array - so a closure defined inside another closure unfolds
through its own dynamic_func_defs. Both directions; the unsupportedPayload
refusal for closures is gone.
New fixture tests/OpCache/fixtures/closures.php holds an arrow function and an
anonymous function in a global function, a closure nested inside another
closure, and a scoped arrow function inside a static method.
Acceptance evidence:
- ClosureRelocationTest::testClosurePayloadRoundTripsByteIdentical -
byte-for-byte round trip of the compiled fixture
- testUnmodifiedResaveStillExecutes / testPatchedClosureFixtureExecutesFromCache
- the re-serialized (and patched) binary executes all closures from the
cache ('cl:42:42:...' proves arrow, anonymous, nested and method-scoped defs)
- full default suite, --group opcache --fail-on-skipped (host and
z-engine-php:debug84 container), phpstan level max and php-cs-fixer all green
Fixes #115
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
…losures feat(opcache): relocate closures/arrow functions (dynamic_func_defs) in cache binaries
…s payloads Port the iterator_funcs_ptr and arrayaccess_funcs_ptr branches of zend_file_cache_serialize_class / zend_file_cache_unserialize_class (ext/opcache/zend_file_cache.c, PHP-8.4.19): each zf_* zend_function pointer and the struct pointer itself are relocated in both directions, keeping the exact C ordering (serialize converts the members through the still-real struct pointer first and the struct pointer last; unserialize the reverse). NULL zf_* members stay NULL, as in the C macros. The get_iterator <-> HOOKED_ITERATOR_PLACEHOLDER swap of the 8.4 load path is deliberately not mirrored: the image is never executed in this process, so the placeholder is preserved verbatim like every other execution-only field and the written file keeps the exact bytes the engine expects (documented on unserializeIteratorFuncs). New fixture tests/OpCache/fixtures/iterators.php compiles classes implementing Iterator, IteratorAggregate and ArrayAccess. Plain compiles store classes unlinked (zend_compile does not early-bind classes that implement interfaces), so both pointers are NULL in such payloads; the crafted-buffer test drives the ported walk itself against an image whose class carries both structs with serialized offsets. Acceptance evidence: - IteratorFuncsRelocationTest::testIteratorPayloadRoundTripsByteIdentical - byte-for-byte round trip of the compiled fixture - testUnmodifiedResaveStillExecutes / testPatchedIteratorFixtureExecutesFromCache - the re-serialized (and patched) binary executes foreach over Iterator and IteratorAggregate plus ArrayAccess reads/writes from the cache - testCraftedIteratorFuncsRelocateAndSerializeBack - both structs and every zf_* slot (NULLs included) relocate to real addresses and serialize back to the exact original bytes - full default suite, --group opcache --fail-on-skipped (host and z-engine-php:debug84 container), phpstan level max and php-cs-fixer all green Fixes #116 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
…terators feat(opcache): relocate iterator-aware and ArrayAccess classes in cache binaries
…load Port the prop->hooks branch of zend_file_cache_serialize_prop_info / zend_file_cache_unserialize_prop_info (ext/opcache/zend_file_cache.c, PHP-8.4.19): the zend_function*[ZEND_PROPERTY_HOOK_COUNT] array is relocated, and each non-NULL hook slot is converted and its op_array walked in both directions (hook bodies shared with the class function_table return early through the existing opcodes guard, as in C). NULL get/set slots stay NULL. The last payload-shape refusal is gone; only the platform refusals (Windows/32-bit, ZTS - issues #119/#118) remain. The hooked-class get_iterator field holds HOOKED_ITERATOR_PLACEHOLDER in the file; it is execution-only and preserved verbatim (see the #116 notes on unserializeIteratorFuncs), so the placeholder round-trips untouched. New fixture tests/OpCache/fixtures/property-hooks.php compiles a property with both get and set hooks, a get-only virtual property and a set-only backed property. Acceptance evidence: - PropertyHookRelocationTest::testPropertyHookPayloadRoundTripsByteIdentical - byte-for-byte round trip of the compiled fixture - testUnmodifiedResaveStillExecutes / testPatchedPropertyHookFixtureExecutesFromCache - the re-serialized (and patched) binary executes get/set hooks from the cache ('0:40:gauge-40:0:...' proves both hooks, the virtual getter and the set-only clamp ran) - full default suite, --group opcache --fail-on-skipped (host and z-engine-php:debug84 container), opcache-runner mode (opcache.enable_cli=1, --exclude-group performance/internal/opcache-incompatible), phpstan level max and php-cs-fixer all green Fixes #113 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
…roperty-hooks feat(opcache): relocate PHP 8.4 property hooks in cache binaries
…nder-opcache fix(reflection): free swap-minted resources of SHM-shared bodies; pin the optimizer-inlining semantics of redefine under opcache
setup-php installs PHP through Homebrew on the macOS runners, and brew by default runs a full `brew update` before every install plus a cleanup pass after it. On these jobs that is tens of seconds of unrelated formula churn wrapped around a single PHP install - "Set up PHP" was measured at 36-53s on macOS arm64 and 78-101s on macOS x64, dominated by that churn rather than the install itself. Set HOMEBREW_NO_AUTO_UPDATE=1 and HOMEBREW_NO_INSTALL_CLEANUP=1 at job level on every macOS runner: tests-macos and header-drift-darwin in ci.yml, and the generate job of the darwin header workflow. The runner images already ship a recent brew and these jobs install nothing but PHP, so both passes are pure overhead. The Windows jobs are left untouched - setup-php uses Chocolatey there, not Homebrew, so these variables would be inert. No caching action is added (extension caching was considered and declined): this is env-only tuning that changes nothing about what the jobs build or test. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
…swap-wiring # Conflicts: # docs/opcache-binary.md
…eedup ci: skip Homebrew auto-update/cleanup on the macOS setup-php jobs
…ring feat(opcache): apply patched cache images to already-loaded code (CacheImageSync)
Lift the ZTS refusal in PayloadRelocator (isSupported() and the constructor throw). The refusal was precautionary, not structural: zend_file_cache.c (PHP-8.4.19) contains no thread-safety conditionals, and a field-by-field diff of the generated layouts.json for linux-x64-nts vs linux-x64-zts shows every struct the walker dereferences (zend_persistent_script, zend_file_cache_metainfo, zend_op_array, zend_class_entry, zend_string, Bucket, zval, ...) is byte-identical - only zend_executor_globals, zend_compiler_globals and zend_module_entry differ, none of which appear in a payload. No layout-dependent walking needed adapting. Config/docs follow: composer.json's test:opcache-zts drops the --exclude-group opcache-relocator exclusion (name kept as the alias CI's ZTS legs call), ci.yml's ZTS matrix comments/gates and the debug-job ZTS opcache_args now cover the full opcache group, AGENTS.md and docs/opcache-binary.md describe ZTS as supported, and the relocator tests' skip message no longer names ZTS. Acceptance evidence: - z-engine-php:debug84-zts container (PHP 8.4.24 ZTS DEBUG, built from tools/docker/php-debug.Dockerfile with PHP_TS=zts): `phpunit --group opcache --fail-on-skipped` with NO relocator exclusion - OK (47 tests, 318 assertions), zero skips: every relocator test ran and passed on ZTS, byte-identical round trips included - host NTS: full default suite (519 tests, skip/incomplete counts unchanged), --group opcache --fail-on-skipped OK (47), z-engine-php:debug84 NTS container OK (47), phpstan level max clean, php-cs-fixer clean Fixes #118 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
…pwire Scope finding for issue #119 (macOS/arm64 relocator support): the per-opline absolute-address branches of zend_file_cache_(un)serialize_op_array (op1/op2.zv SERIALIZE_PTR and the jmp_addr switch) are compiled in only when ZEND_USE_ABS_CONST_ADDR / ZEND_USE_ABS_JMP_ADDR are 1, and zend_compile.h (PHP-8.4.19) defines both as 1 exactly when SIZEOF_SIZE_T == 4. Darwin x64 and arm64 are 64-bit builds and use the same relative addressing as linux - there is no darwin-specific opline walking to port. Implementing those branches would be dead code on every build the relocator supports, so they are deliberately NOT implemented; 32-bit builds (the only ones that use absolute addressing) stay refused by the PHP_INT_SIZE === 8 predicate. What lands instead: - OpcodeAddressingModelTest: proves the relative model on a real compiled payload - every IS_CONST operand in the file is a literal-table index and every JMP-family operand lands on an opline of its own op_array. The test runs (does not skip) on every supported build, darwin CI legs included, and FAILS loudly if a build ever produces absolute operands. - fixtures/addressing-probe.php: top-level code built around getenv() so SCCP cannot fold away the ?: and !== branches - IS_CONST operands and conditional jumps are guaranteed in the main op_array. - The opcodes comment in PayloadRelocator and docs/opcache-binary.md now state the invariant and its source precisely. Darwin execution of these tests happens on the PR's tests-macos CI legs (the relocator group already gates there since #118 removed the ZTS exclusion); no local darwin validation is possible from this environment. Acceptance evidence: - host: full default suite (520 tests, baseline skip counts), --group opcache --fail-on-skipped OK (48 tests, 338 assertions) - z-engine-php:debug84 and z-engine-php:debug84-zts containers: same opcache gate OK (48/338 each) - phpstan level max clean, php-cs-fixer clean Fixes #119 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
The from-scratch writer sketched in issue #117 as ScriptSerializer: a two-pass port of zend_persist_calc -> zend_persist (ext/opcache/zend_persist.c, PHP-8.4.19) fused with the offset-encoding stage of zend_file_cache_serialize. Pass 1 walks the (possibly mutated) live graph from the zend_persistent_script, deduplicating every reachable allocation unit through an xlat table (the zend_shared_alloc_*_xlat_entry port) and summing ZEND_MM_ALIGNED sizes; pass 2 emits a fresh contiguous region - units copied byte-verbatim, every pointer field rewritten, late references (scopes, prototypes, hook prop_info back-references, magic-method slots, IS_INDIRECT interior pointers) resolved against the finished xlat like zend_persist.c's late lookups. The emitted region is a valid relocated image whose on-disk offset encoding is delegated to the existing PayloadRelocator serialize stage, so the offset format has exactly one implementation. Walkers cover the full 8.4 payload surface: op_arrays (static vars, literals, opcodes, arg_info incl. the arg_info[-1] return slot, vars, live ranges, try/catch, attributes, dynamic_func_defs), classes (unlinked and linked-parent, constants, properties incl. hooks, interface/trait names, aliases, precedences, iterator/arrayaccess funcs), zvals/arrays/constant ASTs, warnings and early bindings. script->size is re-stamped (it is the loader's IS_SERIALIZED bound); zend_hash_persist's sparse-table compaction is deliberately skipped (optimization only) and every string is region-copied exactly like a file_cache_only child (nothing is accel-interned there), stamping zend_set_str_gc_flags' interned bits on sources that lack them. The API seam: ReflectionOpcacheFile::addFunctionFrom()/addMethodFrom() graft op_arrays from DONOR cache binaries (compiled by a real opcache child, so their oplines are already file-form - handler-table indexes and literal-index operands are not derivable in-process without unexported engine helpers), regrowing the target hashtable outside the buffer with a faithful re-implementation of the persisted-table insert (hash slots ahead of arData, bucket-index chains, HT_SIZE_TO_MASK = -(2*size); persisted data blocks must never be touched by zend_hash_add). BinaryCacheFile::save() routes grown graphs through the serializer automatically and keeps the byte-exact derelocate() path for in-place edits. Whole added classes and in-process compiled op_arrays remain out of scope and are refused loudly. Also fixes a latent relocator bug found by the serializer's byte checks: _ZSTR_HEADER_SIZE is XtOffsetOf(zend_string, val) = sizeof - 8, not sizeof - 1, which made emitInterned over-copy 7 bytes per emission. Acceptance evidence (GraphGrowingSerializerTest): - issue #117 acceptance: a brand-new function AND a new method grafted into the cached answer.php execute from the file cache in fresh workers ('added-fn', 'added-method-ok'), alongside the original entries; the grown binary passes checksum and round-trips byte-identically through the relocator - rebuild coverage: all seven fixture payloads (attributes/statics, type lists, traits, closures, property hooks, iterators, jump/const probe) re-emitted from scratch execute from the cache and round-trip byte-identical - refusal paths: unknown donor entries and duplicate keys throw dedicated OpCacheException factories - full default suite (530 tests, baseline skips), --group opcache --fail-on-skipped OK (58 tests, 437 assertions) on host, debug84 and debug84-zts containers; phpstan level max clean (the two new pointer-surgery zones carry the same scoped ignores as PayloadRelocator); php-cs-fixer clean Fixes #117 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
…ust model Defense-in-depth for the file-cache reader (issue #123). PayloadRelocator turned every stored offset into base + stored and drove its loops off count fields read straight from the payload, without range checks - so a crafted .bin carrying the current build's system_id (a build fingerprint, not an authenticator; adler32 is forgeable) fed into getReflection() was an FFI arbitrary read/write primitive. Now every stored value is validated before it becomes an address the engine walks, in the relocate() (untrusted-input) path: - requireOffset: interior-pointer offsets against [0, memSize] - requireStringOffset: tagged interned-string offsets against [0, strSize), plain string offsets against [0, memSize] - requireSpan / requireCount: scriptOffset and every count-driven element array - hashtable buckets/packed data, literals, arg_info (incl. the arg_info[-1] return slot), vars, type lists, attribute args, class property tables, properties_info_table, class/trait names, trait alias/precedence NUL-terminated arrays and their exclude lists, property hooks, dynamic_func_defs, ast children/nodes, warnings and early bindings A violation throws OpCacheException::malformedPayload - a loud refusal, never an out-of-bounds walk. The derelocate()/serialize() path and the #117 graph ScriptSerializer operate on an already-relocated in-process image and inherit this validation. The tagged-offset bound tracks the string section serialize() just rebuilt (re-pinned from the header str_size), so the relocate() inside derelocate() validates against the section it produced, not the stale size. docs/opcache-binary.md gains a "Trust model" section: .bin input must come from a trusted source; system_id is a build fingerprint (ABI guard), not authenticity; adler32 catches accidental corruption, not tampering; the bounds checks turn a memory-safety catastrophe into a clean exception but are not a licence to load untrusted code. The optional keyed-MAC for distributing protected binaries is described as a deploy-side responsibility and flagged as a possible follow-up, deliberately not built in (key management belongs to the application). BoundsValidationTest proves the loud refusal on a truncated buffer, an out-of-range scriptOffset, a hostile pointer field, a hostile hash count and an out-of-range interned-string offset - and that a well-formed image still relocates and round-trips (no false positives). Run in the debug84 and debug84-zts containers too, where an unguarded out-of-bounds read segfaults loudest: no crashes, all refusals clean. Acceptance evidence: - full default suite (536 tests, baseline skips), --group opcache --fail-on-skipped OK (64 tests, 472 assertions) on host, debug84 and debug84-zts; phpstan level max clean; php-cs-fixer clean Fixes #123 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
…apes
The opcache pointer-surgery files were held at level max only through
path-scoped ignore blocks in phpstan.dist.neon (property.nonObject,
binaryOp.invalid, cast.int, argument.type, assignOp.invalid) because their
multi-hop FFI\CData reads resolve to mixed after the first hop. Narrow those
reads to the generated ZEngine\Generated\* engine-struct stubs the rest of the
codebase already uses, surfaced through the docblock boundary-narrowing
convention (native `object`, `@param`/`@var`/`@return` stub type), so the
walkers type-check without any behaviour change.
Removed the path-scoped ignore blocks for all four files:
- src/OpCache/PayloadRelocator.php (property.nonObject, binaryOp.invalid,
cast.int, argument.type)
- src/OpCache/ScriptSerializer.php (property.nonObject, binaryOp.invalid,
cast.int, argument.type)
- src/OpCache/ReflectionOpcacheFile.php (property.nonObject, argument.type,
binaryOp.invalid, cast.int, assignOp.invalid)
- tests/OpCache/BoundsValidationTest.php (argument.type)
How the errors were retired:
- pointerAtAddress('T *', ...) string casts -> the stub class-string form
(pointerAtAddress(T::class, ...)), which the TypedEntryPointReturnExtension
types as the stub while the runtime value stays FFI\CData;
- generic walker params typed to the owning stub via @param/@var docblocks
(native type stays `object` - the stubs are analysis-only and would raise a
runtime TypeError if used as native types);
- IS_PTR bucket reads go through the typed zend_value->lval accessor;
- raw uintptr_t slot dereferences go through a single readSlot() primitive
guarded by assert(is_int(...)), matching Core::threadLocalStorageBase();
- FFI::string/FFI::memcpy sizes stated non-negative with max(...,0), matching
HashTable::count()'s analyser clamp;
- the trivial unserializeType/serializeType field wrappers inlined to
unserializeTypeStruct($x->type) on the now-typed owners.
Justified surviving inline @PHPStan-Ignore argument.type (8 total), each an
FFI::addr() on a pointer field that must stay inline to yield the field SLOT
address (a by-value hop through Core::addr() addresses a pointer copy - proven
by BoundsValidationTest) and cannot be CData-typed because the field name is
dynamic: PayloadRelocator ptrValue/writePtrField, ScriptSerializer
ptrValue/put/defer, ReflectionOpcacheFile addMethodFrom (scope re-point), and
BoundsValidationTest's two filename-slot corruptions. This mirrors the existing
Compiler.php precedent for inline FFI::addr ignores.
phpstan-baseline.neon is untouched. Full suite is byte-for-byte identical:
536 tests / 5408 assertions / 5 skipped / 5 incomplete before and after, in
default, opcache-runner and release --group opcache modes, and in the
debug84 container.
Fixes #126
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
The rebase of #126 onto #122 kept both #126's standalone `private readonly object $script` declaration and #122's promoted constructor property of the same name, a fatal redeclaration. Keep #122's promoted property (per the merge resolution) and move #126's `@var zend_persistent_script` narrowing onto the promoted param so PHPStan types $this->script as the stub rather than the FFI\CData|zend_persistent_script @PARAM union, restoring #126's zero-path-scoped-ignore struct-shape typing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BDcCQiYqbMkjRPyhWgLL6M
feat(opcache): ZTS support for the file-cache relocator
…ocator feat(opcache): macOS/arm64 support for the relocator (addressing-model tripwire)
…alizer feat(opcache): graph-growing cache serializer (add functions/methods to a cached script)
…idation opcache: bounds-validate relocation offsets and document the trust model
…apes refactor(opcache): replace path-scoped PHPStan ignores with struct shapes
6 tasks
Owner
|
All 8.4 PRs are merged, so this cascade is now resolved manually in #273 ( Generated by Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated cascade merge of
8.4intomaster(branch flow defined in.github/branch-flow.json).Resolve conflicts in favour of the newer engine structures where they touch
include/- regenerate headers on the target branch instead of merging them textually. See AGENTS.md.