Conversation
github-actions
Bot
force-pushed
the
master
branch
4 times, most recently
from
6850c08 to
3b6f413
Compare
github-actions
Bot
force-pushed
the
master
branch
9 times, most recently
from
73281bd to
c3ee91f
Compare
- Hourly upstream sync from postgres/postgres (24x daily) - AI-powered PR reviews using AWS Bedrock Claude Sonnet 4.5 - Multi-platform CI via existing Cirrus CI configuration - Cost tracking and comprehensive documentation Features: - Automatic issue creation on sync conflicts - PostgreSQL-specific code review prompts (C, SQL, docs, build) - Cost limits: $15/PR, $200/month - Inline PR comments with security/performance labels - Skip draft PRs to save costs Documentation: - .github/SETUP_SUMMARY.md - Quick setup overview - .github/QUICKSTART.md - 15-minute setup guide - .github/PRE_COMMIT_CHECKLIST.md - Verification checklist - .github/docs/ - Detailed guides for sync, AI review, Bedrock See .github/README.md for complete overview Complete Phase 3: Windows builds + fix sync for CI/CD commits Phase 3: Windows Dependency Build System - Implement full build workflow (OpenSSL, zlib, libxml2) - Smart caching by version hash (80% cost reduction) - Dependency bundling with manifest generation - Weekly auto-refresh + manual triggers - PowerShell download helper script - Comprehensive usage documentation Sync Workflow Fix: - Allow .github/ commits (CI/CD config) on master - Detect and reject code commits outside .github/ - Merge upstream while preserving .github/ changes - Create issues only for actual pristine violations Documentation: - Complete Windows build usage guide - Update all status docs to 100% complete - Phase 3 completion summary All three CI/CD phases complete (100%): ✅ Hourly upstream sync with .github/ preservation ✅ AI-powered PR reviews via Bedrock Claude 4.5 ✅ Windows dependency builds with smart caching Cost: $40-60/month total See .github/PHASE3_COMPLETE.md for details Fix sync to allow 'dev setup' commits on master The sync workflow was failing because the 'dev setup v19' commit modifies files outside .github/. Updated workflows to recognize commits with messages starting with 'dev setup' as allowed on master. Changes: - Detect 'dev setup' commits by message pattern (case-insensitive) - Allow merge if commits are .github/ OR dev setup OR both - Update merge messages to reflect preserved changes - Document pristine master policy with examples This allows personal development environment commits (IDE configs, debugging tools, shell aliases, Nix configs, etc.) on master without violating the pristine mirror policy. Future dev environment updates should start with 'dev setup' in the commit message to be automatically recognized and preserved. See .github/docs/pristine-master-policy.md for complete policy See .github/DEV_SETUP_FIX.md for fix summary Optimize CI/CD costs by skipping builds for pristine commits Add cost optimization to Windows dependency builds to avoid expensive builds when only pristine commits are pushed (dev setup commits or .github/ configuration changes). Changes: - Add check-changes job to detect pristine-only pushes - Skip Windows builds when all commits are dev setup or .github/ only - Add comprehensive cost optimization documentation - Update README with cost savings (~40% reduction) Expected savings: ~$3-5/month on Windows builds, ~$40-47/month total through combined optimizations. Manual dispatch and scheduled builds always run regardless.
Review every PR (including drafts) with two jobs that authenticate to AWS Bedrock (Claude Opus 4.8) via GitHub OIDC (vars.AWS_ROLE_ARN); no static AWS credentials are stored in the repo. - ocr-review: runs Alibaba Open Code Review through an ephemeral LiteLLM proxy bridging OCR's OpenAI protocol to Bedrock, and posts inline review comments. Uses output_config.effort=xhigh (Opus 4.8 adaptive thinking). Path-scoped rules (.github/ocr/rule.json) encode PostgreSQL community review standards plus reviewer discipline (verify against the diff, don't hallucinate, state confidence, be blunt, accuracy over approval). - pg-history: OCR cannot call MCP, so a separate Bedrock tool-use agent (.github/ocr/pg-history.py) queries the Agora MCP server (pg.ddx.io) to tie the change to git + pgsql-hackers history, and upserts a comment linking threads as https://pg.ddx.io/m/pgsql-hackers/<message-id>.
The pg-history workflow job has been failing every run with 'Bedrock call failed: The read operation timed out' -- botocore's default 60s read timeout on bedrock-runtime is too short for a multi-round (MAX_ROUNDS=14) tool-use loop against a large PR diff on a reasoning-capable model; a single converse() call alone can take several minutes under load (the sibling ocr-review job's own LLM pass over a similarly large diff took 30-40 minutes). Confirmed via two consecutive live runs against PR #26. Set read_timeout=900s (15 min) explicitly via botocore.config.Config; leave connect_timeout short since a stuck TCP handshake is a different, cheaper-to-detect failure mode that shouldn't wait as long.
github-actions
Bot
force-pushed
the
master
branch
from
c3ee91f to
c7e7712
Compare
Sparsemap is a memory-efficient data structure for maintaining sparse sets of integers using hierarchical bitmaps. It supports O(1) set/get operations and efficient iteration over set bits while using far less memory than a dense bitmap for sparse populations. The implementation provides: - sparsemap_set/get/is_set for individual bit manipulation - sparsemap_scan for efficient forward iteration - sparsemap_select for rank-based selection - Configurable initial capacity with automatic growth Used by the UNDO subsystem for tracking allocated pages and by RECNO for free-space management within relation forks. Includes a TAP regression test module (test_sparsemap) exercising all public API operations.
Header-only implementation of a probabilistic skip list providing O(log n) insert, delete, and lookup operations with O(n) space. Compared to rbtree, skip lists offer simpler implementation, better cache locality for sequential scans, and lock-free read potential. The implementation provides: - Type-safe macros for defining typed skip lists (DEFINE_SKIPLIST) - Configurable maximum height (up to 32 levels) - Forward iteration via SKIPLIST_FOREACH - Range queries and nearest-neighbor lookup - Memory allocation via palloc (TopMemoryContext by default) Used by the UNDO subsystem for maintaining ordered transaction metadata and by RECNO for HLC-ordered page directories. Includes a TAP regression test module (test_skiplist) exercising insertion, deletion, iteration, and edge cases.
Introduce two TableAmRoutine booleans and the begin_bulk_insert callback that the UNDO subsystem builds on, plus the RelationAmSupportsUndo() accessor index AMs use to gate UNDO record generation on the parent table. am_supports_undo marks an AM that registers an UNDO resource manager and emits UNDO records tagged with its own rmid; the UNDO core stays AM-agnostic and interprets the payload only through that RM's callbacks. am_inplace_update_keeps_tid marks an AM that updates in place and keeps the row's TID, so the executor can skip redundant index re-inserts for unchanged keys. The heap AM leaves both false. This commit only adds the routine fields and the accessor; no AM sets the flags yet.
Add the AM-agnostic UNDO engine: the in-WAL UNDO record format and insertion path, the per-relation RELUNDO fork with its own resource manager, the shared sLog tuple-state map, the rollback apply driver and compensation-log generation, the discard horizon, and the background revert/undo workers. Register the UNDO, ATM, and RELUNDO resource managers and wire the subsystem into transaction start/commit/abort, two-phase commit, recovery, and process startup. The engine interprets UNDO payloads only through per-RM callbacks and the RelUndo*_hook function pointers (defined here, left NULL), so the core has no compile-time knowledge of any specific access method. Heap, vacuum, pruning, reloptions, and executor integration consume only the AM-agnostic interfaces. No UNDO-producing AM is registered yet: RegisterUndoRmgrs() initializes the dispatch table but registers no per-AM handlers. The index-AM apply handlers and the AM that sets am_supports_undo arrive in later commits.
Add the UNDO resource-manager handlers for the nbtree and hash index AMs and register them from RegisterUndoRmgrs(). On rollback of an aborting transaction, the nbtree handler re-descends to the leaf entry by key and heap TID before marking it dead, so a committed entry that shifted onto the recorded slot under concurrent inserts or leaf splits is never killed; entries inside posting-list tuples are left for VACUUM. The hash handler reverses its own inserts analogously. Both are gated by RelationAmSupportsUndo() on the parent table, so they are inert until an UNDO-supporting table AM exists.
Add pg_setxattr/pg_getxattr/pg_removexattr/pg_listxattr wrappers over the platform extended-attribute syscalls (Linux/*BSD/macOS, no-op stubs where unsupported) and build them into libpgport. The transactional file-ops resource manager added next uses these to record and reverse xattr mutations.
Filesystem mutations the server performs outside the buffer manager -- creating and removing directories, copying trees, writing version files, managing symlinks, setting extended attributes -- historically had no WAL coverage and relied on best-effort cleanup callbacks that do not survive a crash. Add FILEOPS: a resource manager (RM_FILEOPS_ID) that records each mutation as a deferred pending operation, executes it at commit, and WAL-logs it so redo reproduces it during crash recovery and standby replay. This commit adds the operation engine, the public FileOps* API, the WAL record formats, the redo handler, and the rmgrdesc descriptors. A README under src/backend/storage/file explains why FILEOPS exists and how to use the API. The rollback (UNDO) side and the rewiring of existing callers follow in subsequent commits.
The FILEOPS engine added in the previous commit makes filesystem mutations crash-safe via WAL redo, but redo alone only reproduces a committed operation -- it cannot reverse one when the surrounding transaction aborts. Operations such as chmod, chown, truncate, and setxattr overwrite prior state in place, so undoing them requires the before-image that was captured when the operation ran. This commit adds fileops_undo.c, which registers a handler with the UNDO machinery (RM_UNDO_ID, subtypes FILEOPS_UNDO_*). For each reversible operation it records the before-image -- the original mode for a chmod, the prior owner for a chown, the original length for a truncate, the previous extended-attribute value for a setxattr, and so on -- and during UNDO application performs the inverse action: unlink a created file, rename back, or restore the saved mode, owner, length, or xattr value. FileopsUndoRmgrInit() registers the handler at startup alongside the other built-in UNDO resource managers. The storage/file build manifests are updated to compile fileops_undo.c, and undo.c now includes storage/fileops.h and calls FileopsUndoRmgrInit() from RegisterUndoRmgrs().
With the FILEOPS engine and its UNDO handlers in place, replace the raw filesystem mutations in the transaction-sensitive code paths with the FileOps* API so they become atomic with the transaction and crash-safe. CREATE/DROP DATABASE (dbcommands.c), CREATE/DROP TABLESPACE and ALTER DATABASE ... SET TABLESPACE (tablespace.c), and the directory-copy helper (copydir.c) now register deferred operations instead of calling mkdir/symlink/rename/rmtree directly, and xact.c drives the pending-op queue at commit, abort, subtransaction boundaries, and PREPARE. pg_waldump gains the fileops rmgr descriptor (fileopsdesc.c symlink plus its rmgrdesc.c table entry) so XLOG_FILEOPS_* records render in waldump output. The change ships user documentation (fileops.sgml and its filelist/postgres wiring, a worked example), a test_fileops contrib module exercising the API, regression coverage (regress/fileops.sql), and recovery TAP tests that crash mid-operation and verify redo and rollback. typedefs.list records the new types.
PostgreSQL's heap appends a new tuple version on every UPDATE and abandons the old one to VACUUM. For update-heavy workloads on narrow hot rows -- counters, queue heads, running aggregates, time-series tail writes -- the resulting version churn dominates buffer traffic, bloats relations, and keeps autovacuum permanently behind. RECNO is a table access method (amname "recno", RM_RECNO_ID) that updates tuples in place. An UPDATE overwrites the committed bytes on the main fork; the prior image is preserved in the per-relation UNDO fork (RELUNDO_FORKNUM) so a ROLLBACK restores it and a concurrent snapshot reader can still reconstruct the version it is entitled to see. Heap is untouched; a relation opts in with USING recno. Visibility is driven by a hybrid logical clock rather than per-tuple xmin/xmax scans. Each tuple header carries an HLC commit stamp; a snapshot captures an HLC horizon and a tuple is visible when its stamp precedes the horizon and its writer has committed. Transient state (uncommitted / deleted / updated) lives in header flag bits that UNDO clears on rollback through the RelUndoClearTransientFlags hook, and in-place delta reversal runs through RelUndoReverseDelta; RECNO installs both at init via RecnoRelUndoInstallHooks() so the UNDO core never sees a RECNO tuple layout. Same-page updates rewrite the slot directly; updates that no longer fit spill to an overflow chain with optional per-attribute dictionary compression (LZ4/ZSTD, ANALYZE-refreshed). A sparsemap-backed dirty map and a partitioned secondary log (sLog) serve before-images to old readers without a separate version-store cache; free space and visibility are tracked in RECNO-private FSM and VM forks. WAL coverage is complete: insert, in-place and out-of-place update, delete, tuple-lock, VACUUM, dict writes, and overflow all log redo records under RM_RECNO_ID with matching recovery routines, and crash recovery cooperates with the UNDO driver to roll back in-place loser transactions. Index AMs over a RECNO table force a heap recheck on index-only scans and tolerate stale index entries left by in-place updates until before-image reclamation retires them. The commit also wires RECNO into the supporting infrastructure: pg_am / pg_proc catalog entries, the recno rmgr description routine, pageinspect coverage (pageinspect 1.13 -> 1.14), the in-tree documentation chapter, and isolation, regression, and crash-recovery test suites exercising MVCC correctness, overflow, dictionary compression, and dual-mode UNDO rollback.
Every committed in-place UPDATE now appends an 8-byte RelUndoRecPtr to its new on-page image, guarded by RECNO_TUPLE_HAS_VERSION_PTR, pointing at the head of the relation's persistent version chain in the UNDO fork. Never-updated rows stay at their base length; a row grows by 8 bytes once on its first UPDATE and stays that size for subsequent same-size updates on the CAS fast path. The CAS fast path now targets base+8 and stamps the freshly reserved RelUndoRecPtr before computing the byte-diff, so the diff and the page memcpy both see the final slot layout. First-time +8 growth is performed on the exclusive-lock path, sized to the full slot with the verptr at the tail and logged as a full image to avoid primary/replica divergence. The widen-to-full-slot fixup runs before the critical section, since its repalloc/pfree are forbidden once START_CRIT_SECTION is entered. Accessors RecnoTupleGetVersionPtr/SetVersionPtr read and write the trailing field via memcpy (unaligned-safe) from the on-page slot tail, keyed on ItemIdGetLength rather than any logical length field. Also fixes two latent reverse-apply bugs that activate once reads rely on reconstruction: size the DELTA_UPDATE reconstruction buffer to the diff's old_total_len (not the current on-page length, which a shrinking update makes too small), and promote the slot-too-small restore skip from DEBUG1 to WARNING. RECNO: reconstruct prior versions from UNDO fork, drain sLog (WS-PVS2/3/4) Implement the CTR paper's sLog/PVS division of labor: the durable per-relation UNDO fork becomes the Persistent Version Store, and MVCC readers reconstruct prior tuple versions on demand from its byte-diffs instead of keeping full before-images in the in-memory sLog. WS-PVS2: RecnoReconstructVisibleVersion (recno_pvs.c) walks a tuple's version chain via its trailing verptr -> RelUndoReadRecord, reverse- applying each RELUNDO_DELTA_UPDATE / RELUNDO_UPDATE record until it reaches the version visible to the reader's snapshot. Replaces the committed-UPDATE branch of the former SLogTupleGetSharedBeforeImage call sites on all fetch paths (seq, TID, plain/IOS index, bitmap). WS-PVS3: stop publishing committed-UPDATE before-images into the sLog. Phase 1 removes the DSA before-image allocation and the shared-image getter; Phase 2 stops retaining committed-UPDATE markers in the flat hash and migrates the lost-update probe onto the fork (RecnoTupleHasCommittedUpdateAfter). Together these drain both the DSA memory limit and the bucket-count table_full saturation. WS-PVS4: complete the RelUndoRecPtr generation-counter machinery so a recycled fork page cannot alias a discarded probe record. Bump the metapage counter on free-list recycle and validate it on read; the counter is durably logged with the page reinit in one WAL record. Tests: recno TAP 5/5, recovery 061/063/065, isolation 148/148 all pass. The pre-existing varlen-growing-UPDATE regress failures and the select_parallel worker-count flake are unchanged from baseline. RECNO: COW-reference unchanged overflow columns on UPDATE (WS-Q2 Layer 0) A narrow-column UPDATE of a wide row re-stored the unchanged wide column into fresh overflow pages, doubling on-disk size until VACUUM caught up. Carry a 32-bit whole-value content hash in RecnoOverflowPtr (reclaimed from the dead ov_padding/ov_flags fields so the struct stays 20 bytes and RECNO_OVERFLOW_PTR_SIZE is unchanged -- growing it would defeat the force-shrink UPDATE recovery path). At the exclusive-path form site, snapshot the old on-page overflow pointers by attnum while the buffer is locked, then for each over-threshold varlena compare length+hash against the old pointer (zero I/O). On a match, byte-verify against the fetched old chain and, if equal, build a pointer varlena that references the old chain verbatim and skip re-storing. The byte-verify makes the hash a pure performance prefilter, never a correctness dependency: a collision merely wastes a fetch and falls through to a normal re-store. Sharing is safe without refcounts because RECNO UPDATE is strictly in-place and VACUUM Pass 1 already unions chain locators from every live HAS_OVERFLOW tuple, so a shared chain stays live while any referencer is. RECNO: test VACUUM retains version diffs a live snapshot needs (WS-PVS4) Committed in-place UPDATEs keep their before-image only as a byte-diff in the per-relation UNDO fork; RelUndoVacuum discards diffs whose updater xid precedes GetOldestNonRemovableTransactionId. A live REPEATABLE READ snapshot holds that horizon back, so VACUUM must leave the diff intact and the snapshot must still reconstruct the prior version. The existing recno-retained-reclamation spec never runs VACUUM and recno-vacuum-concurrent only checks DELETE visibility, so the retention gate had no direct coverage. This spec pins a snapshot, updates+commits+ VACUUMs the row, and asserts the snapshot still reads the original value. RECNO: reserve version-pointer headroom at insert (WS-PVS1 fix) Reserve sizeof(RelUndoRecPtr) trailing headroom on every inserted tuple so the first committed in-place UPDATE is a same-length overwrite rather than an +8-byte growth. Without this, a densely packed page of fixed-width rows has no residual free space for the per-row +8 growth, and a multi-row UPDATE fails with "updated recno tuple does not fit on page" once more than a handful of rows on one page are updated in a single command. Every inserted tuple is now born with RECNO_TUPLE_HAS_VERSION_PTR set and an InvalidRelUndoRecPtr trailing field. Readers already treat an invalid chain head as "no history", so never-updated rows are semantically unchanged; the first UPDATE stamps the real chain-head pointer into the pre-reserved slot via the existing CAS fast path. Patched both the single-insert (recno_tuple_insert) and batch (recno_multi_insert) paths. recno regression expected sizes updated to reflect the +8B/row on-page footprint (HEAP sizes unchanged). recno isolation suite: 149/149.
RecnoVacuumCrossPageDefrag held table buffer content locks (the source page EXCLUSIVE across the whole move loop, plus a re-read SHARE lock on the moved tuple) while calling index_insert. index_insert descends into _bt_search, which takes index leaf locks: a TABLE->INDEX lock order. Concurrent nbtree bottom-up deletion takes INDEX->TABLE (it SHARE-locks the table page from under the index leaf lock in recno_index_delete_ tuples). Buffer content locks are LWLock-style and invisible to the deadlock detector, so the cycle hangs forever rather than being aborted. Defer index maintenance until every table page content lock is released. While the destination page is still locked, copy each moved tuple off-page into a palloc'd buffer and record its new TID; after UnlockReleaseBuffer on both source and destination pages, replay the index inserts from those copies with no table content lock held. This mirrors heap VACUUM, which never calls index_insert under a heap buffer content lock. Safe because overflow-carrying pages are skipped by the defrag, so the copied tuple bytes are self-contained.
A SELECT ... FOR NO KEY UPDATE locker (or an EPQ recheck) parked in XactLockTableWait under RecnoLockTuple holds LOCKTAG_TUPLE for the tid and sleeps on the in-progress writer's xid. The recno_tuple_lock handler path already waits only on conflicting writers via SLogTupleGetWriteConflictXid / SLogTupleGetLockConflictXid, so a lone locker is served. The gap is the CAS fast path in recno_tuple_update: same-size committed-row overwrites took only the per-tuple t_writer spinlock plus a share-exclusive page lock and never touched LOCKTAG_TUPLE. A stream of such writers therefore laps the parked locker -- each writer overwrites the clean-committed row, the locker wakes, re-reads a freshly-uncommitted row, and re-parks on the next writer's xid. The locker is starved until statement_timeout fires. It is a liveness/fairness bug, not a correctness one: the reacquire loop still locks the final committed version. Close the parallel gap. Before the CAS overwrite, probe LOCKTAG_TUPLE with a dontWait acquire. If granted, no locker is queued: proceed and release the tag on both TM_Ok exits. If refused, a locker or slow-path writer is already queued on the tag; drop t_writer, the page content lock, and the buffer, then block on LOCKTAG_TUPLE with wait=true so this writer queues FIFO behind the existing waiter. Once acquired (the waiter ahead has been served) fall through to the exclusive slow path via goto, carrying the held tuple lock in the outer have_tuple_lock so the slow path's !have_tuple_lock-guarded acquire sites do not double-lock and RECNO_RELEASE_TUPLOCK frees it on exit. The probe must be dontWait: a blocking heavyweight acquire while t_writer is held would deadlock; the blocking acquire happens only after t_writer and the page lock are released, which is the ordering the slow path already uses in production. This unifies the two serialization domains for a tid -- lockers, slow-path writers, and fast-path writers -- onto the single LOCKTAG_TUPLE FIFO, matching heap_update, which funnels all contending writers through LockTuple(tuple). Uncontended cost is one dontWait LockTuple acquire + release per CAS UPDATE: measured at ~1.9% single-client tps on a warmed hot row (9505 -> 9322 tps, autovacuum/fsync/full_page_writes off). Tuple locks are not fastpath-eligible, so this is a real lock-partition LWLock cycle; bounded and modest against the alternative of 20s+ starvation stalls. New isolation spec recno-cas-locker-starvation asserts CAS writers block behind a FOR NO KEY UPDATE locker. The writer-vs-writer lapping vector is not expressible in isolationtester (it needs a continuous autocommit storm); that is covered by an out-of-tree deterministic teeth repro which shows LAP_DELTA=0 with the fix and thousands of laps without it.
RelUndoReserve had a Tier 1 lock-free path that advanced the head page's pd_lower via a 32-bit compare-and-swap while holding no buffer content lock, then fell back to a lock-based path that advanced the same field with a plain non-atomic read-add-store under the buffer's exclusive lock. A buffer content lock and a lock-free atomic do not mutually exclude. On a hot head page, the non-atomic store on the lock-based path could straddle a concurrent CAS on the lock-free path, so two reservers could be handed overlapping byte ranges. The overlapping UNDO records then corrupt each other's before-images; on rollback the main table is restored from garbage and the global SUM invariant breaks (observed as the 99999997796 divergence under a 64-client transfer storm). Drop the lock-free tier entirely. Every remaining reserve path advances pd_lower only under the buffer's exclusive content lock, so they are mutually exclusive by construction. The per-backend head page cache is retained as the fast path; it still skips the metapage lock but performs the pd_lower advance under the buffer lock.
The slow UPDATE path runs the committed-update detector once at STAGE 2, before waiting on any in-progress writer. That check races: while the head writer W is still in progress, the detector declines because W's undo record is not yet committed, so control falls through to the writer-wait stage. The buffer exclusive lock does not block W's commit, since W clears its in-progress sLog marker through the LRLock domain and the on-page value is already W's (RECNO updates in place). If W commits in the window between STAGE 2's read and the writer-wait probe, the wait stage sees no in-progress writer and would apply our update over W's now-committed value, silently losing W's update. Re-run the detector after the wait stage resolves, as the last action before the update proper. Every path that reaches the update passes this point with the buffer exclusively locked, so a committer that landed anywhere upstream is caught and the loser is bounced to EvalPlanQual (TM_Updated). EPQ reconcile dedup makes this converge exactly like STAGE 2: bounce once per distinct committed marker; a repeat of the identical (verptr, xid) marker already reconciled falls through and applies. This gives READ COMMITTED and REPEATABLE READ the same lost-update semantics as heap.
A committed RECNO CAS fast-path UPDATE emitted two WAL records: the main-fork page byte-diff (RM_RECNO_ID / XLOG_RECNO_CAS_UPDATE) and the undo before-image on the relation's UNDO fork (RM_RELUNDO_ID / XLOG_RELUNDO_INSERT). That 2:1 insert amplification is the dominant write-path cost delta versus HEAP on hot-row OLTP. RelUndoReserve returns the undo buffer still exclusively locked and pinned, and for a fresh page it also leaves the metapage locked. Both stay held through the CAS critical section, so a single XLogInsert can legally register buffers from both forks. Split RelUndoFinish into RelUndoStage (page mutation, no WAL) plus a thin wrapper that preserves the standalone-record behavior for non-folded callers. Add XLOG_RECNO_CAS_UPDATE_UNDO (0xE0) carrying the redo byte-diff (block 0, main fork) plus the staged undo record (block 1, UNDO fork, WILL_INIT on a new page) and the metapage FPI (block 2, new page only). New-page replay keys off the is_new_page field rather than an ORed init-page opcode, since the RECNO rmgr consumes the whole upper info nibble. A K-UPDATE hot-row workload of separate autocommit txns now emits K combined records instead of 2K. An intra-txn repeat UPDATE folds only the first write; subsequent overwrites of the uncommitted self-write correctly divert to the exclusive slow path. Verified: recno regression, 17 isolation specs, TAP 061/063/065/067, crash-recovery smoke, rollback before-image restore, and pg_waldump insert-count sanity (K=10 -> 10 folded records, 0 standalone undo).
Split the single per-relation UNDO append point into RELUNDO_NUM_HEADS=8
independent head/tail chains, sized to match the common WAL's
NUM_XLOGINSERT_LOCKS. A backend selects its slot with MyProcNumber %
RELUNDO_NUM_HEADS, so concurrent writers no longer serialize on a single
hot UNDO tail page (the residual BufferShareExclusive wall the FOLD
combined-record change exposed).
Striping is transparent to the two consumers that key off physical
coordinates: per-txn rollback threads through urec_prevundorec (a
RelUndoRecPtr) and the reader keys off (counter, blkno, offset); neither
cares which slot a record lands on. free_blkno and the generation counter
stay single/shared, both protected by the metapage exclusive lock.
Slot-aware surfaces: RelUndoReserve / relundo_allocate_page select the
per-slot chain; RelUndoDiscard drives eight per-slot discard/splice passes
and only truncates the fork when every slot's chain is empty; the discard
WAL record carries the slot; relundo_redo_{init,discard,truncate} and the
end-of-recovery fork scan walk all eight chains. Bumps
RELUNDO_METAPAGE_VERSION to 3 (head[8]/tail[8] arrays).
The autoconf build hardcodes USE_RECNO = 1 (src/Makefile.global.in), so RECNO is always compiled on the undo branch. The meson build instead treated recno as an auto feature and only set USE_RECNO when the option was not disabled. CI's SanityCheck job runs meson setup with --auto-features=disabled, which turned recno off. guc_parameters.dat and pg_proc.dat reference recno symbols (recno_tableam_handler, recno_use_hlc, and the other recno GUCs) without any USE_RECNO guard, and rmgrlist.h registers RM_RECNO_ID / recno_desc / recno_identify under #ifdef USE_RECNO. With the feature off the recno sources still compiled but RM_RECNO_ID was undeclared, and even past that the generated GUC table and fmgr table failed to link against the missing recno symbols. Match the autoconf behavior: always declare the recno dependency, always set USE_RECNO, and always compile recnodesc.c. Verified that a build mirroring the SanityCheck config (--auto-features=disabled) now compiles and links, and that the normal feature-enabled config is unchanged.
github-actions
Bot
force-pushed
the
master
branch
from
c7e7712 to
ab6faa8
Compare
…uild) MSVC rejects the GCC __attribute__((aligned(8))) spelling on the exposed sparsemap struct with error C2143, which cascades into recno_dirtymap.c failing to compile on the Windows Visual Studio matrix jobs. Switch the struct to PostgreSQL's portable pg_attribute_aligned() macro, which expands to __attribute__((aligned)) on GCC/Clang and __declspec(align()) on MSVC, and fall back to natural alignment where neither is defined (already 8 on LP64 for two size_t plus a pointer). The vendored sparsemap.c body is kept verbatim, so its __attribute__((aligned(1))) unaligned-access hints are neutralized under MSVC with a scoped #define that affects only the vendored code that follows. All real loads and stores go through memcpy, and every Windows target handles unaligned scalar access in hardware, so the hint is a no-op there.
The RECNO table-AM commit had changed 001_replan_regress.pl to reference src/test/regress/integration_schedule, a file that was never added to the tree. pg_regress bailed out (exit 512) because the schedule did not exist, turning CI red. Restore the reference to parallel_schedule, matching origin/master.
Both source files are listed in src/backend/access/undo/meson.build but were missing from the autoconf Makefile OBJS list, so Meson builds compiled them while Autoconf did not. slog.c references SLogFlatHash* symbols defined in slog_flathash.c, and xlogrecovery.c calls PerformRelUndoRecovery() defined in relundo_recovery.c, so the Autoconf build failed with undefined symbols. Add both objects in the position matching the meson.build ordering.
github-actions
Bot
force-pushed
the
master
branch
2 times, most recently
from
96ac82d to
ea640f1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.