The earlier tests improved read I/O and removed standby acknowledgment waiting, but the primary still waited for local WAL synchronization. Separate, later durable-regime traces located substantial delay in guest-OS write admission. This post returns to the diagnostic where I stopped waiting for commit durability to see what remained.
I used session-level synchronous_commit=off for the same 512-worker, 15+45-minute workload. This allows a successful commit response before WAL is durable: a crash can lose recently acknowledged transactions. Keep in mind that I do this in a lab to investigate PostgreSQL bottlenecks. It deviates from the initial comparison with HorizonDB's built-in high-availability and from the original durable-HA configuration. Still, let's see how far we can increase the throughput in this lab.
WAL, fsync and full_page_writes remained enabled. Both streaming replicas were retained. Automatic Patroni failover was separately paused for the diagnostic.
Verify the mode, then measure
The override was supplied through the workload's PGOPTIONS. Before launch, a test Pgtcl connection using that environment executed:
SHOW synchronous_commit; Its recorded check was:
2026-11-08T08:08:47.692827+00:00 BENCH_COMMIT_MODE_CHECK adapter=Pgtcl effective=off This records the adapter test connection's setting. The workload was launched with the same connection environment, while the server default remained on.
The result was:
2026-11-08T09:08:53.198898+00:00 Vuser 1:TEST RESULT : System achieved 66096 NOPM from 152007 PostgreSQL TPM 93.92% of sampled active-client observations were at LWLock:BufferContent. The question had moved from when WAL becomes durable to which backends needed conflicting access to an in-memory page. I wanted to identify that page before changing shared_buffers.
The page identification below comes from a later synchronous_commit=on follow-up.
What BufferContent protects
BufferContent is a wait for a lightweight lock protecting the contents of a cached relation page, normally 8 KiB. Shared holders can coexist. Exclusive access conflicts with other holders, even when the page is already in memory.
BufferMapping instead concerns associating a block with a buffer slot. BufferIO concerns buffer I/O completion.
For Oracle readers, LWLocks are closer to latches and mutexes than transactional enqueues: they protect shared-memory structures, not business rows until commit. PostgreSQL's heavyweight lock manager is the closer comparison for transaction and object locks. pg_locks and pg_blocking_pids() do not expose a content-LWLock holder.
A buffer pin is different again: it prevents recycling the slot, but does not grant exclusive access to its contents.
Oracle's buffer busy waits includes P1/P2/P3 parameters identifying file, block, and block class. PostgreSQL's pg_stat_activity exposes neither that page identity nor per-wait elapsed time for BufferContent. query_start times the statement, not its current wait. Oracle active-session history is sampled too. Samples and timed waits are different measurements.
Observe active backends
The collector grouped pg_stat_activity by backend type, state, and wait event. This query shows the active-client part:
SELECT wait_event_type, wait_event, count(*) AS sessions FROM pg_stat_activity WHERE backend_type = 'client backend' AND state = 'active' GROUP BY wait_event_type, wait_event ORDER BY sessions DESC; active describes statement execution status and includes statements currently waiting.
These are retained samples from the synchronous_commit=off measured window, November 8, 2026, 08:23:51-09:08:53 UTC:
| Observer timestamp, UTC | Phase | Active clients | At BufferContent | Percentage |
|---|---|---|---|---|
| 08:24:43.424849 | measurement | 513 | 476 | 92.79 |
| 08:34:43.913026 | measurement | 512 | 483 | 94.34 |
| 08:44:44.413830 | measurement | 512 | 491 | 95.90 |
| 08:54:44.923233 | measurement | 513 | 484 | 94.35 |
| 09:08:45.608882 | measurement | 513 | 470 | 91.62 |
The collector included all active client backends, including its own connection, without database/user filtering. These counts therefore differ from the 512 workload workers.
Across all 45 samples taken one minute apart inside the measured interval, there were 23,063 active-client observations:
| Wait | Observations | Share |
|---|---|---|
LWLock:BufferContent | 21,660 | 93.92% |
IO:DataFileRead | 981 | 4.25% |
| No reported wait | 376 | 1.63% |
Client:ClientRead | 25 | 0.11% |
IO:DataFileFlush | 10 | 0.04% |
IO:DataFileWrite | 7 | 0.03% |
IO:DataFileExtend | 2 | 0.01% |
IPC:BufferIo | 2 | 0.01% |
None of the 45 snapshots contained LWLock:WALWrite or IPC:SyncRep. The 93.92% is 21660 / 23063, the share of sampled active-client observations.
Background persistence continued. In-window WAL counters showed approximately 194.46 MiB/s generated and 220.76 syncs/s. WAL must still precede durable data-page writes.
A buffer is not necessarily a table's data page
Shared buffers can hold more than application table rows:
| Page contents | Examples |
|---|---|
| Table heap | Row versions, headers and item identifiers |
| Index | B-tree, GIN, GiST, SP-GiST, hash or BRIN pages, including metadata |
| TOAST relation | Out-of-line values and their index |
| System catalogue | Catalogue heap and index pages |
| Materialized view | Stored rows and indexes |
| Sequence | The sequence relation's state page |
| Relation metadata | Free-space-map and visibility-map pages |
| Unlogged relation | Working pages and initialization-fork pages |
Partitions store their own pages. The partitioned parent does not store heap rows. Temporary relations use backend-local buffers. WAL buffers and SLRU caches, such as transaction-status caches, are separate structures with different synchronization.
Relation forks and page identity
A fork is a physical part of a relation with a specific purpose:
| Number | Fork | File suffix | Contents |
|---|---|---|---|
| 0 | main | none | Heap, index or other primary relation storage |
| 1 | fsm | _fsm | Free-space map |
| 2 | vm | _vm | All-visible and all-frozen information for heap pages |
| 3 | init | _init | Unlogged relation's initialization state after an unclean shutdown |
Not every relation has every fork. Indexes have no visibility map. See the file-layout documentation.
PostgreSQL's BufferTag identifies a page by tablespace, database, filenode, fork, and block number. All five matter: main-fork block 0 and VM-fork block 0 are different pages. Catalog OIDs and physical filenodes are distinct identifiers. A rewrite can change the filenode while preserving the OID.
Separate follow-up: identifying pages with synchronous_commit=on
For page-level diagnosis, I used a separate 512-worker pilot with synchronous_commit=on and normal HA. BufferContent was still visible in the capture-trigger samples at 5.47-18.75%, enough to trace page contention under ordinary durable commits.
1️⃣ Capture the page tag at the lock event. I traced selected PostgreSQL processes by PID with Linux perf. Uprobes are tracing points in userspace machine code. I checked their instruction locations and registers against the installed PostgreSQL executable.
At acquisition, semaphore-wait and release points, the probes recorded the lock address and the full page tag from its buffer descriptor. Pairing required the same page identity at the beginning and end of an interval.
Eight nominal three-second recordings sampled selected backends and background processes. This is a selected-field excerpt of the decoded, aggregated wait output:
{ "tag": [1663, 57465, 57491, 2, 10215], "waited_loop_count": 78, "loop_wait_ns": 2357679631 } In order, that tag means tablespace 1663, database 57465 (tpcc), filenode 57491, fork 2, block 10215. Fork 2 was recorded by the probe.
2️⃣ Resolve the filenode through the catalogs. Connected to tpcc, the collector mapped captured filenodes through pg_class. This reduced lookup resolves the same identity:
SELECT n.nspname AS schema, c.relname AS relation, c.relkind, am.amname, pg_relation_filenode(c.oid) AS filenode FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace LEFT JOIN pg_am am ON am.oid = c.relam WHERE c.oid = pg_filenode_relation(1663::oid, 57491::oid); The retained catalog row, reformatted from the collector's JSON:
schema | relation | relkind | amname | filenode --------+----------+---------+--------+--------- public | stock | r | heap | 57491 The relation is public.stock. Fork 2 identifies its visibility map, so this is stock VM block 10215, not stock heap block 10215.
3️⃣ Keep the block identity in the results. These were the leading measured pages by cumulative completed semaphore-loop elapsed time. All have the same tablespace/database/filenode and VM fork:
| Stock VM block | Completed loops | Cumulative loop ms |
|---|---|---|
| 10215 | 78 | 2,357.680 |
| 10031 | 66 | 2,206.913 |
| 10032 | 63 | 2,076.327 |
| 10033 | 55 | 1,984.848 |
Stock's VM fork was 94,912,512 bytes, or 11,586 8-KiB pages, consistent with those block numbers.
Selected processes contributed these completed loops. The elapsed intervals include semaphore work, rescheduling, and probe overhead. Several loops can belong to one acquisition.
Captured holder operations on these pages included heap_update. The longer waiter intervals remain only partially attributed.
The identification chain in this normal-on pilot is event-time tag -> catalogue lookup -> stock VM block identity. pg_buffercache shows current cache contents, where slots can be reused and the view is not one atomic snapshot.
Why visibility-map pages can contend
The visibility map stores two bits per heap page: all-visible and all-frozen, used by index-only scans and vacuum. With an 8-KiB block and 24-byte page header:
(8192 - 24) bytes * 8 bits / 2 bits per heap page = 32672 heap pages per VM page = 255.25 MiB of heap address space per VM page Vacuum sets these bits. Data modifications can clear them. Sessions updating unrelated rows on different heap pages can therefore need exclusive access to the same VM buffer. One metadata page can serialize updates across many heap pages.
A related PostgreSQL change
A July 2026 correctness fix, authored by Melanie Plageman and Andres Freund and included in PostgreSQL 17.11, registers modified VM blocks in heap WAL records. Previously, incremental backups could omit those changes, and clearing VM bits did not produce the full-page images needed for torn-page protection.
Correct logging requires holding the VM lock through heap/VM modification and WAL insertion, extending the critical section.
In the performance discussion, Melanie tested 32 clients clearing bits for different heap pages covered by one VM page, while a vacuum loop set bits again. She observed no degradation with mostly default settings, but 10-15% lower TPS and more BufferContent waits with synchronous_commit=off. Vacuum timing varied between runs.
Her test illustrates how a longer VM critical section becomes visible as BufferContent contention when commit waits are removed.
Where this test ends
Here is the full-run record, linked back to the decisions behind it:
| Configuration | NOPM | Leading active-client observation |
|---|---|---|
| Premium SSD v2, normal HA | 50,648 | DataFileRead, 79.6% |
| Ultra primary, normal HA | 55,784 | WALWrite, 83.18% |
Ultra primary, commit local | 54,556 | WALWrite, 87.29% |
Ultra primary, commit off | 66,096 | BufferContent, 93.92% |
| Full normal-HA repeat | 54,163 | WALWrite, 83.17% |
The off result is 1.305x the original baseline, a 30.5% increase. The primary used the highest nonbursting Ultra budget documented for that tested VM configuration, while commits no longer waited for local or remote durability.
That result combined higher storage provisioning, weaker commit durability, and temporarily paused automatic failover. The runs reused the dataset, with cache contents, dirty pages, checkpoint phase, and vacuum state evolving between tests.
Mean primary CPU utilization in the off window was 18.1%. Low aggregate CPU utilization can coexist with serialization around a few lock holders. Page-level attribution was collected in the normal-on follow-up and identified stock visibility-map pages.
The progression is from a device-read constraint, through a check of remote commit waiting, into OS writeback admission, and finally to shared-page coordination. Removing replica acknowledgment waiting left WALWrite dominant. Each step narrowed where to investigate next.
At the last boundary, more disk IOPS does not make a content lock grant conflicting access simultaneously or split one shared metadata page into independently locked pieces. More workers can instead add contention. The remaining question concerns the amount of shared work, the duration of critical sections, and how access is coordinated.
Keep durability, move the work
The storage and commit-mode experiments took this workload from 50,648 to 66,096 NOPM, a 30.5% increase. The highest result allowed recent transaction loss after a crash. The PostgreSQL primary still had to coordinate access to shared pages.
Bypassing commit durability was a diagnostic step. The production objective is to preserve durable commits while reducing the work and coordination concentrated on the writer.
One approach is disaggregated PostgreSQL storage. Dedicated services handle WAL durability, replication, and data-page materialization. Moving that persistence work can reduce compute-side writeback and buffer maintenance. That's the HorizonDB architecture, which achieved 8x higher throughput for this workload, with better resilience to failures.
SQL execution, transaction semantics, and page-content synchronization remain in PostgreSQL compute. The writer has more headroom for execution and scale-up, while storage services can scale independently across machines. The important change is where the work runs, with the durability guarantee preserved.
HorizonDB retains PostgreSQL’s foreground page synchronization and visibility-map maintenance, while offloading page persistence, checkpoint I/O, full-page-write protection, and WAL distribution. This can ease pressure on those critical sections and give the writer more capacity for transactions. That may explain why the workload achieves higher throughput than this PostgreSQL configuration can support.
If you have resolved similar WAL or BufferContent contention on PostgreSQL, I'd be interested in the configuration, schema, or implementation change, and the measurements that showed improvement, while retaining durable commits.


