What Really Happens When Your App Queries a Database: CPU, Memory, the OS, Async, and Locks Explained

A ground-up mental model of processes, threads, context switches, the Node.js event loop, PostgreSQL internals, and why “waiting” is the hidden theme of almost every performance problem.

You write await prisma.user.findMany(). A few milliseconds later, rows appear. It feels like one step. In reality that line triggers a chain of events across your application process, the operating system, the network, the CPU, memory, and a database server with its own processes, caches, and locks. When something is slow, the cause is almost always hiding in one of those hand-offs.

This article follows a single query on its journey and explains each layer as it appears. You will learn:

• What the CPU, memory, and the operating system (OS) each do, and how they cooperate.
• The difference between processes, threads, and async tasks, and what a context switch costs.
• How Node.js can handle thousands of requests with one JavaScript thread, and how to accidentally break that.
• How PostgreSQL uses processes, shared memory, and locks, and what “waiting on a lock” physically means.
• How to tell whether a slowdown is caused by CPU, memory, disk, locks, or the application itself.

The central idea is simple: almost every system spends most of its time waiting. Performance work is mostly about finding out who is waiting for what, and why.

Scope. Examples use Node.js and PostgreSQL on Linux, because they are common in backend work. The core ideas (scheduling, memory, blocking, locking) apply to almost every language, database, and operating system, though details and numbers vary. Every number here is a rough, typical figure to build intuition, not a benchmark.

Quick glossary

Skim this table first and return to it whenever a term is unfamiliar. Experienced readers can skip it.

1. The big picture: who talks to whom

Here is the cast of characters on a typical backend deployment. The diagram shows an application server and a database server, each running an OS kernel on top of its own CPU, memory, and devices.

Read it as layers. Your code never touches hardware directly. It asks the kernel, through system calls, to send bytes over the network. The kernel decides when each program gets CPU time, hands out memory, and moves data to and from the network card and disks. On the database host, the same kernel services the PostgreSQL processes in the same way.

Why speed differences drive everything

The resources in that diagram differ enormously in speed. This table gives typical latencies and, in the last column, a human-scale analogy: if one nanosecond were one second, how long would each operation feel? The goal is to show relative magnitudes, which explain why systems are designed around avoiding the slow things.

The lesson: reaching RAM is thousands of times faster than reaching disk or another machine. So caches exist at every level (CPU caches, the OS page cache, the database buffer cache), and so do techniques for doing something useful instead of sitting idle while a slow operation finishes. Async programming, thread scheduling, and connection pooling are all variations on that theme.

2. The CPU: what “executing” really means

A CPU core executes one stream of instructions at a time. A modern CPU has several cores, so several streams can truly run at the same instant. This is parallelism, and the number of cores is its hard upper limit.

Hardware threads (SMT / Hyper-Threading)

Many CPUs expose each physical core as two (or occasionally more) hardware threads. This is called SMT (simultaneous multithreading), and Intel's version is Hyper-Threading. The two hardware threads share one core's execution units, so when one stalls (for example waiting for data from RAM), the other can use the idle units. The gain is typically a modest increase in throughput, not a doubling. The OS sees each hardware thread as a separate logical CPU, which is why a 4-core machine often shows 8 CPUs.

A CPU core typically supports 1 or 2 hardware threads, depending on its architecture.

CPU-bound versus I/O-bound work

The single most useful classification in performance work:

• CPU-bound work keeps the CPU busy computing: image processing, compression, hashing passwords, parsing a giant JSON document, running a complex SQL aggregation. Adding more concurrency does not help once all cores are busy, because the limit is raw computation.
• I/O-bound work spends most of its time waiting for something external: a database reply, a disk read, an HTTP call. The CPU is mostly idle during those waits, so one core can juggle many such tasks.

Typical web APIs are I/O-bound: a request does a little computation, then waits for the database. That is exactly why a single Node.js thread can serve many users. It is also why the database host is often CPU-bound while the application host is nearly idle. Most real systems contain both kinds of work, and the right scaling tool differs for each.

3. Processes, threads, and the scheduler

Processes and threads

The OS gives every process its own private virtual address space, which is a view of memory that other processes cannot touch. This isolation is why one crashing program does not corrupt another. A process can contain one or more threads. Threads in the same process share its memory, which makes sharing data cheap but creates the risk of race conditions (two threads changing the same data at once).

The scheduler and thread states

There are almost always more runnable threads than CPU cores, so the kernel's scheduler decides which thread runs on which core and for how long. A thread is always in roughly one of three states:

• Running: executing on a core right now.
• Runnable (ready): could run immediately but is waiting for a free core. These threads form the run queue.
• Blocked (sleeping): cannot continue until an event occurs, such as network data arriving or a lock being released. A blocked thread uses no CPU time.

The scheduler gives each running thread a short time slice (on the order of milliseconds). If the thread does not finish or block by then, the kernel preempts it: pauses it, puts it back in the run queue, and runs another. This is how a few cores create the illusion that hundreds of programs run simultaneously.

What is a context switch?

When the CPU stops running one thread and starts another, the kernel performs a context switch:

1. Save the outgoing thread's CPU state: registers, the program counter (which instruction comes next), and the stack pointer.
2. Choose the next thread.
3. Restore that thread's saved state.
4. If the new thread belongs to a different process, also switch the memory mapping (the page tables).

Context switches are essential, but they are not free:

• Direct cost: the switching work itself, roughly on the order of microseconds.
• Indirect cost, usually larger: the CPU's caches and address-translation buffers (the TLB) are warmed up for the old thread's data. After a switch, the new thread starts with a partly cold cache and runs slower until its own data is loaded.

So a few context switches per second are harmless, but tens of thousands per second per core mean the CPU spends a meaningful share of its time on bookkeeping and cache refills instead of useful work. Two flavors are worth knowing. A voluntary switch happens when a thread blocks itself (waiting for I/O or a lock); that is normal for I/O-heavy programs. An involuntary switch happens when the scheduler preempts a thread that still wanted to run; a high rate of these means too many threads are competing for too few cores.

Processes, threads, and async tasks compared

Three different units of “things happening at once” are often mixed up. The table compares them across the dimensions that matter: memory sharing, switching cost, and isolation.

The row to focus on is the last one: async tasks are cheap precisely because the OS does not know they exist. They are the runtime's own bookkeeping, which is why one Node.js process can keep thousands of requests in flight. The price is that they cannot use more than one core, and that they must cooperate (section 6).

Signs of CPU Saturation

When more threads are ready to execute than there are available CPU cores, some threads must wait for CPU time. Unlike I/O-bound threads, these threads are not waiting for a disk, network, or other external operation; they are waiting for the CPU to become available.

Common signs of CPU contention include:

• A long run queue: Many threads are ready to execute but cannot get CPU time immediately.
• High context-switch rates: The operating system frequently switches between threads to share CPU time.
• Increasing latency under load: Tasks take longer to complete as more threads compete for CPU resources.

The load average reported by tools such as top provides another useful indicator. On Linux, it reflects the average number of tasks that are running, waiting to run, or stuck in uninterruptible sleep, typically due to I/O operations.

If the load average consistently exceeds the number of logical CPUs, it may indicate CPU contention. However, it can also indicate significant I/O waiting, so load average alone cannot tell you which resource is the bottleneck. Section 9 explains how to distinguish CPU saturation from I/O bottlenecks.

4. Memory: virtual, shared, and finite

Virtual memory

Programs do not use physical RAM addresses directly. The OS gives each process virtual memory, divided into fixed-size pages (commonly 4 KB), and maps virtual pages to physical RAM using page tables. This provides isolation (a process only sees its own mappings), flexibility (memory need not be contiguous), and the ability to share pages deliberately.

A few related terms:

• Page fault: a program touches a virtual page that is not currently mapped to RAM. A minor fault is cheap (the data is already in memory and just needs mapping). A major fault requires reading from disk and is thousands of times slower.
• Page cache: spare RAM that the OS uses to cache recently read file data. This is why a database that fits in RAM runs far faster than one that has to hit disk.
• Swap: when RAM runs short, the OS can move rarely used pages to disk. A server that actively swaps is typically in serious trouble, because every swapped page costs a disk-speed delay.
• OOM killer: if memory is truly exhausted, Linux may kill a process to free memory. If it kills a PostgreSQL backend, the database treats it as a crash and restarts all backends in crash recovery, dropping every connection. Memory exhaustion is therefore never a gentle failure.

Private versus shared memory, and fork

Processes are isolated by default, but the OS can map the same physical memory into several processes. This is shared memory, and it is how PostgreSQL's separate processes coordinate.

When PostgreSQL accepts a new client connection, its main process, called the postmaster, creates a separate backend process using fork().

On Linux, fork() uses a technique called copy-on-write (CoW). Instead of immediately copying all the parent's memory, the child process initially shares the same memory pages. A page is copied only when the parent or child modifies it. This reduces the memory and processing overhead of creating a new process.

However, process creation still involves overhead, including process initialization and resource allocation. Creating a new PostgreSQL connection is therefore more expensive than reusing an existing connection from a connection pool, which avoids repeatedly creating backend processes.

How PostgreSQL Uses Memory

To understand PostgreSQL's memory usage, you first need to understand the relationship between a database connection and a backend process.

When a client, such as a Node.js application, connects to PostgreSQL, PostgreSQL typically creates a dedicated backend process for that connection. The backend is the server-side process that handles the client's SQL queries, manages its transactions, and returns results.

For example, if your application opens 20 connections, PostgreSQL will typically have 20 backend processes serving those connections. Even when some connections are idle, their backend processes still consume memory and other system resources.

PostgreSQL memory is divided into two main categories: shared memory, which multiple backend processes can access, and private memory, which belongs to an individual backend or a specific query operation.

The following table explains the main memory areas and why they matter when configuring connection pools and query settings.

Why this matters for connection pooling

Consider a Node.js application with 100 database connections. Each connection typically corresponds to a PostgreSQL backend process, so PostgreSQL must maintain resources for 100 backends.

Now suppose many of those connections execute complex queries simultaneously. Each query might perform several memory-intensive operations, such as sorting and hashing. The resulting memory consumption can grow much faster than the connection count alone suggests.

This is why connection limits and per-query memory settings must be configured together. A generous work_mem setting may be safe with a small number of concurrent queries but dangerous when hundreds of queries execute simultaneously.

Two additional details are important:

• PostgreSQL and the operating system both cache data. PostgreSQL uses shared_buffers, while Linux also maintains a filesystem page cache. The same database data can exist in both caches, which is normal. However, the memory implications depend on the database's I/O behavior and configuration.
• work_mem is not a per-connection memory limit. It applies to individual query operations, and one query can have several operations using it. Some operations may also use parallel workers, increasing total memory consumption further.

The practical lesson is that adding database connections does not simply increase the number of clients PostgreSQL can serve. It also increases backend overhead and can increase peak query memory usage. A connection pool helps control concurrency, protect database resources, and prevent memory exhaustion under load.

Memory in your Node.js process

Node.js manages memory with a garbage collector (GC) inside the V8 engine. Memory leaks (data kept reachable forever) grow the heap until the process slows down or crashes. Large garbage-collection pauses also stall the single JavaScript thread, which delays every request in that process. Keeping per-request memory small (for example by paging results instead of loading a million rows) protects both memory and latency.

5. I/O and waiting: where the time actually goes

Most of a backend's time is spent waiting for I/O: network, disk, or another service. Understanding what happens during a wait explains why blocking is cheap for the CPU but expensive for everything else.

What happens when a thread waits for data

Suppose a thread calls read on a network socket and no data has arrived yet:

1. The thread makes a system call, switching from user mode to kernel mode.
2. The kernel sees there is no data. It marks the thread blocked and removes it from the run queue.
3. The scheduler runs another runnable thread on that core, via a context switch. The CPU keeps doing useful work.
4. Later the network card receives the data and signals the CPU with an interrupt. The kernel copies the data into the socket's buffer and marks the sleeping thread runnable.
5. When the scheduler next picks it, the thread returns from read with its data.

While blocked, the thread consumed no CPU. That is why a machine can show low CPU usage while requests are slow: threads are asleep, waiting on something else.

Waiting is free for the CPU, but not for resources

Blocking costs the CPU nothing, but the waiting task usually still holds something valuable. This table summarizes the most common waits.

The moral of the table: waiting converts a CPU problem into a resource-holding problem. A request waiting on a slow query holds a connection. If enough requests do that, the pool is exhausted, the queue grows, and the system fails even though CPU graphs look healthy. This is the mechanism behind the connection-pool failures discussed in the companion article on pools and timeouts.

The Disk Path and Durability

When a database modifies data, the changes are initially made in memory. To ensure those changes survive a crash or power failure, PostgreSQL uses a mechanism called write-ahead logging (WAL).

Before a transaction is considered durably committed, PostgreSQL must write its WAL records to stable storage. It typically uses fsync or an equivalent mechanism to ensure the operating system has flushed the required data to the storage device.

With the default synchronous_commit = on, PostgreSQL normally waits for the relevant WAL records to be flushed before confirming the commit to the client. This means commit latency depends partly on how quickly the storage system can complete the flush. When storage is slow, this wait can become a bottleneck.

PostgreSQL's synchronous_commit setting can reduce this wait by allowing a commit to be acknowledged before its WAL records are durably flushed. This improves performance but creates a risk: if the server crashes during that window, recently acknowledged transactions may be lost. This does not, by itself, imply database corruption.

The key takeaway is that durable commits require coordination with persistent storage, which is generally much slower than accessing RAM. This is why storage latency, flush performance, and commit settings can significantly affect database throughput and response times.

6. Concurrency and async in Node.js

Concurrency is not parallelism

Picture a cook with six pots. The cook starts the rice, and while it simmers, chops vegetables, then checks the rice again. One person, many dishes progressing: that is concurrency. Now hire five more cooks, each with their own stove, working at the same instant: that is parallelism. Concurrency is about structure (managing many tasks that overlap in time). Parallelism is about simultaneous execution (which needs several cores).

Node.js is the one-cook kitchen. It runs your JavaScript on a single thread, yet it can keep thousands of requests in progress, because most of each request is spent waiting, and waiting does not need a cook.

The Event Loop

When your JavaScript code awaits a database query, it might look as though execution stops until the database responds. In reality, await suspends the current async function, not the JavaScript thread. This allows Node.js to handle other work while the query is in progress.

Here is what happens step by step:

1. Send the query: Your JavaScript calls the database driver, which sends the SQL query through a network socket. Because the socket uses non-blocking I/O, Node.js does not need to wait for the database response before continuing other work.

2. Register the I/O operation: Node.js uses its I/O engine, libuv, to monitor the socket through operating-system mechanisms such as epoll on Linux, kqueue on macOS, or I/O Completion Ports (IOCP) on Windows.

3. Suspend the async function: When execution reaches await, the function pauses until the promise settles. The JavaScript thread can return to the event loop and process other ready callbacks, timers, or incoming requests.

4. Resume when the result arrives: When the database responds, the operating system signals that the socket has data available. Node.js processes the response, and the database driver's promise is settled. The event loop and microtask scheduling then allow the suspended function to resume after the await with the query result.

This is one of the key reasons Node.js can handle many concurrent I/O operations without creating a separate JavaScript thread for every request.

However, asynchronous I/O does not make CPU-intensive JavaScript non-blocking. If your code performs a long-running calculation on the main JavaScript thread, it can still prevent the event loop from processing other work promptly.

For a step-by-step explanation of the call stack, event loop, and asynchronous execution, read Why Your JavaScript Runs in the Wrong Order.

Three requests interleave on one thread like this:

The diagram shows the key insight. The JS thread is busy only for the short “handle” and “reply” slices. The long waits happen outside it, in the OS, the network, and the database. That is why a single Node.js process can serve many I/O-bound requests, and it is also why that process uses almost no CPU while its requests are slow.

Sequential versus concurrent awaits

A runnable TypeScript script demonstrates the performance difference between sequential await calls and concurrent Promise.all execution using mocked asynchronous operations with 1-second delays.

// Sequential: roughly 3 seconds const user = await getUser(id); const orders = await getOrders(id); const invoices = await getInvoices(id); // Concurrent: roughly 1 second const [user, orders, invoices] = await Promise.all([ getUser(id), getOrders(id), getInvoices(id), ]); 

The sequential version waits for each operation to finish before starting the next, so the total time is approximately the sum of all three durations. In contrast, Promise.all starts all three operations immediately and waits for all of them to settle successfully, making the total time approximately that of the slowest operation.

However, concurrency consumes resources. If each query requires a separate database connection, three concurrent queries can occupy three connections at once. With a connection pool of five, two requests executing this pattern could consume all five connections if each request runs three queries concurrently.

Also, if any promise passed to Promise.all rejects, the combined promise rejects immediately, but the other operations continue running unless they are explicitly cancelled.

The takeaway: concurrency can significantly reduce latency, but in production systems, you must balance it against database connection limits, query load, and failure handling.

What Is Not Running on the JavaScript Thread?

Node.js uses more than the main JavaScript thread to handle certain types of work. One important example is libuv's thread pool, which has four threads by default. You can change its size using the UV_THREADPOOL_SIZE environment variable.

The thread pool handles operations that cannot be handled efficiently through non-blocking operating-system interfaces. Common examples include:

• File-system operations: Many asynchronous file-system APIs use the pool to perform disk operations.
• DNS lookups: dns.lookup() uses the operating system's name-resolution facilities, which typically involve the thread pool.
• Cryptographic operations: Functions such as crypto.pbkdf2() and crypto.scrypt() can use the pool for computationally intensive work.
• Compression: Asynchronous zlib operations can use the pool to perform compression and decompression.

However, network sockets, including database connections, do not use this thread pool to wait for responses. Node.js monitors network I/O through operating-system mechanisms and libuv's event loop instead.

An important edge case: Database connection timeouts

Consider an application that creates new PostgreSQL connections on demand. Before connecting, the database driver may need to resolve the database server's hostname using dns.lookup().

Now imagine that all four libuv worker threads are busy performing expensive password-hashing operations or file-system tasks. The DNS lookup may have to wait for an available worker. As a result, establishing a new database connection can take longer, even if PostgreSQL itself is healthy and responding normally.

This can produce a misleading symptom: a connection timeout caused by local thread-pool saturation rather than a database failure.

The distinction matters when troubleshooting production systems. If existing database connections work normally but new connections are delayed, investigate DNS resolution and libuv thread-pool contention alongside database health, network latency, and connection-pool configuration. Increasing UV_THREADPOOL_SIZE may help, but first verify that thread-pool saturation is actually the bottleneck.

Blocking the Event Loop

Node.js executes JavaScript callbacks on a single main thread. This makes it efficient for handling concurrent I/O, but it also creates an important limitation: while synchronous JavaScript is running, the event loop cannot execute other callbacks on that thread.

Consider this example:

app.get('/report', (req, res) => { const result = hugeArray.map(heavyTransform); // 800 ms of pure CPU res.json(result); }); 

Suppose hugeArray.map(heavyTransform) takes 800 milliseconds to complete. During that time, the main JavaScript thread is occupied with computation and cannot process other callbacks.

The impact extends beyond the request generating the report:

• Other requests are delayed: Incoming requests cannot have their JavaScript handlers executed until the thread becomes available.
• Database responses are delayed: Data may arrive through network sockets, but the application cannot process the corresponding callbacks promptly.
• Timers run late: A timer scheduled for five seconds is not guaranteed to execute at exactly five seconds. If the event loop is blocked, its callback must wait until JavaScript can run it.
• Health checks may fail: If the process cannot respond to health checks within the configured timeout, an orchestrator such as Kubernetes may restart it, even though the process itself has not crashed.

Common causes include large JSON.parse() or JSON.stringify() operations, computationally expensive regular expressions, synchronous cryptographic functions, and processing enormous arrays or result sets in a single operation.

Measuring event loop delay

One way to detect event loop blocking is to measure event loop delay: how late the event loop processes scheduled work compared with when it would normally be expected to run.

Node.js provides monitorEventLoopDelay() in the built-in node:perf_hooks module.

import { monitorEventLoopDelay } from 'node:perf_hooks'; const histogram = monitorEventLoopDelay({ resolution: 20 }); histogram.enable(); setInterval(() => { console.log( 'event loop delay p99 (ms):', histogram.percentile(99) / 1e6, ); histogram.reset(); }, 10_000); 

Here's how to interpret the important parts:

• resolution: 20 sets the approximate sampling interval to 20 milliseconds.
• percentile(99) reports the 99th-percentile delay recorded during the measurement window, indicating the delay below which 99% of recorded values fall.
• / 1e6 converts nanoseconds to milliseconds because the histogram reports its values in nanoseconds.
• reset() clears the recorded histogram so the next interval measures a fresh window.

A responsive service typically has low event loop delay, often in the single-digit milliseconds under light load. Sustained delays in the hundreds of milliseconds are a strong warning that the event loop is struggling, although acceptable thresholds depend on the workload and latency requirements.

How to prevent event loop blocking

The right solution depends on what is consuming the CPU or holding large amounts of data:

1. Use worker_threads for CPU-intensive JavaScript. Workers execute JavaScript on separate threads, allowing the main event loop to remain responsive.
2. Move heavy computation to a separate service. This is useful when workloads need independent scaling or process isolation.
3. Process data in chunks. Break large computations into smaller units and yield between chunks so other callbacks can run.
4. Stream instead of buffering. Process large inputs incrementally rather than loading and transforming the entire dataset in memory.
5. Push suitable aggregation into the database. Use SQL filtering, grouping, and aggregation to avoid transferring and processing unnecessary records in Node.js.

The key distinction is that asynchronous I/O allows Node.js to handle waiting efficiently, but it does not automatically make CPU-intensive JavaScript asynchronous. If a computation monopolizes the main thread, every other task that depends on that event loop can experience increased latency.

Which kind of work runs where?

This table summarizes the execution venues. Use it to choose a scaling tool: the wrong one (for example, more pool connections for a CPU-bound handler) does nothing.

To use several cores, you run several Node.js processes (via containers, a process manager, or the cluster module). Each process has its own pool, which is exactly why the per-process connection arithmetic matters.

Single-threaded does not mean race-free

JavaScript never runs two lines at once in the same process, but an await lets other requests run in the middle of your function. That creates classic check-then-act races:

const stock = await getStock(productId); // Other requests can run here. if (stock > 0) { await setStock(productId, stock - 1); // And here. } 

At first glance, this code looks correct: it checks whether stock is available before reducing the quantity. However, it contains a race condition because reading and updating the stock are two separate database operations.

Imagine that a product has only one item left and two customers try to buy it simultaneously:

• Request A reads the stock: 1.

• Request B reads the stock: 1.

• Both requests pass the stock > 0 check.

• Both requests update the stock based on their earlier reads.

Both customers may receive a successful purchase confirmation, even though only one item was available. This is known as overselling.

The problem is not that JavaScript executes two statements simultaneously on the same thread. Rather, each await allows other work to proceed while the current function is suspended. Another request can therefore execute its own database operations before the first request finishes.

Where should the fix go?

Because the stock is stored in the database and must remain correct across concurrent requests, the database should enforce the operation's correctness.

One solution is to combine the check and update into a single atomic SQL statement:

UPDATE products SET stock = stock - 1 WHERE id = $1 AND stock > 0 RETURNING stock; 

The database evaluates the condition and updates the matching row as one statement. Concurrent updates to the same row are coordinated by PostgreSQL, preventing both requests from decrementing the same unit of stock successfully.

Your application should check the result:

• One row returned: The stock was successfully decremented.

• No rows returned: The product does not exist or has no stock available.

Other approaches include database constraints and explicit locking within a transaction, depending on the business rules and complexity of the operation. See Section 8 for more about locks.

The same principle applies to in-memory JavaScript state. Variables can become logically inconsistent when multiple asynchronous operations interleave across await points. In addition, in-memory variables are not automatically shared between separate Node.js processes or application replicas.

The key takeaway is that when correctness depends on shared state, enforce the required invariants at the system that owns that state. For inventory, that usually means using atomic database operations, appropriate constraints, and transactions where necessary, rather than relying on application-level checks alone.

Below is a clearer, more cohesive version of Sections 7–12. I've preserved the transaction-locking image, code examples, SQL queries, and diagnostic tables while clarifying the relationships between PostgreSQL processes, resource contention, and application latency.

7. Inside PostgreSQL: What Happens When a Query Arrives?

The PostgreSQL process model

Postgre...