🔮 The 2027+ Google SRE NALSD Playbook: 10 Operational Frontier Simulations

"If you think System Design is about drawing a load balancer and a database, you are preparing for a 2022 interview."

Hyperscale infrastructure has shifted. The era of "Design a URL shortener" is dead. Modern Site Reliability Engineering at Google, Meta, and AWS requires sizing infrastructure for Agentic AI loops, power constraints, and post-quantum network fragmentation.

In modern Non-Abstract Large System Design (NALSD) rounds, drawing a global architecture without doing the physical bandwidth and IOPS math is an instant L3 (Junior) down-level. Hiring committees are now testing Operational Physics.

Below are 10 live interview simulations of the exact scenarios dominating senior hiring loops right now. To demonstrate how the Hiring Committee evaluates candidates, we have open-sourced the first 3 phases (The Math & Feasibility Checks) of these scenarios.

📑 Table of Contents

Part 1: The Computational & Data Core

1. Global LLM Inference Platform (KV-Cache Sizing, Prefill/Decode Disaggregation)
2. Multi-Tenant AI Accelerator Fleet (GPU Bin-Packing, N+1 Pod Loss)
3. Global RAG & Vector Retrieval Platform (10B Vectors, HNSW Index RAM vs. NVMe)
4. Token-Aware AI API Rate Limiter (Two-Phase Reservation, Overshoot Bounds)
5. Global Distributed Lock Service (Sharded Paxos, Speed-of-Light Latency Floors)

Part 2: The Operational Frontiers

1. Agentic “Infinite Loop” DDoS (Semantic Shedding, Queue Deadlines)
2. Multi-Modal Live Edge Streaming (Consistent-Hash GPU Pinning, Reconnect Buffers)
3. Post-Quantum Cryptography Fleet Rollout (Certificate Bloat, MTU Fragmentation)
4. Datacenter Power & Thermal Shedding (3 MW Breaker Trip, Hardware-Aware Suspension)
5. Sovereign Cloud Data Telemetry Sync (Air-Gapped Store-and-Forward, Aggregation)

🏗️ NALSD Scenario Simulation 1: Google SRE NALSD Round Mock Interview (Global LLM Inference Platform)

Why include this: This is our strongest flagship NALSD scenario for this entire Google SRE bundle. Google's public AI infrastructure direction makes it a credible forward-looking practice problem, although the exact interview prompt is unverified. Senior-level Google SRE-style NALSD mock interview for 2027+ preparation: global LLM inference capacity planning, token throughput, KV-cache sizing, accelerator provisioning, regional failure reserves, and infrastructure BOM.

The math that matters: Input/output token throughput, concurrent streams, accelerator throughput, KV-cache footprint, prefill/decode capacity, and regional failure reserves.

Interview simulation · Senior infrastructure engineering · Capacity sizing and BOM

We will treat this as a live, interviewer-led NALSD round. The candidate will reason aloud, calculate resource requirements, justify architectural decisions, and respond to follow-up questions as they arise.

One distinction upfront: this is a realistic simulated interview, not a transcript of an actual Google interview. The prompt and constraints are supplied; accelerator throughput, model configuration details, and hardware prices are not. Where those missing inputs affect the BOM, the candidate will state assumptions rather than invent benchmark results.

The objective is to demonstrate how a strong candidate approaches the problem in the interview—not to recite a prewritten architecture.

Phase 1 — Clarify the problem (Minutes 0–5)

INTERVIEWER

"Design a globally distributed LLM inference platform with hierarchical KV-cache management for Gemini-class models. We need 100,000 QPS globally, 4,000 input tokens on average, 500 output tokens, P99 TTFT under 300 milliseconds, P99 TPOT under 25 milliseconds, and 99.95% availability. How would you design it?"

CANDIDATE

"I would start by establishing the workload and the resource constraints. At this scale, the critical question is not just how many requests we can accept, but how much prefill computation, decode throughput, accelerator memory, and network capacity those requests consume.

I have three clarifications before sizing the system.

First, is 100,000 QPS the peak aggregate across all regions, and should I assume that traffic can be unevenly distributed?

Second, are the input and output token counts averages for initial and subsequent chat turns combined, or should I model separate workload classes?

Third, should I assume a particular accelerator and benchmarked model throughput, or can I use explicit planning assumptions and derive a conditional BOM?"

INTERVIEWER — SIMULATED RESPONSE

"Treat 100,000 QPS as peak global traffic. Traffic is uneven, and multi-turn sessions reuse prefixes. Use a representative accelerator configuration, but state your assumptions. We expect regional failure tolerance."

Candidate's interpretation: I will design for peak traffic rather than average daily traffic. I will distinguish logical conversation history from reusable KV-cache state. I will also avoid assuming that all requests consume the same amount of accelerator time.

The SLOs are requirements to validate, not performance guarantees that can be claimed before sizing and testing.

Phase 2 — Workload math and the first capacity boundary (Minutes 5–12)

The first calculations establish the amount of work entering the platform.

1. Input and output token throughput

Given:

• Peak request rate: 100,000 requests/second.
• Average input: 4,000 tokens/request.
• Average output: 500 tokens/request.

Input-token rate:

100,000 × 4,000 = 400,000,000

That is 400 million input tokens per second before accounting for cache hits.

Output-token rate:

100,000 × 500 = 50,000,000

That is 50 million generated tokens per second.

These are logical workload rates before prefix-cache savings, not measured accelerator throughput.

2. Concurrent generation streams

If we assume an average request generates 500 tokens at an average of 25 milliseconds per token, the estimated generation duration is:

500 × 0.025 = 12.5 seconds

Using Little's Law, the approximate number of concurrent generation streams is:

100,000 × 12.5 = 1,250,000

So the initial planning estimate is 1.25 million concurrent generation streams.

But I would explicitly tell the interviewer that this estimate assumes the stated 25 milliseconds is representative of average token-generation time. The actual requirement is a P99 TPOT bound, so the average cannot be derived from the P99 alone. We need a measured or separately specified average to finalize concurrency.

3. The first interviewer challenge

INTERVIEWER

"You have calculated 1.25 million streams. Does that mean you need 1.25 million independent accelerator allocations?"

CANDIDATE

"No. That is the concurrency across the serving fleet, not the number of accelerators. Continuous batching allows an accelerator to serve multiple sequences concurrently, subject to memory capacity, memory bandwidth, and the latency target.

I need measured per-accelerator throughput and a validated batch configuration to translate these streams into an accelerator count. I will calculate memory demand next, because the KV cache can limit the feasible concurrency even when compute capacity appears sufficient."

Phase 3 — KV-cache sizing: the first major physical constraint (Minutes 12–20)

Candidate's whiteboard calculation

"I'll use an explicit example KV-cache configuration so the estimate is reproducible. Assume 64 transformer layers, 8 KV heads, a head dimension of 128, and one byte per KV value. These are illustrative assumptions, not a claim about Gemini's internal architecture."

For each token, the cache holds both keys and values:

KV_bytes = 2 × L × H_KV × D × B

= 2 × 64 × 8 × 128 × 1

= 131,072 bytes

That is 128 KiB per token before implementation-specific overhead.

For an active request containing approximately 4,250 tokens:

4,250 × 131,072 = 557,056,000 bytes

That is approximately 557 MB per request under this particular configuration.

This number is large, but it follows directly from the stated assumptions. It is not a universal KV-cache footprint.

Estimate aggregate cache demand

Using the illustrative concurrency estimate of 1.25 million active requests:

KV_total = 1.25 × 10⁶ × 557.056 MB

≈ 696 PB

This is a workload-derived theoretical footprint, not a proposed HBM allocation.

The estimate assumes the example KV format and dimensions apply to all requests, all estimated token positions remain resident, and prefix sharing provides no reduction. Actual cache residency can differ substantially.

Interviewer challenge

INTERVIEWER

"You have just calculated hundreds of petabytes of KV state. Are you proposing to buy that much accelerator memory?"

CANDIDATE

"No. The calculation exposes why we cannot retain every request's full KV state in accelerator HBM. It also tells me that I must validate the workload assumptions before committing to the hardware budget.

I would separate three things: active decode state that must be immediately accessible, reusable prefix state that can be cached, and conversation history that can be persisted and recomputed when necessary.

I would also verify the actual model's KV dimensions and precision. If the interviewer expects a feasible production BOM, I need benchmarked per-accelerator throughput and a realistic cache-residency target. Otherwise, the aggregate figure is a useful upper-bound scenario, not a purchase order."

(Interviewer Note: This is an important NALSD moment. The candidate has used math to expose a physical constraint instead of jumping to a familiar architecture).

The interviewer pushes further

INTERVIEWER

"Fine. The platform must still support multi-turn conversations. How will you manage the cache, how many accelerator nodes do you need across prefill and decode, and what happens to our latency budget when a primary region goes dark?"

🔒 End of Preview: Sizing the Architecture

You have reached the end of the public preview (Phases 1–3).

Most candidates fail NALSD interviews at this exact moment: they freeze when faced with physical hardware ceilings like the 696 PB KV-cache footprint, or they throw arbitrary GPU counts at the board without deriving capacity. Passing an L5/L6 Google SRE round requires using mathematical constraints to dictate the architectural boundary before placing a single server.

🗺️ What’s Inside the Full Walkthrough: Phases 4 through 6

In the complete interview session included in our bundle, the candidate steps through the rest of the 45-minute loop:

• Phase 4 — Compute, Accelerator, and Network Sizing (Minutes 20–30)

  • Quantized Model Footprint: Sharding a 70B parameter model across 8-accelerator hosts (640 GB aggregate physical HBM) and calculating real usable capacity.
  • Throughput Sizing Formulas: Deriving exact prefill (N_prefill) and decode (N_decode) node counts at target utilization (u = 60%).
  • Network Fabric Math: Calculating the 1.6 Gbps+ outbound payload rate and isolating the prefill-to-decode KV transfer bandwidth to prevent congestion-driven tail latency.

• Phase 5 — Architecture Derived from Sizing Constraints (Minutes 30–37)

  • End-to-End Request Path: Designing the global routing layer, admission control, and prefix-aware scheduling across prefill and decode pools.
  • Hierarchical Cache Retention: Engineering the HBM → Host DRAM → Durable conversation history storage pipeline.
  • Regional Failover Math: Sizing surviving regions for the exact +33.3% load shift during an N-1 failure without exceeding TTFT/TPOT latency bounds.

• Phase 6 — Final BOM, SLO Validation, and Conclusion (Minutes 37–45)

  • The Complete Production BOM: Sizing criteria across accelerator hosts, HBM, host DRAM, regional gateways, and the control plane.
  • Tail-Latency Validation Strategy: Empirical testing plans for P99 TTFT, inter-token generation latency distributions, and retry amplification safeguards.
  • Scoring Breakdown: The exact rubrics Google interviewers use to distinguish an L5 (Senior) from an L6 (Staff) candidate.

🚀 Get the Full Walkthrough & Complete Interview Bundle

This scenario is just one part of the end-to-end curriculum designed to build your operational reflexes under real interview pressure.

👉 Unlock the Full Walkthrough in the Google SRE Interview Career Launchpad

What is inside the complete system:

• Full-Length NALSD & Troubleshooting Transcripts: The complete, unabridged walkthroughs, mathematical proofs, and architectural blueprints for this multi-tenant accelerator fleet and other flagship scenarios.
• 20+ "War Room" Interview Simulations: Realistic mock scenarios equipped with the hidden Interviewer Scorecards, follow-up traps, and rubric criteria.
• Linux Internals & Reliability Deep Dives: Practical drills covering D-state processes, TCP connection backlogs, kernel memory pressure, and distributed consensus.
• 70+ Production-Grade Coding Drills: Systems-focused Python & Go drills emphasizing concurrency, streaming, backpressure, and defensive design.
• The Offer Maximizer: Negotiation frameworks and word-for-word scripts to secure top-of-band L5/L6 compensation packages.

Stop guessing the math under pressure. Master the simulation.

🏗️ Google SRE NALSD Scenario Simulation 2: Multi-Tenant AI Accelerator Fleet Capacity & Scheduling

Why include this: This scenario targets the core reality of enterprise AI infrastructure: managing a mixed, multi-tenant fleet of 2,000 heterogeneous workloads. Rather than assuming homogeneous single-model serving, this interview stresses multi-accelerator topology constraints, bin-packing fragmentation, gang scheduling, and pod-level correlated failure domains.

The math that matters: Accelerator-equivalent demand aggregation, compound utilization and bin-packing efficiency factors (0.70 × 0.85), N+1 pod failure derivation, physical host counts, and per-accelerator working memory limits.

Interview simulation · Senior infrastructure engineering · Fleet capacity, bin packing, and N+1 resilience

We will treat this as a live, interviewer-led NALSD round. The candidate will reason aloud, calculate resource requirements, justify architectural decisions, and respond to follow-up questions as they arise.

One distinction upfront: this is a realistic simulated interview, not a transcript of an actual Google interview. Hardware performance metrics, pod geometries, and utilization ceilings are explicit planning assumptions. The objective is to demonstrate how a strong candidate establishes capacity envelopes and prevents correlated outage cascades—not to guess vendor specs.

Phase 1 — Define the problem and workload dimensions

INTERVIEWER

"Design a shared accelerator fleet serving 2,000 enterprise AI workloads. Each workload has different throughput, latency, memory, and priority requirements. Calculate the accelerator count, memory capacity, scheduling overhead, and spare capacity needed to survive the loss of an entire accelerator pod."

CANDIDATE

"Before choosing the number of accelerators, I need to establish the workload distribution, the performance of each accelerator configuration, the memory footprint of each workload, and the failure domain.

Two thousand workloads do not necessarily mean two thousand simultaneously active workloads. Some may be continuously serving inference traffic, some may execute periodic batch jobs, and others may require accelerators only during training windows.

I will separate the problem into five dimensions:

1. Demand: How much accelerator compute does each workload class require?
2. Memory fit: Can the model, working state, and runtime fit on the available accelerator configuration?
3. Scheduling: How efficiently can heterogeneous workloads be packed onto hosts?
4. Latency and isolation: Can high-priority workloads meet their service-level objectives under contention?
5. Resilience: Can the fleet continue serving its required workload after losing an entire accelerator pod?

I will use explicit exercise assumptions first, then calculate the required capacity. Hardware performance figures will be assumptions for this design, not claims about any particular cloud provider's production fleet."

INTERVIEWER

"Good. Start by defining the fleet and its workload."

Phase 2 — Workload demand aggregation

INTERVIEWER

"You have 2,000 workloads. Give me a concrete demand model."

CANDIDATE

"I will divide the fleet into four workload classes. These are exercise assumptions that represent different resource profiles."

"An accelerator-equivalent represents the compute capacity of one reference accelerator under our baseline benchmark assumptions. Fractional values represent aggregated demand across time-sliced or batched workloads; they do not mean a physical GPU can be divided arbitrarily without scheduling and memory isolation consequences.

The fleet's aggregate peak demand is:"

D = 120 + 300 + 320 + 160 = 900

"Therefore, the fleet must deliver 900 reference accelerator-equivalents of compute during the modeled peak."

INTERVIEWER

"Why not simply provision 900 accelerators?"

CANDIDATE

"Because 900 is the modeled compute demand, not the installed fleet size. We still need to account for operational utilization limits, fragmented capacity caused by bin packing, and the loss of an entire accelerator pod.

I also need to distinguish average demand from peak demand. If the workloads have synchronized peaks, aggregate demand approaches 900 accelerator-equivalents. If peaks are statistically independent, multiplexing may reduce the capacity required—but only after measuring demand distributions and acceptable queueing latency.

For this initial sizing, I will provision against the stated aggregate peak rather than assume that statistical multiplexing will work in our favor."

Phase 3 — Hardware baseline and capacity loss factors

INTERVIEWER

"What hardware are you designing around?"

CANDIDATE

"I will use a reference host containing eight accelerators, each with 80 GB of accelerator memory.

For the capacity calculation, I will apply these explicit assumptions:"

INTERVIEWER

"Explain the difference between those last two percentages."

CANDIDATE

"The utilization ceiling (70%) limits how heavily we plan to load capacity that can be allocated, leaving headroom for traffic bursts and latency-sensitive scheduling.

The bin-packing efficiency (85%) accounts for capacity that exists physically but cannot be assigned because of discrete workload shapes, memory limits, or topology placement fragmentation.

Because they describe completely different losses, I apply them multiplicatively to find the effective usable capacity:"

C_effective = N × 0.70 × 0.85 = N × 0.595

Phase 4 — Sizing the N+1 resilient fleet

INTERVIEWER

"How many accelerators do you need to survive losing one pod?"

CANDIDATE

"The surviving fleet must deliver at least 900 effective accelerator-equivalents after losing one complete 64-accelerator pod.

If the fleet has N installed accelerators, the surviving capacity is:"

N_surviving = N - 64

"Applying our combined efficiency factor (0.595):"

(N - 64) × 0.70 × 0.85 ≥ 900

(N - 64) × 0.595 ≥ 900

N - 64 ≥ 900 / 0.595 ≈ 1,512.61

N ≥ 1,576.61 accelerators

"Because we must provision full 64-accelerator pods, we round up to whole pods:"

N_pods = ⌈1,576.61 / 64⌉ = 25 pods

N = 25 × 64 = 1,600 accelerators

"After losing a 64-accelerator pod, 1,536 × 0.595 = 913.92 effective accelerators remain, clearing our 900-unit demand by 13.92 accelerator-equivalents.

However, a 1.55% surplus margin is razor-thin. I would not call this design production-ready without validating placement fragmentation, startup times, and rack-level topology."

Phase 5 — Translating into hosts and memory topology

INTERVIEWER

"Convert that accelerator count into physical hosts and memory capacity."

CANDIDATE

"At 8 accelerators per host, we need:"

N_hosts = 1,600 / 8 = 200 hosts

"The raw fleet memory is:"

M_fleet = 1,600 × 80 GB = 128,000 GB = 128 TB

M_surviving = 1,536 × 80 GB = 122.88 TB

INTERVIEWER

"Does 128 TB of fleet memory prove that all 2,000 workloads will fit?"

CANDIDATE

"No. Aggregate memory capacity is an illusion.

A workload requiring 140 GB cannot fit on a single 80 GB device. It requires two accelerators that must reside on the same high-speed PCIe or NVLink domain. If the cluster has 200 GB of free memory scattered as 25 GB fragments across eight different machines, that 140 GB job cannot schedule.

Furthermore, for the 40 large training jobs requiring 4 accelerators each, we cannot allocate them piecemeal. We must enforce gang scheduling: all required accelerators, host memory, and fabric ports must be acquired simultaneously. Otherwise, partially allocated jobs will idle on scarce GPUs, creating cluster-wide resource deadlocks."

(Interviewer Note: This is an exceptional NALSD answer. The candidate immediately identifies that compute and memory cannot be treated as scalar pools without considering physical topology and deadlock risks).

The interviewer pushes further

INTERVIEWER

"You've proven the physical limits of bin packing and memory topology. Now, how do you actually design the scheduling system to orchestrate 2,000 competing workloads? How do you isolate latency-sensitive inference from batch training, manage control plane overhead, and orchestrate recovery without triggering a cold-start storm when an entire pod goes dark?"

🔒 End of Preview: Capacity Sizing & Topology Constraints

You have reached the end of the public preview (Phases 1–5).

Most candidates fail this interview by treating GPUs like interchangeable CPU cores—ignoring interconnect topologies, fragmentation losses, and correlated restart storms. In an L5/L6 Google SRE round, proving the physical failure boundary is only the first half of the problem.

🗺️ What’s Inside the Full Walkthrough: Phases 6 through 10

In the complete interview session included in our bundle, the candidate steps through the remaining 25 minutes of the mock loop:

• Phase 6 — Heterogeneous Scheduler Architecture (Minutes 20–28)

  • Control Plane vs. Data Plane Disaggregation: Designing the admission control, placement engine, and distributed agent runtime.
  • Workload Isolation & Preemption: Implementing priority tiers, bounded queues, and checkpoint-aware preemption to protect latency-sensitive inference from batch monopolization.
  • Multi-Dimensional Constraint Matching: Sizing placement engines that evaluate compute, memory, and topology (NVLink/RoCE) simultaneously.

• Phase 7 — Control Plane Sizing & Queueing Dynamics (Minutes 28–34)

  • Throughput Sizing: Sizing the scheduler for an initial burst (λ_submission ≈ 3.33 req/s) and steady-state lifecycle events (λ_events = 100 req/s).
  • Latency Breakdown: Separating scheduling decision latency from resource queueing latency under heavy cluster contention.
  • High-Availability Control Plane: Deploying a quorum-based 3-replica active scheduler fabric across distinct failure domains.

• Phase 8 — Correlated Pod Failure & Cold-Start Storm Recovery (Minutes 34–39)

  • Correlated Failure Mechanics: Handling the simultaneous loss of 8 hosts, 64 accelerators, and 5.12 TB of resident memory.
  • Avoiding Recovery Cascades: Why instantaneous restarts trigger secondary failure storms through simultaneous model weights pulls and checkpoint reads.
  • Staged Recovery Pipelines: Rate-limited warm-up queues, priority-tier restoration, and admission throttling.

• Phase 9 — Final Production BOM & Infrastructure Sizing (Minutes 39–43)

  • The Complete Infrastructure Footprint: Sizing 25 pods (1,600 accelerators, 200 hosts), high-bandwidth network fabric, and checkpoint storage throughput.
  • Four-Step Validation Roadmap: Benchmark verification, simulated bin-packing audits, pod-loss drills, and thermal/power density planning.

• Phase 10 — Final Interviewer Challenge & Scoring Rubric (Minutes 43–45)

  • The Single Greatest Risk: Unpacking the danger of treating scalar capacity as fungible capacity.
  • Official Interviewer Scorecard: The exact rubric Google interviewers use to grade workload sizing, failure domain awareness, and operational realism.

🚀 Get the Full Walkthrough & Complete Interview Bundle

This scenario is just one part of the end-to-end curriculum designed to build your operational reflexes under real interview pressure.

👉 Unlock the Full Walkthrough in the Google SRE Interview Career Launchpad

Stop guessing the math under pressure. Master the simulation.

🏗️ Google SRE NALSD Scenario Simulation 3: Global RAG and Vector Retrieval Platform

Why include this: Vector retrieval at the 10-billion scale is the foundational backbone of enterprise Generative AI and RAG architectures. This scenario tests whether a candidate understands that vector databases are memory-bandwidth-bound and scatter-gather-bound systems, rather than conventional disk-backed relational stores.

The math that matters: FP16 vs. FP32 representation, HNSW graph index overheads (+30%), metadata footprint, shard partition sizing, scatter-gather query amplification (naive 20M QPS vs. routed 1M QPS), network candidate bandwidth, and regional N+1 capacity sizing.

Interview simulation · Senior infrastructure engineering · Vector search, memory residency, and global latency

We will treat this as a live, interviewer-led NALSD round. The candidate will reason aloud, calculate resource requirements, justify architectural decisions, and respond to follow-up questions as they arise.

One distinction upfront: this is a realistic simulated interview, not a transcript of an actual Google interview. Data representations, graph overheads, and network margins are explicit planning assumptions. The objective is to demonstrate how a strong candidate establishes capacity envelopes and mitigates tail latency—not to guess vendor specs.

Phase 1 — Define the problem and operational scale

INTERVIEWER

"Design a globally distributed retrieval platform containing 10 billion embedding vectors, each with 1,536 dimensions. Serve 100,000 queries per second with P99 retrieval latency below 100 milliseconds. Calculate RAM, index overhead, storage, replication, network bandwidth, and the final Bill of Materials."

CANDIDATE

"Before choosing a vector database or deciding how many servers to provision, I need to establish the data representation, retrieval algorithm, query distribution, and geographic failure requirements.

Ten billion vectors is a massive dataset, but the vector count alone does not determine the infrastructure requirement. The result depends on bytes per dimension, index overhead, the number of candidates examined per query, the degree of memory residency, and whether retrieval requires one or multiple stages.

I will establish a quantitative model using these assumptions:

• The dataset contains 10 billion vectors, each with 1,536 dimensions.
• The global retrieval service must sustain 100,000 queries per second.
• End-to-end retrieval latency must remain below 100 milliseconds at P99.
• The primary index uses approximate nearest-neighbor search (ANN), with optional exact reranking.
• The platform uses geographically distributed query-serving capacity and replicated data.
• Capacity calculations will separate vector payload, index structures, metadata, replication, and operational headroom.

I will calculate the storage footprint first, derive the shard layout, size query throughput and network traffic, and then determine the compute capacity and final BOM."

INTERVIEWER

"Good. Start with raw storage. Do not choose a server count yet."

Phase 2 — Calculate raw vector storage: FP32 vs. FP16

INTERVIEWER

"Each vector has 1,536 dimensions. How much space do 10 billion vectors require?"

CANDIDATE

"The first decision is numeric precision. If every dimension uses standard 32-bit floating-point (FP32), each vector requires:"

1,536 × 4 bytes = 6,144 bytes

"For 10 billion vectors, that requires:"

10¹⁰ × 6,144 = 61.44 × 10¹² bytes ≈ 61.44 TB (55.9 TiB)

"For a more cost-effective and memory-bandwidth-efficient design, I will evaluate 16-bit floating-point (FP16) storage, provided offline recall and ranking-quality benchmarks show negligible degradation relative to FP32.

The FP16 vector size is:"

1,536 × 2 bytes = 3,072 bytes/vector

"The total raw payload becomes:"

10¹⁰ × 3,072 = 30.72 × 10¹² bytes = 30.72 TB

INTERVIEWER

"Does that mean the entire platform requires only 30.72 TB of RAM?"

CANDIDATE

"No. That number covers only the vector payload. The platform also needs the search index graph, vector identifiers, metadata, runtime buffers, and operational headroom.

I also need to distinguish storage capacity from query-serving capacity. A system may keep its searchable index in RAM while storing original vectors or document content on SSD. Alternatively, a fully memory-resident design may be justified if predictable P99 latency is more important than minimizing memory cost.

I will initially evaluate a memory-resident search index because the service has a strict P99 latency target of 100 ms. I will validate that decision with representative retrieval benchmarks rather than assume that all data must always reside in RAM."

Phase 3 — Index overhead, graph links, and metadata sizing

INTERVIEWER

"What index would you use, and how much additional memory does it require?"

CANDIDATE

"I would evaluate an HNSW-style (Hierarchical Navigable Small World) graph index because it provides an excellent recall-versus-latency curve for approximate nearest-neighbor search. However, the final algorithm depends on the required recall, update rate, ingestion pattern, and memory budget.

For the initial capacity model, I will use these explicit assumptions:"

"The 30% index overhead is an illustrative planning assumption, representing graph connectivity (e.g., M = 16 or 32 links per node), implementation overhead, and indexing buffers.

The index memory estimate is:"

M_index = 30.72 TB × 0.30 = 9.216 TB

"The metadata estimate is:"

M_metadata = 10¹⁰ × 128 bytes = 1.28 × 10¹² bytes = 1.28 TB

"Therefore, the estimated working set for one complete dataset copy is:"

S_copy = 30.72 + 9.216 + 1.28 = 41.216 TB

"This estimate excludes original raw document bodies, full-text inverted indexes, write-ahead logs, and external reranking models, which belong to separate downstream microservices."

Phase 4 — Replicated storage and memory residency targets

INTERVIEWER

"Calculate the storage requirement for three copies."

CANDIDATE

"With a baseline logical replication factor of 3:"

S_replicated = 41.216 TB × 3 = 123.648 TB

"Adding 25% operational headroom to account for memory fragmentation, runtime growth, and background index rebuilding:"

S_planned = 123.648 TB × 1.25 = 154.56 TB

INTERVIEWER

"How much RAM do you need?"

CANDIDATE

"If every replica keeps its vector payload, graph index, and metadata resident in memory, the aggregate usable memory target across all three copies is 154.56 TB.

This is usable memory, not total installed physical RAM. Physical servers require additional memory for OS kernel page caches, network socket buffers, query aggregations, and process overhead."

INTERVIEWER

"Is full memory residency always the right answer?"

CANDIDATE

"No. A hybrid architecture (such as DiskANN) can keep the high-level navigation graph in RAM while fetching the actual FP16 vectors from high-throughput NVMe SSDs via parallel asynchronous I/O.

That dramatically reduces RAM spend, but SSD read amplification and tail-latency jitter can jeopardize our strict P99 target of 100 ms. I will evaluate the memory-resident baseline first to guarantee tail latency, then benchmark hybrid NVMe offloading as an optimization."

Phase 5 — Shard layout and partitioning

INTERVIEWER

"Ten billion vectors cannot fit on one machine. How would you partition the dataset?"

CANDIDATE

"I will partition the dataset into 200 logical primary shards, each holding 50 million vectors:"

N_shards = 10,000,000,000 / 50,000,000 = 200 shards

"Let's calculate the working set per shard:

• Raw vector payload per shard: 50,000,000 × 3,072 bytes = 153.6 GB
• Index overhead per shard (30%): 153.6 × 0.30 = 46.08 GB
• Metadata per shard: 50,000,000 × 128 bytes = 6.4 GB

The total working set per logical shard is:"

S_shard = 153.6 + 46.08 + 6.4 = 206.08 GB

"Across three copies, the cluster hosts 600 shard replicas (200 × 3)."

INTERVIEWER

"Does that mean every user query must visit all 200 primary shards?"

CANDIDATE

"Not necessarily. If every incoming query fans out to all 200 shards, the system suffers severe scatter-gather request amplification.

100,000 queries per second fanning out to 200 shards creates 20,000,000 internal shard searches per second. That creates massive CPU contention and network congestion.

Instead, I would introduce a two-tier routing architecture using a coarse centroid index (e.g., IVF clustering) or tenant/metadata partitioning to direct queries only to candidate shards.

For our initial model, I will assume queries route to an average of 10 candidate shards. However, this introduces an immediate tradeoff: aggressive pruning reduces query load, but poor routing drops relevant neighbors and degrades recall."

(Interviewer Note: This is a critical NALSD insight. The candidate connects database partitioning directly to network request amplification and retrieval recall).

The interviewer pushes further

INTERVIEWER

"You've highlighted the fan-out danger. Now prove how the network fabric handles candidate merging and reranking, show how tail latency behaves across parallel shard fan-outs, and size the compute fleet to survive the loss of an entire cloud region without breaching our 100 ms P99 SLO."

🔒 End of Preview: Vector Footprint & Shard Partitioning

You have reached the end of the public preview (Phases 1–5).

Most candidates fail this interview by treating vector search like a standard database—forgetting that scatter-gather tail latencies compound exponentially (P = pⁿ), and underestimating the network fabric needed to merge millions of candidate vectors per second.

🗺️ What’s Inside the Full Walkthrough: Phases 6 through 10

In the complete interview session included in our bundle, the candidate steps through the remaining 25 minutes of the mock loop:

• Phase 6 — Query Throughput & Scatter-Gather Amplification (Minutes 18–24)

  • Fan-Out Math: Contrasting naive broadcast (20M shard searches/s) with 10-way routed retrieval (1M shard searches/s).
  • Per-Shard Load Balancing: Sizing each logical shard for 5,000 searches/s while handling embedding clustering hot spots.

• Phase 7 — Network Bandwidth & Candidate Merge Sizing (Minutes 24–29)

  • Ingress & Egress Traffic: Sizing the 400 MB/s user ingress against the 4 GB/s (32 Gbps) internal fan-out request volume.
  • Candidate Merge Payload: Sizing 100M candidates/sec yielding 6.4 GB/s (51.2 Gbps) of candidate-response serialization across the coordinator layer.

• Phase 8 — P99 Tail Latency Budget & The Multi-Shard Fan-Out Penalty (Minutes 29–34)

  • End-to-End Latency Slicing: Allocating budgets across ingress (10ms), embedding (15ms), parallel search (40ms), rerank (15ms), and delivery (10ms).
  • The Tail Probability Trap: Proving why independent 99th-percentile shards compound to only a 90.4% success rate over a 10-way fan-out (0.99¹⁰ ≈ 0.904), and designing hedging/speculative retry strategies.

• Phase 9 — Multi-Region Replication & Disaster Recovery Math (Minutes 34–38)

  • Regional Failover Load: Proving why surviving regions in a 3-region cluster must absorb a +50% load surge during a full regional outage.
  • Rebuild Mechanics: Deriving the network transfer floor (55 minutes at 100 Gbps line rate) and preventing foreground query starvation during recovery.

• Phase 10 — Compute Fleet Sizing & Final Bill of Materials (Minutes 38–45)

  • Memory-Bound vs. Compute-Bound: Determining whether 500 throughput-sized hosts or 600 memory-placement hosts govern cluster sizing.
  • The N+1 Regional Sizing: Scaling the fleet to 900 query-serving hosts (230.4 TB installed RAM) to survive a complete regional catastrophe.
  • Official Interviewer Scorecard: The exact grading matrix Google interviewers use to evaluate vector database capacity, tail-latency resilience, and distributed system trade-offs.

🚀 Get the Full Walkthrough & Complete Interview Bundle

This scenario is just one part of the end-to-end curriculum designed to build your operational reflexes under real interview pressure.

👉 Unlock the Full Walkthrough in the Google SRE Interview Career Launchpad

Stop guessing the math under pressure. Master the simulation.

🏗️ Google SRE NALSD Scenario Simulation 4: Global Token-Aware AI API Rate Limiter

Why include this: Standard rate limiters count simple scalar requests (HTTP QPS). In modern GenAI platforms, request count is meaningless—a 100-token prompt and a 50,000-token multi-document prompt consume vastly different amounts of compute and financial budget. This scenario tests distributed admission control, two-phase token reservations, and cross-region coordination without synchronous WAN bottlenecks.

The math that matters: Weighted average token consumption (2,850 tokens/req), 14.25 billion tokens/sec peak accounting rate, in-memory tenant state vs. durable ledger volumes, RPC network payload (15.36 Gbps), bounded quota lease allocations, and regional N+1 capacity sizing (188 instances).

Interview simulation · Senior infrastructure engineering · Distributed admission control, token accounting, and fail-safe quotas

We will treat this as a live, interviewer-led NALSD round. The candidate will reason aloud, calculate resource requirements, justify architectural decisions, and respond to follow-up questions as...