I build Google Sheets and Apps Script tools alongside small Python services, and I keep running into the same two walls as row counts grow.

The first is the 03:06 AM Google Sheets failure email. An internal Google Apps Script that synced 400 orders or CRM leads in January now has 8,500 rows to process. Google kills the morning run halfway through with Exceeded maximum execution time. The script stops with half the spreadsheet updated, after firing duplicate API webhooks and breaking formulas.

The second wall is a quiet n8n container restart on a 2 GB or 4 GB VPS. A self-hosted n8n workflow runs SplitInBatches over a 15,000-row CSV or Shopify catalog and dies with Exit Code 137 or FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory.

Teams respond by renting more VPS RAM for n8n or splitting a Google Sheet across five tabs. Both workarounds buy a little breathing room while leaving the real bottlenecks intact: per-cell network round trips and growing execution state.

Anatomy of the two failure modes

Google's V8 Apps Script sandbox and n8n's Node.js worker fail in different places, though both punish row-by-row loops:

Bottleneck layer Google Apps Script (V8 Runtime) Self-hosted or cloud n8n (Node.js Runtime)
Hard ceiling 360 seconds (6 minutes) hard kill per execution; 90 min/day total trigger quota on consumer Gmail (6 hr/day on Workspace). Node.js V8 heap limit (default ~1.5 to 2 GB unless you raise NODE_OPTIONS=--max-old-space-size) or Docker cgroup OOMKilled (137).
Main antipattern Calling sheet.getRange(r, c).setValue(...) or UrlFetchApp.fetch() one by one inside a for loop (~80 to 150 ms of IPC overhead per cell). Looping 10,000 items inside a single workflow with SplitInBatches, since n8n retains item history for each node across loop iterations in RAM.
Common trap Two simultaneous doPost(e) webhooks read getLastRow() at the same millisecond and overwrite each other's row. Keeping binary files like PDFs or 25 MB CSV exports in N8N_DEFAULT_BINARY_DATA_MODE=default (RAM) instead of filesystem.

Part 1: Engineering around Google Apps Script's 6-minute wall

Fix 1A: Eliminate per-cell RPC calls (getValues() / setValues() + UrlFetchApp.fetchAll())

In Apps Script, each getValue() or setValue() call crosses an internal network boundary to Google Sheets. At 80 to 150 ms a call, 3,000 setValue() writes eat most of your 6-minute budget on round trips alone.

You can batch outbound HTTP traffic the same way. Pass an array of request objects to UrlFetchApp.fetchAll(requests) and Google runs those requests in parallel on its own infrastructure:

/** * BAD: O(N) Sheet RPCs + sequential HTTP = ~340 seconds for 400 rows (times out!) * GOOD: Single 2D matrix read + parallel fetchAll chunks + single matrix write = ~9.4 seconds! */ function enrichSheetBatchFast(sheet, startRow, batchSize, apiEndpoint, apiKey) { const values = sheet.getRange(startRow, 1, batchSize, 4).getValues(); // [domain, status, email, tech] const pendingIndices = []; const httpRequests = []; for (let i = 0; i < values.length; i++) { const domain = String(values[i][0] || "").trim(); const status = String(values[i][1] || "").trim(); if (!domain || status === "DONE") continue; pendingIndices.push(i); httpRequests.push({ url: `${apiEndpoint}?domain=${encodeURIComponent(domain)}`, method: "get", headers: { "Authorization": `Bearer ${apiKey}`, "Accept": "application/json" }, muteHttpExceptions: true, }); } if (httpRequests.length === 0) return 0; // Execute HTTP calls concurrently in chunks of 25 const CHUNK = 25; for (let offset = 0; offset < httpRequests.length; offset += CHUNK) { const reqSlice = httpRequests.slice(offset, offset + CHUNK); const idxSlice = pendingIndices.slice(offset, offset + CHUNK); const responses = UrlFetchApp.fetchAll(reqSlice); for (let j = 0; j < responses.length; j++) { const rowIdx = idxSlice[j]; const code = responses[j].getResponseCode(); if (code >= 200 && code < 300) { const data = JSON.parse(responses[j].getContentText()); values[rowIdx][1] = "DONE"; values[rowIdx][2] = data.primary_email || ""; values[rowIdx][3] = (data.tech_stack || []).join(", "); } else { values[rowIdx][1] = `ERR_${code}`; } } } // Commit the entire updated block in ONE single Spreadsheet write call sheet.getRange(startRow, 1, values.length, 4).setValues(values); SpreadsheetApp.flush(); return httpRequests.length; } 

Fix 1B: Stateful 270-second continuation checkpointing + LockService

Batched reads and writes hit the same 6-minute limit once a sheet holds 45,000 rows or calls a slow external ERP API that needs 25 minutes end to end. Checkpointing does not lift the daily quota either: a consumer account gets 20,000 UrlFetchApp calls per day (100,000 on Workspace), so a job that calls an API for every row of a 45,000-row sheet has to spread over more than one day on a consumer account.

Do not wait until second 359 for Google to kill the script mid-write. Set a soft wall-clock budget of 270 seconds (4.5 minutes) and save cursor_row in PropertiesService.getScriptProperties() after each batch. Before exiting, clear old continuation triggers to prevent duplicate trigger storms, then schedule a single trigger to resume 60 seconds later:

const CURSOR_KEY = "SYNC_CURSOR_ROW"; const TRIGGER_HANDLER = "runResumablePipeline"; const SOFT_LIMIT_MS = 270 * 1000; // 4.5 minutes (90s safety buffer before Google's 6m kill) const BATCH_ROWS = 200; function runResumablePipeline() { // 1. Prevent overlapping executions (e.g. manual click + scheduled trigger colliding) const lock = LockService.getScriptLock(); if (!lock.tryLock(5000)) { console.log("Another pipeline instance holds the script lock. Exiting cleanly."); return; } const startedAt = Date.now(); const props = PropertiesService.getScriptProperties(); try { clearContinuationTriggers_(TRIGGER_HANDLER); const ss = SpreadsheetApp.getActiveSpreadsheet(); const sheet = ss.getSheetByName("Leads"); const lastRow = sheet.getLastRow(); let cursor = parseInt(props.getProperty(CURSOR_KEY) || "2", 10); while (cursor <= lastRow) { // Check soft wall-clock budget BEFORE starting the next batch if (Date.now() - startedAt >= SOFT_LIMIT_MS) { props.setProperty(CURSOR_KEY, String(cursor)); scheduleNextContinuation_(TRIGGER_HANDLER, 60 * 1000); console.log(`Soft time limit reached. Saved checkpoint at row ${cursor}. Resuming in 60s.`); return; } const remaining = lastRow - cursor + 1; const currentBatch = Math.min(BATCH_ROWS, remaining); enrichSheetBatchFast( sheet, cursor, currentBatch, props.getProperty("ENRICH_API_URL"), props.getProperty("ENRICH_API_KEY") ); cursor += currentBatch; props.setProperty(CURSOR_KEY, String(cursor)); } // Completed all rows cleanly: reset cursor for tomorrow's scheduled run! props.deleteProperty(CURSOR_KEY); console.log(`Pipeline completed all ${lastRow - 1} rows with zero timeouts.`); } finally { lock.releaseLock(); } } function clearContinuationTriggers_(handlerName) { const triggers = ScriptApp.getProjectTriggers(); for (const t of triggers) { if (t.getHandlerFunction() === handlerName) { ScriptApp.deleteTrigger(t); } } } function scheduleNextContinuation_(handlerName, delayMs) { ScriptApp.newTrigger(handlerName).timeBased().after(delayMs).create(); } 

You also want a lock on doPost(e) webhook receivers. Wrap the write in const lock = LockService.getScriptLock(); lock.waitLock(15000); try { ... } finally { lock.releaseLock(); }. The overwrite happens when the handler computes its target row with getLastRow() + 1 and then calls setValues(): 5 Stripe or Shopify webhooks arriving in the same second can all read the same last row. The lock also covers any read-then-write logic, such as checking for a duplicate order ID before appending.

Part 2: Stopping n8n out-of-memory (OOM) crashes

A 2 GB self-hosted n8n instance can crash on a 12 MB JSON payload or a 5,000-row SplitInBatches loop because its execution engine snapshots output items at each node boundary. Those snapshots let the UI canvas render intermediate outputs and resolve expressions like $('Previous Node').item.json. Pass a 12 MB array through 8 transformation nodes in one execution and V8 ends up holding several serialized copies, eating hundreds of MB of heap.

Keeping SplitInBatches inside one monolithic workflow compounds the problem. The parent execution holds onto previous batch iterations in its memory tree until the entire run finishes.

The 3-step n8n memory fix

1. Move binary payloads off RAM (filesystem mode)

In your docker-compose.yml or .env for self-hosted n8n, switch binary handling off default (RAM) if your workflows touch CSV exports, PDFs, audio files, or scraped HTML snapshots:

environment: - N8N_DEFAULT_BINARY_DATA_MODE=filesystem - EXECUTIONS_DATA_SAVE_ON_SUCCESS=none - EXECUTIONS_DATA_SAVE_ON_ERROR=all - EXECUTIONS_DATA_PRUNE=true - EXECUTIONS_DATA_MAX_AGE=72 - NODE_OPTIONS=--max-old-space-size=2560 # 4 GB VPS; on a 2 GB VPS keep it around 1536 

Setting EXECUTIONS_DATA_SAVE_ON_SUCCESS=none on high-volume worker workflows stops n8n from serializing 100 MB JSON execution blobs into SQLite or Postgres at completion.

2. Isolate loop batches in a child sub-workflow

Move those 12 enrichment and CRM nodes out of your parent SplitInBatches loop and split the work across two workflows:

  1. The parent workflow queries lightweight record IDs or 200-row slices (SELECT id, domain FROM leads WHERE synced = false LIMIT 200) and calls the Execute Sub-workflow node (Mode: Run once for all items per 200-row batch, Wait For Sub-Workflow Completion: ON).
  2. The child sub-workflow takes those 200 rows and runs the heavy enrichment, from OpenAI Structured Output parsing to CRM upserts. At the end, it returns a small summary object: {"processed": 200, "failed": 0}.

Once the child sub-workflow finishes and hands back that 32-byte summary, V8 can garbage-collect the child execution's intermediate data before batch #2 starts. The process holds one batch in memory at a time instead of growing with the total row count.

3. Offload heavy scraping and deduplication to a 35 MB async Python sidecar

n8n fits stateful orchestration and webhook routing, while 5,000-URL crawls and fuzzy CSV deduplication bog down its Node worker. Once a Code node in n8n passes 80 lines of nested loops or cheerio HTML parsing, I move that compute step into a stateless FastAPI (Python 3.11) container on the same Docker network, or call an Apify Actor over a webhook:

# Lightweight FastAPI chunk worker (`~34 MB` RSS) called by n8n HTTP Request node from fastapi import FastAPI from pydantic import BaseModel, Field import httpx import asyncio app = FastAPI(title="n8n Offload Chunk Worker") class BatchRequest(BaseModel): domains: list[str] = Field(..., max_length=250) concurrency: int = Field(default=15, ge=1, le=40) @app.post("/v1/probe-batch") async def probe_domains_batch(payload: BatchRequest) -> dict: sem = asyncio.Semaphore(payload.concurrency) results: list[dict] = [] async def _probe(client: httpx.AsyncClient, domain: str) -> None: url = domain if domain.startswith("http") else f"https://{domain}" async with sem: try: r = await client.get(url, timeout=6.0) results.append({"domain": domain, "status": r.status_code, "ok": r.status_code < 400}) except Exception as exc: results.append({"domain": domain, "status": 0, "ok": False, "err": type(exc).__name__}) async with httpx.AsyncClient(follow_redirects=True) as client: await asyncio.gather(*(_probe(client, d) for d in payload.domains)) # Return compact structured summary back to n8n — zero raw HTML enters n8n's V8 heap!  return { "total": len(results), "alive_count": sum(1 for x in results if x["ok"]), "items": results, } 

What changes, in short

These are expected behaviors from how each runtime manages limits; I did not benchmark the four setups side by side on a client system.

Architecture Large sheet / batch When it fails Recovery
Naive Apps Script (setValue loop) Hits the 6-minute kill partway through Mid-write, half the sheet updated Manual cleanup
Checkpointed Apps Script (batched writes + 270s cursor + LockService) Finishes across chained runs Each run stops at the 270s budget Resumes from the saved row cursor
Monolithic n8n (SplitInBatches in one workflow, in-memory binaries) Memory grows on each loop OS can OOM-kill the container (exit 137) Partial, unlogged state
n8n parent/child sub-workflow + async Python sidecar Memory stays bounded by batch size One batch fails while the rest of the run continues Per-batch retry + alert

Templates and help

If your Google Sheets scripts hit the 6-minute execution limit, or your n8n workflows drop webhooks and crash under load, you can reuse a few pieces instead of rebuilding from scratch.

I published async Python hidden-API and automation starter templates on Gumroad as the free Python & Playwright Hidden-API Starter Kit Lite ($0+). The full 6-engine set with importable n8n and Apps Script blueprints is in the Pro Toolkit.

To move B2B lead enrichment, Shopify or WooCommerce price monitoring, US healthcare clinic DSO screening, EU hospitality scraping, or 2-Opt TSP route optimization out of n8n RAM, you can call my 256 MB pay-per-event Actors on Apify Store (@thanh81646): https://apify.com/thanh81646.

For a broken workflow or a custom pipeline, I refactor and build Python, Playwright, n8n, FastAPI, and Google Apps Script automations on small fixed-price milestones. My portfolio and contact info are at trungthanhtran.vercel.app.