Backend from First Principles, Week 1: How the Web Actually Talks

This is the first entry in a weekly series where I turn backend
concepts into small experiments instead of stopping at definitions.

A request can look almost too simple:

const response = await fetch("https://api.example.com/products"); const products = await response.json();

Two lines of JavaScript. But those lines hide a surprising amount of
work: the browser applies its security rules, a hostname must be
resolved, a connection is established, HTTP metadata travels with the
request, a server chooses a route, application code runs, and a response
makes its way back.

When something goes wrong, the browser might show a CORS error, a 401,
a 304, or a response that seems to have disappeared. Those messages
can feel unrelated. They become easier to debug when we stop treating an
API call as a single action and start tracing the steps around it.

That is the goal of this week's lab: make the invisible parts of an
HTTP request observable.

I am learning these concepts as part of my backend-from-first-principles
study. The explanations below are written in my own words, and the
examples are designed to be reproducible. The most useful part is not
memorising every header; it is learning what to inspect when the
behaviour differs from what I expected.

1. Follow one request from the browser to the server

Imagine a frontend requesting:

GET /api/products/1 HTTP/1.1 Host: api.example.com Accept: application/json

A typical request path might look like this:

DNS lookup: the client resolves api.example.com to an IP address. The answer may already be cached, so a DNS lookup is not necessarily performed for every request.
Connection and transport: the client establishes or reuses a connection. With HTTP/1.1 or HTTP/2, this commonly means TCP; HTTPS also involves TLS. HTTP/3 uses QUIC over UDP instead.
Network edge: depending on the deployment, the request may pass through a firewall, load balancer, CDN, or reverse proxy such as Nginx.
Application server: the request reaches the process listening for it. The application matches the method and path to a handler.
Application work: the handler validates input, applies business rules, and may read from or write to a database or another service.
Response: the server sends a status code, headers, and usually a body. The client then decides how to handle that response.

This is a mental model, not a fixed architecture. A small local Node
server may skip the proxy and cloud infrastructure entirely. A
production system may have several services between the browser and the
code that finally handles the request.

The practical debugging question is: which step did the request
reach? The browser's Network panel, server logs, and a command-line
client each reveal a different part of the story.

2. Why split frontend and backend?

Frontend code is delivered to a user's device and runs in an environment
the developer does not control. A user can inspect that code and modify
what happens locally. That makes the browser the wrong place to keep
secrets or enforce rules that must be trusted.

The backend is where a system can keep private credentials, enforce
permissions, validate important operations, and communicate with
databases using server-side connections. It can also perform work that
would be expensive or inappropriate for a user's device.

A useful rule of thumb:

The frontend handles presentation and user interaction.
The backend owns trusted application rules and mediates access to protected resources.
The database stores and retrieves persistent data, but should not be exposed directly to an untrusted browser.

This is not because frontend code is incapable of doing calculations. It
is because calculations and decisions that must be trusted need to be
enforced somewhere the user cannot simply rewrite.

CORS is related to browser security, but it is not a replacement for
authentication or authorization. We'll test that distinction shortly.

3. Read the HTTP message, not just the JSON

HTTP gives the client and server a shared format for describing a
request and its response. A simplified request might be:

PUT /api/users/123 HTTP/1.1 Host: example.com Content-Type: application/json Authorization: Bearer <token> {"name":"Akhil"}

It has a request line, headers, a blank line, and an optional body. The
response has a status line, headers, a blank line, and usually a body:

HTTP/1.1 200 OK Content-Type: application/json Cache-Control: no-store {"id":123,"name":"Akhil"}

The JSON is only the body. The status and headers tell the client how to
interpret it.

A few headers worth recognising:

Content-Type describes the representation being sent, such as application/json.
Accept tells the server what response formats the client can accept.
Authorization carries credentials or a token.
Cache-Control describes caching rules.
ETag identifies a particular representation version for cache validation.
Content-Encoding describes a transformation such as gzip compression.

Headers are not decoration. They are part of the protocol, and changing
one can change how a browser or server behaves.

4. HTTP methods: intent matters

The common methods express different kinds of operations:

Method Typical intent Idempotent?

GET Read a representation Yes

POST Submit data or create a Not generally
resource

PUT Create or replace the Yes
target resource with

the supplied

representation

PATCH Apply a partial Depends on the patch
modification operation

DELETE Remove the target Yes
resource

Idempotent does not mean "returns the same response every time." It
means that repeating the same request has the same intended effect on
server state as making it once.

For example, setting a product's price to 600 repeatedly can leave the
product at 600. Creating an order repeatedly can create a new order
each time. A repeated DELETE can return a different status on the
second call while still being idempotent if the resource remains
deleted.

This distinction matters when designing retry behaviour. A network can
fail after a server has processed a request but before the client
receives the response. The client may not know whether the operation
happened. Blindly retrying a non-idempotent operation can create
duplicates; real payment or order systems often need explicit
idempotency keys and server-side handling.

5. CORS: the browser may hide a response the server sent

Browsers apply the same-origin policy. An origin is the combination of
scheme, host, and port, so http://localhost:5173 and
http://localhost:3001 are different origins even though both use
localhost.

Cross-Origin Resource Sharing (CORS) is a mechanism through which a
server tells a browser which origins may read a response. For a request
that does not require preflight, the browser can send the request and
then check the response's CORS headers before exposing the response to
JavaScript.

That creates a useful distinction:

The server may receive a request and send a response.
The browser may still refuse to expose that response to the page's JavaScript.

Some requests trigger a preflight: the browser first sends an
OPTIONS request asking whether the origin, method, and requested
headers are allowed. A JSON PUT or a request with an Authorization
header commonly triggers this check.

A simplified preflight looks like this:

OPTIONS /api/products/1 HTTP/1.1 Origin: http://localhost:5173 Access-Control-Request-Method: PUT Access-Control-Request-Headers: content-type

If the server approves it, the browser can proceed with the actual
request. If the preflight fails, the browser normally does not send that
actual request.

CORS is not authentication. A command-line tool such as curl does
not enforce browser CORS rules. A server must still authenticate callers
and check their permissions; CORS alone does not protect an API from
direct requests.

6. Status codes are clues, not complete explanations

The status code gives the client a broad category of outcome:

2xx --- success: 200 OK, 201 Created, 204 No Content.
3xx --- redirection or cache validation: 301, 302, 304 Not Modified.
4xx --- request or client-side problem: 400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 409 Conflict, 429 Too Many Requests.
5xx --- server-side failure or temporary unavailability: 500 Internal Server Error, 503 Service Unavailable.

Two common confusions are worth clearing up:

401 means valid authentication credentials are missing or unacceptable; 403 means the server refuses the action. In real applications, exact behaviour depends on the authentication scheme and API design.
A 404 does not prove that a server is down. It often means the server is reachable but the requested route or resource was not found.

A status code is one clue. Combine it with the response body, request
headers, server logs, and the route that handled the request.

7. Caching: sometimes the best request is no request

Caching allows a client to reuse a response rather than download the
same data repeatedly. For example:

Cache-Control: max-age=10, public ETag: "abc123"

max-age=10 says that the response can be considered fresh for ten
seconds under the applicable caching rules. While a response is fresh,
the browser may reuse it without contacting the server.

After it becomes stale, the client can revalidate it:

If-None-Match: "abc123"

If the representation has not changed, the server can reply with
304 Not Modified. That response has no representation body; the client
reuses the body it already has. If the representation has changed, the
server sends a fresh response, commonly with 200 OK and a new ETag.

There are two behaviours to distinguish when debugging:

Fresh-cache reuse: the browser may make no network request at all.
Revalidation: the browser contacts the server, which can reply 304 or return a new representation.

Browser APIs such as fetch can make cache behaviour less obvious than
it looks in the Network panel. The lab below logs requests on the server
so we can compare what JavaScript receives with what actually reached
the server.

8. Representation, compression, uploads, and streaming

HTTP can carry more than one kind of payload. The headers help both
sides agree on how to interpret it.

Content negotiation: Accept and related headers express preferences about response formats, languages, or encodings. The server may choose among supported representations.
Compression: Accept-Encoding tells the server which content encodings the client supports. If the server compresses a suitable response, it identifies the encoding with Content-Encoding. Compression can reduce transferred bytes, but the result depends on the data, algorithm, and configuration.
Multipart uploads: multipart/form-data packages form fields and file content into parts separated by a boundary. It is common for browser file uploads.
Streaming: instead of waiting for the entire response body, a client can consume chunks as they arrive. Server-Sent Events (SSE) use text/event-stream for one-way updates from server to client.

Streaming is especially useful when waiting for the complete result
would make an interface feel slow. The client can begin displaying
progress or partial output while the server continues working. It is not
the same as simply returning a large JSON object: with ordinary JSON,
the client generally needs the complete body before parsing it as one
JSON value.

9. Routing and API versioning

A route maps an HTTP method and a path to a handler. The path can
identify a collection or a particular resource, while query parameters
often control filtering or pagination.

GET /api/books GET /api/users/123 GET /api/search?q=backend&page=2&limit=10 GET /api/users/7/products/1

The path and query string serve different purposes. /api/users/123
identifies a user; ?page=2&limit=10 commonly describes how to retrieve
a slice of a collection.

APIs also evolve. If clients depend on a response containing name and
price, changing those fields in place can break them. One option is to
introduce a new version:

/api/v1/products/1 returns id, name, and price.
/api/v2/products/1 returns id, title, cost, and currency.

Versioning is not free: it adds maintenance work and old versions need a
deprecation plan. The point is to make incompatible changes deliberately
rather than accidentally breaking every existing client.

The experiment: make the request visible

Definitions are useful, but they do not always predict what the browser
will show. For the rest of this post, I use a small local playground to
test a few behaviours that are easy to misunderstand:

A request can reach the server even when browser JavaScript cannot read the response.
A preflight can fail before the actual request is sent.
A cache hit and a 304 revalidation are different events.
Repeating an idempotent request does not necessarily produce an identical response.
Compression and streaming are visible in the bytes transferred and the timing of chunks.

The lab is deliberately small and uses Node's built-in modules. It is
not a production server; it is a controlled environment for learning how
HTTP behaves.

The server below is intentionally compact: it includes a small router,
an in-memory product, example status endpoints, cache validation, gzip
compression, an SSE endpoint, and a request log. The data is reset
whenever the process restarts.

One important limitation: the /api/status/:code route is a
demonstration tool that returns the requested status code. Clicking
403 or 409 there does not reproduce every real-world permission or
conflict scenario; it lets you inspect how a client displays different
status codes. The /api/secure route is also only a toy example and
does not validate a real token.

server.js

// backend-lab/server.js // Zero-dependency starter for the "Backend from First Principles" series. // // node server.js // API -> http://localhost:3001 // Page -> http://localhost:5173 (a different origin on purpose, so CORS is real) const http = require('http'); const fs = require('fs'); const path = require('path'); const zlib = require('zlib'); const crypto = require('crypto'); const API_PORT = Number(process.env.API_PORT) || 3001; const WEB_PORT = Number(process.env.WEB_PORT) || 5173; const ALLOWED_ORIGIN = process.env.ALLOWED_ORIGIN || `http://localhost:${WEB_PORT}`; // ---------- in-memory "database" ---------- let product = { id: 1, name: 'iPhone', price: 500 }; const orders = []; const catalog = Array.from({ length: 25 }, (_, i) => ({ id: i + 1, name: `Item ${i + 1}`, price: 100 + i * 10, })); const requestLog = []; // what the server really saw (browsers hide some of it) // ---------- helpers ---------- const etagOf = (s) => '"' + crypto.createHash('sha1').update(s).digest('hex').slice(0, 8) + '"'; // Send JSON. Compresses with gzip when the client allows it and the body is big. function send(req, res, status, data, headers = {}) { const raw = typeof data === 'string' ? data : JSON.stringify(data); let body = Buffer.from(raw); const h = { 'Content-Type': 'application/json', ...headers }; if (body.length > 1024 && /\bgzip\b/.test(req.headers['accept-encoding'] || '')) { body = zlib.gzipSync(body); h['Content-Encoding'] = 'gzip'; } res.rawBytes = Buffer.byteLength(raw); res.sentBytes = body.length; res.writeHead(status, h); res.end(body); } function readJson(req) { return new Promise((resolve, reject) => { let raw = ''; req.on('data', (c) => (raw += c)); req.on('end', () => { if (!raw) return resolve({}); try { resolve(JSON.parse(raw)); } catch { reject(Object.assign(new Error('Invalid JSON'), { status: 400 })); } }); }); } // ---------- tiny router: method + pattern -> handler ---------- const routes = []; function route(method, pattern, handler) { const keys = []; const regex = new RegExp( '^' + pattern.replace(/:(\w+)/g, (_, k) => (keys.push(k), '([^/]+)')) + '$' ); routes.push({ method, regex, keys, handler }); } // Static, path params, query params, nested resources route('GET', '/api/v1/products', ({ req, res, query }) => { const page = Math.max(1, Number(query.get('page')) || 1); const limit = Math.min(50, Math.max(1, Number(query.get('limit')) || 10)); send(req, res, 200, { page, limit, total: catalog.length, items: catalog.slice((page - 1) * limit, page * limit), }); }); // v1 supports HTTP caching (ETag + Cache-Control) route('GET', '/api/v1/products/:id', ({ req, res, params }) => { if (Number(params.id) !== 1) return send(req, res, 404, { error: 'Product not found' }); const body = JSON.stringify(product); const etag = etagOf(body); if (req.headers['if-none-match'] === etag) { res.writeHead(304, { ETag: etag }); return res.end(); } send(req, res, 200, body, { 'Cache-Control': 'max-age=10, public', ETag: etag }); }); // v2: same resource, new shape. Old clients keep using v1. route('GET', '/api/v2/products/:id', ({ req, res, params }) => { if (Number(params.id) !== 1) return send(req, res, 404, { error: 'Product not found' }); send(req, res, 200, { id: product.id, title: product.name, cost: product.price, currency: 'USD', }); }); // PUT replaces the whole resource (idempotent) route('PUT', '/api/v1/products/:id', ({ req, res, body }) => { if (typeof body.name !== 'string' || typeof body.price !== 'number') { return send(req, res, 400, { error: 'name (string) and price (number) are required' }); } product = { id: 1, name: body.name, price: body.price }; send(req, res, 200, product); }); // POST creates something new every time (not idempotent) route('POST', '/api/orders', ({ req, res }) => { orders.push({ id: orders.length + 1 }); send(req, res, 201, { totalOrders: orders.length }); }); route('GET', '/api/users/:userId/products/:productId', ({ req, res, params }) => send(req, res, 200, params) ); // Return any status code on demand route('GET', '/api/status/:code', ({ req, res, params }) => { const code = Number(params.code); if (!(code >= 200 && code <= 599)) return send(req, res, 400, { error: 'Use 200-599' }); if (code === 204 || code === 304) { res.writeHead(code); return res.end(); } send(req, res, code, { status: code }); }); // 401 without an Authorization header, 200 with one route('GET', '/api/secure', ({ req, res }) => { if (!req.headers.authorization) { return send(req, r...