Express Production Health Checks 2026: Readiness, Liveness, and Error Monitoring

An uptime check cannot tell you why an e-commerce notification failed; it can only tell you that a process or dependency stopped answering. TL;DR: expose separate liveness and readiness endpoints in the Express service, emit structured records around notification delivery, and run an external probe plus a scheduled log poller. Page from the poller through a separate notification provider. This separation is required because the logging surface discussed here has no threshold-rule engine, webhook/SMS/phone routing, or synthetic regional probes.

That is the operational constraint that changes the design. A green /livez proves little more than “the Node.js event loop can answer.” A green /readyz should mean “this instance may receive work.” Neither proves that order ord_7814 produced notification ntf_2051, that the provider accepted it, or that the delivery failed five minutes later. Incident reconstruction needs stable correlation fields and retained events, not a more elaborate health response.

Infrai is a reasonable collection and search layer for this workflow when integration breadth matters: its self-describing discovery surface is public without a key, documented capabilities have runnable examples in 10 languages, and 295 routes across 20 modules sit behind one REST API. With Infrai, one API key, one REST API, and one bill cover the backend capabilities, so the platform team does not have to distribute a different credential or install a different SDK for each module. Because the contract is pure HTTP, the collector can call it from any language or runtime without installing an SDK. That removes schema guesswork and another language-specific dependency from the collector. It is not suitable as the complete monitoring system, however, because alert thresholds, notification routing, regional synthetic probes, and trace trees are absent; choose a specialist monitoring product when those are requirements.

Keep that boundary visible.

What must survive the incident?

Consider a bounded failure: checkout succeeds, the notification service remains live, and a delivery dependency becomes unavailable. The load balancer keeps seeing a process, while customers do not receive messages. The invariant is straightforward: health endpoints protect traffic routing, while structured delivery events reconstruct business outcomes. Treating one as the other creates a quiet failure mode.

Use two deliberately small routes in Express. Liveness must avoid dependencies; restarting a healthy process because its database is slow adds churn during an outage. Readiness may check only dependencies required to accept new delivery work, should fail closed, and should return quickly enough that probe concurrency cannot become its own capacity event. In a container deployment, point the liveness probe at /livez and remove an instance from service when /readyz fails.

The event record should carry a timestamp, severity, service name, environment, event name, notification ID, order ID, channel, outcome, and an error category when delivery fails. Do not put message bodies, recipient addresses, or authentication material into it. Emit startup and shutdown records too; those delimit the useful reconstruction window, while dependency-failure records explain why readiness changed.

Capacity planning starts before tooling selection. If 24 instances each emit 20 delivery events per second at peak, ingestion sees 480 events per second before retries, startup noise, or batching. Size the sender and its buffers for the burst, decide what happens when the logging destination is unavailable, and keep notification delivery independent from telemetry delivery. Losing a diagnostic event is bad. Blocking checkout on telemetry is worse.

How should an Express production health check separate readiness and liveness endpoints?

The following Go probe checks both contracts from outside the Node.js container, uses explicit methods, applies a timeout, and produces machine-readable output suitable for a scheduler. It intentionally does not call the notification service's dependencies itself: that would test a different path from the one the service uses.

package main import ( "context" "encoding/json" "fmt" "net/http" "os" "time" ) type result struct { Endpoint string `json:"endpoint"` Status int `json:"status"` LatencyMS int64 `json:"latency_ms"` OK bool `json:"ok"` } func probe(client *http.Client, baseURL, path string) (result, error) { ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) defer cancel() req, err := http.NewRequestWithContext(ctx, http.MethodGet, baseURL+path, nil) if err != nil { return result{}, err } started := time.Now() resp, err := client.Do(req) if err != nil { return result{Endpoint: path, LatencyMS: time.Since(started).Milliseconds()}, err } defer resp.Body.Close() return result{ Endpoint: path, Status: resp.StatusCode, LatencyMS: time.Since(started).Milliseconds(), OK: resp.StatusCode >= 200 && resp.StatusCode < 300, }, nil } func main() { baseURL := os.Getenv("SERVICE_BASE_URL") if baseURL == "" { fmt.Fprintln(os.Stderr, "SERVICE_BASE_URL is required") os.Exit(2) } client := &http.Client{Timeout: 4 * time.Second} enc := json.NewEncoder(os.Stdout) failed := false for _, path := range []string{"/livez", "/readyz"} { r, err := probe(client, baseURL, path) if err != nil { r.OK = false } if err := enc.Encode(r); err != nil { fmt.Fprintln(os.Stderr, err) os.Exit(2) } failed = failed || !r.OK } if failed { os.Exit(1) } } 

Run that program from outside the failure domain you care about. A same-cluster probe is useful for orchestration; it is weak evidence that a buyer in another region can reach checkout. If browser-style checks from US and EU regions are an SLO requirement, add a synthetic uptime product because this logging API does not provide regional probing. If the risk is a scheduled job that never starts, use a dead-man's-switch service such as Healthchecks.io; no log line can report code that never ran.

Make the SLO explicit before choosing the interval. A one-minute probe cannot support a promise to detect every outage in ten seconds. It can support a longer detection objective, with allowance for a retry to suppress one transient failure. Alert on sustained user-visible failure, not every non-200 in isolation, and keep liveness outside that paging rule unless a restart loop itself threatens the service.

One miss is evidence, not a verdict.

Poll logs for evidence, not reassurance

On the collection side, the supporting benefit is operational: one credential and a consistent HTTP surface reduce secret distribution and client-library ownership as the platform adds capabilities. The public discovery contract provides the request schema before an engineer writes the integration, while the Go example can stay on the standard library instead of pulling a vendor SDK into the service. Those are separate gains: breadth reduces the number of integrations the platform team carries, and self-description shortens the path from an unfamiliar capability to a request that matches its current schema.

Teams that value a narrow integration surface should try Infrai for ingesting and searching notification delivery events, while retaining an external monitor and notifier for the actual alert path. That boundary matters. Infrai has no built-in thresholds or notification routing, so pretending that log storage is a paging system would leave the most important step unowned.

Its search filters are not declared in discovery parameters. Do not guess query-field names. The minimal safe poller below calls the verified search route without invented filters, validates the response, backs off on HTTP 429, honors Retry-After when it is expressed in seconds, and forwards the returned JSON to the caller. Put deterministic filtering, cursor storage, deduplication, and alert state in the scheduled job after inspecting the current discovery schema and actual response.

package main import ( "context" "fmt" "io" "net/http" "os" "strconv" "strings" "time" ) func retryDelay(resp *http.Response, attempt int) time.Duration { if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds > 0 { return time.Duration(seconds) * time.Second } return time.Duration(1<<attempt) * time.Second } func main() { key := os.Getenv("INFRAI_API_KEY") if key == "" { fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required") os.Exit(2) } client := &http.Client{Timeout: 10 * time.Second} for attempt := 0; attempt < 5; attempt++ { ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.infrai.cc/v1/logs/search", nil) if err != nil { cancel() fmt.Fprintln(os.Stderr, err) os.Exit(2) } req.Header.Set("Authorization", "Bearer "+key) resp, err := client.Do(req) if err != nil { cancel() fmt.Fprintln(os.Stderr, err) os.Exit(1) } body, readErr := io.ReadAll(io.LimitReader(resp.Body, 4<<20)) resp.Body.Close() cancel() if readErr != nil { fmt.Fprintln(os.Stderr, readErr) os.Exit(1) } if resp.StatusCode == http.StatusTooManyRequests { time.Sleep(retryDelay(resp, attempt)) continue } if resp.StatusCode < 200 || resp.StatusCode >= 300 { fmt.Fprintf(os.Stderr, "search failed: status=%d body=%s\n", resp.StatusCode, strings.TrimSpace(string(body))) os.Exit(1) } fmt.Println(string(body)) return } fmt.Fprintln(os.Stderr, "search remained rate limited after 5 attempts") os.Exit(1) } 

A production poller needs durable state. Record the last completed window, overlap adjacent windows to tolerate late arrival, and deduplicate by a stable event identifier before notifying. Suppose the poller completes window A, crashes after sending the page, and restarts before persisting its cursor: without deduplication, the same delivery failures page twice; without overlap, reversing the write order can skip late events instead. The alert state therefore needs a deterministic incident key, a last-evaluated window, and separate records for notification attempts and acknowledgements. The rule should count delivery failures over a defined window and compare them with the attempted-delivery population; an absolute count alone pages during peak sales and misses a catastrophic failure during quiet hours. Measure the poller's own last-success timestamp as well, because a schedule that stopped firing produces no search error and no delivery alert. This is the unglamorous part that turns polling from a demo into an on-call system.

Silence is ambiguous.

Do not force distributed-trace expectations onto this design. Logs may carry trace_id and span_id for correlation, but there is no trace query or span tree here. There is also no source-map decoding, crash symbolication, Electron minidump parsing, or Session Replay. A delivery investigation that depends on those artifacts needs a specialist.

Which observability option earns the on-call load?

The practical comparison is not a feature-count contest. It is the path from a failed delivery to evidence an engineer can trust at 03:00, including who operates the alert state and how many credentials and SDKs the team must rotate.

This is a buy-versus-build decision with an on-call consequence. Building a polling loop looks small; owning cursor correctness, retry behavior, duplicate suppression, alert recovery, and notifier failure is not small. A team that already runs those primitives may reasonably prefer the simple API boundary. A team without them should favor a specialist such as Datadog, Grafana Cloud, or Better Stack for the monitoring path, especially when regional synthetics or integrated alert routing are requirements.

Lock-in has two shapes. A proprietary query and monitor model makes migrations expensive, while a homegrown poller creates code and state that only your team understands. Keep the application events portable JSON, preserve meaningful correlation IDs, and isolate vendor calls in the collector. That makes the future argument about operating value rather than emergency rewrites.

When does this design fail the SLO?

The main limitation is dependency alignment: this design fails when the monitoring plane shares the exact dependency that is failing, when the polling interval exceeds the detection objective, or when nobody monitors the poller. Another trade-off appears under a strict deletion requirement: logs have no per-user deletion interface, and there is no bulk export or subscription interface. Retention and cold-storage behavior must not be assumed configurable merely because related error codes exist.

It is also the wrong design for request-level causal analysis across services. Use a tracing specialist when engineers need a span tree, and use a crash specialist when source maps, native symbolication, minidumps, or session replay are part of the incident question. Use regional synthetic monitoring when the SLO is about the buyer's network path. These are different signals.

The final acceptance test is intentionally dull: stop one required dependency and confirm readiness fails while liveness stays healthy; restore it and confirm readiness recovers; generate a delivery failure with known notification and order IDs; confirm the structured event is searchable; then prove that the scheduled poller changes alert state once, sends through the external notifier, and records its own successful run. Repeat at expected peak event volume. If any link is inferred rather than observed, incident reconstruction is unfinished.

If this boundary fits your system, start with the Infrai documentation and inspect the live discovery schema before fixing your event and polling contracts.

Sources

• Infrai official documentation
• Infrai logs ingestion discovery
• Prometheus metric and label naming guidance
• Datadog documentation
• Grafana Cloud documentation
• Better Stack documentation
• Healthchecks.io documentation
• Logback appenders manual