Payment requests travel over networks that drop things. A phone on a train loses signal for a second, or a mobile SDK stops waiting. The request can reach your server, your server can charge the card, and the answer can vanish on the way back. The client sees an error and does the sensible thing: it sends the request again.
Retrying is the right reaction: the client can't tell a request that never arrived from one whose answer got lost. The trouble is on the server, which without an idempotency key can't tell a retry from a new payment. A payment is a POST, and a server that treats every POST as new work charges the card once per attempt.
So the customer taps Pay once, sees one receipt, and finds two charges of 25.00 on their statement. Your logs show two successful payments a second apart, each correct on its own, and the fix is a refund, an apology and possibly a chargeback, a dispute raised through the customer's bank.
The general fix is an idempotency key : a unique value the client generates once per payment and sends with every attempt. The server does the work the first time it sees a key, records the result, and answers every repeat with that same result.
The key needs a home that every copy of your server can reach and that can settle a race between two requests carrying it. If your payments already live in PostgreSQL, a table with a primary key on the idempotency key does both.
In this tutorial, you'll implement idempotency keys in Go with PostgreSQL for a small payments API, break each version on purpose, and load-test the result against a fake card provider that keeps its own ledger. It's a sequel to Distributed Locks in Redis with Go. Everything was tested with Go 1.27.2, pgx v5.11.0 and PostgreSQL 18.6.
Prerequisites
To follow along, you need:
- Go 1.25 or later, which pgx v5.11.0, the PostgreSQL driver, requires. The code also uses
crypto/rand.Text(Go 1.24) and method patterns such as"POST /payments"inhttp.HandleFunc, which arrived in Go 1.22. - Docker, to run PostgreSQL locally.
- Working knowledge of
net/http,context.Contextand SQL. No payments experience is needed. - Ports 5432, 8080 and 8090 free, and three terminal windows open in the project directory.
Step 1: Set Up a Fake Card Provider and a Checkout Client
This step builds a fake card provider on port 8090 and a checkout client that retries like a phone. The provider and the API each keep a ledger, because the bug lives in the gap between what the shop believes it charged and what the card was actually charged.
Start PostgreSQL and wait until it reports accepting connections:
docker run -d --name pg -e POSTGRES_PASSWORD=postgres -p 5432:5432 postgres:18 docker exec pg pg_isready -U postgres Output
/var/run/postgresql:5432 - accepting connections The fallback password is for this container only
The API reads DATABASE_URL and the provider reads PROVIDER_DATABASE_URL. When they're unset, both fall back to this container's postgres:postgres URL, so every command here runs as shown. In a real deployment, set the variables from a secret store.
Create one database per ledger:
docker exec pg psql -U postgres -c "CREATE DATABASE shop" -c "CREATE DATABASE provider" Output
CREATE DATABASE CREATE DATABASE Create the Go module and add pgx:
mkdir payments && cd payments go mod init example.com/payments go get github.com/jackc/pgx/v5@v5.11.0 Output
go: creating new go.mod: module example.com/payments go: to add module requirements and sums: go mod tidy go: added github.com/jackc/pgpassfile v1.0.0 go: added github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 go: added github.com/jackc/pgx/v5 v5.11.0 go: added golang.org/x/text v0.29.0 In the shop's schema, charges is the ledger and idempotency_keys holds one row per key. Its primary key is (customer_id, key), so two customers who send the same key never see each other's payments.
schema.sql
-- The shop's own ledger: one row per payment it believes it took. CREATE TABLE charges ( id bigserial PRIMARY KEY, customer_id text NOT NULL, amount bigint NOT NULL CHECK (amount > 0), provider_id text NOT NULL, created_at timestamptz NOT NULL DEFAULT now() ); -- One row per idempotency key: who claimed it, for which request, and what we answered. CREATE TABLE idempotency_keys ( customer_id text NOT NULL, key text NOT NULL, request_hash bytea NOT NULL, status text NOT NULL DEFAULT 'started', response_code int, response_body bytea, locked_at timestamptz NOT NULL DEFAULT now(), created_at timestamptz NOT NULL DEFAULT now(), PRIMARY KEY (customer_id, key) ); The provider's table is its record of what it actually charged. Its idempotency_key column is UNIQUE, and because a UNIQUE column treats NULLs as distinct by default, a charge sent without a key never conflicts with anything:
provider.sql
-- The fake card processor's own records: what it actually charged. CREATE TABLE card_charges ( id text PRIMARY KEY DEFAULT 'ch_' || substr(md5(random()::text), 1, 10), idempotency_key text UNIQUE, card text NOT NULL, amount bigint NOT NULL, created_at timestamptz NOT NULL DEFAULT now() ); Load both schemas:
docker exec -i pg psql -U postgres -d shop < schema.sql docker exec -i pg psql -U postgres -d provider < provider.sql Output
CREATE TABLE CREATE TABLE CREATE TABLE The fake provider takes one second per charge and declines the test card tok_declined with 402 Payment Required. With an Idempotency-Key, it behaves like Stripe: it takes a PostgreSQL advisory lock (a lock PostgreSQL holds on a number, here a hash of the key, until the transaction ends), answers 409 if another request with the same key is still running, and returns the original charge if the key was already used.
cmd/provider/main.go
// Command provider is a stand-in for a card processor such as Stripe or Adyen. // It charges slowly, declines one test card, and honors an Idempotency-Key. package main import ( "context" "encoding/json" "log" "net/http" "os" "time" "github.com/jackc/pgx/v5/pgxpool" ) type chargeRequest struct { Amount int64 `json:"amount"` Card string `json:"card"` } func main() { url := os.Getenv("PROVIDER_DATABASE_URL") if url == "" { url = "postgres://postgres:postgres@localhost:5432/provider?pool_max_conns=50" // the local Docker container; one connection per charge in flight } db, err := pgxpool.New(context.Background(), url) if err != nil { log.Fatal(err) } http.HandleFunc("POST /charges", func(w http.ResponseWriter, r *http.Request) { var req chargeRequest if err := json.NewDecoder(r.Body).Decode(&req); err != nil { http.Error(w, "bad request", http.StatusBadRequest) return } ctx := r.Context() tx, err := db.Begin(ctx) if err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) return } defer tx.Rollback(ctx) // Like Stripe, answer 409 while another request with the same key is running, // and return the original charge once it has finished. var key *string if k := r.Header.Get("Idempotency-Key"); k != "" { key = &k var locked bool tx.QueryRow(ctx, `SELECT pg_try_advisory_xact_lock(hashtext($1))`, k).Scan(&locked) if !locked { writeJSON(w, http.StatusConflict, map[string]string{"error": "request_in_progress"}) return } var id string if tx.QueryRow(ctx, `SELECT id FROM card_charges WHERE idempotency_key = $1`, k).Scan(&id) == nil { writeJSON(w, http.StatusCreated, map[string]string{"id": id}) return } } time.Sleep(time.Second) // the card network is slow if req.Card == "tok_declined" { writeJSON(w, http.StatusPaymentRequired, map[string]string{"error": "card_declined"}) return } // Without a key (NULL), nothing conflicts: every call is a new charge. var id string err = tx.QueryRow(ctx, ` INSERT INTO card_charges (idempotency_key, card, amount) VALUES ($1, $2, $3) RETURNING id`, key, req.Card, req.Amount).Scan(&id) if err == nil { err = tx.Commit(ctx) } if err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) return } writeJSON(w, http.StatusCreated, map[string]string{"id": id}) }) log.Println("provider listening on :8090") log.Fatal(http.ListenAndServe(":8090", nil)) } func writeJSON(w http.ResponseWriter, code int, v any) { w.Header().Set("Content-Type", "application/json") w.WriteHeader(code) json.NewEncoder(w).Encode(v) } payments/server.go holds the API's shared pieces. A payment carries an amount in cents and a card token, a string the provider's card form issues so your server never sees a card number. customerID reads a header as a stand-in for authentication; a real API answers 401 when there's no authenticated customer and caps the body with http.MaxBytesReader.
payments/server.go
// Package payments is a small payments API that charges a card through a provider // and records each payment in PostgreSQL. package payments import ( "bytes" "context" "encoding/json" "errors" "fmt" "net/http" "time" "github.com/jackc/pgx/v5/pgxpool" ) // PaymentRequest is what the checkout page sends. Amount is in cents; Card is a // token from the provider's card form, never a card number. type PaymentRequest struct { Amount int64 `json:"amount"` Card string `json:"card"` } // Payment is what the API returns for a successful charge. type Payment struct { ID int64 `json:"id"` Amount int64 `json:"amount"` ProviderID string `json:"provider_charge_id"` } // Server holds the API's dependencies. type Server struct { DB *pgxpool.Pool Provider *Provider } // ErrDeclined means the provider refused the card. That is a final answer, not a fault. var ErrDeclined = errors.New("card declined") // ErrInProgress means the provider is still working on a charge with the same key. var ErrInProgress = errors.New("charge in progress at the provider") // Provider is a client for the card processor. type Provider struct { URL string HTTP *http.Client } // Charge asks the provider to take amount from card. A non-empty key is sent as // the provider's Idempotency-Key, so repeating the call cannot charge twice. func (p *Provider) Charge(ctx context.Context, amount int64, card, key string) (string, error) { body, _ := json.Marshal(PaymentRequest{Amount: amount, Card: card}) req, err := http.NewRequestWithContext(ctx, http.MethodPost, p.URL+"/charges", bytes.NewReader(body)) if err != nil { return "", err } req.Header.Set("Content-Type", "application/json") if key != "" { req.Header.Set("Idempotency-Key", key) } resp, err := p.HTTP.Do(req) if err != nil { return "", err } defer resp.Body.Close() switch resp.StatusCode { case http.StatusCreated: var out struct{ ID string } err := json.NewDecoder(resp.Body).Decode(&out) return out.ID, err case http.StatusPaymentRequired: return "", ErrDeclined case http.StatusConflict: return "", ErrInProgress default: return "", fmt.Errorf("provider returned %s", resp.Status) } } // NewProvider returns a client whose timeout bounds every provider call. func NewProvider(url string) *Provider { return &Provider{URL: url, HTTP: &http.Client{Timeout: 5 * time.Second}} } // customerID stands in for authentication: a real API takes it from the session or token. func customerID(r *http.Request) string { return r.Header.Get("Customer-ID") } func writeJSON(w http.ResponseWriter, code int, v any) { w.Header().Set("Content-Type", "application/json") w.WriteHeader(code) json.NewEncoder(w).Encode(v) } func writeError(w http.ResponseWriter, code int, msg string) { writeJSON(w, code, map[string]string{"error": msg}) } func decode(r *http.Request) (PaymentRequest, error) { var req PaymentRequest if err := json.NewDecoder(r.Body).Decode(&req); err != nil { return req, err } if req.Amount <= 0 || req.Card == "" { return req, errors.New("amount and card are required") } return req, nil } The checkout client plays the phone. Its first attempt gives up after 600 ms; later attempts wait 3 seconds, with a 300 ms backoff that doubles, for up to 4 attempts. -key sends one key per payment (the default), one per attempt (new) or none, and -parallel fires identical copies at once.
cmd/pay/main.go
// Command pay is the checkout client: it sends one payment and retries the way a // mobile app does when the network is slow. package main import ( "bytes" "crypto/rand" "flag" "fmt" "io" "net/http" "strings" "sync" "time" ) func main() { amount := flag.Int("amount", 2500, "amount in cents") card := flag.String("card", "tok_visa", "card token") keyMode := flag.String("key", "once", "once: one key per payment; new: a new key per attempt; none: no key") keyValue := flag.String("key-value", "", "use this key instead of a random one") firstTimeout := flag.Duration("first-timeout", 600*time.Millisecond, "the first attempt gives up after this long (the phone loses signal)") timeout := flag.Duration("timeout", 3*time.Second, "how long later attempts wait for an answer") attempts := flag.Int("attempts", 4, "attempts before giving up") parallel := flag.Int("parallel", 1, "send this many copies at the same moment") flag.Parse() key := *keyValue if key == "" { key = rand.Text() // 26 random characters from crypto/rand } body := fmt.Sprintf(`{"amount": %d, "card": %q}`, *amount, *card) send := func(attempt int) (int, string, error) { req, _ := http.NewRequest(http.MethodPost, "http://localhost:8080/payments", strings.NewReader(body)) req.Header.Set("Content-Type", "application/json") req.Header.Set("Customer-ID", "cus_1001") switch *keyMode { case "once": req.Header.Set("Idempotency-Key", key) case "new": req.Header.Set("Idempotency-Key", rand.Text()) } wait := *timeout if attempt == 1 { wait = *firstTimeout } resp, err := (&http.Client{Timeout: wait}).Do(req) if err != nil { return 0, "", err } defer resp.Body.Close() b, _ := io.ReadAll(resp.Body) extra := ""Source：https://dev.to/shahidyousuf/how-to-implement-idempotency-keys-in-go-with-postgresql-52eRelated posts
I read all 308 open jobs at xAI: 54% are in its data center team, and most of them are in Memphis
Claude Code marketing analytics with an app-scoped key
I red-teamed my own support bot with 30 scenarios. It failed 7 — here's the harness.
The Log That Told Everyone the Password: Redacting Secrets in n8n Execution Logs
Your AI Portal Isn't Ready Until a Stranger Can Recover from Failure
