Caddy 2.11.6 turns a silent config failure on /load into an HTTP 400

I posted a broken load-balancer config to Caddy's admin API and it told me the reload had worked. The server behind it stopped answering requests entirely. The API still said 200.

That was on Caddy 2.11.4. I wanted to know whether the two patch releases that followed it, 2.11.6 (1 October) and 2.11.7 (3 October), actually closed that gap, because their changelogs claim "stricter config validation" and a fix for a Unix socket that used to hang on reload. Both claims turned out to be true, and more precise than I expected on the first one and more dramatic than I expected on the second.

The config that lied about succeeding

Caddy's admin API has a /load endpoint: POST a JSON config, it replaces the running one. I sent this, which gives a weighted_round_robin policy a negative weight:

"load_balancing": { "selection_policy": { "policy": "weighted_round_robin", "weights": [1, -3, 2] } } 

On Caddy 2.11.4:

$ curl -s -o /tmp/out.json -w 'HTTP %{http_code}\n' -X POST \ -H 'Content-Type: application/json' --data @bad_weighted.json \ http://localhost:2019/load HTTP 200 

Then I curled the server's actual listener on port 8080:

$ curl -s -o /dev/null -w 'HTTP %{http_code}\n' http://localhost:8080/ curl: (52) Empty reply from server 

The admin API said the new config was live. The thing it was supposedly serving had stopped responding. On Caddy 2.11.6 and 2.11.7, the same POST gets this instead:

HTTP 400 {"error":"loading config: loading new config: loading http app module: provision http: server srv0: ... http.reverse_proxy.selection_policies.weighted_round_robin: invalid configuration: weighted_round_robin requires at least one upstream with a positive weight"} 

And the server that was already running kept running. I reloaded a known-good config on 2.11.7, confirmed it was serving, sent the broken one, got the 400, and checked again:

If you have anything automated that pushes Caddy config over this endpoint and only checks for a 2xx, on 2.11.4 and earlier that check was lying to you.

Caddy logs every admin API request, which is the other way to catch this if you're watching logs rather than status codes. On 2.11.4, the log line for the bad load reads:

{"logger":"admin.api","msg":"load complete"} 

On 2.11.7, the same POST produces:

{"level":"error","logger":"admin.api","msg":"request error","error":"... weighted_round_robin: invalid configuration: weighted_round_robin requires at least one upstream with a positive weight","status_code":400} 

"Load complete" is exactly what it sounds like: as far as 2.11.4 is concerned, nothing went wrong. A log-based alert watching for admin.api errors would have caught this on the newer version and found nothing at all to catch on the older one, same as checking the status code would.

It's a specific list, not a general re-check

The changelog names several newly-rejected categories, and I tested two more of them directly, because I wanted to know whether "stricter" meant "all of it" or "this list, no more."

A Caddyfile with a duplicate uri inside forward_auth:

forward_auth 127.0.0.1:9099 { uri /auth1 uri /auth2 } 

On 2.11.4, caddy adapt accepts this with no warning and silently keeps the second value. On 2.11.7:

$ caddy adapt --config Caddyfile.forwardauth {"level":"error","msg":"parsing caddyfile tokens for 'forward_auth': cannot re-declare uri: /auth2, at /Caddyfile.forwardauth:4"} 

Exit code 1. A map handler with a destination placeholder missing its closing brace behaved the same way: 200 on 2.11.4, and on 2.11.7:

{"error":"... http.handlers.map: destination 0 must be a placeholder and only a placeholder, but got '{http.vars.unterminated'"} 

What I got wrong on the way

I assumed "stricter validation" meant any previously-accepted mistake would now be caught, so I tried two more cases expecting the same result and got surprised twice.

First, duplicate @id values on two routes. I expected this to be a new catch. It wasn't — Caddy 2.11.4 already rejects it:

{"error":"loading config: indexing config: duplicate ID 'r1' found at /config/apps/http/servers/srv0/routes/0 and /config/apps/http/servers/srv0/routes/1"} 

Same error, same HTTP 400, on both versions. Second, a file_limit set to 3.5 where the schema wants an integer. Also already rejected on 2.11.4, by Go's own JSON decoder rather than anything Caddy added:

{"error":"... json: cannot unmarshal number 3.5 into Go struct field Browse.browse.file_limit of type int"} 

So the fix is a named list of specific gaps — duplicate named_routes, duplicate forward_auth subdirectives, bad weighted_round_robin weights, ambiguous map inputs, and a handful of others — not a blanket pass over every handler. Worth knowing before you assume a config mistake you haven't tested is now covered.

I also checked a boundary case on the weight fix, because "stricter" sometimes means "stricter than it should be." A weight of 0, as opposed to negative, is accepted on 2.11.7:

"weights": [1, 0, 2] 

returns 200. The rule is "at least one positive weight," not "no zeroes," so taking an upstream fully out of rotation with a zero weight still works.

The Unix socket that took two minutes to let go

This is the one I didn't expect to be able to measure cleanly, and it was the most striking result of the day.

The 2.11.7 changelog says a reload that moves a server's listener off a Unix socket used to leave the old socket file in place and leave clients connecting to it hanging for "about two minutes," fixed by closing it immediately. I set up Caddy 2.11.6 listening on unix//tmp/sock_a, confirmed it answered, then POSTed a new config that moves the same server to unix//tmp/sock_b:

$ curl -s -o /dev/null -w 'HTTP %{http_code}\n' -X POST --data @b2.json \ http://localhost:2019/load HTTP 200 

Then I polled the old socket path every half second or so. For roughly the first 100 seconds, every attempt timed out rather than failing fast:

curl: (28) Operation timed out after 1001 milliseconds with 0 bytes received 

Around the 110-second mark it flipped to a connection reset, and by about 120 seconds it was a clean, instant refusal. The socket file itself (/tmp/sock_a) was still sitting on disk the entire time — ls kept finding it right up until the connections started failing fast. That is close enough to the changelog's "about two minutes" that I don't think I got lucky with the timing; it's a real, repeatable window.

On Caddy 2.11.7, running the identical reload:

$ ls /tmp/sock_a ls: cannot access '/tmp/sock_a': No such file or directory $ curl -s --unix-socket /tmp/sock_a http://localhost/ curl: (7) Failed to connect to localhost port 80 after 0 ms: Couldn't connect to server 

The file was gone and the refusal was instant, checked immediately after the /load call returned and ten more times over the next three seconds. No timeout anywhere in that window.

If you reload Caddy in a loop as part of a deploy pipeline and something downstream reconnects on a fixed interval, the old version could leave that something stuck for two minutes against a socket nobody is serving on any more. That is a real production cost, not a cosmetic log message.

Run it yourself

This needs Docker and about ten minutes once the images are pulled.

docker pull caddy:2.11.4 && docker pull caddy:2.11.7 cat > good.json <<'EOF' {"apps":{"http":{"servers":{"srv0":{"listen":[":8080"],"routes":[ {"handle":[{"handler":"static_response","body":"hello-good"}]} ]}}}}} EOF # Deliberately invalid: a negative weight cat > bad_weighted.json <<'EOF' {"apps":{"http":{"servers":{"srv0":{"listen":[":8080"],"routes":[{"handle":[{ "handler":"reverse_proxy", "load_balancing":{"selection_policy":{"policy":"weighted_round_robin","weights":[1,-3,2]}}, "upstreams":[{"dial":"127.0.0.1:9001"},{"dial":"127.0.0.1:9002"},{"dial":"127.0.0.1:9003"}] }]}]}}}}} EOF for v in 2.11.4 2.11.7; do docker run -d --name caddy-$v caddy:$v caddy run --resume=false >/dev/null sleep 1 docker cp good.json caddy-$v:/good.json docker cp bad_weighted.json caddy-$v:/bad_weighted.json echo "== $v ==" docker exec caddy-$v curl -s -o /dev/null -w 'load good: HTTP %{http_code}\n' \ -X POST -H 'Content-Type: application/json' --data @/good.json http://localhost:2019/load docker exec caddy-$v curl -s -o /dev/null -w 'traffic after good: HTTP %{http_code}\n' \ http://localhost:8080/ docker exec caddy-$v curl -s -o /dev/null -w 'load bad: HTTP %{http_code}\n' \ -X POST -H 'Content-Type: application/json' --data @/bad_weighted.json http://localhost:2019/load docker exec caddy-$v curl -s -o /dev/null -w 'traffic after bad: HTTP %{http_code}\n' \ http://localhost:8080/ docker rm -f caddy-$v >/dev/null echo done 

This is the actual output from running it:

== 2.11.4 == load good: HTTP 200 traffic after good: HTTP 200 load bad: HTTP 200 traffic after bad: HTTP 000 == 2.11.7 == load good: HTTP 200 traffic after good: HTTP 200 load bad: HTTP 400 traffic after bad: HTTP 200 

On 2.11.4, the bad config reports 200 and the server stops answering (000, connection failure). On 2.11.7, the bad config is refused with 400 and the server carries on serving hello-good exactly as before. The /load response alone tells you which version you're running, without needing a second request against the data path at all.

What to do with this

If anything you run pushes configuration to Caddy's admin API — a CI job, a control plane, a sidecar — and it treats a 2xx from /load as "the new config is live and correct," go check which version you're running. Before 2.11.6, that check could pass while the server underneath it was already broken. After 2.11.6, it still can't tell you about every possible mistake, only the specific ones listed in the changelog, so don't drop your own post-reload health check just because the API got stricter.

And if you reload Caddy on a schedule and something else in your infrastructure dials a Unix socket Caddy used to own, check your Caddy version before assuming a two-minute stall is your code's fault.