Our stock API has one endpoint that marketplace partners call every few minutes: GET /stock with a sku parameter. We documented it as a comma separated list, sku=A,B,C, and the Flask handler read request.args.get('sku') and split it on commas.

In October our biggest marketplace partner moved to a new version of their integration library. It serialises a list the way Python's requests library does when you pass one as a parameter, by repeating the key: sku=A&sku=B&sku=C. Nobody told us, and nothing required them to.

Flask's request.args is a Werkzeug MultiDict. It keeps every value of a repeated key, but get returns only the first, and so does indexing it with square brackets. Our handler received A, split it on commas, found one SKU, and returned stock for one SKU with a 200. The partner's integration treated any SKU missing from our response as zero stock. By ten in the morning three hundred and eighty of our listings were marked out of stock, and the only error anywhere was a sales report that looked quiet.

Our logs showed nothing wrong, because nothing was wrong as far as the handler knew. The access log had the full query string, and that is where we saw it, an hour later, by comparing what was asked with what we sent back.

The handler now reads request.args.getlist('sku') and splits each value on commas, so both forms work and the partner did not have to change anything. That fixed the symptom. The more important change was in the response. It now contains an entry for every SKU that was requested, and a SKU we do not recognise comes back with a status of unknown rather than being left out. A partner reading our answer can no longer mistake silence for zero. The endpoint also rejects any query parameter it does not know, with a 400 that names it, so a renamed parameter fails loudly instead of being ignored.

We added a contract test for each partner that calls the endpoint with the same library and version they run in production, and they tell us now before they upgrade it.

Two clients asked the same question in two spellings. We answered one of them correctly and the other one partly, and a partial answer looked exactly like a complete one.

– Sergey Shinder