The data scientist who built it has moved to another company, and the model came to your team in the reorganisation. It predicts which customer orders are likely to be returned, and the warehouse uses those predictions every morning. You are a backend developer. You have never trained a model in your life, and now this one is yours.

Your first thought might be to sign up for a course and start with the mathematics. That will help eventually. It is not the right first step, because the theory will not tell you anything about this particular model, and this particular model is the thing people rely on tomorrow morning.

Start instead by learning how it is judged. Somewhere there should be a record of how well it performed when it was built. Find the data it was tested on, the numbers that were reported and what those numbers mean in plain terms. If it says the model is right ninety percent of the time, ask what the other ten percent looks like. Wrongly flagging a good order costs something different from missing a bad one.

Then follow the data in and out. Where do the inputs come from each night, and what happens if one of them is missing? Where do the predictions go, and who reads them? Treat it like any other system you inherit. Most problems with models in production are not clever mathematical problems. They are a column that changed meaning, a feed that stopped updating, or a world that slowly drifted away from the data the model learned from.

Talk to the people who use it. The warehouse supervisor will know when the predictions feel wrong, often long before any metric does. Ask what they do when they disagree with it.

If the person who built it is still reachable, send a short, specific message. People are usually glad to help with something they made.

Now take the course. The theory will land much better once you have a real model in front of you, and every chapter will answer a question you have already asked.

Learning a new field through something you are responsible for is uncomfortable. It is also one of the fastest ways to learn it well.

– Asael Shinder