In a lot of LLM codebases, prompts live in f-strings scattered across the business logic. Someone tweaks a sentence to fix one case, another feature quietly gets worse, and nobody can say which version of the prompt is actually running in production.
Prompts are code. They change behaviour, they have bugs, and they deserve the same tooling as the rest of your application. Here's the setup I use.
1. Move prompts out of the business logic
Keep prompts in their own files with named placeholders, and load them by name and version:
from pathlib import Path from string import Template PROMPT_DIR = Path(__file__).parent / "prompts" def load_prompt(name: str, version: str = "v1") -> Template: return Template((PROMPT_DIR / f"{name}.{version}.txt").read_text()) prompt = load_prompt("summarise_ticket", "v2").substitute(ticket=ticket_text) Now a prompt change is a file change: easy to find, easy to diff, easy to roll back.
2. Version every change
Never edit a live prompt in place. Create summarise_ticket.v3.txt, switch to it, and keep v2 around until you're confident. Log the prompt name and version with every model call, so when an odd output shows up you know exactly which prompt produced it.
3. Build a small evaluation set
You don't need a research-grade benchmark. Twenty to fifty real inputs, each with the properties a good answer must have, will catch most regressions. Assert on properties, not exact text, because model output varies:
import pytest from app.llm import summarise_ticket CASES = [ ("Refund not received after 10 days", "refund"), ("App crashes when uploading a PDF", "crash"), ] @pytest.mark.parametrize("ticket, keyword", CASES) def test_summary_mentions_core_issue(ticket: str, keyword: str) -> None: summary = summarise_ticket(ticket) assert keyword in summary.lower() assert len(summary.split()) <= 40 Run it before and after every prompt change. If a "small wording fix" breaks three cases, you find out in review, not from users.
4. Review prompt diffs like code diffs
Prompt changes go through pull requests like everything else. The diff shows exactly which words changed, and the evaluation results show what that did. "Made the prompt better" isn't a review comment anyone can verify. "Fixed 4 failing cases, no regressions" is.
5. Keep model settings next to the prompt
A prompt tuned for one model at temperature 0.2 can behave very differently on another model at 0.8. Store the model name, temperature and max tokens alongside the prompt version so they always change together.
The payoff
None of this is complicated, but together it turns prompt work from guesswork into normal engineering: you can see what changed, measure whether it helped, and roll back in seconds when it didn't.
How do you manage prompts in your projects? I'd love to hear what's worked for you.
![UI Practice - Duolingo [1]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F8uawvr6a0e07ilktpkk7.png)

