Every AI retrofit I've been on started in the same meeting. Someone senior says the core isn't "AI-ready," and the roadmap grows a rewrite before it grows a feature.
That framing is almost always wrong. On systems with 3+ years of production code, the thing blocking an AI feature has never been the age or shape of the core — it's been the data path feeding it and the release cadence around it.
An AI retrofit lands on three seams that your system already has:
- The data path — how records get in, validated and normalized.
- The async boundary — where expensive work leaves the request cycle.
- The output surface — where a probabilistic answer meets a user.
Audit those three in that order. The model call is the cheap part.
The rebuild instinct
Rebuilding a core to make it "AI-ready" is the most expensive way to test the cheapest hypothesis.
The cost asymmetry is brutal. Three seams are a few weeks of work each and they're independently revertible; a core rewrite is multiple quarters, freezes your roadmap, and you still don't know whether the AI feature is any good at the end of it.
And most of these features don't survive contact with users. S&P Global Market Intelligence found that 42% of companies scrapped most of their AI initiatives in 2025, up from 17% the year before, across a survey of 1,000+ respondents in North America and Europe — with an average of 46% of proofs-of-concept killed before they ever reached production. MIT's NANDA initiative put it harder in State of AI in Business 2025: only 5% of organizations were translating GenAI pilots into measurable operational or financial impact, based on 52 executive interviews, 153 survey responses and 300+ deployment reviews.
Read those two numbers together and the sequencing decides itself: if roughly half your PoCs die, you want the cheap, reversible version first.
There are real cases where the core is the blocker. I'll get to the four conditions at the end — they're narrower than the meeting suggests.
Seam one — ingestion is the actual project
The ingestion path, not the model call, is the expensive part of an AI retrofit. Getting data in, validated, normalized and replayable is where the months go. Swapping one model provider for another is an afternoon.
The data is where teams are actually stuck, and they know it. dbt Labs' 2025 State of Analytics Engineering found 56% of respondents named data quality as a problem while 80% of data practitioners were already using AI in their workflows. The 2026 State of Data Integrity and AI Readiness study from Drexel LeBow and Precisely — 505 data and analytics leaders at enterprises with 1,000+ employees — found 43% naming data readiness as the single biggest barrier to aligning AI with business objectives, even though 88% claimed they had that readiness. That gap between claimed and actual readiness is the retrofit's real schedule risk.
DORA's 2025 AI Capabilities Model reached the same place from the delivery side: "healthy data ecosystems" — high-quality, accessible, unified internal data — "substantially amplifies the positive influence of AI adoption on organizational performance." No amount of architecture fixes an ingest path that drops rows.
What a workable ingestion seam looks like:
- Staged, not monolithic. Upload → validation → processing → calculation, each stage its own unit with its own failure mode.
- Event-driven between stages. A queue or topic between steps, so a slow stage backs up instead of taking down the one before it.
- Replayable. Raw input is retained so you can re-run stage 3 after fixing stage 3, without asking the customer to re-upload.
- Idempotent. Every stage keyed so a redelivery is a no-op, because at-least-once delivery is what you actually get.
On a carbon management platform — 3 full-stack developers, German client, engaged since 2023 — this was the whole shape of the work. Users needed to stop entering emission data product-by-product, so the pipeline takes bulk CSV uploads through automated validation, processing and calculation stages on AWS Lambda, SNS, SQS and DynamoDB. As the case describes it, "data goes through all the stages, from a simple CSV file to ready-to-use calculations."
The idempotency guard is the piece most retrofits skip:
// One handler per stage. The dedupe key is the contract.
export const handler = async (event: SQSEvent) => {
for (const record of event.Records) {
const { uploadId, stage, rowHash } = JSON.parse(record.body);
const key = ${uploadId}#${stage}#${rowHash};
try {
await ddb.put({
TableName: 'pipeline_progress',
Item: { pk: key, startedAt: Date.now(), ttl: ttlIn(30, 'days') },
ConditionExpression: 'attribute_not_exists(pk)',
});
} catch (e) {
if (e.name === 'ConditionalCheckFailedException') continue; // already done
throw e;
}
const normalized = await validateAndNormalize(record);
await sns.publish({ TopicArn: NEXT_STAGE, Message: JSON.stringify(normalized) });
}
};
The thing to look at is the conditional put: it makes replay safe, which is what lets you re-run six months of ingest after you fix a normalization bug. Without it, "replay" means "double-count."
Limits, honestly: serverless staging fits bursty batch ingest. It does not fit sustained streaming or sub-second budgets, and AWS Lambda caps a single invocation at 900 seconds (default 3). If a stage can't be chunked under that, it belongs on a long-running worker instead.
Multi-tenancy makes the data path harder than the model
White-label and per-tenant deployment is where ingestion gets genuinely hard, and it has nothing to do with AI.
The carbon platform had to deploy across separate clusters depending on each client's location. That was handled with AWS TypeScript CDK plus SAM, so infrastructure configuration ships as code to any appropriate cluster rather than being hand-assembled per customer.
For a retrofit, the tenancy questions come before the model questions:
- Can you scope every training or context record to exactly one tenant, provably?
- Can a new tenant's pipeline be stood up from config, not from a runbook?
- Does a per-tenant deploy mean a per-tenant model config, prompt version and cost line?
If those three are answerable, your "not AI-ready" core is fine. If they aren't, that's a data-boundary problem — and rewriting the app layer won't touch it.
Seam two — inference belongs off the request path
Any expensive computation blocking a user interaction is one problem with known fixes. Inference is just the newest thing to fall into that bucket.
The budget is not negotiable. Google's Core Web Vitals threshold for Interaction to Next Paint is 200 ms at the 75th percentile for "good," and anything above 500 ms is "poor." No LLM call fits in that. So it doesn't go there.
The fixes are old and they all still apply:
- Queues and workers — the request enqueues and returns a job id.
- Precomputation — run inference on write or on a schedule, serve from a materialized result.
- Caching — key by input hash plus model version; a surprising share of prompts repeat.
- Virtualization — never render 10k rows to show 20.
- Payload reduction — send the delta, not the document.
- Parallel loading — fetch independent resources concurrently instead of in sequence.
A non-AI case makes the point cleanly, and I want to be explicit that there was no AI in it: on Vitus, a Danish platform built around large 3D models and heavy data volumes, where 3 front-end developers have worked alongside the in-house team since 2019, moving heavy computation to web workers, virtualizing lists with React Virtuoso, caching locally in IndexedDB, migrating to the SVF2 format and loading models in parallel produced published results of 5x smaller models, 2.5x faster load times and 5x faster multi-model loading — all from conventional front-end work, none of it AI. Cited for the pattern only: the same moves apply when the expensive thing is a model call instead of a 3D model.
The boundary itself is boring, which is the point:
// POST /insights -> 202, never blocks the interaction
app.post('/insights', async (req, res) => {
const jobId = crypto.randomUUID();
await jobs.insert({ id: jobId, tenantId: req.tenant.id, status: 'queued', input: req.body });
await queue.send({ jobId });
res.status(202).json({ jobId, poll: /insights/${jobId} });
});
// Worker: the only place the model is called
async function work({ jobId }) {
const job = await jobs.claim(jobId); // status -> running, with a lease
try {
const out = await model.infer(job.input, { timeout: 30_000 });
await jobs.complete(jobId, { output: out, modelVersion: MODEL_VERSION });
} catch (err) {
await jobs.fail(jobId, { error: err.message, attempt: job.attempt + 1 });
}
}
Note modelVersion written next to the output. When you swap models — and you will, repeatedly — that column is how you explain why last month's recommendations looked different.
What async costs you, honestly:
- Synchronous UX expectations break. Someone has to design the pending state, and "spinner forever" is not a design.
- Retries become your problem: leases, attempt caps, dead-letter handling.
- Partial results need a contract. Half the insights arriving is a product decision, not an implementation detail.
Seam three — put probabilistic output where a wrong answer is cheap
Retrofits survive when AI output sits beside deterministic data as a recommendation, not on top of it as a replacement.
Your users have already calibrated. In the 2025 Stack Overflow Developer Survey, 46% of developers actively distrusted the accuracy of AI tools against 33% who trusted it, with just 3% "highly trusting" — and 66% named "AI solutions that are almost right, but not quite" as their single biggest frustration. DORA's 2025 report, drawing on nearly 5,000 technology professionals, found 30% reporting little or no trust in AI-generated code even at 90% adoption. Design the surface for a user who assumes the output may be wrong, because that's who shows up.
Concretely, the output surface that works:
- The recommendation renders next to the existing chart or record, not instead of it.
- The deterministic path stays intact underneath and remains the system of record.
- Access is role-scoped, so a probabilistic answer only reaches people qualified to overrule it.
- Every recommendation carries its inputs, so "why did it say that" is answerable.
GazeHealth, a US healthcare platform we've worked on since 2019, is shaped this way. It's an EMR plus an analytics tool with about 15 charts — including a custom D3 Sankey and timeline, line and calendar tracking views — and AI/Big Data treatment recommendations were added alongside that existing analytics surface, not layered over it. Access is routed by role and company through AWS Cognito, to the point that pages a user can't reach aren't added to the routing object at all. Stack is React/TypeScript/D3 over NestJS and PostgreSQL.
The team composition is the detail I'd point at: 6 full-stack developers to 2 AI data scientists. A 3:1 ratio is the honest shape of a retrofit — it is mostly product engineering, with a minority of the effort in the model work everyone talks about.
Limits: no outcome metrics are published for that platform, so treat it as a structural example rather than an efficacy claim. And in regulated domains the output surface is where validation, auditability and explainability cost lands — budget for it as a workstream, not a checkbox.
The constraint teams misdiagnose as architecture
AI features need many iterations, so release safety gates a retrofit harder than code structure does. You will ship prompt changes, threshold changes and model swaps weekly. If a release takes two weeks and a rollback takes a day, the architecture isn't your bottleneck — the pipeline is.
MINT, an Italian adtech platform we've worked with since 2019, is the topology I'd copy for this. It grew to up to 40 engineers across 14 development teams, each with at least one QA professional, independent sandboxes per team and a dedicated UAT environment to absorb business changes during release cycles — supporting several releases per day.
Sourcing note, because it matters: our scope there was QA and team scaling — dedicated QA, dedicated dev team, staff augmentation. MINT's own ML and automation across 16 channels and 350+ integrations is the client's product, not our work. What transfers is the cadence and the team shape, nothing else.
The limit is real. Cadence without evaluation harnesses just ships regressions faster — and DORA's 2025 data already shows AI adoption carrying a negative relationship with software delivery stability. A golden set of inputs with expected outputs, run in CI on every prompt or model change, is the thing that makes fast releases safe rather than merely fast.
When rebuilding is right
I'd stop arguing and back a rewrite when any of these is true:
- No event stream or change data capture anywhere. If nothing in the system can tell you what changed and when, there's no data path to retrofit — you'd be building one inside a system that resists it.
- Structurally entangled tenant data. If tenant isolation can't be proven at the query level, per-tenant context and per-tenant models aren't safe to build.
- Hard real-time latency budgets. If the feature must answer inside a sub-second, non-deferrable interaction, the async boundary isn't available to you.
- Residency or compliance boundaries the deployment model can't express. If you can't deploy and store per jurisdiction, config-as-code won't rescue it.
Notice what isn't on the list: monolith, old framework, no microservices, "legacy."
What to measure first
Before the model bake-off, instrument these:
- Data completeness and freshness per tenant — what fraction of expected records arrived, and how stale is the newest.
- Ingestion failure rate by stage, so you know which stage is lying to you.
- p95 end-to-end latency from user action to available result, not model latency in isolation.
- Cost per inference per tenant, because pricing and rate limits are tenant-level problems.
- Human override rate — how often users reject the recommendation. It's your live accuracy proxy.
- Release frequency for prompt and model changes, which tells you whether iteration is actually possible.
Run the diagnostic in order: ingestion first, then the async boundary, then the output surface, then cadence. In most systems those four are weeks of work with reversible commits, and they answer whether the feature is worth having before you've spent a quarter on architecture.
If one of the four rewrite conditions holds, rebuild — deliberately, and for that reason. "The core isn't AI-ready" is not that reason. It's usually a data-quality and release-cadence complaint wearing an architecture costume.
