AI & ML
Mitra-v2: Scaling synthetic pretraining for tabular foundation models
Amazon Science0 views
Much of the data that businesses run on lives in tables: the rows might represent customers, transactions, or interactions, and their “attributes” are listed in the columns. Predicting missing attributes — such as likelihood of customer retention or how much demand to expect next week — is one of the most common machine learning tasks in industry. The standard tools for this job, such as gradient-boosted decision trees, produce models tailored to specific tables. Every new table means choosing an algorithm, engineering features, tuning hyperparameters, and setting up a processing pipeline. Tabular foundation models promise to end that cycle: one pretrained model reads the labeled rows of a previously unseen table and predicts the rest, with no training run in-between. Last year we introduced Mitra, a tabular foundation model pretrained entirely on synthetic tables. It worked well on small and mid-sized tables but ran out of room on large ones, wide ones, and problems with many classes. Today we're introducing Mitra-v2, the second generation of the model, for both classification and regression. We describe it in a technical report, and we've released versions fine-tuned for both classification and regression, along with the inference and fine-tuning code, under the Apache-2.0 license on Hugging Face. Amazon Connect Customer, which provides agentic-AI systems for core business functions, led the effort. Amazon Connect Customer is adopting Mitra-v2 for prediction on structured business and customer-experience data, such as estimating how likely a business-relevant event is to occur given a customer's interaction history and operational context. On TabArena, a public benchmark of 51 real-world datasets, Mitra-v2 leads the single-model field. It produces a better result than Mitra-v1 on 32 of the 33 datasets where both models run natively, and it completes all 51 datasets within the benchmark's one-hour budget, which Mitra-v1 failed to do on more than a third of them. On ScoringBench, a public benchmark for regression models that predict a range of likely outcomes rather than a single number, fine-tuned Mitra-v2 ranks first. A large language model is pretrained on huge volumes of data, so when you give it a few examples in a prompt, it can generalize from them with no additional training. Tabular foundation models work the same way. We pretrain Mitra-v2 on millions of synthetic prediction problems, and at deployment time the labeled rows of a real table serve as the examples. The model reads them and predicts the unlabeled rows, a technique called in-context learning. Every pretraining table is synthetic: we use no real data, and in particular, no customer data. The model learns general principles about how columns relate to one another rather than the contents of any real dataset. The lesson of the original Mitra was that the quality and diversity of those synthetic tables are central to how well the model generalizes. Mitra-v2 takes that lesson further. The architecture is unchanged: a Transformer that attends both across the rows of a table and across its columns. What has changed is the size of the examples the model sees during pretraining. Mitra-v2 is pretrained on tables with 10 times as many rows and roughly three times as many columns as those used to train Mitra-v1, so it has seen tables the size of the ones businesses actually use. We also added a new family of synthetic tables. To train the original Mitra, we generated tables from structural causal models (SCMs), which are graphs of cause-and-effect relationships between columns, and from tree-based models like the ones data scientists fit in practice. Mitra-v2's new hybrid SCM prior builds a single causal graph in which each relationship can be a small neural network, a decision tree, a smooth function drawn from a Gaussian process, or a time-dependent autoregressive process, so one synthetic table can carry the mix of relationships found in real tables. A new hybrid optimizer keeps training stable across this wider range. At deployment time, two lightweight wrappers extend the model beyond its pretraining limits. For very wide tables, we use singular-value decomposition to produce a new table that fits in the allotted space, populated with synthetic values that capture most of the variance in the original data. Mitra-v2’s output layer has only 10 nodes, corresponding to 10 classes, so for tasks with more classes than that, we send each node’s output toa a hierarchical tree, which further refines the classification. By default, Mitra-v2 is briefly, automatically fine-tuned on each new table it sees, producing eight copies whose predictions are averaged, a standard technique called bagging. So it isn't quite a zero-shot model. The adaptation adds some computational overhead, but in return, it delivers higher accuracy and gives customers additional options. We evaluated Mitra-v2 on TabArena, a living benchmark maintained by an independent academic team. TabArena ranks methods by Elo, the rating system used in chess, here computed from head-to-head wins across datasets under a fixed one-hour budget. Mitra-v2 leads the single-model field, as shown below, with large gains over Mitra-v1 on both classification and regression. It also ranks among the leading methods on the independent TALENT benchmark of roughly 300 datasets, where it's strongest on classification tasks with more than 10 classes. Full results, including per-dataset comparisons, are in the technical report. For regression, Mitra-v2 predicts a full range of likely values for each row rather than a single number, so users get prediction intervals along with the point estimate. That’s the approach that ScoringBench evaluates, rewarding forecasts that are both accurate and well calibrated. On the ScoringBench leaderboard, fine-tuned Mitra-v2 again ranks first. Mitra-v2 shows that a tabular foundation model can make substantial gains by widening the diversity and range of its synthetic pretraining, without a deeper network. We expect richer synthetic priors and adaptive mixtures to push this further. The fine-tuned classification and regression models and code are open on Hugging Face and ready to use, and we invite researchers and practitioners to try Mitra-v2 on their own tables. Learn more Mitra-v2 classifier Mitra-v2 regressor Inference and fine-tuning code Mitra-v2 technical report Acknowledgments Yefan Tao, Raja Ghanta, Xiyuan Zhang, Xinyi Liu, Boran Han, Danielle Maddix Robinson, Haoyang Fang, Zhen Han, Jiading Gai, Xuanqing Liu, Michael Bohlke-Schneider, Bernie Wang, Gerald Friedland, Kevan Mah, Chris Lee
Source:https://www.amazon.science/blog/mitra-v2-scaling-synthetic-pretraining-for-tabular-foundation-modelsPreviousNew CNCF report: 75% of cloud-native developers ship code with AI. Jonathan Bryce explains why.NextDone with Debian Linux distros? Try Fedora next – 7 reasons why
Related posts
300 行以内搭一个「模型路由」:贵模型走实时,便宜模型走闲时
0dev.to Python

How we taught an AI to reply to YouTube comments in the creator's own voice
0dev.to Python
How 4 AI Agents Burned $47,000 in 11 Days: The Distributed Consensus Failure Nobody Talks About
0dev.to DevOps

Rockstar plans to make GTA VI available on Xbox Cloud Gaming at launch next week, likely restricting streaming exclusively to consoles; GTA VI runs at 30fps (Tom Warren/The Verge)
0Techmeme

Reddit launches Reddit Reviews, a built-in hub using AI to turn subreddit comments into product pages with pros, cons, and sentiment, starting with 10K items (Mia Sato/The Verge)
0Techmeme

Sources: Goldman Sachs expects Anthropic revenue of ~$52B in 2026 and $600B+ by 2030, operating profit in 2029, and positive free cash flow in 2030 (Financial Times)
0Techmeme