ML.NET
A deep-dive walkthrough of ML.NET — Microsoft's open-source, cross-platform machine learning framework for .NET — covering MLContext, data loading and IDataView, preprocessing and feature engineering, how training pipelines (estimators and transformers) actually work, model training and evaluation, saving/loading models, making predictions safely in production, AutoML, and complete worked examples for classification, regression, clustering, recommendation systems, and anomaly detection.
Table of Contents
- Introduction
- ML.NET Fundamentals
- MLContext
- Data Loading
- IDataView
- Data Preprocessing
- Feature Engineering
- Training Pipelines: Estimators and Transformers
- Model Training
- Model Evaluation
- Model Saving and Loading
- Model Prediction
- Classification
- Regression
- Clustering
- Recommendation Systems
- Anomaly Detection
- AutoML
- Common Pitfalls
- Quick Reference Table
- Conclusion
Introduction
ML.NET lets .NET developers build, train, and run machine learning models entirely in C# or F#, without switching to Python or leaving the .NET ecosystem. A model trained with ML.NET is just a file you load into an ordinary .NET application — a web API, a worker service, a desktop app — and call like any other dependency.
var mlContext = new MLContext(seed: 0); IDataView data = mlContext.Data.LoadFromTextFile<HouseData>("houses.csv", hasHeader: true, separatorChar: ','); var pipeline = mlContext.Transforms.Concatenate("Features", "Size", "Bedrooms") .Append(mlContext.Regression.Trainers.Sdca(labelColumnName: "Price")); ITransformer model = pipeline.Fit(data); // TRAINING var engine = mlContext.Model.CreatePredictionEngine<HouseData, HousePrediction>(model); var result = engine.Predict(new HouseData { Size = 1800, Bedrooms = 3 }); // INFERENCE Those six lines contain the whole ML.NET workflow: load data → build a pipeline → fit it → predict. Everything else in this guide is detail on one of those steps. If the vocabulary here (features, labels, training vs. inference, overfitting, precision/recall) is unfamiliar, this series' AI/ML Fundamentals guide covers the concepts; this guide focuses on how to apply them in .NET.
1. ML.NET Fundamentals
What ML.NET is — and where it fits
ML.NET is a machine learning library for .NET. It provides data loading, transformation, a catalog of training algorithms, evaluation tools, and model persistence, all behind a consistent API. It runs on Windows, Linux, and macOS, and is designed for integrating ML into .NET applications rather than for research.
Strong fit: - Classic ML on tabular and text data (classification, regression, clustering, recommendation, anomaly detection, time series) - Teams that are .NET-first and want ML inside their existing services - Training and serving in the same language, with no separate Python service - Using models trained elsewhere via ONNX import, then serving them from .NET Weaker fit: - Cutting-edge deep learning research and large-scale neural network training (the Python ecosystem — PyTorch, TensorFlow — is far richer here) - Needing the very latest model architectures as soon as they're published An honest framing: ML.NET is the pragmatic choice for classic ML inside .NET applications. For deep learning, a common pattern is to train in Python, export to ONNX, and consume the model from .NET (ML.NET and ONNX Runtime both support this).
The tasks ML.NET supports
Task Example question Catalog ------------------------ ---------------------------------------- --------------------------- Binary classification Is this review positive? mlContext.BinaryClassification Multiclass classification Which category is this ticket? mlContext.MulticlassClassification Regression What will this house sell for? mlContext.Regression Clustering Which customers behave alike? mlContext.Clustering Recommendation Which movies will this user like? mlContext.Recommendation() Anomaly detection Is this data point unusual? mlContext.AnomalyDetection / Transforms.Detect* Ranking / Forecasting Order results; predict future values mlContext.Ranking / Forecasting Installing
dotnet add package Microsoft.ML # core: data, transforms, common trainers dotnet add package Microsoft.ML.AutoML # AutoML (Section 17) dotnet add package Microsoft.ML.Recommender # matrix factorization (Section 15) dotnet add package Microsoft.ML.TimeSeries # time-series anomaly detection (Section 16) dotnet add package Microsoft.ML.FastTree # gradient-boosted tree trainers dotnet add package Microsoft.Extensions.ML # PredictionEnginePool for ASP.NET Core (Section 11) The universal workflow
1. Create an MLContext 2. Load data into an IDataView 3. Split into train / test sets 4. Build a pipeline: preprocessing transforms + a trainer 5. Fit the pipeline on the training set -> a trained model (ITransformer) 6. Evaluate on the test set 7. Save the model to a file 8. Load it in your application and make predictions 2. MLContext
The single entry point to everything in ML.NET
var mlContext = new MLContext(seed: 0); MLContext is the factory and the catalog root. Every operation hangs off it:
mlContext.Data -> loading, saving, splitting, filtering data mlContext.Transforms -> preprocessing and feature engineering mlContext.Regression -> regression trainers and evaluation mlContext.BinaryClassification / MulticlassClassification mlContext.Clustering / AnomalyDetection / Ranking / Forecasting mlContext.Recommendation() -> recommendation trainers mlContext.Model -> save / load / create prediction engines mlContext.Auto() -> AutoML (needs Microsoft.ML.AutoML) The seed parameter matters for reproducibility
Passing a seed makes operations that involve randomness (data shuffling in splits, some trainers' initialization) produce repeatable results — which is essential when comparing two pipeline variants, or when a teammate needs to reproduce your numbers. Without a seed, results can differ slightly run to run. MLContext also exposes logging (mlContext.Log += ...) for observing what trainers are doing, and it should generally be created once and reused rather than constructed repeatedly.
3. Data Loading
Describe your data as a C# class, then load it
public class HouseData { [LoadColumn(0)] public float Size { get; set; } [LoadColumn(1)] public float Bedrooms { get; set; } [LoadColumn(2)] public string Neighborhood { get; set; } [LoadColumn(3)] public float Price { get; set; } // the label we want to predict } public class HousePrediction { [ColumnName("Score")] public float Price { get; set; } // regression trainers write predictions to "Score" } IDataView data = mlContext.Data.LoadFromTextFile<HouseData>( path: "houses.csv", hasHeader: true, separatorChar: ','); Notes worth knowing precisely:
- [LoadColumn(n)] maps a class property to the n-th column in the file (zero-based). [LoadColumn(1, 5)] maps a RANGE of columns into a float[] . - Use float, not double, for numeric columns — it's ML.NET's native numeric type. - [ColumnName("Label")] renames a column; ML.NET trainers look for a column named "Label" by default (and "Features" for the feature vector). Other sources
// From an in-memory collection (great for tests and small data) IDataView fromList = mlContext.Data.LoadFromEnumerable(houses); // From a database (requires the Microsoft.ML package's database loader support) var loader = mlContext.Data.CreateDatabaseLoader<HouseData>(); var dbSource = new DatabaseSource(SqlClientFactory.Instance, connectionString, "SELECT Size, Bedrooms, Neighborhood, Price FROM Houses"); IDataView fromDb = loader.Load(dbSource); Loading from SQL Server is a natural fit for .NET teams — your training data is often already in a database (see this series' SQL guides). Whichever source you use, the result is the same type: an IDataView.
Splitting into train and test sets
var split = mlContext.Data.TrainTestSplit(data, testFraction: 0.2, seed: 1); IDataView trainData = split.TrainSet; IDataView testData = split.TestSet; Split BEFORE any fitting, so the test set never influences training. For time-ordered data, do NOT use a random split — split by time instead (train on the past, test on the future), or you'll leak future information. 4. IDataView
ML.NET's lazy, schema-aware, columnar table abstraction
IDataView is the type that flows through every part of ML.NET — what loaders produce, what transforms consume and emit, and what trainers learn from. Think of it as a read-only, lazily-evaluated table with a schema.
Key properties: - LAZY: nothing is read or computed until something iterates over it. Building a pipeline doesn't touch the data; only Fit/Transform-and-iterate does. - FORWARD-ONLY cursor access: designed to stream data, so datasets larger than memory can be processed. - SCHEMA-ful: every column has a name and a type (float, string, vector, key...). - IMMUTABLE: transforms produce a NEW IDataView layered on the previous one; the original is never modified. Inspecting an IDataView
// Peek at the first rows and the schema — the standard debugging move var preview = data.Preview(maxRows: 5); foreach (var row in preview.RowView) Console.WriteLine(string.Join(", ", row.Values.Select(kv => $"{kv.Key}={kv.Value}"))); // Pull a single column out IEnumerable<float> prices = data.GetColumn<float>("Price"); // Convert back to a strongly typed C# collection var houses = mlContext.Data.CreateEnumerable<HouseData>(data, reuseRowObject: false).ToList(); Why laziness matters (and bites)
Because an IDataView is lazy, a transform that is expensive (text featurization, say) is RE-COMPUTED every time something iterates the data — and iterative trainers iterate many times. Cache after expensive steps: pipeline.AppendCacheCheckpoint(mlContext).Append(trainer) (See Section 7. Skip caching for very large data that won't fit in memory.) Also: reuseRowObject: false in CreateEnumerable is essential if you store the results — with true, the same object is overwritten on each iteration, so a collected list would contain many references to one final row's values.
5. Data Preprocessing
Cleaning and converting raw columns into model-ready form
Preprocessing transforms live in mlContext.Transforms. Each one is an estimator that you chain into a pipeline (Section 7).
// Missing values: replace with the column's mean (default) before training mlContext.Transforms.ReplaceMissingValues("Size", replacementMode: MissingValueReplacingEstimator.ReplacementMode.Mean) // Categorical text -> numbers: one-hot encode mlContext.Transforms.Categorical.OneHotEncoding("NeighborhoodEncoded", "Neighborhood") // Scale numeric features to a comparable range mlContext.Transforms.NormalizeMinMax("Features") // squashes to [0, 1] mlContext.Transforms.NormalizeMeanVariance("Features") // zero mean, unit variance // Map a string label to the "key" type multiclass trainers require mlContext.Transforms.Conversion.MapValueToKey("Label", "Category") Filtering rows
// Drop obviously bad rows (e.g. impossible prices) BEFORE training IDataView cleaned = mlContext.Data.FilterRowsByColumn( data, columnName: "Price", lowerBound: 10_000, upperBound: 5_000_000); Why these steps exist
- Models consume NUMBERS. Strings (neighborhood names) must be encoded. - Missing values can crash or silently distort a trainer; decide a policy explicitly. - Features on wildly different scales (Size ~ 2000, Bedrooms ~ 3) can make gradient-based trainers converge slowly or favor the large-valued feature. Normalization puts them on equal footing. - Tree-based trainers (FastTree, LightGBM) are far less sensitive to scale, so normalization matters most for linear models and neural-network-style learners. 6. Feature Engineering
Turning raw columns into the single Features vector trainers consume
Every ML.NET trainer expects one vector column (named Features by default) containing all the model's inputs. Concatenate builds it:
mlContext.Transforms.Concatenate("Features", "Size", "Bedrooms", "NeighborhoodEncoded") Text featurization — raw text to numeric features in one step
mlContext.Transforms.Text.FeaturizeText("Features", "ReviewText") FeaturizeText tokenizes, normalizes, and converts text into a numeric vector of word and character n-gram features — it's the standard starting point for any text classification problem, such as sentiment analysis (Section 12).
Other useful feature-engineering transforms
Transforms.NormalizeBinning(...) -> bucket a numeric column into ranges Transforms.Categorical.OneHotHashEncoding -> hash-encode HIGH-cardinality categories (thousands of distinct values) compactly Transforms.Text.ProduceWordBags(...) -> bag-of-words / n-grams Transforms.Conversion.ConvertType(...) -> change a column's data type Transforms.SelectColumns / DropColumns -> keep only what you need Transforms.FeatureSelection.* -> select the most informative features Transforms.CustomMapping(...) -> your own C# logic (with a caveat below) A caveat on CustomMapping
CustomMapping lets you run arbitrary C# per row — flexible, but a model that contains one CANNOT simply be saved and loaded in another process unless the mapping is registered through a contract assembly. Prefer a built-in transform, or do the derived-feature computation in your data-loading code BEFORE ML.NET sees the data, so the saved model stays self-contained. Feature engineering is usually where the gains are
Good features typically move model quality more than choosing a fancier algorithm: - Derive "price per square foot" or "age of house" from raw columns - Extract "day of week" or "hour" from a timestamp - Combine related fields into ratios or flags Do this thoughtfully, and avoid features that wouldn't exist at prediction time (data leakage — see this series' AI/ML Fundamentals guide). 7. Training Pipelines: Estimators and Transformers
The two core abstractions
IEstimator<T> — a RECIPE for a step. It hasn't seen data yet. It may need to LEARN something from data (the mean to normalize by, the vocabulary to encode, a model's weights). ITransformer — the RESULT of fitting an estimator. It has learned its parameters and can now TRANSFORM data. estimator.Fit(trainingData) -> ITransformer transformer.Transform(data) -> IDataView with new/changed columns A pipeline is a chain of estimators
var pipeline = mlContext.Transforms.ReplaceMissingValues("Size") .Append(mlContext.Transforms.Categorical.OneHotEncoding("NeighborhoodEncoded", "Neighborhood")) .Append(mlContext.Transforms.Concatenate("Features", "Size", "Bedrooms", "NeighborhoodEncoded")) .Append(mlContext.Transforms.NormalizeMinMax("Features")) .AppendCacheCheckpoint(mlContext) // cache the preprocessed data .Append(mlContext.Regression.Trainers.Sdca(labelColumnName: "Price", featureColumnName: "Features")); ITransformer model = pipeline.Fit(trainData); // fits EVERY step, in order, on trainData Why this design is valuable
1. ONE object captures preprocessing AND the model. When you save the trained model (Section 10), the normalization parameters, encoders, and trainer all travel together — so predictions in production apply EXACTLY the same transformations as training. This eliminates a classic bug: preprocessing in production subtly differing from preprocessing at training time. 2. Fitting happens on the TRAINING data only. The normalization mean/range is learned from the training set and merely APPLIED to the test set — preventing test-set information from leaking into training. 3. Pipelines are lazy and composable: define once, Fit, Transform, reuse. AppendCacheCheckpoint stores the data computed up to that point in memory so iterative trainers don't recompute upstream transforms on every pass. Place it after the expensive preprocessing and before the trainer. For datasets too large for memory, omit it.
8. Model Training
Fitting the pipeline is the training step
ITransformer model = pipeline.Fit(trainData); Fit runs the whole pipeline over the training data: transforms learn their parameters, and the trainer runs its optimization to learn the model's weights. The returned ITransformer is your trained model (preprocessing + predictor together).
Choosing a trainer
ML.NET's catalog offers many trainers per task. A practical way to choose:
Start simple and fast: - Linear trainers (Sdca*, Lbfgs*, Sgd*): fast, a good baseline, work well on sparse/text data; benefit from normalization. Then try stronger: - Tree-ensemble trainers (FastTree*, FastForest*, LightGbm*): often the best accuracy on tabular data; less sensitive to scaling; handle non-linear patterns. // Swapping a trainer is a ONE-LINE change — the rest of the pipeline is untouched .Append(mlContext.Regression.Trainers.Sdca(labelColumnName: "Price")) // linear baseline .Append(mlContext.Regression.Trainers.FastTree(labelColumnName: "Price")) // gradient-boosted trees That one-line interchangeability is a major practical strength: you can compare several algorithms against the same preprocessing and the same test set with minimal code (or let AutoML do it, Section 17).
Training takes the label by name
Trainers take labelColumnName and featureColumnName. If your label column is literally "Label" and features are in "Features", you can omit them. Otherwise pass them explicitly — a mismatched column name is the most common "Schema mismatch" error beginners hit. 9. Model Evaluation
Evaluate on the held-out test set, using the task-appropriate metrics
IDataView predictions = model.Transform(testData); // run the trained model on the TEST data RegressionMetrics metrics = mlContext.Regression.Evaluate(predictions, labelColumnName: "Price"); Console.WriteLine($"R²: {metrics.RSquared:F3}"); Console.WriteLine($"RMSE: {metrics.RootMeanSquaredError:F0}"); Console.WriteLine($"MAE: {metrics.MeanAbsoluteError:F0}"); Each task has its own evaluator and metrics:
Task Evaluate call Key metrics ---------------------- ---------------------------------------------- ----------------------------------------- Regression mlContext.Regression.Evaluate RSquared, RootMeanSquaredError, MeanAbsoluteError Binary classification mlContext.BinaryClassification.Evaluate Accuracy, AreaUnderRocCurve, F1Score, PositivePrecision, PositiveRecall Multiclass mlContext.MulticlassClassification.Evaluate MicroAccuracy, MacroAccuracy, LogLoss Clustering mlContext.Clustering.Evaluate AverageDistance, DaviesBouldinIndex Anomaly detection mlContext.AnomalyDetection.Evaluate AreaUnderRocCurve, DetectionRateAtFalsePositiveCount Cross-validation for a more reliable estimate
var cvResults = mlContext.Regression.CrossValidate( data, pipeline, numberOfFolds: 5, labelColumnName: "Price"); double avgR2 = cvResults.Average(r => r.Metrics.RSquared); A single split can be lucky or unlucky; cross-validation trains and evaluates across several folds and lets you look at both the average and the spread.
Evaluation principles (the same ones from this series' AI/ML Fundamentals guide)
- NEVER evaluate on the training data — it measures memorization. - Compare against a baseline; a "good" metric means little without one. - Look at the train-vs-test gap: great training metrics with poor test metrics means overfitting. - On imbalanced classification, don't trust accuracy alone — check precision, recall, F1, and AUC. 10. Model Saving and Loading
A trained model is a .zip file you can ship
// Save: the model AND the schema of the data it was trained on mlContext.Model.Save(model, trainData.Schema, "model.zip"); // Load (in the same app, or — typically — in a different one) var loadMlContext = new MLContext(); ITransformer loadedModel = loadMlContext.Model.Load("model.zip", out DataViewSchema inputSchema); The saved file contains the entire pipeline — preprocessing steps with their learned parameters, plus the trained predictor — so the consuming application doesn't need the training data or the training code, only the file and the matching C# input/output classes.
Practical guidance: - Treat model.zip as a build/deployment artifact, versioned like any other. Name or folder it by version (model-v3.zip) so a rollback is trivial. - Keep the input/output classes (HouseData, HousePrediction) in a shared project referenced by both the training and the consuming app, so the schema can't drift apart. - Models can also be saved/loaded via Stream — useful for storing them in blob storage or a database rather than on local disk. 11. Model Prediction
Single predictions: PredictionEngine
var engine = mlContext.Model.CreatePredictionEngine<HouseData, HousePrediction>(model); HousePrediction result = engine.Predict(new HouseData { Size = 1800, Bedrooms = 3, Neighborhood = "Westside" }); Console.WriteLine($"Predicted price: {result.Price:C0}"); ⚠️ PredictionEngine is NOT thread-safe
A PredictionEngine holds internal state and must not be shared across threads. Creating a new one per request is wasteful (it's relatively expensive to build). In a web application, using a single shared PredictionEngine instance — a very common mistake — produces race conditions and wrong results under concurrent load. The ASP.NET Core answer: PredictionEnginePool
// Program.cs (requires the Microsoft.Extensions.ML package) builder.Services.AddPredictionEnginePool<HouseData, HousePrediction>() .FromFile(modelName: "HouseModel", filePath: "model.zip", watchForChanges: true); // In a controller / minimal API endpoint — inject the pool app.MapPost("/predict", (HouseData input, PredictionEnginePool<HouseData, HousePrediction> pool) => { var prediction = pool.Predict(modelName: "HouseModel", example: input); return Results.Ok(prediction.Price); }); The pool manages thread-safe engine reuse for you, integrates with dependency injection, and with watchForChanges: true automatically reloads the model when the file changes — so you can deploy a retrained model without restarting the app.
Batch predictions: Transform
IDataView batch = mlContext.Data.LoadFromEnumerable(newHouses); IDataView scored = model.Transform(batch); var results = mlContext.Data.CreateEnumerable<HousePrediction>(scored, reuseRowObject: false).ToList(); For scoring many rows at once (nightly jobs, bulk imports), Transform over an IDataView is much more efficient than looping a PredictionEngine.
12. Classification
Binary classification: a yes/no question — sentiment analysis
public class SentimentData { Source:https://dev.to/rhuturaj_takle/mlnet-5h4oRelated posts
