Deploying a ML.NET model to a Web API for real-time predictions

A trained machine learning model becomes useful when an application can call it at the moment a decision is needed. In a .NET environment, ML.NET and ASP.NET Core provide a practical route from a prepared dataset to an HTTP endpoint that returns a prediction in milliseconds. This pattern suits fraud screening, demand forecasting, customer classification, equipment monitoring, and recommendation services.

For Australian teams, deployment details matter as much as model accuracy. A retail platform serving Sydney and Melbourne may need predictable latency across Azure Australia East and Australia Southeast, while a mining operation near Perth may have intermittent connectivity and strict data-handling rules. The conference programme reflects the wider Microsoft development ecosystem in which these production concerns sit, from cloud architecture to DevOps and applied AI.

Prepare the model for serving

The deployment journey starts with a model that has a stable input contract. In ML.NET, that contract is commonly represented by a class containing feature properties, while the prediction result uses a second class containing the score, label, probability, or forecast value. The same schema must be understood by training code, the web API, and the client sending requests.

A binary classification example might use fields such as transaction amount, merchant category, account age, and transaction hour. A regression model could accept temperature, stock levels, and historical sales to estimate demand. Keep preprocessing inside the ML.NET pipeline where possible. Normalisation, one-hot encoding, missing-value handling, and feature concatenation then travel with the model instead of being duplicated in separate application code.

After training, save the ITransformer and its schema to a file or another controlled artefact store. A simplified training step looks like this:

DataViewSchema schema;
ITransformer model = pipeline.Fit(trainingData);

mlContext.Model.Save(model, trainingData.Schema, "model.zip");

The saved file should be versioned alongside the code and evaluated against a fixed test set. Record the training data range, feature definitions, model metrics, and ML.NET package version. A prediction endpoint should never silently load an untested replacement simply because a file was copied into a server directory.

Build a prediction endpoint in ASP.NET Core

A web API should load the model once when the application starts, rather than reading model.zip for every request. Register MLContext, load the model, and create a prediction engine or a reusable prediction service. For moderate traffic, PredictionEngine<TInput, TOutput> is straightforward. For higher concurrency, use PredictionEnginePool or an approach based on batched IDataView processing.

A service registration in a modern ASP.NET Core application can look like this:

builder.Services.AddPredictionEnginePool<ModelInput, ModelOutput>()
    .FromFile("model.zip");

The controller can then validate the request and obtain a prediction:

[ApiController]
[Route("api/predictions")]
public class PredictionsController : ControllerBase
{
    private readonly PredictionEnginePool<ModelInput, ModelOutput> _pool;

    public PredictionsController(PredictionEnginePool<ModelInput, ModelOutput> pool)
    {
        _pool = pool;
    }

    [HttpPost]
    public ActionResult<ModelOutput> Predict(ModelInput input)
    {
        if (!ModelState.IsValid)
            return ValidationProblem(ModelState);

        return Ok(_pool.Predict(input));
    }
}

The exact registration varies with the ML.NET version, so package versions should be pinned and tested together. Return a clear response contract instead of exposing internal model objects. A classification response might include Prediction, Probability, ModelVersion, and a timestamp. Avoid returning sensitive source fields such as full customer identifiers when the client only needs a decision.

Real-time does not mean every prediction must be synchronous. A sub-100-millisecond request may suit a checkout or call-centre workflow, while a complex document model may belong in a queue. Set cancellation tokens, request limits, and timeouts. Use structured logging for latency, validation failures, model version, and correlation ID, while excluding raw personal data from logs.

Choose the right hosting and configuration

The API can run in Azure App Service, an ASP.NET Core container, Azure Kubernetes Service, or a Windows or Linux virtual machine. App Service is often a sensible starting point for a small prediction service because deployment, TLS, scaling, and diagnostics are managed with relatively little operational overhead. Containers become attractive when the team needs consistent runtime dependencies across development, testing, and production.

Keep the model path and settings outside the code. Configuration should specify the model location, maximum request size, confidence threshold, and feature flags. In Azure, store secrets in Key Vault and use managed identity rather than embedding credentials in appsettings.json. The model itself can be packaged with the application for simple releases or fetched from Blob Storage when a model registry controls promotion.

For an Australian service, Azure Australia East in New South Wales or Australia Southeast in Victoria may be preferable when data residency and response time are important. A customer portal used across Brisbane, Adelaide, and Melbourne can usually tolerate modest network variation, while an industrial system in Western Australia may require local buffering or an edge component. The right choice depends on the data classification, users, failover plan, and service availability requirements.

Design for cold starts and horizontal scaling. If each instance loads a large model, startup time and memory usage can become significant during autoscaling. Health checks should distinguish between a process that is running and an instance that has successfully loaded a usable model. A readiness probe can keep traffic away until the model and its dependencies are available.

Make predictions reliable in production

A prediction endpoint needs the same engineering discipline as any other public API. Validate numerical ranges, reject unexpected values, and establish a policy for missing fields. For example, a negative product quantity or an impossible temperature should produce a useful 400 Bad Request, not a misleading score. Use API versioning when the input schema changes, since a retrained model may require new features.

Security is especially important when predictions influence access, finance, employment, or health-related decisions. Apply authentication and authorisation, rate-limit public routes, and use HTTPS. Under the Australian Privacy Principles, organisations should consider how personal information is collected, retained, disclosed, and secured. A model endpoint that receives names, addresses, or behavioural data should have a clear retention policy and an access trail.

The service should expose operational metrics. Track p50 and p95 latency, throughput, error codes, timeout frequency, and the proportion of low-confidence outcomes. Compare incoming feature distributions with the training distribution to detect drift. A model trained on summer purchasing patterns may behave differently during a major sale, while a transport model can change when fuel prices or road conditions shift.

Use a fallback that is explicit and safe. A rules-based threshold, cached recommendation, or manual review queue may be appropriate when the model is unavailable or its confidence is low. Do not quietly substitute yesterday’s model without recording that event. In a regional Australian setting, an intermittent link from a remote site should result in a controlled retry or local queue rather than duplicated transactions when connectivity returns.

Test, release, and monitor the service

Testing should cover the complete path from JSON input to prediction response. Unit tests can verify validation and mapping logic, while integration tests load the actual model and send representative requests through the HTTP pipeline. Include malformed JSON, missing values, extreme values, concurrent requests, and model-file failures. The expected result should include acceptable score ranges rather than a brittle exact floating-point value.

Load testing reveals whether the selected prediction mechanism is safe under concurrent traffic. Test realistic payload sizes and a gradual increase in requests, then observe CPU, memory, garbage collection, and response time. A service that performs well on a developer laptop may struggle on a small App Service plan once several callers submit predictions simultaneously.

A deployment pipeline can package the API, run tests, scan the container or dependencies, publish the model artefact, and deploy to a staging slot. Smoke tests should verify that the endpoint is healthy and that the model version returned by the service is the expected one. A slot swap or progressive rollout limits the blast radius if a new model produces poor outcomes.

Model monitoring extends beyond application monitoring. Store prediction outcomes when a legitimate label becomes available, then compare actual results with prior estimates. Review false positives and false negatives with domain experts. For a Melbourne retailer, this might mean checking whether stock recommendations miss local events; for a Sydney financial service, it may involve checking whether a threshold disadvantages a particular customer group. Retraining should follow evidence from drift and outcome data, not an arbitrary calendar alone.

When these practices are combined, ML.NET provides a compact path from a .NET training pipeline to a dependable HTTP prediction service. The API remains easy for web, mobile, and internal clients to consume, while versioned models, Australian hosting choices, validation, and monitoring make the deployment suitable for real operational workloads.