Adding Telemetry to .NET Apps with OpenTelemetry and Grafana

Modern .NET applications generate a steady stream of operational signals. Web requests, background jobs, database calls, message queues and external APIs all affect whether a service feels fast and dependable. Without consistent telemetry, developers often discover faults through customer complaints, support tickets or a sudden alert from a separate platform.

OpenTelemetry provides a vendor-neutral way to collect traces, metrics and logs from C# and ASP.NET Core applications. Grafana then turns those signals into dashboards, alerts and trace views that help teams understand what is happening across a distributed system.

The subject fits naturally within the cloud, DevOps and Microsoft ecosystem covered by the conference programme. For Australian development teams, the same practices are especially useful when applications serve users across large distances, depend on cloud regions in Sydney or Melbourne, and must meet strict expectations around privacy and service availability.

Why Telemetry Matters In Distributed .NET Systems

A log message saying that an order failed gives limited information. A trace can show that the request entered an ASP.NET Core controller, waited 900 milliseconds for a SQL query, called a payment provider, retried an HTTP request and eventually returned a 500 response. Metrics can then reveal whether that delay affects one request or thousands of customers.

OpenTelemetry groups observability into three primary signals. Traces describe the path of an operation through services. Metrics represent measurements such as request duration, error rate, CPU use and queue depth. Logs provide detailed event records. When these signals share a trace identifier, an engineer can move from a Grafana panel to the exact request and related log entries without manually matching timestamps.

This approach is valuable in an Australian environment where a service might run in an Azure Australia East region while customers connect from Perth, Brisbane, regional New South Wales or Tasmania. Network latency, mobile connectivity and regional traffic patterns can produce symptoms that are invisible in a developer’s local environment. Telemetry makes those differences measurable rather than anecdotal.

A useful starting point is to define service-level objectives. For example, an online service might target 99.9 per cent availability, a p95 API latency below 400 milliseconds and an error rate under 0.5 per cent. These targets give dashboards and alerts a business meaning instead of turning observability into a collection of attractive graphs.

Instrumenting ASP.NET Core And C# Workloads

The .NET OpenTelemetry libraries can automatically instrument common framework components. ASP.NET Core instrumentation records incoming requests, route information, response status and duration. HttpClient instrumentation follows outbound calls, while packages for Entity Framework Core, SQL Client, Redis and messaging systems can add visibility into dependencies.

A typical setup begins with resource information that identifies the service:

builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource
        .AddService(
            serviceName: "orders-api",
            serviceVersion: "2.4.0",
            serviceInstanceId: Environment.MachineName))
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSqlClientInstrumentation()
        .AddOtlpExporter())
    .WithMetrics(metrics => metrics
        .AddAspNetCoreInstrumentation()
        .AddRuntimeInstrumentation()
        .AddProcessInstrumentation()
        .AddOtlpExporter());

The exact package names and exporter configuration depend on the application’s .NET version and deployment model. The important design choice is to identify the service consistently. A name such as orders-api should remain stable, while environment, version and instance can vary. This allows Grafana queries to compare production, staging and individual releases.

Automatic instrumentation is a strong foundation, yet it does not understand every business operation. Manual activities can describe actions such as reserving stock, calculating freight or approving a refund:

using var activity = ActivitySource.StartActivity("reserve-stock");
activity?.SetTag("warehouse.region", "sydney");
activity?.SetTag("order.type", "express");

Tags should explain the operation without exposing secrets or excessive personal information. An order identifier may be useful in a controlled diagnostic system, while a full address, access token or payment detail should never become a default span attribute.

Designing A Reliable Telemetry Pipeline

Applications can send data directly to Grafana-compatible back ends, but an OpenTelemetry Collector usually creates a cleaner architecture. The application exports OTLP data to the collector, which can batch, filter, sample, enrich and route signals. This reduces coupling between application code and a particular storage product.

A collector can send traces to Grafana Tempo, metrics to Prometheus or Grafana Mimir, and logs to Grafana Loki. Grafana provides a shared interface for exploring them. Grafana Alloy is another option for collecting and forwarding telemetry in modern environments, especially where teams want a single agent for multiple signal types.

The collector should be treated as production infrastructure. Configure retry behaviour, queue limits and memory protection so a temporary back-end outage does not exhaust application resources. Telemetry is important, but it must not prevent an API from serving customers. Exporting asynchronously and applying bounded queues helps keep observability overhead predictable.

Sampling also deserves deliberate attention. Recording every trace may be useful during a short investigation, but it can become expensive at scale. Head-based sampling makes an early decision, while tail-based sampling allows the collector to retain slow or failed requests after seeing their complete journey. A practical policy often keeps all errors, a percentage of successful requests and selected high-value business flows.

Australian organisations should also consider where telemetry is stored and processed. Customer data may be subject to contractual requirements, the Privacy Act and sector-specific controls. Keeping telemetry in an Australian cloud region can simplify governance, yet location alone does not make data safe. Teams still need retention rules, access controls, encryption and a review of what attributes are being exported.

Connecting Grafana Dashboards To Real Decisions

A useful Grafana dashboard answers a specific operational question. An executive view might show availability, successful transactions and customer-facing latency. A service dashboard can display request rate, error rate, p50 and p95 duration, saturation and dependency health. A database panel might focus on connection pool pressure, slow queries and lock waits.

The four golden signals—latency, traffic, errors and saturation—are a practical foundation for HTTP services. Pair them with RED metrics for request-driven systems: rate, errors and duration. For infrastructure and resource-heavy workloads, USE metrics—utilisation, saturation and errors—help reveal CPU, memory, disk and network constraints.

Metric labels require care. Labels such as route, method and status code are generally manageable. Labels containing user IDs, order numbers or unrestricted URLs can create high cardinality, increasing storage costs and reducing dashboard performance. Prefer route templates such as /orders/{id} over raw paths, and aggregate business dimensions into a small, documented set.

Trace exemplars can connect a metric spike to an actual distributed trace. For instance, a p95 latency panel may contain a link to a slow request in Tempo. From that trace, the operator can inspect an SQL span, follow a downstream HTTP call and open related Loki logs. This workflow is much faster than searching each system independently.

Alerts should describe impact and provide a useful response path. An alert for “CPU above 80 per cent” may be noisy if the service is healthy. An alert for elevated failed requests, sustained queue age or an SLO burn rate is more meaningful. Each alert should identify the affected service, environment, likely severity and the dashboard or runbook that supports investigation.

Applying Observability Across Delivery And Operations

Telemetry belongs in the software delivery lifecycle, not just in production. During development, engineers can verify that new endpoints create useful spans and that sensitive attributes are absent. In test environments, integration tests can check that important metrics and trace relationships appear as expected.

Release comparisons are especially valuable. After deploying a new version, Grafana can compare latency, error rates and dependency calls with the previous release. A small change to serialisation, database queries or retry logic may look harmless in code review but become clear in a before-and-after view.

For Australian teams working across Sydney, Melbourne and remote locations, shared dashboards reduce the friction of handovers. A developer finishing an afternoon shift can leave a trace link and a clear observation for a colleague beginning the next morning. Plain language helps too: “The checkout API is slow for Brisbane traffic because the inventory call is timing out” is more useful than a vague note that “the system seems crook.”

Telemetry can support capacity planning as well. Retail traffic around Australian public holidays, end-of-financial-year processing, major sporting events and seasonal travel can produce unusual demand. Historical metrics reveal when to scale services, increase database capacity or review third-party limits before a peak arrives.

Teams should document ownership, retention and access. Engineers need enough detail to troubleshoot, while customer support and business users may need carefully filtered dashboards. Role-based access in Grafana, separate production permissions and masked log fields help protect operational data without blocking legitimate incident response.

Choosing Signals And Grafana Components

The right observability design depends on the question being answered, the cost of retention and the operational maturity of the team. Traces are excellent for following an individual request, metrics are efficient for trends and alerting, and logs preserve detailed context. A balanced implementation uses each signal for its strongest purpose rather than sending everything everywhere.

The following mapping provides a practical starting point for a .NET service:

A sensible rollout starts with one customer-facing ASP.NET Core service. Add resource metadata, inbound and outbound tracing, runtime metrics and structured logs. Build a small dashboard around traffic, errors, latency and saturation, then test an incident scenario such as a slow database query or unavailable external API.

Once the basics are stable, extend instrumentation to background workers, message consumers and scheduled jobs. Review the cost and privacy profile after each stage. With consistent OpenTelemetry conventions and Grafana’s cross-signal navigation, .NET teams gain a dependable view of application behaviour—from a local development environment to services operating across Australia and the wider cloud.