Tracing microservices in .NET with OpenTelemetry

The shift towards microservices and cloud-native architectures has reshaped how Australian engineering teams build and operate software. Platforms like the Australian Taxation Office's whole-of-government systems, the New Payments Platform, and Atlassian's globally distributed products all depend on dozens of independently deployed services that must work together seamlessly. When a single user request fans out across multiple services, understanding the path it takes — and where latency or errors creep in — becomes genuinely hard. Traditional logging, even at scale, rarely gives engineers the end-to-end picture they need.

This is where distributed tracing earns its place. By assigning a unique identifier to each request and propagating it across every hop, tracing stitches together the work done by separate processes, containers, and serverless functions into a coherent narrative. OpenTelemetry, the vendor-neutral observability framework that merged the OpenTracing and OpenCensus projects, has become the de facto standard for this work. Its .NET SDK offers first-class support for ASP.NET Core, HttpClient, gRPC, and a growing list of data stores.

For developers attending the conference, this topic sits at the intersection of several hot tracks: .NET internals, cloud computing on Microsoft Azure, and DevOps. The remainder of this article walks through the practical steps of building a tracing pipeline in .NET, from initial configuration through to exporter choice and operational considerations specific to Australian cloud deployments.

Why distributed tracing matters in modern .NET systems

A typical Sydney-based fintech might run its customer-facing API in one App Service, its risk engine in an Azure Kubernetes Service cluster, and its settlement worker on an Azure Function. A single login request touches identity verification, fraud scoring, and account lookup — each owned by a different team. When latency spikes, the on-call engineer must answer a deceptively simple question: where did the time go?

Logs alone are a poor fit for that question because they are indexed by service, not by request. Metrics tell you that latency is up, but not which span is responsible. Tracing fills the gap by recording the duration and metadata of each unit of work, known as a span, and linking them through a shared trace identifier. With that link in place, an engineer can pivot from a slow transaction to a flame chart that shows every contributing service.

Beyond debugging, traces feed architectural decision-making. Teams in Melbourne and Brisbane frequently use trace data to spot accidental chatty services, unnecessary cross-region calls, or synchronous chains that should be asynchronous. Tracing also dovetails nicely with structured logging: a trace identifier can be added to every log line so that operators can jump between views.

OpenTelemetry fundamentals and the .NET SDK

OpenTelemetry is built around three signals: traces, metrics, and logs. For tracing specifically, the SDK provides an API for creating activities — the .NET term for spans — and a set of automatic instrumentations that hook into well-known libraries. The NuGet packages follow a predictable pattern: OpenTelemetry.Extensions.Hosting for the host integration, OpenTelemetry.Instrumentation.AspNetCore for inbound HTTP, and OpenTelemetry.Instrumentation.Http for outbound calls.

Configuration happens through the OpenTelemetryTracingBuilder extension methods. A minimal setup adds ASP.NET Core and HttpClient instrumentation, sets a resource builder to decorate the service with its name and version, and registers an exporter. The SDK reads the standard OTEL_* environment variables, which means the same application binary can be deployed across multiple environments — useful when promoting builds through an Australian developer's dev, test, and production tiers in the Azure Australia East region.

Resource attributes are worth populating thoughtfully. Beyond the service name, including the deployment environment, the cloud region, and a build commit hash makes traces far easier to slice later. Many teams add a custom attribute for the responsible team or product squad, which mirrors the squad model common in Australian banks like CBA and ANZ.

Instrumenting ASP.NET Core applications for tracing

Automatic instrumentation handles the bulk of common scenarios, but custom spans are often needed for business logic. Inside a request handler, an ActivitySource is created from a static field and used to start spans around meaningful units of work — for example, a pricing calculation or a database-heavy report. Each span should carry attributes that describe what it is doing, not how. An attribute like order.value is more useful than internal.buffer.size.

Spans also accept events and links. Events are timestamped annotations attached to a span, useful for marking milestones such as "cache hit" or "queue dispatched". Links connect a span to another trace, which is handy when a worker processes a batch of related messages that originated from separate traces. A practical example is a Sydney retailer's nightly reconciliation job, where each batch item traces back to its originating order event.

Baggage is the third leg of the tracing API. Unlike attributes, baggage is serialised and propagated across process boundaries alongside the trace identifier. It is the right place for tenant identifiers, feature flags, or correlation keys that downstream services need. The .NET SDK exposes baggage through Activity.Current.Baggage or the higher-level Baggage API.

Propagating context across service boundaries

Propagation is what makes a distributed trace distributed. The W3C Trace Context standard, which OpenTelemetry supports by default, carries two headers: traceparent and tracestate. Most modern proxies and service meshes in Azure — including Application Gateway and the Linkerd service mesh — preserve these headers automatically, so engineers rarely need to touch them directly.

The cases that bite are usually legacy integrations. Calling out to a SOAP endpoint, a partner REST API, or a mainframe job runner rarely propagates headers by itself. Teams writing such integrations need to explicitly inject the current activity's context into the outbound request, then extract it on the receiving side. The Propagators.DefaultTextMapPropagator does the heavy lifting for HTTP, while message-based systems like Azure Service Bus or AWS SQS — increasingly used by Australian SaaS companies — require custom propagators that serialise context into message properties.

A subtle pitfall in Australian deployments is cross-region traffic. A trace that hops from a Sydney API to a Melbourne function and back will still display as a single trace, but the network latency between regions is non-trivial. Annotating the region as a span attribute makes these round trips visible, which is helpful when justifying co-location or evaluating Azure Australia's paired regions.

Sampling strategies for production workloads

Recording every span on a busy service is rarely feasible. A Sydney payments processor might handle tens of thousands of requests per minute, and storing each one would quickly overwhelm even a generous observability budget. Sampling addresses this by deciding, at collection time, which traces to keep.

OpenTelemetry supports several sampler types, each suited to different scenarios:

• ParentBased(AlwaysOn) — the default and safest choice, which samples the entire trace once the root span is sampled.
• TraceIdRatioBased — samples a fixed percentage of traces, useful when storage costs need a hard ceiling.
• Custom rule-based samplers — drop health-check traffic, prioritise traces carrying a specific baggage entry, or boost sampling for known problem tenants.

In practice, head-based sampling — where the decision is made at the start of the trace — works well for steady-state traffic, while tail-based sampling evaluates complete traces and is better for catching rare errors. Tail-based sampling requires a collector that buffers spans, which adds operational complexity but pays dividends when debugging the kind of intermittent failures that keep an on-call rotation busy during AEST business hours.

Exporters and backend integration choices

The .NET SDK ships with several exporters built in, and more are available as separate packages. The OTLP exporter is the most portable, sending traces over gRPC or HTTP to any compatible backend, including the OpenTelemetry Collector. Jaeger and Zipkin exporters remain for teams that have invested in those tools. For Azure-centric shops, the Application Insights exporter integrates directly with the broader APM experience.

The choice of backend tends to follow existing investments. A team already using Datadog for logs and metrics will probably export to Datadog's OTLP endpoint. A startup running entirely on Grafana Cloud — increasingly common in Australian scale-ups — will point at Tempo. Atlassian-style organisations running self-hosted observability often deploy an OpenTelemetry Collector in front of a storage backend like Jaeger or Tempo.

A comparison of common choices for Australian .NET teams:

Operationalising tracing in Australian cloud environments

Data residency is a recurring concern for Australian organisations. The Privacy Act and APRA's CPS 234 standard push many financial services and government-adjacent workloads towards in-region storage. Azure Australia East (Sydney) and Australia Southeast (Melbourne) cover most needs, but exporters that ship data overseas — even for processing — should be configured carefully. The OpenTelemetry Collector can be deployed as an in-region agent that buffers and forwards traces only when the destination is reachable and approved.

Compliance also touches what is recorded. PII like names, emails, and Tax File Numbers should never appear in span attributes. The SDK exposes processors that can scrub or redact attributes before export, and resource detectors can be limited so that hostnames and IP addresses do not leak. A short checklist worth following:

• Audit span attributes for PII during code review.
• Configure resource detectors to omit host metadata where unnecessary.
• Use secret managers for any exporter credentials.
• Document the trace data flow in the team's privacy impact assessment.

Cultural habits matter too. Australian engineering teams often blend follow-the-sun support across Sydney, Brisbane, and Perth, and traces make handovers smoother because the on-call engineer in another state can pick up exactly where the previous shift left off. Good trace hygiene — consistent span naming, attribute schemas, and service naming conventions — turns this from a chore into a routine. When the same conventions appear in runbooks, post-mortems, and dashboards, the cost of an incident drops measurably across the whole team.