Configure Azure Monitor OpenTelemetry

This article covers configuration settings for the Azure Monitor OpenTelemetry distro.

Connection string

A connection string in Application Insights defines the target location for sending telemetry data.

Use one of the following three ways to configure the connection string:

  • Add UseAzureMonitor() to your program.cs file:

    var builder = WebApplication.CreateBuilder(args);
    
    // Add the OpenTelemetry telemetry service to the application.
    // This service will collect and send telemetry data to Azure Monitor.
    builder.Services.AddOpenTelemetry().UseAzureMonitor(options => {
        options.ConnectionString = "<Your Connection String>";
    });
    
    var app = builder.Build();
    
    app.Run();
    
  • Set an environment variable.

    APPLICATIONINSIGHTS_CONNECTION_STRING=<Your Connection String>
    
  • Add the following section to your appsettings.json config file.

    {
      "AzureMonitor": {
          "ConnectionString": "<Your Connection String>"
      }
    }
    

Note

If you set the connection string in more than one place, we adhere to the following precedence:

  1. Code
  2. Environment Variable
  3. Configuration File

Set the Cloud Role Name and the Cloud Role Instance

For supported languages, the Azure Monitor OpenTelemetry Distro automatically detects the resource context and provides default values for the Cloud Role Name and the Cloud Role Instance properties of your component. However, you might want to override the default values to something that makes sense to your team. The cloud role name value appears on the Application Map as the name underneath a node.

Set the Cloud Role Name and the Cloud Role Instance via Resource attributes. Cloud Role Name uses service.namespace and service.name attributes, although it falls back to service.name if service.namespace isn't set. Cloud Role Instance uses the service.instance.id attribute value. For information on standard attributes for resources, see OpenTelemetry Semantic Conventions.

// Setting role name and role instance

// Create a dictionary of resource attributes.
var resourceAttributes = new Dictionary<string, object> {
    { "service.name", "my-service" },
    { "service.namespace", "my-namespace" },
    { "service.instance.id", "my-instance" }};

// Create a new ASP.NET Core web application builder.
var builder = WebApplication.CreateBuilder(args);

// Add the OpenTelemetry telemetry service to the application.
// This service will collect and send telemetry data to Azure Monitor.
builder.Services.AddOpenTelemetry()
    .UseAzureMonitor()
    // Configure the ResourceBuilder to add the custom resource attributes to all signals.
    // Custom resource attributes should be added AFTER AzureMonitor to override the default ResourceDetectors.
    .ConfigureResource(resourceBuilder => resourceBuilder.AddAttributes(_testResourceAttributes));

// Build the ASP.NET Core web application.
var app = builder.Build();

// Start the ASP.NET Core web application.
app.Run();

Enable Sampling

You might want to enable sampling to reduce your data ingestion volume, which reduces your cost. Azure Monitor provides a custom fixed-rate sampler that populates events with a sampling ratio, which Application Insights converts to ItemCount. The fixed-rate sampler ensures accurate experiences and event counts. The sampler is designed to preserve your traces across services, and it's interoperable with older Application Insights Software Development Kits (SDKs). For more information, see Learn More about sampling.

Note

Metrics and Logs are unaffected by sampling.

The sampler expects a sample rate of between 0 and 1 inclusive. A rate of 0.1 means approximately 10% of your traces are sent.

// Create a new ASP.NET Core web application builder.
var builder = WebApplication.CreateBuilder(args);

// Add the OpenTelemetry telemetry service to the application.
// This service will collect and send telemetry data to Azure Monitor.
builder.Services.AddOpenTelemetry().UseAzureMonitor(o =>
{
    // Set the sampling ratio to 10%. This means that 10% of all traces will be sampled and sent to Azure Monitor.
    o.SamplingRatio = 0.1F;
});

// Build the ASP.NET Core web application.
var app = builder.Build();

// Start the ASP.NET Core web application.
app.Run();

Tip

When using fixed-rate/percentage sampling and you aren't sure what to set the sampling rate as, start at 5% (i.e., 0.05 sampling ratio) and adjust the rate based on the accuracy of the operations shown in the failures and performance panes. A higher rate generally results in higher accuracy. However, ANY sampling will affect accuracy so we recommend alerting on OpenTelemetry metrics, which are unaffected by sampling.

Live metrics

Live metrics provides a real-time analytics dashboard for insight into application activity and performance.

Important

See the Supplemental Terms of Use for Azure Previews for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

This feature is enabled by default.

Users can disable Live Metrics when configuring the Distro.

builder.Services.AddOpenTelemetry().UseAzureMonitor(options => {
	// Disable the Live Metrics feature.
    options.EnableLiveMetrics = false;
});

Enable Microsoft Entra ID (formerly Azure AD) authentication

You might want to enable Microsoft Entra authentication for a more secure connection to Azure, which prevents unauthorized telemetry from being ingested into your subscription.

We support the credential classes provided by Azure Identity.

  • We recommend DefaultAzureCredential for local development.
  • We recommend ManagedIdentityCredential for system-assigned and user-assigned managed identities.
    • For system-assigned, use the default constructor without parameters.
    • For user-assigned, provide the client ID to the constructor.
  • We recommend ClientSecretCredential for service principals.
    • Provide the tenant ID, client ID, and client secret to the constructor.
  1. Install the latest Azure.Identity package:

    dotnet add package Azure.Identity
    
  2. Provide the desired credential class:

    // Create a new ASP.NET Core web application builder.    
    var builder = WebApplication.CreateBuilder(args);
    
    // Add the OpenTelemetry telemetry service to the application.
    // This service will collect and send telemetry data to Azure Monitor.
    builder.Services.AddOpenTelemetry().UseAzureMonitor(options => {
        // Set the Azure Monitor credential to the DefaultAzureCredential.
        // This credential will use the Azure identity of the current user or
        // the service principal that the application is running as to authenticate
        // to Azure Monitor.
        options.Credential = new DefaultAzureCredential();
    });
    
    // Build the ASP.NET Core web application.
    var app = builder.Build();
    
    // Start the ASP.NET Core web application.
    app.Run();
    

Offline Storage and Automatic Retries

To improve reliability and resiliency, Azure Monitor OpenTelemetry-based offerings write to offline/local storage by default when an application loses its connection with Application Insights. It saves the application telemetry to disk and periodically tries to send it again for up to 48 hours. In high-load applications, telemetry is occasionally dropped for two reasons. First, when the allowable time is exceeded, and second, when the maximum file size is exceeded or the SDK doesn't have an opportunity to clear out the file. If we need to choose, the product saves more recent events over old ones. Learn More

The Distro package includes the AzureMonitorExporter, which by default uses one of the following locations for offline storage (listed in order of precedence):

  • Windows
    • %LOCALAPPDATA%\Microsoft\AzureMonitor
    • %TEMP%\Microsoft\AzureMonitor
  • Non-Windows
    • %TMPDIR%/Microsoft/AzureMonitor
    • /var/tmp/Microsoft/AzureMonitor
    • /tmp/Microsoft/AzureMonitor

To override the default directory, you should set AzureMonitorOptions.StorageDirectory.

// Create a new ASP.NET Core web application builder.
var builder = WebApplication.CreateBuilder(args);

// Add the OpenTelemetry telemetry service to the application.
// This service will collect and send telemetry data to Azure Monitor.
builder.Services.AddOpenTelemetry().UseAzureMonitor(options =>
{
    // Set the Azure Monitor storage directory to "C:\\SomeDirectory".
    // This is the directory where the OpenTelemetry SDK will store any telemetry data that cannot be sent to Azure Monitor immediately.
    options.StorageDirectory = "C:\\SomeDirectory";
});

// Build the ASP.NET Core web application.
var app = builder.Build();

// Start the ASP.NET Core web application.
app.Run();

To disable this feature, you should set AzureMonitorOptions.DisableOfflineStorage = true.

Enable the OTLP Exporter

You might want to enable the OpenTelemetry Protocol (OTLP) Exporter alongside the Azure Monitor Exporter to send your telemetry to two locations.

Note

The OTLP Exporter is shown for convenience only. We don't officially support the OTLP Exporter or any components or third-party experiences downstream of it.

  1. Install the OpenTelemetry.Exporter.OpenTelemetryProtocol package in your project.

    dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol
    
  2. Add the following code snippet. This example assumes you have an OpenTelemetry Collector with an OTLP receiver running. For details, see the example on GitHub.

    // Create a new ASP.NET Core web application builder.
    var builder = WebApplication.CreateBuilder(args);
    
    // Add the OpenTelemetry telemetry service to the application.
    // This service will collect and send telemetry data to Azure Monitor.
    builder.Services.AddOpenTelemetry().UseAzureMonitor();
    
    // Add the OpenTelemetry OTLP exporter to the application.
    // This exporter will send telemetry data to an OTLP receiver, such as Prometheus
    builder.Services.AddOpenTelemetry().WithTracing(builder => builder.AddOtlpExporter());
    builder.Services.AddOpenTelemetry().WithMetrics(builder => builder.AddOtlpExporter());
    
    // Build the ASP.NET Core web application.
    var app = builder.Build();
    
    // Start the ASP.NET Core web application.
    app.Run();
    

OpenTelemetry configurations

The following OpenTelemetry configurations can be accessed through environment variables while using the Azure Monitor OpenTelemetry Distros.

Environment variable Description
APPLICATIONINSIGHTS_CONNECTION_STRING Set it to the connection string for your Application Insights resource.
APPLICATIONINSIGHTS_STATSBEAT_DISABLED Set it to true to opt out of internal metrics collection.
OTEL_RESOURCE_ATTRIBUTES Key-value pairs to be used as resource attributes. For more information about resource attributes, see the Resource SDK specification.
OTEL_SERVICE_NAME Sets the value of the service.name resource attribute. If service.name is also provided in OTEL_RESOURCE_ATTRIBUTES, then OTEL_SERVICE_NAME takes precedence.

Frequently asked questions

This section provides answers to common questions.

What is OpenTelemetry?

It's a new open-source standard for observability. Learn more at OpenTelemetry.

Why is Microsoft Azure Monitor investing in OpenTelemetry?

Microsoft is investing in OpenTelemetry for the following reasons:

  • It's vendor-neutral and provides consistent APIs/SDKs across languages.
  • Over time, we believe OpenTelemetry will enable Azure Monitor customers to observe applications written in languages beyond our supported languages.
  • It expands the types of data you can collect through a rich set of instrumentation libraries.
  • OpenTelemetry Software Development Kits (SDKs) tend to be more performant at scale than their predecessors, the Application Insights SDKs.
  • OpenTelemetry aligns with Microsoft's strategy to embrace open source.

What's the status of OpenTelemetry?

See OpenTelemetry Status.

What is the Azure Monitor OpenTelemetry Distro?

You can think of it as a thin wrapper that bundles together all the OpenTelemetry components for a first-class experience on Azure. This wrapper is also called a distribution in OpenTelemetry.

Why should I use the Azure Monitor OpenTelemetry Distro?

There are several advantages to using the Azure Monitor OpenTelemetry Distro over native OpenTelemetry from the community:

In the spirit of OpenTelemetry, we designed the distro to be open and extensible. For example, you can add:

  • An OpenTelemetry Protocol (OTLP) exporter and send to a second destination simultaneously
  • Other instrumentation libraries not included in the distro

Because the Distro provides an OpenTelemetry distribution, the Distro supports anything supported by OpenTelemetry. For example, you can add more telemetry processors, exporters, or instrumentation libraries, if OpenTelemetry supports them.

Note

The Distro sets the sampler to a custom, fixed-rate sampler for Application Insights. You can change this to a different sampler, but doing so might disable some of the Distro's included capabilities. For more information about the supported sampler, see the Enable Sampling section of Configure Azure Monitor OpenTelemetry.

For languages without a supported standalone OpenTelemetry exporter, the Azure Monitor OpenTelemetry Distro is the only currently supported way to use OpenTelemetry with Azure Monitor. For languages with a supported standalone OpenTelemetry exporter, you have the option of using either the Azure Monitor OpenTelemetry Distro or the appropriate standalone OpenTelemetry exporter depending on your telemetry scenario. For more information, see When should I use the Azure Monitor OpenTelemetry exporter?.

How can I test out the Azure Monitor OpenTelemetry Distro?

Check out our enablement docs for .NET, Java, JavaScript (Node.js), and Python.

Should I use OpenTelemetry or the Application Insights SDK?

We recommend using the OpenTelemetry Distro unless you require a feature that is only available with formal support in the Application Insights SDK.

Adopting OpenTelemetry now prevents having to migrate at a later date.

When should I use the Azure Monitor OpenTelemetry exporter?

For ASP.NET Core, Java, Node.js, and Python, we recommend using the Azure Monitor OpenTelemetry Distro. It's one line of code to get started.

For all other .NET scenarios, including classic ASP.NET, console apps, Windows Forms (WinForms), etc., we recommend using the .NET Azure Monitor OpenTelemetry exporter: Azure.Monitor.OpenTelemetry.Exporter.

For more complex Python telemetry scenarios that require advanced configuration, we recommend using the Python Azure Monitor OpenTelemetry Exporter.

What's the current release state of features within the Azure Monitor OpenTelemetry Distro?

The following chart breaks out OpenTelemetry feature support for each language.

Feature .NET Node.js Python Java
Distributed tracing
Custom metrics
Standard metrics
Fixed-rate sampling
Offline storage and automatic retries
Exception reporting
Logs collection ⚠️
Custom Events ⚠️ ⚠️ ⚠️
Live metrics
Live Metrics Filtering
Detect Resource Context for VM/VMSS and App Service
Detect Resource Context for Azure Kubernetes Service (AKS) and Functions
Availability Testing Events generated using the Track Availability API
Filter requests, dependencies, logs, and exceptions by anonymous user ID and synthetic source
Filter dependencies, logs, and exceptions by operation name
Adaptive sampling
Profiler ⚠️
Snapshot Debugger

Key

  • ✅ This feature is available to all customers with formal support.
  • ⚠️ This feature is available as a public preview. See Supplemental terms of use for Azure previews.
  • ❌ This feature isn't available or isn't applicable.

Can OpenTelemetry be used for web browsers?

Yes, but we don't recommend it and Azure doesn't support it. OpenTelemetry JavaScript is heavily optimized for Node.js. Instead, we recommend using the Application Insights JavaScript SDK.

When can we expect the OpenTelemetry SDK to be available for use in web browsers?

The OpenTelemetry web SDK doesn't have a determined availability timeline. We're likely several years away from a browser SDK that is a viable alternative to the Application Insights JavaScript SDK.

Can I test OpenTelemetry in a web browser today?

The OpenTelemetry web sandbox is a fork designed to make OpenTelemetry work in a browser. It's not yet possible to send telemetry to Application Insights. The SDK doesn't define general client events.

Is running Application Insights alongside competitor agents like AppDynamics, DataDog, and NewRelic supported?

This practice isn't something we plan to test or support, although our Distros allow you to export to an OTLP endpoint alongside Azure Monitor simultaneously.

Can I use preview features in production environments?

We don't recommend it. See Supplemental terms of use for Azure previews.

What's the difference between manual and automatic instrumentation?

See the OpenTelemetry Overview.

Can I use the OpenTelemetry Collector?

Some customers use the OpenTelemetry Collector as an agent alternative, even though Microsoft doesn't officially support an agent-based approach for application monitoring yet. In the meantime, the open-source community contributed an OpenTelemetry Collector Azure Monitor Exporter that some customers are using to send data to Azure Monitor Application Insights. This is not supported by Microsoft.

What's the difference between OpenCensus and OpenTelemetry?

OpenCensus is the precursor to OpenTelemetry. Microsoft helped bring together OpenTracing and OpenCensus to create OpenTelemetry, a single observability standard for the world. The current production-recommended Python SDK for Azure Monitor is based on OpenCensus. Microsoft is committed to making Azure Monitor based on OpenTelemetry.

In Grafana, why do I see Status: 500. Can't visualize trace events using the trace visualizer?

You could be trying to visualize raw text logs rather than OpenTelemetry traces.

In Application Insights, the 'Traces' table stores raw text logs for diagnostic purposes. They aid in identifying and correlating traces associated with user requests, other events, and exception reports. However, the 'Traces' table doesn't directly contribute to the end-to-end transaction view (waterfall chart) in visualization tools like Grafana.

With the growing adoption of cloud-native practices, there's an evolution in telemetry collection and terminology. OpenTelemetry became a standard for collecting and instrumenting telemetry data. In this context, the term 'Traces' took on a new meaning. Rather than raw logs, 'Traces' in OpenTelemetry refer to a richer, structured form of telemetry that includes spans, which represent individual units of work. These spans are crucial for constructing detailed transaction views, enabling better monitoring and diagnostics of cloud-native applications.

Troubleshooting

Step 1: Enable diagnostic logging

The Azure Monitor Exporter uses EventSource for its internal logging. The exporter logs are available to any EventListener by opting in to the source that's named OpenTelemetry-AzureMonitor-Exporter. For troubleshooting steps, see OpenTelemetry Troubleshooting on GitHub.

Step 2: Test connectivity between your application host and the ingestion service

Application Insights SDKs and agents send telemetry to get ingested as REST calls at our ingestion endpoints. To test connectivity from your web server or application host computer to the ingestion service endpoints, use cURL commands or raw REST requests from PowerShell. For more information, see Troubleshoot missing application telemetry in Azure Monitor Application Insights.

Known issues

The following items are known issues for the Azure Monitor OpenTelemetry Exporters:

  • The operation name is missing from dependency telemetry. The missing operation name causes failures and adversely affects performance tab experience.

  • The device model is missing from request and dependency telemetry. The missing device model adversely affects device cohort analysis.

Support

Select a tab for the language of your choice to discover support options.

OpenTelemetry feedback

To provide feedback: