Add and modify Azure Monitor OpenTelemetry for .NET, Java, Node.js, and Python applications

  • Article

This guide provides instructions on integrating and customizing OpenTelemetry (OTel) instrumentation within Azure Monitor Application Insights.

To learn more about OpenTelemetry concepts, see the OpenTelemetry overview or OpenTelemetry FAQ.

Automatic data collection

The distros automatically collect data by bundling OpenTelemetry instrumentation libraries.

Included instrumentation libraries

The following OpenTelemetry Instrumentation libraries are included as part of the Azure Monitor Application Insights Distro. For more information, see Azure SDK for JavaScript.

Requests

Dependencies

Logs

Instrumentations can be configured using AzureMonitorOpenTelemetryOptions:

// Import Azure Monitor OpenTelemetry
const { useAzureMonitor, AzureMonitorOpenTelemetryOptions } = require("@azure/monitor-opentelemetry");
// Import OpenTelemetry HTTP Instrumentation to get config type
const { HttpInstrumentationConfig } = require("@azure/monitor-opentelemetry");
    // Import HTTP to get type
const { IncomingMessage } = require("http");

// Specific Instrumentation configs could be added
const httpInstrumentationConfig: HttpInstrumentationConfig = {
    ignoreIncomingRequestHook: (request: IncomingMessage) => {
        return false; //Return true if you want to ignore a specific request 
    },
    enabled: true
};
// Instrumentations configuration
const options: AzureMonitorOpenTelemetryOptions = {
instrumentationOptions: {
    http: httpInstrumentationConfig,
    azureSdk: { enabled: true },
    mongoDb: { enabled: true },
    mySql: { enabled: true },
    postgreSql: { enabled: true },
    redis: { enabled: true },
    redis4: { enabled: true },
}
};

// Enable Azure Monitor integration
useAzureMonitor(options);

Footnotes

  • ¹: Supports automatic reporting of unhandled/uncaught exceptions
  • ²: Supports OpenTelemetry Metrics
  • ³: By default, logging is only collected at INFO level or higher. To change this setting, see the configuration options.
  • ⁴: By default, logging is only collected when that logging is performed at the WARNING level or higher.

Note

The Azure Monitor OpenTelemetry Distros include custom mapping and logic to automatically emit Application Insights standard metrics.

Tip

All OpenTelemetry metrics whether automatically collected from instrumentation libraries or manually collected from custom coding are currently considered Application Insights "custom metrics" for billing purposes. Learn more.

Add a community instrumentation library

You can collect more data automatically when you include instrumentation libraries from the OpenTelemetry community.

Caution

We don't support or guarantee the quality of community instrumentation libraries. To suggest one for our distro, post or up-vote in our feedback community. Be aware, some are based on experimental OpenTelemetry specs and might introduce future breaking changes.

Other OpenTelemetry Instrumentations are available here and could be added using TraceHandler in ApplicationInsightsClient:

   // Import the Azure Monitor OpenTelemetry plugin and OpenTelemetry API
   const { useAzureMonitor } = require("@azure/monitor-opentelemetry");
   const { metrics, trace, ProxyTracerProvider } = require("@opentelemetry/api");

   // Import the OpenTelemetry instrumentation registration function and Express instrumentation
   const { registerInstrumentations } = require( "@opentelemetry/instrumentation");
   const { ExpressInstrumentation } = require('@opentelemetry/instrumentation-express');

   // Get the OpenTelemetry tracer provider and meter provider
   const tracerProvider = (trace.getTracerProvider() as ProxyTracerProvider).getDelegate();
   const meterProvider = metrics.getMeterProvider();

   // Enable Azure Monitor integration
   useAzureMonitor();
   
   // Register the Express instrumentation
   registerInstrumentations({
     // List of instrumentations to register
     instrumentations: [
       new ExpressInstrumentation(), // Express instrumentation
     ],
   // OpenTelemetry tracer provider
     tracerProvider: tracerProvider,
     // OpenTelemetry meter provider
     meterProvider: meterProvider
   });

Collect custom telemetry

This section explains how to collect custom telemetry from your application.

Depending on your language and signal type, there are different ways to collect custom telemetry, including:

  • OpenTelemetry API
  • Language-specific logging/metrics libraries
  • Application Insights Classic API

The following table represents the currently supported custom telemetry types:

Language Custom Events Custom Metrics Dependencies Exceptions Page Views Requests Traces
ASP.NET Core
   OpenTelemetry API Yes Yes Yes Yes
   ILogger API Yes
   AI Classic API
Java
   OpenTelemetry API Yes Yes Yes Yes
   Logback, Log4j, JUL Yes Yes
   Micrometer Metrics Yes
   AI Classic API Yes Yes Yes Yes Yes Yes Yes
Node.js
   OpenTelemetry API Yes Yes Yes Yes
Python
   OpenTelemetry API Yes Yes Yes Yes
   Python Logging Module Yes
   Events Extension Yes Yes

Note

Application Insights Java 3.x listens for telemetry that's sent to the Application Insights Classic API. Similarly, Application Insights Node.js 3.x collects events created with the Application Insights Classic API. This makes upgrading easier and fills a gap in our custom telemetry support until all custom telemetry types are supported via the OpenTelemetry API.

Add custom metrics

In this context, the custom metrics term refers to manually instrumenting your code to collect additional metrics beyond what the OpenTelemetry Instrumentation Libraries automatically collect.

The OpenTelemetry API offers six metric "instruments" to cover various metric scenarios and you need to pick the correct "Aggregation Type" when visualizing metrics in Metrics Explorer. This requirement is true when using the OpenTelemetry Metric API to send metrics and when using an instrumentation library.

The following table shows the recommended aggregation types for each of the OpenTelemetry Metric Instruments.

OpenTelemetry Instrument Azure Monitor Aggregation Type
Counter Sum
Asynchronous Counter Sum
Histogram Min, Max, Average, Sum, and Count
Asynchronous Gauge Average
UpDownCounter Sum
Asynchronous UpDownCounter Sum

Caution

Aggregation types beyond what's shown in the table typically aren't meaningful.

The OpenTelemetry Specification describes the instruments and provides examples of when you might use each one.

Tip

The histogram is the most versatile and most closely equivalent to the Application Insights GetMetric Classic API. Azure Monitor currently flattens the histogram instrument into our five supported aggregation types, and support for percentiles is underway. Although less versatile, other OpenTelemetry instruments have a lesser impact on your application's performance.

Histogram example

// Import the Azure Monitor OpenTelemetry plugin and OpenTelemetry API
const { useAzureMonitor } = require("@azure/monitor-opentelemetry");
const { metrics } = require("@opentelemetry/api");

// Enable Azure Monitor integration
useAzureMonitor();

// Get the meter for the "testMeter" namespace
const meter =  metrics.getMeter("testMeter");

// Create a histogram metric
let histogram = meter.createHistogram("histogram");

// Record values to the histogram metric with different tags
histogram.record(1, { "testKey": "testValue" });
histogram.record(30, { "testKey": "testValue2" });
histogram.record(100, { "testKey2": "testValue" });

Counter example

// Import the Azure Monitor OpenTelemetry plugin and OpenTelemetry API
const { useAzureMonitor } = require("@azure/monitor-opentelemetry");
const { metrics } = require("@opentelemetry/api");

// Enable Azure Monitor integration
useAzureMonitor();

// Get the meter for the "testMeter" namespace
const meter =  metrics.getMeter("testMeter");

// Create a counter metric
let counter = meter.createCounter("counter");

// Add values to the counter metric with different tags
counter.add(1, { "testKey": "testValue" });
counter.add(5, { "testKey2": "testValue" });
counter.add(3, { "testKey": "testValue2" });

Gauge example

// Import the useAzureMonitor function and the metrics module from the @azure/monitor-opentelemetry and @opentelemetry/api packages, respectively.
const { useAzureMonitor } = require("@azure/monitor-opentelemetry");
const { metrics } = require("@opentelemetry/api");

// Enable Azure Monitor integration.
useAzureMonitor();

// Get the meter for the "testMeter" meter name.
const meter = metrics.getMeter("testMeter");

// Create an observable gauge metric with the name "gauge".
let gauge = meter.createObservableGauge("gauge");

// Add a callback to the gauge metric. The callback will be invoked periodically to generate a new value for the gauge metric.
gauge.addCallback((observableResult: ObservableResult) => {
    // Generate a random number between 0 and 99.
    let randomNumber = Math.floor(Math.random() * 100);

    // Set the value of the gauge metric to the random number.
    observableResult.observe(randomNumber, {"testKey": "testValue"});
});

Add custom exceptions

Select instrumentation libraries automatically report exceptions to Application Insights. However, you might want to manually report exceptions beyond what instrumentation libraries report. For instance, exceptions caught by your code aren't ordinarily reported. You might wish to report them to draw attention in relevant experiences including the failures section and end-to-end transaction views.

The Node.js SDK will only export these manually recorded span-based exceptions to Application Insights as exceptions if they are recorded on the children of remote, internal spans, or if the exception is recorded on a top level span.

// Import the Azure Monitor OpenTelemetry plugin and OpenTelemetry API
const { useAzureMonitor } = require("@azure/monitor-opentelemetry");
const { trace } = require("@opentelemetry/api");

// Enable Azure Monitor integration
useAzureMonitor();

// Get the tracer for the "testTracer" namespace
const tracer = trace.getTracer("testTracer");

// Start a span with the name "hello"
let span = tracer.startSpan("hello");

// Try to throw an error
try {
    throw new Error("Test Error");
}

// Catch the error and record it to the span
catch(error){
    span.recordException(error);
}

Add custom spans

You might want to add a custom span in two scenarios. First, when there's a dependency request not already collected by an instrumentation library. Second, when you wish to model an application process as a span on the end-to-end transaction view.

// Import the Azure Monitor OpenTelemetry plugin and OpenTelemetry API
const { useAzureMonitor } = require("@azure/monitor-opentelemetry");
const { trace } = require("@opentelemetry/api");

// Enable Azure Monitor integration
useAzureMonitor();

// Get the tracer for the "testTracer" namespace
const tracer = trace.getTracer("testTracer");

// Start a span with the name "hello"
let span = tracer.startSpan("hello");

// End the span
span.end();

Send custom telemetry using the Application Insights Classic API

We recommend you use the OpenTelemetry APIs whenever possible, but there might be some scenarios when you have to use the Application Insights Classic API.

If you want to add custom events or access the Application Insights API, replace the @azure/monitor-opentelemetry package with the applicationinsights v3 Beta package. It offers the same methods and interfaces, and all sample code for @azure/monitor-opentelemetry applies to the v3 Beta package.

You need to use the applicationinsights v3 Beta package to send custom telemetry using the Application Insights classic API. (https://www.npmjs.com/package/applicationinsights/v/beta)

// Import the TelemetryClient class from the Application Insights SDK for JavaScript.
const { TelemetryClient } = require("applicationinsights");

// Create a new TelemetryClient instance.
const telemetryClient = new TelemetryClient();

Then use the TelemetryClient to send custom telemetry:

Events

// Create an event telemetry object.
let eventTelemetry = {
    name: "testEvent"
};

// Send the event telemetry object to Azure Monitor Application Insights.
telemetryClient.trackEvent(eventTelemetry);

Logs

// Create a trace telemetry object.
let traceTelemetry = {
    message: "testMessage",
    severity: "Information"
};

// Send the trace telemetry object to Azure Monitor Application Insights.
telemetryClient.trackTrace(traceTelemetry);

Exceptions

// Try to execute a block of code.
try {
    ...
}

// If an error occurs, catch it and send it to Azure Monitor Application Insights as an exception telemetry item.
catch (error) {
    let exceptionTelemetry = {
    exception: error,
    severity: "Critical"
    };
    telemetryClient.trackException(exceptionTelemetry);
}

Modify telemetry

This section explains how to modify telemetry.

Add span attributes

These attributes might include adding a custom property to your telemetry. You might also use attributes to set optional fields in the Application Insights schema, like Client IP.

Add a custom property to a Span

Any attributes you add to spans are exported as custom properties. They populate the customDimensions field in the requests, dependencies, traces, or exceptions table.

// Import the necessary packages.
const { useAzureMonitor } = require("@azure/monitor-opentelemetry");
const { ReadableSpan, Span, SpanProcessor } = require("@opentelemetry/sdk-trace-base");
const { SemanticAttributes } = require("@opentelemetry/semantic-conventions");

// Create a new SpanEnrichingProcessor class.
class SpanEnrichingProcessor implements SpanProcessor {
  forceFlush(): Promise<void> {
    return Promise.resolve();
  }

  shutdown(): Promise<void> {
    return Promise.resolve();
  }

  onStart(_span: Span): void {}

  onEnd(span: ReadableSpan) {
    // Add custom dimensions to the span.
    span.attributes["CustomDimension1"] = "value1";
    span.attributes["CustomDimension2"] = "value2";
  }
}

// Enable Azure Monitor integration.
const options: AzureMonitorOpenTelemetryOptions = {
    // Add the SpanEnrichingProcessor
    spanProcessors: [new SpanEnrichingProcessor()] 
}
useAzureMonitor(options);

Set the user IP

You can populate the client_IP field for requests by setting an attribute on the span. Application Insights uses the IP address to generate user location attributes and then discards it by default.

Use the custom property example, but replace the following lines of code:

...
// Import the SemanticAttributes class from the @opentelemetry/semantic-conventions package.
const { SemanticAttributes } = require("@opentelemetry/semantic-conventions");

// Create a new SpanEnrichingProcessor class.
class SpanEnrichingProcessor implements SpanProcessor {

    onEnd(span) {
    // Set the HTTP_CLIENT_IP attribute on the span to the IP address of the client.
    span.attributes[SemanticAttributes.HTTP_CLIENT_IP] = "<IP Address>";
    }
}

Set the user ID or authenticated user ID

You can populate the user_Id or user_AuthenticatedId field for requests by using the following guidance. User ID is an anonymous user identifier. Authenticated User ID is a known user identifier.

Important

Consult applicable privacy laws before you set the Authenticated User ID.

Use the custom property example, but replace the following lines of code:

...
// Import the SemanticAttributes class from the @opentelemetry/semantic-conventions package.
import { SemanticAttributes } from "@opentelemetry/semantic-conventions";

// Create a new SpanEnrichingProcessor class.
class SpanEnrichingProcessor implements SpanProcessor {

    onEnd(span: ReadableSpan) {
    // Set the ENDUSER_ID attribute on the span to the ID of the user.
    span.attributes[SemanticAttributes.ENDUSER_ID] = "<User ID>";
    }
}

Add log attributes

const { useAzureMonitor } = require("@azure/monitor-opentelemetry");
const bunyan = require('bunyan');

// Instrumentations configuration
const options: AzureMonitorOpenTelemetryOptions = {
    instrumentationOptions: {
        // Instrumentations generating logs
        bunyan: { enabled: true },
    }
};

// Enable Azure Monitor integration
useAzureMonitor(options);

var log = bunyan.createLogger({ name: 'testApp' });
log.info({
    "testAttribute1": "testValue1",
    "testAttribute2": "testValue2",
    "testAttribute3": "testValue3"
}, 'testEvent');

Get the trace ID or span ID

You can obtain the Trace ID and Span ID of the currently active Span using following steps.

Get the request trace ID and the span ID in your code:

// Import the trace module from the OpenTelemetry API.
const { trace } = require("@opentelemetry/api");

// Get the span ID and trace ID of the active span.
let spanId = trace.getActiveSpan().spanContext().spanId;
let traceId = trace.getActiveSpan().spanContext().traceId;

Next steps