Telemetry processors (preview) - Azure Monitor Application Insights for Java

Application Insights Java 3.x can process telemetry data before the data is exported.

Some use cases:

• Mask sensitive data.
• Conditionally add custom dimensions.
• Update the span name, which is used to aggregate similar telemetry in the Azure portal.
• Drop specific span attributes to control ingestion costs.
• Filter out some metrics to control ingestion costs.

Terminology

Before you learn about telemetry processors, you should understand the terms span and log.

A span is a type of telemetry that represents one of:

• An incoming request.
• An outgoing dependency (for example, a remote call to another service).
• An in-process dependency (for example, work being done by subcomponents of the service).

A log is a type of telemetry that represents:

• log data captured from Log4j, Logback, and java.util.logging

For telemetry processors, these span/log components are important:

• Name
• Body
• Attributes

The span name is the primary display for requests and dependencies in the Azure portal. Span attributes represent both standard and custom properties of a given request or dependency.

The trace message or body is the primary display for logs in the Azure portal. Log attributes represent both standard and custom properties of a given log.

Telemetry processor types

Currently, the four types of telemetry processors are

• Attribute processors
• Span processors
• Log processors
• Metric filters

An attribute processor can insert, update, delete, or hash attributes of a telemetry item (span or log). It can also use a regular expression to extract one or more new attributes from an existing attribute.

A span processor can update the telemetry name of requests and dependencies. It can also use a regular expression to extract one or more new attributes from the span name.

A log processor can update the telemetry name of logs. It can also use a regular expression to extract one or more new attributes from the log name.

A metric filter can filter out metrics to help control ingestion cost.

Getting started

To begin, create a configuration file named applicationinsights.json. Save it in the same directory as applicationinsights-agent-*.jar. Use the following template.

{
  "connectionString": "InstrumentationKey=00000000-0000-0000-0000-000000000000",
  "preview": {
    "processors": [
      {
        "type": "attribute",
        ...
      },
      {
        "type": "attribute",
        ...
      },
      {
        "type": "span",
        ...
      },
      {
        "type": "log",
        ...
      },
      {
        "type": "metric-filter",
        ...
      }
    ]
  }
}

Attribute processor

The attribute processor modifies attributes of a span or a log. It can support the ability to include or exclude span or log. It takes a list of actions that are performed in the order that the configuration file specifies. The processor supports these actions:

• insert
• update
• delete
• hash
• extract
• mask

insert

The insert action inserts a new attribute in telemetry item where the key doesn't already exist.

"processors": [
  {
    "type": "attribute",
    "actions": [
      {
        "key": "attribute1",
        "value": "value1",
        "action": "insert"
      }
    ]
  }
]

The insert action requires the following settings:

• key
• Either value or fromAttribute
• action: insert

update

The update action updates an attribute in telemetry item where the key already exists.

"processors": [
  {
    "type": "attribute",
    "actions": [
      {
        "key": "attribute1",
        "value": "newValue",
        "action": "update"
      }
    ]
  }
]

The update action requires the following settings:

• key
• Either value or fromAttribute
• action: update

delete

The delete action deletes an attribute from a telemetry item.

"processors": [
  {
    "type": "attribute",
    "actions": [
      {
        "key": "attribute1",
        "action": "delete"
      }
    ]
  }
]

The delete action requires the following settings:

• key
• action: delete

hash

The hash action hashes (SHA1) an existing attribute value.

"processors": [
  {
    "type": "attribute",
    "actions": [
      {
        "key": "attribute1",
        "action": "hash"
      }
    ]
  }
]

The hash action requires the following settings:

• key
• action: hash

extract

The extract action extracts values by using a regular expression rule from the input key to target keys that the rule specifies. If a target key already exists, the extract action overrides the target key. This action behaves like the span processor toAttributes setting, where the existing attribute is the source.

"processors": [
  {
    "type": "attribute",
    "actions": [
      {
        "key": "attribute1",
        "pattern": "<regular pattern with named matchers>",
        "action": "extract"
      }
    ]
  }
]

The extract action requires the following settings:

• key
• pattern
• action: extract

mask

The mask action masks attribute values by using a regular expression rule specified in the pattern and replace.

"processors": [
  {
    "type": "attribute",
    "actions": [
      {
        "key": "attributeName",
        "pattern": "<regular expression pattern>",
        "replace": "<replacement value>",
        "action": "mask"
      }
    ]
  }
]

The mask action requires the following settings:

• key
• pattern
• replace
• action: mask

pattern can contain a named group placed between ?< and >:. Example: (?<userGroupName>[a-zA-Z.:\/]+)\d+? The group is (?<userGroupName>[a-zA-Z.:\/]+) and userGroupName is the name of the group. pattern can then contain the same named group placed between ${ and } followed by the mask. Example where the mask is **: ${userGroupName}**.

See Telemetry processor examples for masking examples.

Include criteria and exclude criteria

Attribute processors support optional include and exclude criteria. An attribute processor is applied only to telemetry that matches its include criteria (if it's available) and don't match its exclude criteria (if it's available).

To configure this option, under include or exclude (or both), specify at least one matchType and either spanNames or attributes. The include or exclude configuration allows more than one specified condition. All specified conditions must evaluate to true to result in a match.

• Required fields:

  • matchType controls how items in spanNames arrays and attributes arrays are interpreted. Possible values are regexp and strict. Regular expression matches are performed against the entire attribute value, so if you want to match a value that contains abc anywhere in it, then you need to use .*abc.*.

• Optional fields:

  • spanNames must match at least one of the items.
  • attributes specifies the list of attributes to match. All of these attributes must match exactly to result in a match.

Sample usage

"processors": [
  {
    "type": "attribute",
    "include": {
      "matchType": "strict",
      "spanNames": [
        "spanA",
        "spanB"
      ]
    },
    "exclude": {
      "matchType": "strict",
      "attributes": [
        {
          "key": "redact_trace",
          "value": "false"
        }
      ]
    },
    "actions": [
      {
        "key": "credit_card",
        "action": "delete"
      },
      {
        "key": "duplicate_key",
        "action": "delete"
      }
    ]
  }
]

For more information, see Telemetry processor examples.

Span processor

The span processor modifies either the span name or attributes of a span based on the span name. It can support the ability to include or exclude spans.

Name a span

The name section requires the fromAttributes setting. The values from these attributes are used to create a new name, concatenated in the order that the configuration specifies. The processor changes the span name only if all of these attributes are present on the span.

The separator setting is optional. This setting is a string, and you can use split values.

"processors": [
  {
    "type": "span",
    "name": {
      "fromAttributes": [
        "attributeKey1",
        "attributeKey2",
      ],
      "separator": "::"
    }
  }
] 

Extract attributes from the span name

The toAttributes section lists the regular expressions to match the span name against. It extracts attributes based on subexpressions.

The rules setting is required. This setting lists the rules that are used to extract attribute values from the span name.

Extracted attribute names replace the values in the span name. Each rule in the list is a regular expression (regex) pattern string.

Here's how extracted attribute names replace values:

1. The span name is checked against the regex.
2. All named subexpressions of the regex are extracted as attributes if the regex matches.
3. The extracted attributes are added to the span.
4. Each subexpression name becomes an attribute name.
5. The subexpression matched portion becomes the attribute value.
6. The extracted attribute name replaces the matched portion in the span name. If the attributes already exist in the span, they're overwritten.

This process is repeated for all rules in the order they're specified. Each subsequent rule works on the span name that's the output of the previous rule.

"processors": [
  {
    "type": "span",
    "name": {
      "toAttributes": {
        "rules": [
          "rule1",
          "rule2",
          "rule3"
        ]
      }
    }
  }
]

Common span attributes

This section lists some common span attributes that telemetry processors can use.

HTTP spans

Java Database Connectivity spans

The following table describes attributes that you can use in Java Database Connectivity (JDBC) spans:

Include criteria and exclude criteria

Span processors support optional include and exclude criteria. A span processor is applied only to telemetry that matches its include criteria (if it's available) and don't match its exclude criteria (if it's available).

To configure this option, under include or exclude (or both), specify at least one matchType and either spanNames or span attributes. The include or exclude configuration allows more than one specified condition. All specified conditions must evaluate to true to result in a match.

• Required fields:

  • matchType controls how items in spanNames arrays and attributes arrays are interpreted. Possible values are regexp and strict. Regular expression matches are performed against the entire attribute value, so if you want to match a value that contains abc anywhere in it, then you need to use .*abc.*.

• Optional fields:

  • spanNames must match at least one of the items.
  • attributes specifies the list of attributes to match. All of these attributes must match exactly to result in a match.

Sample usage

"processors": [
  {
    "type": "span",
    "include": {
      "matchType": "strict",
      "spanNames": [
        "spanA",
        "spanB"
      ]
    },
    "exclude": {
      "matchType": "strict",
      "attributes": [
        {
          "key": "attribute1",
          "value": "attributeValue1"
        }
      ]
    },
    "name": {
      "toAttributes": {
        "rules": [
          "rule1",
          "rule2",
          "rule3"
        ]
      }
    }
  }
]

For more information, see Telemetry processor examples.

Log processor

The log processor modifies either the log message body or attributes of a log based on the log message body. It can support the ability to include or exclude logs.

Update Log message body

The body section requires the fromAttributes setting. The values from these attributes are used to create a new body, concatenated in the order that the configuration specifies. The processor changes the log body only if all of these attributes are present on the log.

The separator setting is optional. This setting is a string. You can specify it to split values.

"processors": [
  {
    "type": "log",
    "body": {
      "fromAttributes": [
        "attributeKey1",
        "attributeKey2",
      ],
      "separator": "::"
    }
  }
] 

Extract attributes from the log message body

The toAttributes section lists the regular expressions to match the log message body. It extracts attributes based on subexpressions.

The rules setting is required. This setting lists the rules that are used to extract attribute values from the body.

Extracted attribute names replace the values in the log message body. Each rule in the list is a regular expression (regex) pattern string.

Here's how extracted attribute names replace values:

1. The log message body is checked against the regex.
2. All named subexpressions of the regex are extracted as attributes if the regex matches.
3. The extracted attributes are added to the log.
4. Each subexpression name becomes an attribute name.
5. The subexpression matched portion becomes the attribute value.
6. The extracted attribute name replaces the matched portion in the log name. If the attributes already exist in the log, they're overwritten.

This process is repeated for all rules in the order they're specified. Each subsequent rule works on the log name that's the output of the previous rule.

"processors": [
  {
    "type": "log",
    "body": {
      "toAttributes": {
        "rules": [
          "rule1",
          "rule2",
          "rule3"
        ]
      }
    }
  }
]

Include criteria and exclude criteria

Log processors support optional include and exclude criteria. A log processor is applied only to telemetry that matches its include criteria (if it's available) and don't match its exclude criteria (if it's available).

To configure this option, under include or exclude (or both), specify the matchType and attributes. The include or exclude configuration allows more than one specified condition. All specified conditions must evaluate to true to result in a match.

• Required field:
  • matchType controls how items in attributes arrays are interpreted. Possible values are regexp and strict. Regular expression matches are performed against the entire attribute value, so if you want to match a value that contains abc anywhere in it, then you need to use .*abc.*.
  • attributes specifies the list of attributes to match. All of these attributes must match exactly to result in a match.

Sample usage

"processors": [
  {
    "type": "log",
    "include": {
      "matchType": "strict",
      "attributes": [
        {
          "key": "attribute1",
          "value": "value1"
        }
      ]
    },
    "exclude": {
      "matchType": "strict",
      "attributes": [
        {
          "key": "attribute2",
          "value": "value2"
        }
      ]
    },
    "body": {
      "toAttributes": {
        "rules": [
          "rule1",
          "rule2",
          "rule3"
        ]
      }
    }
  }
]

For more information, see Telemetry processor examples.

Metric filter

Metric filters are used to exclude some metrics in order to help control ingestion cost.

Metric filters only support exclude criteria. Metrics that match its exclude criteria aren't exported.

To configure this option, under exclude, specify the matchType one or more metricNames.

• Required field:
  • matchType controls how items in metricNames are matched. Possible values are regexp and strict. Regular expression matches are performed against the entire attribute value, so if you want to match a value that contains abc anywhere in it, then you need to use .*abc.*.
  • metricNames must match at least one of the items.

Sample usage

The following sample shows how to exclude metrics with names "metricA" and "metricB":

"processors": [
  {
    "type": "metric-filter",
    "exclude": {
      "matchType": "strict",
      "metricNames": [
        "metricA",
        "metricB"
      ]
    }
  }
]

The following sample shows how to turn off all metrics including the default autocollected performance metrics like cpu and memory.

"processors": [
  {
    "type": "metric-filter",
    "exclude": {
      "matchType": "regexp",
      "metricNames": [
        ".*"
      ]
    }
  }
]

Default metrics captured by Java agent

Next steps

• To review frequently asked questions (FAQ), see Telemetry processors FAQ