Azure Event Hubs monitoring data reference

  • Article

This article contains all the monitoring reference information for this service.

See Monitor Azure Event Hubs for details on the data you can collect for Event Hubs and how to use it.

Azure Event Hubs creates monitoring data using Azure Monitor, which is a full stack monitoring service in Azure. Azure Monitor provides a complete set of features to monitor your Azure resources. It can also monitor resources in other clouds and on-premises.

Azure Event Hubs collects the same kinds of monitoring data as other Azure resources that are described in Monitoring data from Azure resources.

Metrics

This section lists all the automatically collected platform metrics for this service. These metrics are also part of the global list of all platform metrics supported in Azure Monitor.

For information on metric retention, see Azure Monitor Metrics overview.

Supported metrics for Microsoft.EventHub/clusters

The following table lists the metrics available for the Microsoft.EventHub/clusters resource type.

Table headings

  • Metric - The metric display name as it appears in the Azure portal.
  • Name in Rest API - Metric name as referred to in the REST API.
  • Unit - Unit of measure.
  • Aggregation - The default aggregation type. Valid values: Average, Minimum, Maximum, Total, Count.
  • Dimensions - Dimensions available for the metric.
  • Time Grains - Intervals at which the metric is sampled. For example, PT1M indicates that the metric is sampled every minute, PT30M every 30 minutes, PT1H every hour, and so on.
  • DS Export- Whether the metric is exportable to Azure Monitor Logs via Diagnostic Settings. For information on exporting metrics, see Create diagnostic settings in Azure Monitor.
Metric Name in REST API Unit Aggregation Dimensions Time Grains DS Export
ActiveConnections

Total Active Connections for Microsoft.EventHub.		 ActiveConnections Count Average <none> PT1M No
Available Memory

Available memory for the Event Hub Cluster as a percentage of total memory.		 AvailableMemory Percent Maximum Role PT1M No
Capture Backlog.

Capture Backlog for Microsoft.EventHub.		 CaptureBacklog Count Total (Sum) <none> PT1M No
Captured Bytes.

Captured Bytes for Microsoft.EventHub.		 CapturedBytes Bytes Total (Sum) <none> PT1M No
Captured Messages.

Captured Messages for Microsoft.EventHub.		 CapturedMessages Count Total (Sum) <none> PT1M No
Connections Closed.

Connections Closed for Microsoft.EventHub.		 ConnectionsClosed Count Average <none> PT1M No
Connections Opened.

Connections Opened for Microsoft.EventHub.		 ConnectionsOpened Count Average <none> PT1M No
CPU

CPU utilization for the Event Hub Cluster as a percentage		 CPU Percent Maximum Role PT1M No
Incoming Bytes.

Incoming Bytes for Microsoft.EventHub.		 IncomingBytes Bytes Total (Sum) <none> PT1M Yes
Incoming Messages

Incoming Messages for Microsoft.EventHub.		 IncomingMessages Count Total (Sum) <none> PT1M Yes
Incoming Requests

Incoming Requests for Microsoft.EventHub.		 IncomingRequests Count Total (Sum) <none> PT1M Yes
Outgoing Bytes.

Outgoing Bytes for Microsoft.EventHub.		 OutgoingBytes Bytes Total (Sum) <none> PT1M Yes
Outgoing Messages

Outgoing Messages for Microsoft.EventHub.		 OutgoingMessages Count Total (Sum) <none> PT1M Yes
Quota Exceeded Errors.

Quota Exceeded Errors for Microsoft.EventHub.		 QuotaExceededErrors Count Total (Sum) OperationResult PT1M No
Server Errors.

Server Errors for Microsoft.EventHub.		 ServerErrors Count Total (Sum) OperationResult PT1M No
Size

Size of an EventHub in Bytes.		 Size Bytes Average, Minimum, Maximum Role PT1M No
Successful Requests

Successful Requests for Microsoft.EventHub.		 SuccessfulRequests Count Total (Sum) OperationResult PT1M No
Throttled Requests.

Throttled Requests for Microsoft.EventHub.		 ThrottledRequests Count Total (Sum) OperationResult PT1M No
User Errors.

User Errors for Microsoft.EventHub.		 UserErrors Count Total (Sum) OperationResult PT1M No

Supported metrics for Microsoft.EventHub/Namespaces

The following table lists the metrics available for the Microsoft.EventHub/Namespaces resource type.

Table headings

  • Metric - The metric display name as it appears in the Azure portal.
  • Name in Rest API - Metric name as referred to in the REST API.
  • Unit - Unit of measure.
  • Aggregation - The default aggregation type. Valid values: Average, Minimum, Maximum, Total, Count.
  • Dimensions - Dimensions available for the metric.
  • Time Grains - Intervals at which the metric is sampled. For example, PT1M indicates that the metric is sampled every minute, PT30M every 30 minutes, PT1H every hour, and so on.
  • DS Export- Whether the metric is exportable to Azure Monitor Logs via Diagnostic Settings. For information on exporting metrics, see Create diagnostic settings in Azure Monitor.
Metric Name in REST API Unit Aggregation Dimensions Time Grains DS Export
ActiveConnections

Total Active Connections for Microsoft.EventHub.		 ActiveConnections Count Maximum, Minimum, Average <none> PT1M No
Capture Backlog.

Capture Backlog for Microsoft.EventHub.		 CaptureBacklog Count Total (Sum) EntityName PT1M No
Captured Bytes.

Captured Bytes for Microsoft.EventHub.		 CapturedBytes Bytes Total (Sum) EntityName PT1M No
Captured Messages.

Captured Messages for Microsoft.EventHub.		 CapturedMessages Count Total (Sum) EntityName PT1M No
Connections Closed.

Connections Closed for Microsoft.EventHub.		 ConnectionsClosed Count Maximum EntityName PT1M No
Connections Opened.

Connections Opened for Microsoft.EventHub.		 ConnectionsOpened Count Maximum EntityName PT1M No
Archive backlog messages (Deprecated)

Event Hub archive messages in backlog for a namespace (Deprecated)		 EHABL Count Total (Sum) <none> PT1M Yes
Archive message throughput (Deprecated)

Event Hub archived message throughput in a namespace (Deprecated)		 EHAMBS Bytes Total (Sum) <none> PT1M Yes
Archive messages (Deprecated)

Event Hub archived messages in a namespace (Deprecated)		 EHAMSGS Count Total (Sum) <none> PT1M Yes
Incoming bytes (Deprecated)

Event Hub incoming message throughput for a namespace (Deprecated)		 EHINBYTES Bytes Total (Sum) <none> PT1M Yes
Incoming bytes (obsolete) (Deprecated)

Event Hub incoming message throughput for a namespace. This metric is deprecated. Please use Incoming bytes metric instead (Deprecated)		 EHINMBS Bytes Total (Sum) <none> PT1M Yes
Incoming Messages (Deprecated)

Total incoming messages for a namespace (Deprecated)		 EHINMSGS Count Total (Sum) <none> PT1M Yes
Outgoing bytes (Deprecated)

Event Hub outgoing message throughput for a namespace (Deprecated)		 EHOUTBYTES Bytes Total (Sum) <none> PT1M Yes
Outgoing bytes (obsolete) (Deprecated)

Event Hub outgoing message throughput for a namespace. This metric is deprecated. Please use Outgoing bytes metric instead (Deprecated)		 EHOUTMBS Bytes Total (Sum) <none> PT1M Yes
Outgoing Messages (Deprecated)

Total outgoing messages for a namespace (Deprecated)		 EHOUTMSGS Count Total (Sum) <none> PT1M Yes
Failed Requests (Deprecated)

Total failed requests for a namespace (Deprecated)		 FAILREQ Count Total (Sum) <none> PT1M Yes
Incoming Bytes.

Incoming Bytes for Microsoft.EventHub.		 IncomingBytes Bytes Total (Sum) EntityName PT1M Yes
Incoming Messages

Incoming Messages for Microsoft.EventHub.		 IncomingMessages Count Total (Sum) EntityName PT1M Yes
Incoming Requests

Incoming Requests for Microsoft.EventHub.		 IncomingRequests Count Total (Sum) EntityName PT1M Yes
Incoming Messages (obsolete) (Deprecated)

Total incoming messages for a namespace. This metric is deprecated. Please use Incoming Messages metric instead (Deprecated)		 INMSGS Count Total (Sum) <none> PT1M Yes
Incoming Requests (Deprecated)

Total incoming send requests for a namespace (Deprecated)		 INREQS Count Total (Sum) <none> PT1M Yes
Internal Server Errors (Deprecated)

Total internal server errors for a namespace (Deprecated)		 INTERR Count Total (Sum) <none> PT1M Yes
Other Errors (Deprecated)

Total failed requests for a namespace (Deprecated)		 MISCERR Count Total (Sum) <none> PT1M Yes
CPU

CPU usage metric for Premium SKU namespaces.		 NamespaceCpuUsage Percent Maximum, Minimum, Average Replica PT1M No
Memory Usage

Memory usage metric for Premium SKU namespaces.		 NamespaceMemoryUsage Percent Maximum, Minimum, Average Replica PT1M No
Outgoing Bytes.

Outgoing Bytes for Microsoft.EventHub.		 OutgoingBytes Bytes Total (Sum) EntityName PT1M Yes
Outgoing Messages

Outgoing Messages for Microsoft.EventHub.		 OutgoingMessages Count Total (Sum) EntityName PT1M Yes
Outgoing Messages (obsolete) (Deprecated)

Total outgoing messages for a namespace. This metric is deprecated. Please use Outgoing Messages metric instead (Deprecated)		 OUTMSGS Count Total (Sum) <none> PT1M Yes
Quota Exceeded Errors.

Quota Exceeded Errors for Microsoft.EventHub.		 QuotaExceededErrors Count Total (Sum) EntityName, OperationResult PT1M No
ReplicationLagCount

Replication lag by message count		 ReplicationLagCount Count Maximum, Minimum, Average EntityName PT1M No
ReplicationLagDuration

Replication lag by time duration		 ReplicationLagDuration Seconds Maximum, Minimum, Average EntityName PT1M Yes
Server Errors.

Server Errors for Microsoft.EventHub.		 ServerErrors Count Total (Sum) EntityName, OperationResult PT1M No
Size

Size of an EventHub in Bytes.		 Size Bytes Average, Minimum, Maximum EntityName PT1M No
Successful Requests

Successful Requests for Microsoft.EventHub.		 SuccessfulRequests Count Total (Sum) EntityName, OperationResult PT1M No
Successful Requests (Deprecated)

Total successful requests for a namespace (Deprecated)		 SUCCREQ Count Total (Sum) <none> PT1M Yes
Server Busy Errors (Deprecated)

Total server busy errors for a namespace (Deprecated)		 SVRBSY Count Total (Sum) <none> PT1M Yes
Throttled Requests.

Throttled Requests for Microsoft.EventHub.		 ThrottledRequests Count Total (Sum) EntityName, OperationResult PT1M No
User Errors.

User Errors for Microsoft.EventHub.		 UserErrors Count Total (Sum) EntityName, OperationResult PT1M No

The following tables list all the automatically collected platform metrics collected for Azure Event Hubs. The resource provider for these metrics is Microsoft.EventHub/clusters or Microsoft.EventHub/namespaces.

Request metrics count the number of data and management operations requests. This table provides more information about values from the preceding tables.

Metric name Description
Incoming Requests The number of requests made to the Event Hubs service over a specified period. This metric includes all the data and management plane operations.
Successful Requests The number of successful requests made to the Event Hubs service over a specified period.
Throttled Requests The number of requests that were throttled because the usage was exceeded.

This table provides more information for message metrics from the preceding tables.

Metric name Description
Incoming Messages The number of events or messages sent to Event Hubs over a specified period.
Outgoing Messages The number of events or messages received from Event Hubs over a specified period.
Captured Messages The number of captured messages.
Incoming Bytes Incoming bytes for an event hub over a specified period.
Outgoing Bytes Outgoing bytes for an event hub over a specified period.
Size Size of an event hub in bytes.

Note

  • These values are point-in-time values. Incoming messages that are consumed immediately after that point-in-time might not be reflected in these metrics.
  • The Incoming Requests metric includes all the data and management plane operations. The Incoming Messages metric gives you the total number of events that are sent to the event hub. For example, if you send a batch of 100 events to an event hub, it counts as 1 incoming request and 100 incoming messages.

This table provides more information for capture metrics from the preceding tables.

Metric name Description
Captured Messages The number of captured messages.
Captured Bytes Captured bytes for an event hub.
Capture Backlog Capture backlog for an event hub.

This table provides more information for connection metrics from the preceding tables.

Metric name Description
Active Connections The number of active connections on a namespace and on an entity (event hub) in the namespace. Value for this metric is a point-in-time value. Connections that were active immediately after that point-in-time might not be reflected in the metric.
Connections Opened The number of open connections.
Connections Closed The number of closed connections.

This table provides more information for error metrics from the preceding tables.

Metric name Description
Server Errors The number of requests not processed because of an error in the Event Hubs service over a specified period.
User Errors The number of requests not processed because of user errors over a specified period.
Quota Exceeded Errors The number of errors caused by exceeding quotas over a specified period.

The following two types of errors are classified as user errors:

  1. Client-side errors (In HTTP that would be 400 errors).
  2. Errors that occur while processing messages.

Note

Logic Apps creates epoch receivers. Receivers can be moved from one node to another depending on the service load. During those moves, ReceiverDisconnection exceptions might occur. They are counted as user errors on the Event Hubs service side. Logic Apps can collect failures from Event Hubs clients so that you can view them in user logs.

Metric dimensions

For information about what metric dimensions are, see Multi-dimensional metrics.

This service has the following dimensions associated with its metrics.

Dimension name Description
EntityName Name of the event hub. With the 'Incoming Requests' metric, the Entity Name dimension has a value of -NamespaceOnlyMetric- in addition to all your event hubs. It represents the requests that were made at the namespace level. Examples include a request to list all event hubs in the namespace or requests to entities that failed authentication or authorization.
OperationResult Either indicates success or the appropriate error state, such as serverbusy, clienterror or quotaexceeded.

Adding dimensions to your metrics is optional. If you don't add dimensions, metrics are specified at the namespace level.

Note

When you enable metrics in a diagnostic setting, dimension information isn't currently included as part of the information sent to a storage account, event hub, or log analytics.

Resource logs

This section lists the types of resource logs you can collect for this service. The section pulls from the list of all resource logs category types supported in Azure Monitor.

Supported resource logs for Microsoft.EventHub/Namespaces

Category Category display name Log table Supports basic log plan Supports ingestion-time transformation Example queries Costs to export
ApplicationMetricsLogs Application Metrics Logs AZMSApplicationMetricLogs

Captures application metrics(incoming/outgoing, successful/failed, etc. message delivery) for Azure Event Hubs and Azure Service Bus.

 Yes No Yes
ArchiveLogs Archive Logs AZMSArchiveLogs

Captures information about Event Hubs capture operations, specifically, logs related to capture errors.

 Yes No No
AutoScaleLogs Auto Scale Logs AZMSAutoscaleLogs

Captures auto-inflate operations done on an Event Hubs namespace.

 Yes No No
CustomerManagedKeyUserLogs Customer Managed Key Logs No No No
DataDRLogs DataDR Logs AzureDiagnostics

Logs from multiple Azure resources.

 No No Queries Yes
DiagnosticErrorLogs Diagnostic Error Logs AZMSDiagnosticErrorLogs

Captures aggregated diagnostic information such as client errors , server busy errors and quota exceeded errors for various data plane access operations (such as send or receive messages) in Azure Event Hubs and Azure Service Bus.

 Yes No Queries Yes
EventHubVNetConnectionEvent VNet/IP Filtering Connection Logs AZMSVnetConnectionEvents

Captures all virtual network and IP filtering logs for Azure Event Hubs and Azure Service Bus. These would only be emitted if namespace allows access from selected networks or from specific IP address (IP Filter rules).

 Yes No Queries No
KafkaCoordinatorLogs Kafka Coordinator Logs AZMSKafkaCoordinatorLogs

Captures kafka coordinator operations related to Event Hubs.

 Yes No No
KafkaUserErrorLogs Kafka User Error Logs AZMSKafkaUserErrorLogs

Captures information about kafka APIs called on Event Hubs.

 Yes No No
OperationalLogs Operational Logs AZMSOperationalLogs

Captures all management operations that are performed on the Azure Event Hubs/Azure Service Bus namespace and its entities.

 Yes No Queries No
RuntimeAuditLogs Runtime Audit Logs AZMSRunTimeAuditLogs

Captures aggregated diagnostic information for various data plane access operations (such as send or receive messages) in Azure Event Hubs and Azure Service Bus. Runtime audit logs are currently available only in premium tier.

 Yes No Queries Yes

Azure Monitor Logs tables

This section lists the Azure Monitor Logs tables relevant to this service, which are available for query by Log Analytics using Kusto queries. The tables contain resource log data and possibly more depending on what is collected and routed to them.

Event Hubs Microsoft.EventHub/namespaces

Event Hubs resource logs

Azure Event Hubs now has the capability to dispatch logs to either of two destination tables: Azure Diagnostic or Resource specific tables in Log Analytics. You could use the toggle available on Azure portal to choose destination tables.

Screenshot of dialog box to set destination table.

Azure Event Hubs uses Kusto tables from Azure Monitor Logs. You can query these tables with Log Analytics. For a list of Kusto tables the service uses, see Azure Monitor Logs table reference.

You can view our sample queries to get started with different log categories.

Important

Dimensions aren't exported to a Log Analytics workspace.

Runtime audit logs

Runtime audit logs capture aggregated diagnostic information for all data plane access operations (such as send or receive events) in Event Hubs.

Note

Runtime audit logs are available only in premium and dedicated tiers.

Runtime audit logs include the elements listed in the following table:

Name Description Supported in Azure Diagnostics Supported in Resource Specific table
ActivityId A randomly generated UUID that ensures uniqueness for the audit activity. Yes Yes
ActivityName Runtime operation name. Yes Yes
ResourceId Resource associated with the activity. Yes Yes
Timestamp Aggregation time. Yes No
TimeGenerated [UTC] Time of executed operation (in UTC) No Yes
Status Status of the activity (success or failure). Yes Yes
Protocol Type of the protocol associated with the operation. Yes Yes
AuthType Type of authentication (Microsoft Entra ID or SAS Policy). Yes Yes
AuthKey Microsoft Entra ID application ID or SAS policy name that's used to authenticate to a resource. Yes Yes
NetworkType Type of the network access: Public or Private. Yes Yes
ClientIP IP address of the client application. Yes Yes
Count Total number of operations performed during the aggregated period of 1 minute. Yes Yes
Properties Metadata that are specific to the data plane operation. Yes Yes
Category Log category Yes No
Provider Name of Service emitting the logs, such as EventHubs No Yes
Type Type of logs emitted No Yes

Here's an example of a runtime audit log entry:

AzureDiagnostics:

{
    "ActivityId": "<activity id>",
    "ActivityName": "ConnectionOpen | Authorization | SendMessage | ReceiveMessage",
    "ResourceId": "/SUBSCRIPTIONS/xxx/RESOURCEGROUPS/<Resource Group Name>/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/<Event Hubs namespace>/eventhubs/<event hub name>",
    "Time": "1/1/2021 8:40:06 PM +00:00",
    "Status": "Success | Failure",
    "Protocol": "AMQP | KAFKA | HTTP | Web Sockets", 
    "AuthType": "SAS | Azure Active Directory", 
    "AuthId": "<AAD application name | SAS policy name>",
    "NetworkType": "Public | Private", 
    "ClientIp": "x.x.x.x",
    "Count": 1,
    "Category": "RuntimeAuditLogs"
 }

Resource specific table entry:

{
    "ActivityId": "<activity id>",
    "ActivityName": "ConnectionOpen | Authorization | SendMessage | ReceiveMessage",
    "ResourceId": "/SUBSCRIPTIONS/xxx/RESOURCEGROUPS/<Resource Group Name>/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/<Event Hubs namespace>/eventhubs/<event hub name>",
    "TimeGenerated (UTC)": "1/1/2021 8:40:06 PM +00:00",
    "Status": "Success | Failure",
    "Protocol": "AMQP | KAFKA | HTTP | Web Sockets", 
    "AuthType": "SAS | Azure Active Directory", 
    "AuthId": "<AAD application name | SAS policy name>",
    "NetworkType": "Public | Private", 
    "ClientIp": "x.x.x.x",
    "Count": 1,
    "Type": "AZMSRuntimeAUditLogs",
    "Provider":"EVENTHUB"
 }

Application metrics logs

Application metrics logs capture the aggregated information on certain metrics related to data plane operations. The captured information includes the following runtime metrics.

Note

Application metrics logs are available only in premium and dedicated tiers.

Name Description
ConsumerLag Indicate the lag between consumers and producers.
NamespaceActiveConnections Details of active connections established from a client to the event hub.
GetRuntimeInfo Obtain run time information from Event Hubs.
GetPartitionRuntimeInfo Obtain the approximate runtime information for a logical partition of an event hub.
IncomingMessages Details of number of messages published to Event Hubs.
IncomingBytes Details of Publisher throughput sent to Event Hubs
OutgoingMessages Details of number of messages consumed from Event Hubs.
OutgoingBytes Details of Consumer throughput from Event Hubs.
OffsetCommit Number of offset commit calls made to the event hub
OffsetFetch Number of offset fetch calls made to the event hub.

Diagnostic Error Logs

Diagnostic error logs capture error messages for any client side, throttling, and Quota exceeded errors. They provide detailed diagnostics for error identification.

Diagnostic Error Logs include elements listed in following table:

Name Description Supported in Azure Diagnostics Supported in AZMSDiagnosticErrorLogs (Resource specific table)
ActivityId A randomly generated UUID that ensures uniqueness for the audit activity. Yes Yes
ActivityName Operation name Yes Yes
NamespaceName Name of Namespace Yes yes
EntityType Type of Entity Yes Yes
EntityName Name of Entity Yes Yes
OperationResult Type of error in Operation (clienterror or serverbusy or quotaexceeded) Yes Yes
ErrorCount Count of identical errors during the aggregation period of 1 minute. Yes Yes
ErrorMessage Detailed Error Message Yes Yes
ResourceProvider Name of Service emitting the logs. Possible values: Microsoft.EventHub and Microsoft.ServiceBus Yes Yes
Time Generated (UTC) Operation time No Yes
EventTimestamp Operation Time Yes No
Category Log category Yes No
Type Type of Logs emitted No Yes

Here's an example of Diagnostic error log entry:

{
    "ActivityId": "0000000000-0000-0000-0000-00000000000000",
    "SubscriptionId": "<Azure Subscription Id",
    "NamespaceName": "Name of Event Hubs Namespace",
    "EntityType": "EventHub",
    "EntityName": "Name of Event Hub",
    "ActivityName": "SendMessage",
    "ResourceId": "/SUBSCRIPTIONS/xxx/RESOURCEGROUPS/<Resource Group Name>/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/<Event hub namespace name>",,
    "OperationResult": "ServerBusy",
    "ErrorCount": 1,
    "EventTimestamp": "3/27/2024 1:02:29.126 PM +00:00",
    "ErrorMessage": "the request was terminated because the entity is being throttled by the application group with application group name <application group name> and policy name <throttling policy name>.error code: 50013.",
    "category": "DiagnosticErrorLogs"
 }

Resource specific table entry:

{
    "ActivityId": "0000000000-0000-0000-0000-00000000000000",
    "NamespaceName": "Name of Event Hubs Namespace",
    "EntityType": "Event Hub",
    "EntityName": "Name of Event Hub",
    "ActivityName": "SendMessage",
    "ResourceId": "/SUBSCRIPTIONS/xxx/RESOURCEGROUPS/<Resource Group Name>/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/<Event hub namespace name>",,
    "OperationResult": "ServerBusy",
    "ErrorCount": 1,
    "TimeGenerated [UTC]": "1/27/2024 4:02:29.126 PM +00:00",
    "ErrorMessage": "The request was terminated because the entity is being throttled by the application group with application group name <application group name> and policy name <throttling policy name>.error code: 50013.",
    "Type": "AZMSDiagnosticErrorLogs"
 }

Activity log

The linked table lists the operations that may be recorded in the activity log for this service. This is a subset of all the possible resource provider operations in the activity log.

For more information on the schema of activity log entries, see Activity Log schema.

Related content