从 Splunk 到 Azure Monitor 日志查询Splunk to Azure Monitor log query

本文旨在帮助熟悉 Splunk 的用户通过学习 Kusto 查询语言，在 Azure Monitor 中编写日志查询。This article is intended to assist users who are familiar with Splunk to learn the Kusto query language to write log queries in Azure Monitor. 其中将两者做了直接的比较，让用户了解它们的主要差异和相似之处，并抉机利用现有的知识。Direct comparisons are made between the two to understand key differences and also similarities where you can leverage your existing knowledge.

结构和概念Structure and concepts

下表比较了 Splunk 和 Azure Monitor 日志的概念与数据结构。The following table compares concepts and data structures between Splunk and Azure Monitor logs.

概念Concept SplunkSplunk Azure MonitorAzure Monitor 注释Comment
部署单元Deployment unit clustercluster clustercluster Azure Monitor 允许跨群集进行任意查询，Azure Monitor allows arbitrary cross cluster queries. Splunk 则不允许。Splunk does not.
数据缓存Data caches 存储桶buckets 缓存和保留策略Caching and retention policies 控制数据的保留期和缓存级别。Controls the period and caching level for the data. 此设置直接影响查询性能和部署成本。This setting directly impacts the performance of queries and cost of the deployment.
数据的逻辑分区Logical partition of data indexindex databasedatabase 允许数据的逻辑隔离。Allows logical separation of the data. 这两个实现都允许跨这些分区的联合与联接。Both implementations allow unions and joining across these partitions.
结构化事件元数据Structured event metadata 不适用N/A 表table Splunk 没有向事件元数据搜索语言公开的概念。Splunk does not have the concept exposed to the search language of event metadata. Azure Monitor 日志具有表的概念，表包含列。Azure Monitor logs has the concept of a table, which has columns. 每个事件实例映射到行。Each event instance is mapped to a row.
数据记录Data record eventevent 行row 仅限术语变化。Terminology change only.
数据记录属性Data record attribute 字段field 列column 在 Azure Monitor 中，此概念预定义为表结构的一部分。In Azure Monitor, this is predefined as part of the table structure. 在 Splunk 中，每个事件有自身的字段集。In Splunk, each event has its own set of fields.
类型Types 数据类型datatype 数据类型datatype Azure Monitor 数据类型更明确，因为它们是在列中设置的。Azure Monitor datatypes are more explicit as they are set on the columns. 两者都能动态处理数据类型，数据类型集（包括 JSON 支持）大致相同。Both have the ability to work dynamically with data types and roughly equivalent set of datatypes including JSON support.
查询和搜索Query and search 搜索search 查询query Azure Monitor 和 Splunk 的概念在本质上相同。Concepts are essentially the same between both Azure Monitor and Splunk.
数据引入时间Event ingestion time 系统时间System Time ingestion_time()ingestion_time() 在 Splunk 中，每个事件将获取编制事件索引时的系统时间戳。In Splunk, each event gets a system timestamp of the time that the event was indexed. 在 Azure Monitor 中，可以定义名为 ingestion_time 的策略，用于公开可通过 ingestion_time() 函数引用的系统列。In Azure Monitor, you can define a policy called ingestion_time that exposes a system column that can be referenced through the ingestion_time() function.

函数Functions

下表指定了 Azure Monitor 中等效于 Splunk 函数的函数。The following table specifies functions in Azure Monitor that are equivalent to Splunk functions.

SplunkSplunk Azure MonitorAzure Monitor 注释Comment
strcatstrcat strcat()strcat() (1)(1)
splitsplit split()split() (1)(1)
ifif iff()iff() (1)(1)
tonumbertonumber todouble()todouble()
tolong()tolong()
toint()toint()		 (1)(1)
upperupper
lowerlower		 toupper()toupper()
tolower()tolower()		 (1)(1)
replacereplace replace()replace() (1)(1)
另请注意，尽管这两个产品中的 replace() 都采用三个参数，但这些参数不同。Also note that while replace() takes three parameters in both products, the parameters are different.
substrsubstr substring()substring() (1)(1)
另请注意，Splunk 使用从 1 开始的索引。Also note that Splunk uses one-based indices. Azure Monitor 记录从 0 开始的索引。Azure Monitor notes zero-based indices.
tolowertolower tolower()tolower() (1)(1)
touppertoupper toupper()toupper() (1)(1)
matchmatch matches regexmatches regex (2)(2)
regexregex matches regexmatches regex 在 Splunk 中，regex 是运算符。In Splunk, regex is an operator. 在 Azure Monitor 中，它是关系运算符。In Azure Monitor, it's a relational operator.
searchmatchsearchmatch == 在 Splunk 中，searchmatch 允许搜索确切的字符串。In Splunk, searchmatch allows searching for the exact string.
randomrandom rand()rand()
rand(n)rand(n)		 Splunk 的函数返回从 0 到 231-1 的数字。Splunk's function returns a number from zero to 231-1. Azure Monitor 返回介于 0.0 和 1.0 之间的数字；如果提供了参数，则返回介于 0 和 n-1 之间的数字。Azure Monitor' returns a number between 0.0 and 1.0, or if a parameter provided, between 0 and n-1.
nownow now()now() (1)(1)
relative_timerelative_time totimespan()totimespan() (1)(1)
在 Azure Monitor 中，与 Splunk 的 relative_time(datetimeVal, offsetVal) 等效的函数是 datetimeVal + totimespan(offsetVal)。In Azure Monitor, Splunk's equivalent of relative_time(datetimeVal, offsetVal) is datetimeVal + totimespan(offsetVal).
例如，search | eval n=relative_time(now(), "-1d@d") 变成了 ... | extend myTime = now() - totimespan("1d")。For example, search | eval n=relative_time(now(), "-1d@d") becomes ... | extend myTime = now() - totimespan("1d").

(1) 在 Splunk 中，使用 eval 运算符调用该函数。(1) In Splunk, the function is invoked with the eval operator. 在 Azure Monitor 中，它用作 extend 或 project 的一部分。In Azure Monitor, it is used as part of extend or project.
(2) 在 Splunk 中，使用 eval 运算符调用该函数。(2) In Splunk, the function is invoked with the eval operator. 在 Azure Monitor 中，可以结合 where 运算符使用该函数。In Azure Monitor, it can be used with the where operator.

运算符Operators

以下部分通过示例演示 Splunk 和 Azure Monitor 如何使用不同的运算符。The following sections give examples of using different operators between Splunk and Azure Monitor.

Note

在以下示例中，Splunk 字段 rule 映射到 Azure Monitor 中的某个表，Splunk 的默认时间戳映射到 Azure Monitor 的 ingestion_time() 列。For the purpose of the following example, the Splunk field rule maps to a table in Azure Monitor, and Splunk's default timestamp maps to the Logs Analytics ingestion_time() column.

在 Splunk 中，可以省略 search 关键字，并指定不带引号的字符串。In Splunk, you can omit the search keyword and specify an unquoted string. 在 Azure Monitor 中，必须在每个查询的开头使用 find，不带引号的字符串是列名，查找值必须是带引号的字符串。In Azure Monitor you must start each query with find, an unquoted string is a column name, and the lookup value must be a quoted string.
SplunkSplunk searchsearch search Session.Id="c8894ffd-e684-43c9-9125-42adc25cd3fc" earliest=-24h
Azure MonitorAzure Monitor findfind find Session.Id=="c8894ffd-e684-43c9-9125-42adc25cd3fc" and ingestion_time()> ago(24h)

筛选器Filter

Azure Monitor 日志查询从包含筛选器的表格结果集开始。Azure Monitor log queries start from a tabular result set where the filter. 在 Splunk 中，筛选是针对当前索引执行的默认操作。In Splunk, filtering is the default operation on the current index. 还可以在 Splunk 中使用 where 运算符，但不建议。You can also use where operator in Splunk, but it is not recommended.
SplunkSplunk searchsearch Event.Rule="330009.2" Session.Id="c8894ffd-e684-43c9-9125-42adc25cd3fc" _indextime>-24h
Azure MonitorAzure Monitor wherewhere Office_Hub_OHubBGTaskError
| where Session_Id == "c8894ffd-e684-43c9-9125-42adc25cd3fc" and ingestion_time() > ago(24h)

获取用于检查的 n 个事件/行Getting n events/rows for inspection

Azure Monitor 日志查询还支持将 take 用作 limit 的别名。Azure Monitor log queries also support take as an alias to limit. 在 Splunk 中，如果结果已排序，则 head 将返回前 n 个结果。In Splunk, if the results are ordered, head will return the first n results. 在 Azure Monitor 中，limit 不会排序，而是返回找到的前 n 行。In Azure Monitor, limit is not ordered but returns the first n rows that are found.
SplunkSplunk headhead Event.Rule=330009.2
| head 100
Azure MonitorAzure Monitor limitlimit Office_Hub_OHubBGTaskError
| limit 100

获取按字段/列排序的前 n 个事件/行Getting the first n events/rows ordered by a field/column

对于底部结果，在 Splunk 中可以使用 tail。For bottom results, in Splunk you use tail. 在 Azure Monitor 中，可以使用 asc 指定排序方向。In Azure Monitor you can specify the ordering direction with asc.
SplunkSplunk headhead Event.Rule="330009.2"
| sort Event.Sequence
| head 20
Azure MonitorAzure Monitor 返回页首top Office_Hub_OHubBGTaskError
| top 20 by Event_Sequence

使用新字段/列扩展结果集Extending the result set with new fields/columns

Splunk 还有一个 eval 函数，该函数不能与 eval 运算符进行比较。Splunk also has an eval function, which is not to be comparable with the eval operator. Splunk 中的 eval 运算符与 Azure Monitor 中的 extend 运算符仅支持标量函数和算术运算符。Both the eval operator in Splunk and the extend operator in Azure Monitor only support scalar functions and arithmetic operators.
SplunkSplunk evaleval Event.Rule=330009.2
| eval state= if(Data.Exception = "0", "success", "error")
Azure MonitorAzure Monitor extendextend Office_Hub_OHubBGTaskError
| extend state = iif(Data_Exception == 0,"success" ,"error")

重命名Rename

Azure Monitor 使用相同的运算符来重命名和新建字段。Azure Monitor uses the same operator to rename and to create a new field. Splunk 有两个独立的运算符：eval 和 rename。Splunk has two separate operators, eval and rename.
SplunkSplunk renamerename Event.Rule=330009.2
| rename Date.Exception as execption
Azure MonitorAzure Monitor extendextend Office_Hub_OHubBGTaskError
| extend exception = Date_Exception

设置结果格式/投影Format results/Projection

Splunk 似乎没有类似于 project-away 的运算符。Splunk does not seem to have an operator similar to project-away. 可以使用 UI 来筛选字段。You can use the UI to filter away fields.
SplunkSplunk tabletable Event.Rule=330009.2
| table rule, state
Azure MonitorAzure Monitor projectproject
project-awayproject-away		 Office_Hub_OHubBGTaskError
| project exception, state

聚合Aggregation

有关不同的聚合函数，请参阅“Azure Monitor 日志查询中的聚合”。See the Aggregations in Azure Monitor log queries for the different aggregation functions.
SplunkSplunk statsstats search (Rule=120502.*)
| stats count by OSEnv, Audience
Azure MonitorAzure Monitor summarizesummarize Office_Hub_OHubBGTaskError
| summarize count() by App_Platform, Release_Audience

JoinJoin

Splunk 中的联接具有很强的限制。Join in Splunk has significant limitations. 子查询限制为 10000 条结果（在部署配置文件中设置），联接形式数目也有限制。The subquery has a limit of 10000 results (set in the deployment configuration file), and there a limited number of join flavors.
SplunkSplunk joinjoin Event.Rule=120103* | stats by Client.Id, Data.Alias | join Client.Id max=0 [search earliest=-24h Event.Rule="150310.0" Data.Hresult=-2147221040]
Azure MonitorAzure Monitor joinjoin cluster("OAriaPPT").database("Office PowerPoint").Office_PowerPoint_PPT_Exceptions
| where Data_Hresult== -2147221040
| join kind = inner (Office_System_SystemHealthMetadata
| summarize by Client_Id, Data_Alias)on Client_Id

排序Sort

在 Splunk 中，若要按升序排序，必须使用 reverse 运算符。In Splunk, to sort in ascending order you must use the reverse operator. Azure Monitor 还支持定义 null 值的放置位置：开头或末尾。Azure Monitor also supports defining where to put nulls, at the beginning or at the end.
SplunkSplunk sortsort Event.Rule=120103
| sort Data.Hresult
| reverse
Azure MonitorAzure Monitor order byorder by Office_Hub_OHubBGTaskError
| order by Data_Hresult, desc

多值扩展Multivalue expand

此运算符在 Splunk 和 Azure Monitor 中类似。This is a similar operator in both Splunk and Azure Monitor.
SplunkSplunk mvexpandmvexpand mvexpand foo
Azure MonitorAzure Monitor mvexpandmvexpand mvexpand foo

结果分面、相关字段Results facets, interesting fields

在 Azure 门户的 Log Analytics 中，仅公开第一列。In Log Analytics in the Azure portal, only the first column is exposed. 可通过 API 查看所有列。All columns are available through the API.
SplunkSplunk fieldsfields Event.Rule=330009.2
| fields App.Version, App.Platform
Azure MonitorAzure Monitor facetsfacets Office_Excel_BI_PivotTableCreate
| facet by App_Branch, App_Version

重复数据删除De-duplicate

可以改用 summarize arg_min() 来反转记录的选择顺序。You can use summarize arg_min() instead to reverse the order of which record gets chosen.
SplunkSplunk dedupdedup Event.Rule=330009.2
| dedup device_id sortby -batterylife
Azure MonitorAzure Monitor summarize arg_max()summarize arg_max() Office_Excel_BI_PivotTableCreate
| summarize arg_max(batterylife, *) by device_id

后续步骤Next steps