从 Splunk 到 Azure Monitor 日志查询Splunk to Azure Monitor log query

12/07/2020

本文旨在帮助熟悉 Splunk 的用户通过学习 Kusto 查询语言，以在 Azure Monitor 中编写日志查询。This article is intended to assist users who are familiar with Splunk to learn the Kusto query language to write log queries in Azure Monitor. 其中将两者做了直接的比较，让用户了解它们的主要差异和相似之处，并抉机利用现有的知识。Direct comparisons are made between the two to understand key differences and also similarities where you can leverage your existing knowledge.

结构和概念Structure and concepts

下表比较了 Splunk 和 Azure Monitor 日志的概念与数据结构。The following table compares concepts and data structures between Splunk and Azure Monitor logs.

概念Concept	SplunkSplunk	Azure MonitorAzure Monitor	注释Comment
部署单元Deployment unit	clustercluster	clustercluster	Azure Monitor 允许跨群集进行任意查询，Azure Monitor allows arbitrary cross cluster queries. Splunk 则不允许。Splunk does not.
数据缓存Data caches	存储桶buckets	缓存和保留策略Caching and retention policies	控制数据的保留期和缓存级别。Controls the period and caching level for the data. 此设置直接影响查询性能和部署成本。This setting directly impacts the performance of queries and cost of the deployment.
数据的逻辑分区Logical partition of data	索引index	databasedatabase	允许数据的逻辑隔离。Allows logical separation of the data. 这两个实现都允许跨这些分区的联合与联接。Both implementations allow unions and joining across these partitions.
结构化事件元数据Structured event metadata	空值N/A	表table	Splunk 没有向事件元数据搜索语言公开的概念。Splunk does not have the concept exposed to the search language of event metadata. Azure Monitor 日志具有表的概念，表包含列。Azure Monitor logs has the concept of a table, which has columns. 每个事件实例映射到行。Each event instance is mapped to a row.
数据记录Data record	eventevent	行row	仅限术语变化。Terminology change only.
数据记录属性Data record attribute	字段field	列column	在 Azure Monitor 中，此概念预定义为表结构的一部分。In Azure Monitor, this is predefined as part of the table structure. 在 Splunk 中，每个事件有自身的字段集。In Splunk, each event has its own set of fields.
类型Types	数据类型datatype	数据类型datatype	Azure Monitor 数据类型更明确，因为它们是在列中设置的。Azure Monitor datatypes are more explicit as they are set on the columns. 两者都能动态处理数据类型，数据类型集（包括 JSON 支持）大致相同。Both have the ability to work dynamically with data types and roughly equivalent set of datatypes including JSON support.
查询和搜索Query and search	搜索search	queryquery	Azure Monitor 和 Splunk 的概念在本质上相同。Concepts are essentially the same between both Azure Monitor and Splunk.
数据引入时间Event ingestion time	系统时间System Time	ingestion_time()ingestion_time()	在 Splunk 中，每个事件将获取编制事件索引时的系统时间戳。In Splunk, each event gets a system timestamp of the time that the event was indexed. 在 Azure Monitor 中，可以定义名为 ingestion_time 的策略，用于公开可通过 ingestion_time() 函数引用的系统列。In Azure Monitor, you can define a policy called ingestion_time that exposes a system column that can be referenced through the ingestion_time() function.

函数Functions

下表指定了 Azure Monitor 中等效于 Splunk 函数的函数。The following table specifies functions in Azure Monitor that are equivalent to Splunk functions.

SplunkSplunk	Azure MonitorAzure Monitor	注释Comment
strcatstrcat	strcat()strcat()	(1)(1)
splitsplit	split()split()	(1)(1)
ifif	iff()iff()	(1)(1)
tonumbertonumber	todouble()todouble() tolong()tolong() toint()toint()	(1)(1)
upperupper lowerlower	toupper()toupper() tolower()tolower()	(1)(1)
replacereplace	replace()replace()	(1)(1) 另请注意，尽管这两个产品中的 `replace()` 都采用三个参数，但这些参数不同。Also note that while `replace()` takes three parameters in both products, the parameters are different.
substrsubstr	substring()substring()	(1)(1) 另请注意，Splunk 使用从 1 开始的索引。Also note that Splunk uses one-based indices. Azure Monitor 记录从 0 开始的索引。Azure Monitor notes zero-based indices.
tolowertolower	tolower()tolower()	(1)(1)
touppertoupper	toupper()toupper()	(1)(1)
matchmatch	matches regexmatches regex	(2)(2)
regexregex	matches regexmatches regex	在 Splunk 中，`regex` 是运算符。In Splunk, `regex` is an operator. 在 Azure Monitor 中，它是关系运算符。In Azure Monitor, it's a relational operator.
searchmatchsearchmatch	==	在 Splunk 中，`searchmatch` 允许搜索确切的字符串。In Splunk, `searchmatch` allows searching for the exact string.
randomrandom	rand()rand() rand(n)rand(n)	Splunk 的函数返回从 0 到 2³¹-1 的数字。Splunk's function returns a number from zero to 2³¹-1. Azure Monitor 返回介于 0.0 和 1.0 之间的数字；如果提供了参数，则返回介于 0 和 n-1 之间的数字。Azure Monitor' returns a number between 0.0 and 1.0, or if a parameter provided, between 0 and n-1.
nownow	now()now()	(1)(1)
relative_timerelative_time	totimespan()totimespan()	(1)(1) 在 Azure Monitor 中，与 Splunk 的 relative_time(datetimeVal, offsetVal) 等效的函数是 datetimeVal + totimespan(offsetVal)。In Azure Monitor, Splunk's equivalent of relative_time(datetimeVal, offsetVal) is datetimeVal + totimespan(offsetVal). 例如，`search \| eval n=relative_time(now(), "-1d@d")` 重命名为 `... \| extend myTime = now() - totimespan("1d")`。For example, `search \| eval n=relative_time(now(), "-1d@d")` becomes `... \| extend myTime = now() - totimespan("1d")`.

(1) 在 Splunk 中，使用 eval 运算符调用该函数。(1) In Splunk, the function is invoked with the eval operator. 在 Azure Monitor 中，它用作 extend 或 project 的一部分。In Azure Monitor, it is used as part of extend or project.
(2) 在 Splunk 中，使用 eval 运算符调用该函数。(2) In Splunk, the function is invoked with the eval operator. 在 Azure Monitor 中，可以结合 where 运算符使用该函数。In Azure Monitor, it can be used with the where operator.

运算符Operators

以下部分通过示例演示 Splunk 和 Azure Monitor 如何使用不同的运算符。The following sections give examples of using different operators between Splunk and Azure Monitor.

备注

在以下示例中，Splunk 字段 rule 映射到 Azure Monitor 中的某个表，Splunk 的默认时间戳映射到 Logs Analytics 的 ingestion_time() 列。For the purpose of the following example, the Splunk field rule maps to a table in Azure Monitor, and Splunk's default timestamp maps to the Logs Analytics ingestion_time() column.

搜索Search

在 Splunk 中，可以省略 search 关键字，并指定不带引号的字符串。In Splunk, you can omit the search keyword and specify an unquoted string. 在 Azure Monitor 中，必须在每个查询的开头使用 find，不带引号的字符串是列名，查找值必须是带引号的字符串。In Azure Monitor you must start each query with find, an unquoted string is a column name, and the lookup value must be a quoted string.

	运算符Operator	示例Example
SplunkSplunk	searchsearch	`search Session.Id="c8894ffd-e684-43c9-9125-42adc25cd3fc" earliest=-24h`
Azure MonitorAzure Monitor	findfind	`find Session.Id=="c8894ffd-e684-43c9-9125-42adc25cd3fc" and ingestion_time()> ago(24h)`

筛选器Filter

Azure Monitor 日志查询从包含筛选器的表格结果集开始。Azure Monitor log queries start from a tabular result set where the filter. 在 Splunk 中，筛选是针对当前索引执行的默认操作。In Splunk, filtering is the default operation on the current index. 还可以在 Splunk 中使用 where 运算符，但不建议。You can also use where operator in Splunk, but it is not recommended.

	运算符Operator	示例Example
SplunkSplunk	searchsearch	`Event.Rule="330009.2" Session.Id="c8894ffd-e684-43c9-9125-42adc25cd3fc" _indextime>-24h`
Azure MonitorAzure Monitor	wherewhere	`Office_Hub_OHubBGTaskError \| where Session_Id == "c8894ffd-e684-43c9-9125-42adc25cd3fc" and ingestion_time() > ago(24h)`

获取用于检查的 n 个事件/行Getting n events/rows for inspection

Azure Monitor 日志查询还支持将 take 用作 limit 的别名。Azure Monitor log queries also support take as an alias to limit. 在 Splunk 中，如果结果已排序，则 head 将返回前 n 个结果。In Splunk, if the results are ordered, head will return the first n results. 在 Azure Monitor 中，limit 不会排序，而是返回找到的前 n 行。In Azure Monitor, limit is not ordered but returns the first n rows that are found.

	运算符Operator	示例Example
SplunkSplunk	headhead	`Event.Rule=330009.2 \| head 100`
Azure MonitorAzure Monitor	limitlimit	`Office_Hub_OHubBGTaskError \| limit 100`

获取按字段/列排序的前 n 个事件/行Getting the first n events/rows ordered by a field/column

对于底部结果，在 Splunk 中可以使用 tail。For bottom results, in Splunk you use tail. 在 Azure Monitor 中，可以使用 asc 指定排序方向。In Azure Monitor you can specify the ordering direction with asc.

	运算符Operator	示例Example
SplunkSplunk	headhead	`Event.Rule="330009.2" \| sort Event.Sequence \| head 20`
Azure MonitorAzure Monitor	返回页首top	`Office_Hub_OHubBGTaskError \| top 20 by Event_Sequence`

使用新字段/列扩展结果集Extending the result set with new fields/columns

Splunk 还有一个 eval 函数，该函数不能与 eval 运算符进行比较。Splunk also has an eval function, which is not to be comparable with the eval operator. Splunk 中的 eval 运算符与 Azure Monitor 中的 extend 运算符仅支持标量函数和算术运算符。Both the eval operator in Splunk and the extend operator in Azure Monitor only support scalar functions and arithmetic operators.

	运算符Operator	示例Example
SplunkSplunk	evaleval	`Event.Rule=330009.2 \| eval state= if(Data.Exception = "0", "success", "error")`
Azure MonitorAzure Monitor	extendextend	`Office_Hub_OHubBGTaskError \| extend state = iif(Data_Exception == 0,"success" ,"error")`

重命名Rename

Azure Monitor 使用 project-rename 运算符重命名字段。Azure Monitor uses the project-rename operator to rename a field. project-rename 允许查询利用为字段预先生成的任何索引。project-rename allows the query to take advantage of any indexes pre-built for a field. Splunk 使用 rename 运算符来执行相同的操作。Splunk has a rename operator to do the same.

	运算符Operator	示例Example
SplunkSplunk	renamerename	`Event.Rule=330009.2 \| rename Date.Exception as execption`
Azure MonitorAzure Monitor	project-renameproject-rename	`Office_Hub_OHubBGTaskError \| project-rename exception = Date_Exception`

设置结果格式/投影Format results/Projection

Splunk 似乎没有类似于 project-away 的运算符。Splunk does not seem to have an operator similar to project-away. 可以使用 UI 来筛选字段。You can use the UI to filter away fields.

	运算符Operator	示例Example
SplunkSplunk	tabletable	`Event.Rule=330009.2 \| table rule, state`
Azure MonitorAzure Monitor	projectproject project-awayproject-away	`Office_Hub_OHubBGTaskError \| project exception, state`

聚合Aggregation

有关不同的聚合函数，请参阅 Azure Monitor 日志查询中的聚合。See the Aggregations in Azure Monitor log queries for the different aggregation functions.

	运算符Operator	示例Example
SplunkSplunk	statsstats	`search (Rule=120502.*) \| stats count by OSEnv, Audience`
Azure MonitorAzure Monitor	summarizesummarize	`Office_Hub_OHubBGTaskError \| summarize count() by App_Platform, Release_Audience`

联接Join

Splunk 中的联接具有很强的限制。Join in Splunk has significant limitations. 子查询限制为 10000 条结果（在部署配置文件中设置），联接形式数目也有限制。The subquery has a limit of 10000 results (set in the deployment configuration file), and there a limited number of join flavors.

	运算符Operator	示例Example
SplunkSplunk	joinjoin	`Event.Rule=120103* \| stats by Client.Id, Data.Alias \| join Client.Id max=0 [search earliest=-24h Event.Rule="150310.0" Data.Hresult=-2147221040]`
Azure MonitorAzure Monitor	joinjoin	`cluster("OAriaPPT").database("Office PowerPoint").Office_PowerPoint_PPT_Exceptions \| where Data_Hresult== -2147221040 \| join kind = inner (Office_System_SystemHealthMetadata \| summarize by Client_Id, Data_Alias)on Client_Id`

排序Sort

在 Splunk 中，若要按升序排序，必须使用 reverse 运算符。In Splunk, to sort in ascending order you must use the reverse operator. Azure Monitor 还支持定义 null 值的放置位置：开头或末尾。Azure Monitor also supports defining where to put nulls, at the beginning or at the end.

	运算符Operator	示例Example
SplunkSplunk	sortsort	`Event.Rule=120103 \| sort Data.Hresult \| reverse`
Azure MonitorAzure Monitor	order byorder by	`Office_Hub_OHubBGTaskError \| order by Data_Hresult, desc`

多值扩展Multivalue expand

此运算符在 Splunk 和 Azure Monitor 中类似。This is a similar operator in both Splunk and Azure Monitor.

	运算符Operator	示例Example
SplunkSplunk	mvexpandmvexpand	`mvexpand foo`
Azure MonitorAzure Monitor	mvexpandmvexpand	`mvexpand foo`

在 Azure 门户的 Log Analytics 中，仅公开第一列。In Log Analytics in the Azure portal, only the first column is exposed. 可通过 API 查看所有列。All columns are available through the API.

	运算符Operator	示例Example
SplunkSplunk	字段fields	`Event.Rule=330009.2 \| fields App.Version, App.Platform`
Azure MonitorAzure Monitor	facetsfacets	`Office_Excel_BI_PivotTableCreate \| facet by App_Branch, App_Version`

重复数据删除De-duplicate

可以改用 summarize arg_min() 来反转记录的选择顺序。You can use summarize arg_min() instead to reverse the order of which record gets chosen.

	运算符Operator	示例Example
SplunkSplunk	dedupdedup	`Event.Rule=330009.2 \| dedup device_id sortby -batterylife`
Azure MonitorAzure Monitor	summarize arg_max()summarize arg_max()	`Office_Excel_BI_PivotTableCreate \| summarize arg_max(batterylife, *) by device_id`

后续步骤Next steps

完成关于在 Azure Monitor 中编写日志查询的一课。Go through a lesson on the writing log queries in Azure Monitor.