从 Splunk 到 Azure Monitor 日志查询Splunk to Azure Monitor log query
本文旨在帮助熟悉 Splunk 的用户通过学习 Kusto 查询语言,以在 Azure Monitor 中编写日志查询。This article is intended to assist users who are familiar with Splunk to learn the Kusto query language to write log queries in Azure Monitor. 其中将两者做了直接的比较,让用户了解它们的主要差异和相似之处,并抉机利用现有的知识。Direct comparisons are made between the two to understand key differences and also similarities where you can leverage your existing knowledge.
结构和概念Structure and concepts
下表比较了 Splunk 和 Azure Monitor 日志的概念与数据结构。The following table compares concepts and data structures between Splunk and Azure Monitor logs.
概念Concept | SplunkSplunk | Azure MonitorAzure Monitor | 注释Comment |
---|---|---|---|
部署单元Deployment unit | clustercluster | clustercluster | Azure Monitor 允许跨群集进行任意查询,Azure Monitor allows arbitrary cross cluster queries. Splunk 则不允许。Splunk does not. |
数据缓存Data caches | 存储桶buckets | 缓存和保留策略Caching and retention policies | 控制数据的保留期和缓存级别。Controls the period and caching level for the data. 此设置直接影响查询性能和部署成本。This setting directly impacts the performance of queries and cost of the deployment. |
数据的逻辑分区Logical partition of data | 索引index | databasedatabase | 允许数据的逻辑隔离。Allows logical separation of the data. 这两个实现都允许跨这些分区的联合与联接。Both implementations allow unions and joining across these partitions. |
结构化事件元数据Structured event metadata | 空值N/A | 表table | Splunk 没有向事件元数据搜索语言公开的概念。Splunk does not have the concept exposed to the search language of event metadata. Azure Monitor 日志具有表的概念,表包含列。Azure Monitor logs has the concept of a table, which has columns. 每个事件实例映射到行。Each event instance is mapped to a row. |
数据记录Data record | eventevent | 行row | 仅限术语变化。Terminology change only. |
数据记录属性Data record attribute | 字段field | 列column | 在 Azure Monitor 中,此概念预定义为表结构的一部分。In Azure Monitor, this is predefined as part of the table structure. 在 Splunk 中,每个事件有自身的字段集。In Splunk, each event has its own set of fields. |
类型Types | 数据类型datatype | 数据类型datatype | Azure Monitor 数据类型更明确,因为它们是在列中设置的。Azure Monitor datatypes are more explicit as they are set on the columns. 两者都能动态处理数据类型,数据类型集(包括 JSON 支持)大致相同。Both have the ability to work dynamically with data types and roughly equivalent set of datatypes including JSON support. |
查询和搜索Query and search | 搜索search | queryquery | Azure Monitor 和 Splunk 的概念在本质上相同。Concepts are essentially the same between both Azure Monitor and Splunk. |
数据引入时间Event ingestion time | 系统时间System Time | ingestion_time()ingestion_time() | 在 Splunk 中,每个事件将获取编制事件索引时的系统时间戳。In Splunk, each event gets a system timestamp of the time that the event was indexed. 在 Azure Monitor 中,可以定义名为 ingestion_time 的策略,用于公开可通过 ingestion_time() 函数引用的系统列。In Azure Monitor, you can define a policy called ingestion_time that exposes a system column that can be referenced through the ingestion_time() function. |
函数Functions
下表指定了 Azure Monitor 中等效于 Splunk 函数的函数。The following table specifies functions in Azure Monitor that are equivalent to Splunk functions.
SplunkSplunk | Azure MonitorAzure Monitor | 注释Comment |
---|---|---|
strcatstrcat | strcat()strcat() | (1)(1) |
splitsplit | split()split() | (1)(1) |
ifif | iff()iff() | (1)(1) |
tonumbertonumber | todouble()todouble() tolong()tolong() toint()toint() |
(1)(1) |
upperupper lowerlower |
toupper()toupper() tolower()tolower() |
(1)(1) |
replacereplace | replace()replace() | (1)(1) 另请注意,尽管这两个产品中的 replace() 都采用三个参数,但这些参数不同。Also note that while replace() takes three parameters in both products, the parameters are different. |
substrsubstr | substring()substring() | (1)(1) 另请注意,Splunk 使用从 1 开始的索引。Also note that Splunk uses one-based indices. Azure Monitor 记录从 0 开始的索引。Azure Monitor notes zero-based indices. |
tolowertolower | tolower()tolower() | (1)(1) |
touppertoupper | toupper()toupper() | (1)(1) |
matchmatch | matches regexmatches regex | (2)(2) |
regexregex | matches regexmatches regex | 在 Splunk 中,regex 是运算符。In Splunk, regex is an operator. 在 Azure Monitor 中,它是关系运算符。In Azure Monitor, it's a relational operator. |
searchmatchsearchmatch | == | 在 Splunk 中,searchmatch 允许搜索确切的字符串。In Splunk, searchmatch allows searching for the exact string. |
randomrandom | rand()rand() rand(n)rand(n) |
Splunk 的函数返回从 0 到 231-1 的数字。Splunk's function returns a number from zero to 231-1. Azure Monitor 返回介于 0.0 和 1.0 之间的数字;如果提供了参数,则返回介于 0 和 n-1 之间的数字。Azure Monitor' returns a number between 0.0 and 1.0, or if a parameter provided, between 0 and n-1. |
nownow | now()now() | (1)(1) |
relative_timerelative_time | totimespan()totimespan() | (1)(1) 在 Azure Monitor 中,与 Splunk 的 relative_time(datetimeVal, offsetVal) 等效的函数是 datetimeVal + totimespan(offsetVal)。In Azure Monitor, Splunk's equivalent of relative_time(datetimeVal, offsetVal) is datetimeVal + totimespan(offsetVal). 例如, search | eval n=relative_time(now(), "-1d@d") 重命名为 ... | extend myTime = now() - totimespan("1d") 。For example, search | eval n=relative_time(now(), "-1d@d") becomes ... | extend myTime = now() - totimespan("1d") . |
(1) 在 Splunk 中,使用 eval
运算符调用该函数。(1) In Splunk, the function is invoked with the eval
operator. 在 Azure Monitor 中,它用作 extend
或 project
的一部分。In Azure Monitor, it is used as part of extend
or project
.
(2) 在 Splunk 中,使用 eval
运算符调用该函数。(2) In Splunk, the function is invoked with the eval
operator. 在 Azure Monitor 中,可以结合 where
运算符使用该函数。In Azure Monitor, it can be used with the where
operator.
运算符Operators
以下部分通过示例演示 Splunk 和 Azure Monitor 如何使用不同的运算符。The following sections give examples of using different operators between Splunk and Azure Monitor.
备注
在以下示例中,Splunk 字段 rule 映射到 Azure Monitor 中的某个表,Splunk 的默认时间戳映射到 Logs Analytics 的 ingestion_time() 列。For the purpose of the following example, the Splunk field rule maps to a table in Azure Monitor, and Splunk's default timestamp maps to the Logs Analytics ingestion_time() column.
搜索Search
在 Splunk 中,可以省略 search
关键字,并指定不带引号的字符串。In Splunk, you can omit the search
keyword and specify an unquoted string. 在 Azure Monitor 中,必须在每个查询的开头使用 find
,不带引号的字符串是列名,查找值必须是带引号的字符串。In Azure Monitor you must start each query with find
, an unquoted string is a column name, and the lookup value must be a quoted string.
运算符Operator | 示例Example | |
---|---|---|
SplunkSplunk | searchsearch | search Session.Id="c8894ffd-e684-43c9-9125-42adc25cd3fc" earliest=-24h |
Azure MonitorAzure Monitor | findfind | find Session.Id=="c8894ffd-e684-43c9-9125-42adc25cd3fc" and ingestion_time()> ago(24h) |
筛选器Filter
Azure Monitor 日志查询从包含筛选器的表格结果集开始。Azure Monitor log queries start from a tabular result set where the filter. 在 Splunk 中,筛选是针对当前索引执行的默认操作。In Splunk, filtering is the default operation on the current index. 还可以在 Splunk 中使用 where
运算符,但不建议。You can also use where
operator in Splunk, but it is not recommended.
运算符Operator | 示例Example | |
---|---|---|
SplunkSplunk | searchsearch | Event.Rule="330009.2" Session.Id="c8894ffd-e684-43c9-9125-42adc25cd3fc" _indextime>-24h |
Azure MonitorAzure Monitor | wherewhere | Office_Hub_OHubBGTaskError |
获取用于检查的 n 个事件/行Getting n events/rows for inspection
Azure Monitor 日志查询还支持将 take
用作 limit
的别名。Azure Monitor log queries also support take
as an alias to limit
. 在 Splunk 中,如果结果已排序,则 head
将返回前 n 个结果。In Splunk, if the results are ordered, head
will return the first n results. 在 Azure Monitor 中,limit 不会排序,而是返回找到的前 n 行。In Azure Monitor, limit is not ordered but returns the first n rows that are found.
运算符Operator | 示例Example | |
---|---|---|
SplunkSplunk | headhead | Event.Rule=330009.2 |
Azure MonitorAzure Monitor | limitlimit | Office_Hub_OHubBGTaskError |
获取按字段/列排序的前 n 个事件/行Getting the first n events/rows ordered by a field/column
对于底部结果,在 Splunk 中可以使用 tail
。For bottom results, in Splunk you use tail
. 在 Azure Monitor 中,可以使用 asc
指定排序方向。In Azure Monitor you can specify the ordering direction with asc
.
运算符Operator | 示例Example | |
---|---|---|
SplunkSplunk | headhead | Event.Rule="330009.2" |
Azure MonitorAzure Monitor | 返回页首top | Office_Hub_OHubBGTaskError |
使用新字段/列扩展结果集Extending the result set with new fields/columns
Splunk 还有一个 eval
函数,该函数不能与 eval
运算符进行比较。Splunk also has an eval
function, which is not to be comparable with the eval
operator. Splunk 中的 eval
运算符与 Azure Monitor 中的 extend
运算符仅支持标量函数和算术运算符。Both the eval
operator in Splunk and the extend
operator in Azure Monitor only support scalar functions and arithmetic operators.
运算符Operator | 示例Example | |
---|---|---|
SplunkSplunk | evaleval | Event.Rule=330009.2 |
Azure MonitorAzure Monitor | extendextend | Office_Hub_OHubBGTaskError |
重命名Rename
Azure Monitor 使用 project-rename
运算符重命名字段。Azure Monitor uses the project-rename
operator to rename a field. project-rename
允许查询利用为字段预先生成的任何索引。project-rename
allows the query to take advantage of any indexes pre-built for a field. Splunk 使用 rename
运算符来执行相同的操作。Splunk has a rename
operator to do the same.
运算符Operator | 示例Example | |
---|---|---|
SplunkSplunk | renamerename | Event.Rule=330009.2 |
Azure MonitorAzure Monitor | project-renameproject-rename | Office_Hub_OHubBGTaskError |
设置结果格式/投影Format results/Projection
Splunk 似乎没有类似于 project-away
的运算符。Splunk does not seem to have an operator similar to project-away
. 可以使用 UI 来筛选字段。You can use the UI to filter away fields.
运算符Operator | 示例Example | |
---|---|---|
SplunkSplunk | tabletable | Event.Rule=330009.2 |
Azure MonitorAzure Monitor | projectproject project-awayproject-away |
Office_Hub_OHubBGTaskError |
聚合Aggregation
有关不同的聚合函数,请参阅 Azure Monitor 日志查询中的聚合。See the Aggregations in Azure Monitor log queries for the different aggregation functions.
运算符Operator | 示例Example | |
---|---|---|
SplunkSplunk | statsstats | search (Rule=120502.*) |
Azure MonitorAzure Monitor | summarizesummarize | Office_Hub_OHubBGTaskError |
联接Join
Splunk 中的联接具有很强的限制。Join in Splunk has significant limitations. 子查询限制为 10000 条结果(在部署配置文件中设置),联接形式数目也有限制。The subquery has a limit of 10000 results (set in the deployment configuration file), and there a limited number of join flavors.
运算符Operator | 示例Example | |
---|---|---|
SplunkSplunk | joinjoin | Event.Rule=120103* | stats by Client.Id, Data.Alias | join Client.Id max=0 [search earliest=-24h Event.Rule="150310.0" Data.Hresult=-2147221040] |
Azure MonitorAzure Monitor | joinjoin | cluster("OAriaPPT").database("Office PowerPoint").Office_PowerPoint_PPT_Exceptions |
排序Sort
在 Splunk 中,若要按升序排序,必须使用 reverse
运算符。In Splunk, to sort in ascending order you must use the reverse
operator. Azure Monitor 还支持定义 null 值的放置位置:开头或末尾。Azure Monitor also supports defining where to put nulls, at the beginning or at the end.
运算符Operator | 示例Example | |
---|---|---|
SplunkSplunk | sortsort | Event.Rule=120103 |
Azure MonitorAzure Monitor | order byorder by | Office_Hub_OHubBGTaskError |
多值扩展Multivalue expand
此运算符在 Splunk 和 Azure Monitor 中类似。This is a similar operator in both Splunk and Azure Monitor.
运算符Operator | 示例Example | |
---|---|---|
SplunkSplunk | mvexpandmvexpand | mvexpand foo |
Azure MonitorAzure Monitor | mvexpandmvexpand | mvexpand foo |
结果分面、相关字段Results facets, interesting fields
在 Azure 门户的 Log Analytics 中,仅公开第一列。In Log Analytics in the Azure portal, only the first column is exposed. 可通过 API 查看所有列。All columns are available through the API.
运算符Operator | 示例Example | |
---|---|---|
SplunkSplunk | 字段fields | Event.Rule=330009.2 |
Azure MonitorAzure Monitor | facetsfacets | Office_Excel_BI_PivotTableCreate |
重复数据删除De-duplicate
可以改用 summarize arg_min()
来反转记录的选择顺序。You can use summarize arg_min()
instead to reverse the order of which record gets chosen.
运算符Operator | 示例Example | |
---|---|---|
SplunkSplunk | dedupdedup | Event.Rule=330009.2 |
Azure MonitorAzure Monitor | summarize arg_max()summarize arg_max() | Office_Excel_BI_PivotTableCreate |
后续步骤Next steps
- 完成关于在 Azure Monitor 中编写日志查询的一课。Go through a lesson on the writing log queries in Azure Monitor.