parse operator
Applies to: ✅ Azure Data Explorer ✅ Azure Monitor ✅ Microsoft Sentinel
Evaluates a string expression and parses its value into one or more calculated columns. The calculated columns return null values for unsuccessfully parsed strings. If there's no need to use rows where parsing doesn't succeed, prefer using the parse-where operator.
Syntax
T | parse [ kind=kind [ flags=regexFlags ]] expression with [ * ] stringConstant columnName [: columnType] [ * ] , ...
Learn more about syntax conventions.
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
| T | string |
✔️ | The tabular input to parse. |
| kind | string |
✔️ | One of the supported kind values. The default value is simple. |
| regexFlags | string |
If kind is regex, then you can specify regex flags to be used like U for ungreedy, m for multi-line mode, s for match new line \n, and i for case-insensitive. More flags can be found in Flags. |
|
| expression | string |
✔️ | An expression that evaluates to a string. |
| stringConstant | string |
✔️ | A string constant for which to search and parse. |
| columnName | string |
✔️ | The name of a column to assign a value to, extracted from the string expression. |
| columnType | string |
The scalar value that indicates the type to convert the value to. The default is string. |
Note
- The parse pattern can start with ColumnName in addition to StringConstant.
- Use
*in the pattern to skip junk values. The*can't be used after astringtype column. - If the parsed expression isn't of type
string, it will be converted to typestring. - Use
projectif you also want to drop or rename some columns.
Supported kind values
| Text | Description |
|---|---|
simple |
This is the default value. stringConstant is a regular string value and the match is strict. All string delimiters should appear in the parsed string, and all extended columns must match the required types. |
regex |
stringConstant can be a regular expression and the match is strict. All string delimiters, which can be a regex for this mode, should appear in the parsed string, and all extended columns must match the required types. |
relaxed |
stringConstant is a regular string value and the match is relaxed. All string delimiters should appear in the parsed string, but extended columns might partially match the required types. Extended columns that didn't match the required types get the value null. |
Regex mode
In regex mode, parse translates the pattern to a regex. Use regular expressions to do the matching and use numbered captured groups that are handled internally. For example:
parse kind=regex Col with * <regex1> var1:string <regex2> var2:long
In the parse statement, the regex internally generated by the parse is .*?<regex1>(.*?)<regex2>(\-\d+).
*was translated to.*?.stringwas translated to.*?.longwas translated to\-\d+.
Returns
The input table extended according to the list of columns that are provided to the operator.
Examples
The parse operator provides a streamlined way to extend a table by using multiple extract applications on the same string expression. This result is useful, when the table has a string column that contains several values that you want to break into individual columns. For example, a column that's produced by a developer trace ("printf"/"Console.WriteLine") statement.
Parse and extend results
In the following example, the column EventText of table Traces contains
strings of the form Event: NotifySliceRelease (resourceName={0}, totalSlices={1}, sliceNumber={2}, lockTime={3}, releaseTime={4}, previousLockTime={5}).
The operation extends the table with six columns: resourceName, totalSlices, sliceNumber, lockTime, releaseTime, and previousLockTime.
let Traces = datatable(EventText: string)
[
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=23, lockTime=02/17/2016 08:40:01, releaseTime=02/17/2016 08:40:01, previousLockTime=02/17/2016 08:39:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=15, lockTime=02/17/2016 08:40:00, releaseTime=02/17/2016 08:40:00, previousLockTime=02/17/2016 08:39:00)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=20, lockTime=02/17/2016 08:40:01, releaseTime=02/17/2016 08:40:01, previousLockTime=02/17/2016 08:39:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=22, lockTime=02/17/2016 08:41:01, releaseTime=02/17/2016 08:41:00, previousLockTime=02/17/2016 08:40:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=16, lockTime=02/17/2016 08:41:00, releaseTime=02/17/2016 08:41:00, previousLockTime=02/17/2016 08:40:00)"
];
Traces
| parse EventText with * "resourceName=" resourceName ", totalSlices=" totalSlices: long * "sliceNumber=" sliceNumber: long * "lockTime=" lockTime ", releaseTime=" releaseTime: date "," * "previousLockTime=" previousLockTime: date ")" *
| project resourceName, totalSlices, sliceNumber, lockTime, releaseTime, previousLockTime
Output
| resourceName | totalSlices | sliceNumber | lockTime | releaseTime | previousLockTime |
|---|---|---|---|---|---|
| PipelineScheduler | 27 | 15 | 02/17/2016 08:40:00 | 2016-02-17 08:40:00.0000000 | 2016-02-17 08:39:00.0000000 |
| PipelineScheduler | 27 | 23 | 02/17/2016 08:40:01 | 2016-02-17 08:40:01.0000000 | 2016-02-17 08:39:01.0000000 |
| PipelineScheduler | 27 | 20 | 02/17/2016 08:40:01 | 2016-02-17 08:40:01.0000000 | 2016-02-17 08:39:01.0000000 |
| PipelineScheduler | 27 | 16 | 02/17/2016 08:41:00 | 2016-02-17 08:41:00.0000000 | 2016-02-17 08:40:00.0000000 |
| PipelineScheduler | 27 | 22 | 02/17/2016 08:41:01 | 2016-02-17 08:41:00.0000000 | 2016-02-17 08:40:01.0000000 |
Extract email alias and DNS
In the following example, entries from the Contacts table are parsed to extract the alias and domain from an email address, and the domain from a website URL. The query returns the EmailAddress, EmailAlias, and WebsiteDomain columns, where the fullEmail column combines the parsed email aliases and domains.
let Leads=datatable(Contacts: string)
[
"Event: LeadContact (email=john@contosohotel.com, Website=https:contosohotel.com)",
"Event: LeadContact (email=abi@fourthcoffee.com, Website=https:www.fourthcoffee.com)",
"Event: LeadContact (email=nevena@treyresearch.com, Website=https:treyresearch.com)",
"Event: LeadContact (email=faruk@tailspintoys.com, Website=https:tailspintoys.com)",
"Event: LeadContact (email=ebere@relecloud.com, Website=https:relecloud.com)",
];
Leads
| parse Contacts with * "email=" alias:string "@" domain: string ", Website=https:" WebsiteDomain: string ")"
| project EmailAddress=strcat(alias, "@", domain), EmailAlias=alias, WebsiteDomain
Output
| EmailAddress | EmailAlias | WebsiteDomain |
|---|---|---|
| nevena@treyresearch.com | nevena | treyresearch.com |
| john@contosohotel.com | john | contosohotel.com |
| faruk@tailspintoys.com | faruk | tailspintoys.com |
| ebere@relecloud.com | ebere | relecloud.com |
| abi@fourthcoffee.com | abi | www.fourthcoffee.com |
Regex mode
In the following example, regular expressions are used to parse and extract data from the EventText column. The extracted data is projected into new fields.
let Traces=datatable(EventText: string)
[
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=23, lockTime=02/17/2016 08:40:01, releaseTime=02/17/2016 08:40:01, previousLockTime=02/17/2016 08:39:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=15, lockTime=02/17/2016 08:40:00, releaseTime=02/17/2016 08:40:00, previousLockTime=02/17/2016 08:39:00)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=20, lockTime=02/17/2016 08:40:01, releaseTime=02/17/2016 08:40:01, previousLockTime=02/17/2016 08:39:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=22, lockTime=02/17/2016 08:41:01, releaseTime=02/17/2016 08:41:00, previousLockTime=02/17/2016 08:40:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=16, lockTime=02/17/2016 08:41:00, releaseTime=02/17/2016 08:41:00, previousLockTime=02/17/2016 08:40:00)"
];
Traces
| parse kind=regex EventText with "(.*?)[a-zA-Z]*=" resourceName @", totalSlices=\s*\d+\s*.*?sliceNumber=" sliceNumber: long ".*?(previous)?lockTime=" lockTime ".*?releaseTime=" releaseTime ".*?previousLockTime=" previousLockTime: date "\\)"
| project resourceName, sliceNumber, lockTime, releaseTime, previousLockTime
Output
| resourceName | sliceNumber | lockTime | releaseTime | previousLockTime |
|---|---|---|---|---|
| PipelineScheduler | 15 | 02/17/2016 08:40:00, | 02/17/2016 08:40:00, | 2016-02-17 08:39:00.0000000 |
| PipelineScheduler | 23 | 02/17/2016 08:40:01, | 02/17/2016 08:40:01, | 2016-02-17 08:39:01.0000000 |
| PipelineScheduler | 20 | 02/17/2016 08:40:01, | 02/17/2016 08:40:01, | 2016-02-17 08:39:01.0000000 |
| PipelineScheduler | 16 | 02/17/2016 08:41:00, | 02/17/2016 08:41:00, | 2016-02-17 08:40:00.0000000 |
| PipelineScheduler | 22 | 02/17/2016 08:41:01, | 02/17/2016 08:41:00, | 2016-02-17 08:40:01.0000000 |
Regex mode with regex flags
In the following example resourceName is extracted.
let Traces=datatable(EventText: string)
[
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=23, lockTime=02/17/2016 08:40:01, releaseTime=02/17/2016 08:40:01, previousLockTime=02/17/2016 08:39:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=15, lockTime=02/17/2016 08:40:00, releaseTime=02/17/2016 08:40:00, previousLockTime=02/17/2016 08:39:00)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=20, lockTime=02/17/2016 08:40:01, releaseTime=02/17/2016 08:40:01, previousLockTime=02/17/2016 08:39:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=22, lockTime=02/17/2016 08:41:01, releaseTime=02/17/2016 08:41:00, previousLockTime=02/17/2016 08:40:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=16, lockTime=02/17/2016 08:41:00, releaseTime=02/17/2016 08:41:00, previousLockTime=02/17/2016 08:40:00)"
];
Traces
| parse kind=regex EventText with * "resourceName=" resourceName ',' *
| project resourceName
Output
| resourceName |
|---|
| PipelineScheduler, totalSlices=27, sliceNumber=23, lockTime=02/17/2016 08:40:01, releaseTime=02/17/2016 08:40:01 |
| PipelineScheduler, totalSlices=27, sliceNumber=15, lockTime=02/17/2016 08:40:00, releaseTime=02/17/2016 08:40:00 |
| PipelineScheduler, totalSlices=27, sliceNumber=20, lockTime=02/17/2016 08:40:01, releaseTime=02/17/2016 08:40:01 |
| PipelineScheduler, totalSlices=27, sliceNumber=22, lockTime=02/17/2016 08:41:01, releaseTime=02/17/2016 08:41:00 |
| PipelineScheduler, totalSlices=27, sliceNumber=16, lockTime=02/17/2016 08:41:00, releaseTime=02/17/2016 08:41:00 |
If there are records where resourceName sometimes appears as lower-case and sometimes as upper-case, you might get nulls for some values.
The results in the previous example are unexpected, and include full event data since the default mode is greedy.
To extract only resourceName, run the previous query with the non-greedy U, and disable case-sensitive i regex flags.
let Traces=datatable(EventText: string)
[
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=23, lockTime=02/17/2016 08:40:01, releaseTime=02/17/2016 08:40:01, previousLockTime=02/17/2016 08:39:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=15, lockTime=02/17/2016 08:40:00, releaseTime=02/17/2016 08:40:00, previousLockTime=02/17/2016 08:39:00)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=20, lockTime=02/17/2016 08:40:01, releaseTime=02/17/2016 08:40:01, previousLockTime=02/17/2016 08:39:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=22, lockTime=02/17/2016 08:41:01, releaseTime=02/17/2016 08:41:00, previousLockTime=02/17/2016 08:40:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=16, lockTime=02/17/2016 08:41:00, releaseTime=02/17/2016 08:41:00, previousLockTime=02/17/2016 08:40:00)"
];
Traces
| parse kind=regex flags=Ui EventText with * "RESOURCENAME=" resourceName ',' *
| project resourceName
Output
| resourceName |
|---|
| PipelineScheduler |
| PipelineScheduler |
| PipelineScheduler |
| PipelineScheduler |
| PipelineScheduler |
If the parsed string has newlines, use the flag s, to parse the text.
let Traces=datatable(EventText: string)
[
"Event: NotifySliceRelease (resourceName=PipelineScheduler\ntotalSlices=27\nsliceNumber=23\nlockTime=02/17/2016 08:40:01\nreleaseTime=02/17/2016 08:40:01\npreviousLockTime=02/17/2016 08:39:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler\ntotalSlices=27\nsliceNumber=15\nlockTime=02/17/2016 08:40:00\nreleaseTime=02/17/2016 08:40:00\npreviousLockTime=02/17/2016 08:39:00)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler\ntotalSlices=27\nsliceNumber=20\nlockTime=02/17/2016 08:40:01\nreleaseTime=02/17/2016 08:40:01\npreviousLockTime=02/17/2016 08:39:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler\ntotalSlices=27\nsliceNumber=22\nlockTime=02/17/2016 08:41:01\nreleaseTime=02/17/2016 08:41:00\npreviousLockTime=02/17/2016 08:40:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler\ntotalSlices=27\nsliceNumber=16\nlockTime=02/17/2016 08:41:00\nreleaseTime=02/17/2016 08:41:00\npreviousLockTime=02/17/2016 08:40:00)"
];
Traces
| parse kind=regex flags=s EventText with * "resourceName=" resourceName: string "(.*?)totalSlices=" totalSlices: long "(.*?)lockTime=" lockTime: datetime "(.*?)releaseTime=" releaseTime: datetime "(.*?)previousLockTime=" previousLockTime: datetime "\\)"
| project-away EventText
Output
| resourceName | totalSlices | lockTime | releaseTime | previousLockTime |
|---|---|---|---|---|
| PipelineScheduler |
27 | 2016-02-17 08:40:00.0000000 | 2016-02-17 08:40:00.0000000 | 2016-02-17 08:39:00.0000000 |
| PipelineScheduler |
27 | 2016-02-17 08:40:01.0000000 | 2016-02-17 08:40:01.0000000 | 2016-02-17 08:39:01.0000000 |
| PipelineScheduler |
27 | 2016-02-17 08:40:01.0000000 | 2016-02-17 08:40:01.0000000 | 2016-02-17 08:39:01.0000000 |
| PipelineScheduler |
27 | 2016-02-17 08:41:00.0000000 | 2016-02-17 08:41:00.0000000 | 2016-02-17 08:40:00.0000000 |
| PipelineScheduler |
27 | 2016-02-17 08:41:01.0000000 | 2016-02-17 08:41:00.0000000 | 2016-02-17 08:40:01.0000000 |
Relaxed mode
In the following relaxed mode example, the extended column totalSlices must be of type long. However, in the parsed string, it has the value nonValidLongValue.
For the extended column, releaseTime, the value nonValidDateTime can't be parsed as datetime.
These two extended columns result in null values while the other columns, such as sliceNumber, still result in the correct values.
If you use option kind = simple for the following query, you get null results for all extended columns. This option is strict on extended columns, and is the difference between relaxed and simple mode.
Note
In relaxed mode, extended columns can be partially matched.
let Traces=datatable(EventText: string)
[
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=23, lockTime=02/17/2016 08:40:01, releaseTime=nonValidDateTime 08:40:01, previousLockTime=02/17/2016 08:39:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=15, lockTime=02/17/2016 08:40:00, releaseTime=nonValidDateTime, previousLockTime=02/17/2016 08:39:00)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=nonValidLongValue, sliceNumber=20, lockTime=02/17/2016 08:40:01, releaseTime=nonValidDateTime 08:40:01, previousLockTime=02/17/2016 08:39:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=27, sliceNumber=22, lockTime=02/17/2016 08:41:01, releaseTime=02/17/2016 08:41:00, previousLockTime=02/17/2016 08:40:01)",
"Event: NotifySliceRelease (resourceName=PipelineScheduler, totalSlices=nonValidLongValue, sliceNumber=16, lockTime=02/17/2016 08:41:00, releaseTime=02/17/2016 08:41:00, previousLockTime=02/17/2016 08:40:00)"
];
Traces
| parse kind=relaxed EventText with * "resourceName=" resourceName ", totalSlices=" totalSlices: long ", sliceNumber=" sliceNumber: long * "lockTime=" lockTime ", releaseTime=" releaseTime: date "," * "previousLockTime=" previousLockTime: date ")" *
| project-away EventText
Output
| resourceName | totalSlices | sliceNumber | lockTime | releaseTime | previousLockTime |
|---|---|---|---|---|---|
| PipelineScheduler | 27 | 15 | 02/17/2016 08:40:00 | 2016-02-17 08:39:00.0000000 | |
| PipelineScheduler | 27 | 23 | 02/17/2016 08:40:01 | 2016-02-17 08:39:01.0000000 | |
| PipelineScheduler | 20 | 02/17/2016 08:40:01 | 2016-02-17 08:39:01.0000000 | ||
| PipelineScheduler | 16 | 02/17/2016 08:41:00 | 2016-02-17 08:41:00.0000000 | 2016-02-17 08:40:00.0000000 | |
| PipelineScheduler | 27 | 22 | 02/17/2016 08:41:01 | 2016-02-17 08:41:00.0000000 | 2016-02-17 08:40:01.0000000 |