ipv4_lookup 插件ipv4_lookup plugin

ipv4_lookup 插件会在查找表中查找 IPv4 值,并返回具有匹配值的行。The ipv4_lookup plugin looks up an IPv4 value in a lookup table and returns rows with matched values.

T | evaluate ipv4_lookup(LookupTable, SourceIPv4Key, IPv4LookupKey)

T | evaluate ipv4_lookup(LookupTable, SourceIPv4Key, IPv4LookupKey, return_unmatched = true)

T | evaluate ipv4_lookup(LookupTable, SourceIPv4Key, IPv4LookupKey, ExtraKey1, ExtraKey2, return_unmatched = true)

语法Syntax

T | evaluate ipv4_lookup( LookupTable , SourceIPv4Key , IPv4LookupKey [, ExtraKey1 [..T | evaluate ipv4_lookup( LookupTable , SourceIPv4Key , IPv4LookupKey [, ExtraKey1 [.. , ExtraKeyN [, return_unmatched ]]] ), ExtraKeyN [, return_unmatched ]]] )

参数Arguments

  • T:表格输入,其列 SourceIPv4Key 将用于 IPv4 匹配。T: The tabular input whose column SourceIPv4Key will be used for IPv4 matching.
  • LookupTable:具有 IPv4 查找数据的表或表格表达式,其列 LookupKey 将用于 IPv4 匹配。LookupTable: Table or tabular expression with IPv4 lookup data, whose column LookupKey will be used for IPv4 matching. 可以使用 IP 前缀表示法对 IPv4 值进行掩码操作。IPv4 values can be masked using IP-prefix notation.
  • SourceIPv4Key:T 的列,其中包含的 IPv4 字符串需在 LookupTable 中查找。SourceIPv4Key: The column of T with IPv4 string to be looked up in LookupTable. 可以使用 IP 前缀表示法对 IPv4 值进行掩码操作。IPv4 values can be masked using IP-prefix notation.
  • IPv4LookupKey:LookupTable 列,其中包含的 IPv4 字符串与每个 SourceIPv4Key 值相匹配。IPv4LookupKey: The column of LookupTable with IPv4 string that is matched against each SourceIPv4Key value.
  • ExtraKey1 ..ExtraKey1 .. ExtraKeyN:(可选)用于查找匹配项的其他列引用。ExtraKeyN: (Optional) Additional column references that are used for lookup matches. join 运算类似:具有相等值的记录将被视为匹配。Similar to join operation: records with equal values will be considered matching. 列名引用必须同时存在于源表 TLookupTable 中。Column name references must exist both is source table T and LookupTable.
  • return_unmatched:一个布尔标志,用于定义结果是应包含所有行还是仅包含匹配的行(默认值:false - 仅返回匹配的行)。return_unmatched: A boolean flag that defines if the result should include all or only matching rows (default: false - only matching rows returned).

IP 前缀表示法IP-prefix notation

IP 地址可通过使用斜杠 (/) 字符的 IP-prefix notation 进行定义。IP addresses can be defined with IP-prefix notation using a slash (/) character. 斜杠 (/) 左边的 IP 地址是基本 IP 地址。The IP address to the left of the slash (/) is the base IP address. 斜杠 (/) 右边的数字(1 到 32)是网络掩码中连续位的数目。The number (1 to 32) to the right of the slash (/) is the number of contiguous 1 bit in the netmask.

例如,192.168.2.0/24 将具有关联的网络/子网掩码,其中包含 24 个连续位或点分十进制格式的 255.255.255.0。For example, 192.168.2.0/24 will have an associated net/subnetmask containing 24 contiguous bits or 255.255.255.0 in dotted decimal format.

返回Returns

ipv4_lookup 插件会返回基于 IPv4 密钥进行联接(查找)的结果。The ipv4_lookup plugin returns a result of join (lookup) based on IPv4 key. 该表的架构是源表与查找表的并集,类似于 lookup 运算符的结果。The schema of the table is the union of the source table and the lookup table, similar to the result of the lookup operator.

如果 return_unmatched 参数设置为 true,则生成的表会同时包含匹配的行和不匹配的行(使用 null 进行填充)。If the return_unmatched argument is set to true, the resulting table will include both matched and unmatched rows (filled with nulls).

如果 return_unmatched 参数设置为 false 或已省略(使用默认值 false),则生成的表的记录数将与匹配的结果数相同。If the return_unmatched argument is set to false, or omitted (the default value of false is used), the resulting table will have as many records as matching results. return_unmatched=true 执行相比,此查找变体具有更好的性能。This variant of lookup has better performance compared to return_unmatched=true execution.

备注

  • 此插件涵盖基于 IPv4 的联接的方案,假设查找表较小(10-20 万行),输入表可以较大。This plugin covers the scenario of IPv4-based join, assuming a small lookup table size (100K-200K rows), with the input table optionally having a larger size.
  • 插件的性能将取决于查找表和数据源表的大小、列数和匹配记录数。The performance of the plugin will depend on the sizes of the lookup and data source tables, the number of columns, and number of matching records.

示例Examples

IPv4 查找 - 仅匹配行IPv4 lookup - matching rows only

// IP lookup table: IP_Data
// Partial data from: https://raw.githubusercontent.com/datasets/geoip2-ipv4/master/data/geoip2-ipv4.csv
let IP_Data = datatable(network:string, continent_code:string ,continent_name:string, country_iso_code:string, country_name:string)
[
  "111.68.128.0/17","AS","Asia","JP","Japan",
  "5.8.0.0/19","EU","Europe","RU","Russia",
  "223.255.254.0/24","AS","Asia","SG","Singapore",
  "46.36.200.51/32","OC","Oceania","CK","Cook Islands",
  "2.20.183.0/24","EU","Europe","GB","United Kingdom",
];
let IPs = datatable(ip:string)
[
  '2.20.183.12',   // United Kingdom
  '5.8.1.2',       // Russia
  '192.165.12.17', // Unknown
];
IPs
| evaluate ipv4_lookup(IP_Data, ip, network)
ipip networknetwork continent_codecontinent_code continent_namecontinent_name country_iso_codecountry_iso_code country_namecountry_name
2.20.183.122.20.183.12 2.20.183.0/242.20.183.0/24 EUEU 欧洲Europe GBGB United KingdomUnited Kingdom
5.8.1.25.8.1.2 5.8.0.0/195.8.0.0/19 EUEU 欧洲Europe RURU 俄罗斯Russia

IPv4 查找 - 同时返回匹配行和非匹配行IPv4 lookup - return both matching and non-matching rows

// IP lookup table: IP_Data
// Partial data from: 
// https://raw.githubusercontent.com/datasets/geoip2-ipv4/master/data/geoip2-ipv4.csv
let IP_Data = datatable(network:string,continent_code:string ,continent_name:string ,country_iso_code:string ,country_name:string )
[
    "111.68.128.0/17","AS","Asia","JP","Japan",
    "5.8.0.0/19","EU","Europe","RU","Russia",
    "223.255.254.0/24","AS","Asia","SG","Singapore",
    "46.36.200.51/32","OC","Oceania","CK","Cook Islands",
    "2.20.183.0/24","EU","Europe","GB","United Kingdom",
];
let IPs = datatable(ip:string)
[
    '2.20.183.12',   // United Kingdom
    '5.8.1.2',       // Russia
    '192.165.12.17', // Unknown
];
IPs
| evaluate ipv4_lookup(IP_Data, ip, network, return_unmatched = true)
ipip networknetwork continent_codecontinent_code continent_namecontinent_name country_iso_codecountry_iso_code country_namecountry_name
2.20.183.122.20.183.12 2.20.183.0/242.20.183.0/24 EUEU 欧洲Europe GBGB United KingdomUnited Kingdom
5.8.1.25.8.1.2 5.8.0.0/195.8.0.0/19 EUEU 欧洲Europe RURU 俄罗斯Russia
192.165.12.17192.165.12.17

IPv4 查找 - 在 external_data() 中使用源IPv4 lookup - using source in external_data()

let IP_Data = external_data(network:string,geoname_id:long,continent_code:string,continent_name:string ,country_iso_code:string,country_name:string,is_anonymous_proxy:bool,is_satellite_provider:bool)
    ['https://raw.githubusercontent.com/datasets/geoip2-ipv4/master/data/geoip2-ipv4.csv'];
let IPs = datatable(ip:string)
[
    '2.20.183.12',   // United Kingdom
    '5.8.1.2',       // Russia
    '192.165.12.17', // Sweden
];
IPs
| evaluate ipv4_lookup(IP_Data, ip, network, return_unmatched = true)
ipip networknetwork geoname_idgeoname_id continent_codecontinent_code continent_namecontinent_name country_iso_codecountry_iso_code country_namecountry_name is_anonymous_proxyis_anonymous_proxy is_satellite_provideris_satellite_provider
2.20.183.122.20.183.12 2.20.183.0/242.20.183.0/24 26351672635167 EUEU 欧洲Europe GBGB 英国United Kingdom 00 00
5.8.1.25.8.1.2 5.8.0.0/195.8.0.0/19 20173702017370 EUEU 欧洲Europe RURU 俄罗斯Russia 00 00
192.165.12.17192.165.12.17 192.165.8.0/21192.165.8.0/21 26618862661886 EUEU 欧洲Europe SESE 瑞典Sweden 00 00

IPv4 查找 - 使用额外的列进行匹配IPv4 lookup - using extra columns for matching

let IP_Data = external_data(network:string,geoname_id:long,continent_code:string,continent_name:string ,country_iso_code:string,country_name:string,is_anonymous_proxy:bool,is_satellite_provider:bool)
    ['https://raw.githubusercontent.com/datasets/geoip2-ipv4/master/data/geoip2-ipv4.csv'];
let IPs = datatable(ip:string, continent_name:string, country_iso_code:string)
[
    '2.20.183.12',   'Europe', 'GB', // United Kingdom
    '5.8.1.2',       'Europe', 'RU', // Russia
    '192.165.12.17', 'Europe', '',   // Sweden is 'SE' - so it won't be matched
];
IPs
| evaluate ipv4_lookup(IP_Data, ip, network, continent_name, country_iso_code)
ipip continent_namecontinent_name country_iso_codecountry_iso_code networknetwork geoname_idgeoname_id continent_codecontinent_code country_namecountry_name is_anonymous_proxyis_anonymous_proxy is_satellite_provideris_satellite_provider
2.20.183.122.20.183.12 欧洲Europe GBGB 2.20.183.0/242.20.183.0/24 26351672635167 EUEU 英国United Kingdom 00 00
5.8.1.25.8.1.2 欧洲Europe RURU 5.8.0.0/195.8.0.0/19 20173702017370 EUEU 俄罗斯Russia 00 00