Azure 流分析中 JavaScript 用户定义的函数JavaScript user-defined functions in Azure Stream Analytics

Azure 流分析支持以 JavaScript 编写的用户定义的函数。Azure Stream Analytics supports user-defined functions written in JavaScript. 利用 JavaScript 提供的丰富 StringRegExpMathArrayDate 方法,可以更轻松地创建包含流分析作业的复杂数据转换。With the rich set of String, RegExp, Math, Array, and Date methods that JavaScript provides, complex data transformations with Stream Analytics jobs become easier to create.

概述Overview

JavaScript 用户定义的函数支持仅用于计算的且不需要外部连接的无状态标量函数。JavaScript user-defined functions support stateless, compute-only scalar functions that don't require external connectivity. 函数的返回值只能是标量(单个)值。The return value of a function can only be a scalar (single) value. 将某个 JavaScript 用户定义的函数添加到作业后,可在查询中的任意位置使用该函数,如同内置标量函数一样。After you add a JavaScript user-defined function to a job, you can use the function anywhere in the query, like a built-in scalar function.

下面是 JavaScript 用户定义的函数可派上用场的一些情景:Here are some scenarios where you might find JavaScript user-defined functions useful:

  • 使用 Regexp_Replace()Regexp_Extract() 等正则表达式函数分析和处理字符串Parsing and manipulating strings that have regular expression functions, for example, Regexp_Replace() and Regexp_Extract()
  • 解码和编码数据,例如,二进制到十六进制的转换Decoding and encoding data, for example, binary-to-hex conversion
  • 使用 JavaScript 数学函数进行数学计算Doing mathematic computations with JavaScript Math functions
  • 执行排序、联接、查找和填充等数组操作Doing array operations like sort, join, find, and fill

使用流分析中的 JavaScript 用户定义的函数无法实现以下目的:Here are some things that you can't do with a JavaScript user-defined function in Stream Analytics:

  • 调用外部 REST 终结点,例如,执行反向 IP 查找,或者从外部源提取引用数据Call out external REST endpoints, for example, doing reverse IP lookup or pulling reference data from an external source
  • 针对输入/输出执行自定义事件格式序列化或反序列化Perform custom event format serialization or deserialization on inputs/outputs
  • 创建自定义聚合Create custom aggregates

尽管函数定义中并不禁止 Date.GetDate()Math.random() 等函数,但应该避免使用这些函数。Although functions like Date.GetDate() or Math.random() aren't blocked in the functions definition, you should avoid using them. 这些函数在每次被调用时不会返回相同的结果,并且 Azure 流分析服务不会保留函数调用和返回结果的日记。These functions don't return the same result every time you call them, and the Azure Stream Analytics service doesn't keep a journal of function invocations and returned results. 当你或流分析服务重新启动某个作业时,如果某个函数针对相同的事件返回不同的结果,将无法保证可重复性。If a function returns different result on the same events, repeatability isn't guaranteed when a job is restarted by you or by the Stream Analytics service.

向作业中添加 JavaScript 用户定义的函数Add a JavaScript user-defined function to your job

若要在流分析作业中创建 JavaScript 用户定义的函数,请在“作业拓扑”下选择“函数”。To create a JavaScript user-defined function in your Stream Analytics job, select Functions under Job Topology. 然后,从“+ 添加”下拉菜单中选择“JavaScript UDF”。Then, select JavaScript UDF from the +Add dropdown menu.

添加 JavaScript UDF

然后,你必须提供以下属性并选择“保存”。You must then provide the following properties and select Save.

propertiesProperty 说明Description
函数别名Function alias 输入一个名称以在查询中调用函数。Enter a name to invoke the function in your query.
输出类型Output type JavaScript 用户定义的函数将向流分析查询返回的类型。Type that will be returned by your JavaScript user-defined function to your Stream Analytics query.
函数定义Function definition 每次从查询中调用 UDF 时将执行的 JavaScript 函数的实现。Implementation of your JavaScript function that will be executed each time your UDF gets invoked from your query.

对 JavaScript UDF 进行测试和故障排除Test and troubleshoot JavaScript UDFs

可在任何浏览器中测试和调试 JavaScript UDF 逻辑。You can test and debug your JavaScript UDF logic in any browser. 流分析门户目前不支持调试和测试这些用户定义函数的逻辑。Debugging and testing the logic of these user-defined functions is currently not supported in the Stream Analytics portal. 函数按预期方式运行后,可以将其添加到流分析作业(如上所述),然后直接从查询调用它。Once the function works as expected, you can add it to the Stream Analytics job as mentioned above and then invoke it directly from your query.

JavaScript 运行时错误被视为严重错误,可通过活动日志查看。JavaScript runtime errors are considered fatal, and are surfaced through the Activity log. 如果要检索日志,请在 Azure 门户中转到用户的作业,并选择“活动日志”。To retrieve the log, in the Azure portal, go to your job and select Activity log.

在查询中调用 JavaScript 用户定义的函数Call a JavaScript user-defined function in a query

可以使用以 udf 为前缀的函数别名轻松在查询中调用 JavaScript 函数。You can easily invoke your JavaScript function in your query using the function alias prefixed with udf. 下面是 JavaScript UDF 的一个示例,它将十六进制值转换为在流分析查询中调用的整数。Here is an example of a JavaScript UDF that converts hexadecimal values to integer being invoked in a Stream Analytics query.

    SELECT
        time,
        UDF.hex2Int(offset) AS IntOffset
    INTO
        output
    FROM
        InputStream

支持的 JavaScript 对象Supported JavaScript objects

Azure 流分析 JavaScript 用户定义的函数支持标准的内置 JavaScript 对象。Azure Stream Analytics JavaScript user-defined functions support standard, built-in JavaScript objects. 有关这些对象的列表,请参阅 Global Objects(全局对象)。For a list of these objects, see Global Objects.

流分析和 JavaScript 类型转换Stream Analytics and JavaScript type conversion

流分析查询语言与 JavaScript 支持的类型有差别。There are differences in the types that the Stream Analytics query language and JavaScript support. 下表列出了两者之间的转换映射:This table lists the conversion mappings between the two:

流分析Stream Analytics JavaScriptJavaScript
bigintbigint Number(JavaScript 只能精确呈现最大 2^53 的整数)Number (JavaScript can only represent integers up to precisely 2^53)
DateTimeDateTime Date(JavaScript 仅支持毫秒)Date (JavaScript only supports milliseconds)
Doubledouble NumberNumber
nvarchar(MAX)nvarchar(MAX) 字符串String
RecordRecord 对象Object
ArrayArray ArrayArray
NullNULL NullNull

下面是 JavaScript 到流分析的转换:Here are JavaScript-to-Stream Analytics conversions:

JavaScriptJavaScript 流分析Stream Analytics
NumberNumber 如果数字已舍入并介于 long.MinValue 和 long.MaxValue 之间,则为 Bigint;否则为 doubleBigint (if the number is round and between long.MinValue and long.MaxValue; otherwise, it's double)
DateDate DateTimeDateTime
字符串String nvarchar(MAX)nvarchar(MAX)
对象Object RecordRecord
ArrayArray ArrayArray
Null、UndefinedNull, Undefined NullNULL
其他任何类型(例如函数或错误)Any other type (for example, a function or error) 不支持(导致运行时错误)Not supported (results in runtime error)

JavaScript 语言区分大小写,JavaScript 代码中对象字段的大小写必须与传入数据中字段的大小写匹配。JavaScript language is case-sensitive and casing of the object fields in JavaScript code must match the casing of the fields in the incoming data. 兼容性级别为 1.0 的作业会将 SQL SELECT 语句中的字段转换为小写。Jobs with compatibility level 1.0 will convert fields from SQL SELECT statement to be lowercase. 在兼容性级别 1.1 及更高级别下,SELECT 语句中的字段将具有与 SQL 查询中指定的相同的大小写。Under compatibility level 1.1 and higher, fields from SELECT statement will have the same casing as specified in the SQL query.

JavaScript 用户定义的函数的其他模式Other JavaScript user-defined function patterns

编写要输出的嵌套 JSONWrite nested JSON to output

如果后续处理步骤需要使用流分析作业输出作为输入并且要求采用 JSON 格式,可以编写要输出的 JSON 字符串。If you have a follow-up processing step that uses a Stream Analytics job output as input, and it requires a JSON format, you can write a JSON string to output. 以下示例调用 JSON.stringify() 函数封装输入的所有名称/值对,并将其写入为输出中的单个字符串值。The next example calls the JSON.stringify() function to pack all name/value pairs of the input, and then write them as a single string value in output.

JavaScript 用户定义的函数定义:JavaScript user-defined function definition:

function main(x) {
return JSON.stringify(x);
}

示例查询:Sample query:

SELECT
    DataString,
    DataValue,
    HexValue,
    UDF.jsonstringify(input) As InputEvent
INTO
    output
FROM
    input PARTITION BY PARTITIONID

将字符串强制转换为要处理的 JSON 对象Cast string to JSON object to process

如果你有一个 JSON 字符串字段,并想将其转换为 JSON 对象以在 JavaScript UDF 中进行处理,则可以使用 JSON.parse() 函数来创建一个随后可使用的 JSON 对象。If you have a string field that is JSON and want to convert it to a JSON object for processing in a JavaScript UDF, you can use the JSON.parse() function to create a JSON object that can then be used.

JavaScript 用户定义的函数定义:JavaScript user-defined function definition:

function main(x) {
var person = JSON.parse(x);  
return person.name;
}

示例查询:Sample query:

SELECT
    UDF.getName(input) AS Name
INTO
    output
FROM
    input

使用 try/catch 进行错误处理Use try/catch for error handling

Try/catch 块可帮助确定传递给 JavaScript UDF 的格式错误的输入数据的问题。Try/catch blocks can help you identify problems with malformed input data that are passed into a JavaScript UDF.

JavaScript 用户定义的函数定义:JavaScript user-defined function definition:

function main(input, x) {
    var obj = null;

    try{
        obj = JSON.parse(x);
    }catch(error){
        throw input;
    }
    
    return obj.Value;
}

示例查询:将整个记录作为第一个参数进行传递,以便在出现错误时可将其返回。Sample query: Pass entire record as first parameter so that it can be returned if there is an error.

SELECT
    A.context.company AS Company,
    udf.getValue(A, A.context.value) as Value
INTO
    output
FROM
    input A