优化模型超参数Tune Model Hyperparameters

本文介绍如何使用 Azure 机器学习设计器(预览版)中的“优化模型超参数”模块。This article describes how to use the Tune Model Hyperparameters module in Azure Machine Learning designer (preview). 目标是确定机器学习模型的最佳超参数。The goal is to determine the optimum hyperparameters for a machine learning model. 该模块使用不同的设置组合来生成并测试多个模型。The module builds and tests multiple models by using different combinations of settings. 它将比较所有模型的指标,以获取设置组合。It compares metrics over all models to get the combinations of settings.

术语“参数”和“超参数”可能会造成混淆。 The terms parameter and hyperparameter can be confusing. 模型的参数是在模块右窗格中设置的参数。The model's parameters are what you set in the right pane of the module. 简单而言,此模块将对指定的参数设置执行参数扫描。Basically, this module performs a parameter sweep over the specified parameter settings. 它将会习得一组最佳的超参数,而这些超参数对于每个特定的决策树、数据集或回归方法可能有所不同。It learns an optimal set of hyperparameters, which might be different for each specific decision tree, dataset, or regression method. 查找最佳配置的过程有时称为“优化”。The process of finding the optimal configuration is sometimes called tuning.

该模块支持使用以下方法来查找模型的最佳设置:集成式训练和优化。The module supports the following method for finding the optimum settings for a model: integrated train and tune. 在此方法中,你将配置一组要使用的参数。In this method, you configure a set of parameters to use. 然后,让模块迭代多个组合。You then let the module iterate over multiple combinations. 该模块将不断测量准确度,直到找到“最佳”模型。The module measures accuracy until it finds a "best" model. 对于大多数学习器模块,可以选择在训练过程中要更改的参数,以及要保持固定的参数。With most learner modules, you can choose which parameters should be changed during the training process, and which should remain fixed.

根据优化过程的所需运行时长,你可能决定对所有组合进行穷尽测试。Depending on how long you want the tuning process to run, you might decide to exhaustively test all combinations. 或者,你可能会通过建立参数组合的网格,并测试参数网格的随机化子集,来缩短该过程的时长。Or you might shorten the process by establishing a grid of parameter combinations and testing a randomized subset of the parameter grid.

此方法将生成一个已训练的模型,可以保存该模型以供重复使用。This method generates a trained model that you can save for reuse.

提示

可以执行相关的任务。You can do a related task. 在开始优化之前,请应用特征选择,以确定信息价值最高的列或变量。Before you start tuning, apply feature selection to determine the columns or variables that have the highest information value.

如何配置“优化模型超参数”How to configure Tune Model Hyperparameters

了解机器学习模型的最佳超参数需要大量使用管道。Learning the optimal hyperparameters for a machine learning model requires considerable use of pipelines.

使用参数扫描训练模型Train a model by using a parameter sweep

本部分介绍如何执行基本的参数扫描,以使用“优化模型超参数”模块训练模型。This section describes how to perform a basic parameter sweep, which trains a model by using the Tune Model Hyperparameters module.

  1. 在设计器中,将“优化模型超参数”模块添加到管道。Add the Tune Model Hyperparameters module to your pipeline in the designer.

  2. 将未经训练的模型连接到最左侧的输入。Connect an untrained model to the leftmost input.

    备注

    优化模型超参数只能连接到内置机器学习算法模块,不能支持创建 Python 模型中内置的自定义模型。Tune Model Hyperparameters can only be connect to built-in machine learning algorithm modules, and cannot support customized model built in Create Python Model.

  3. 添加要用于训练的数据集,然后将其连接到“优化模型超参数”的中间输入。Add the dataset that you want to use for training, and connect it to the middle input of Tune Model Hyperparameters.

    (可选)如果有带标签的数据集,可将其连接到最右侧的输入端口(“可选的验证数据集”)。Optionally, if you have a tagged dataset, you can connect it to the rightmost input port (Optional validation dataset). 这样,便可以在训练和优化时测量准确度。This lets you measure accuracy while training and tuning.

  4. 在“优化模型超参数”的右侧面板中,为“参数扫描模式”选择值。In the right panel of Tune Model Hyperparameters, choose a value for Parameter sweeping mode. 此选项控制参数的选择方式。This option controls how the parameters are selected.

    • 整个网格:如果选择此选项,模块将循环访问系统预定义的网格,以尝试不同的组合并识别最佳学习器。Entire grid: When you select this option, the module loops over a grid predefined by the system, to try different combinations and identify the best learner. 如果你不知道最佳的参数设置是什么,并想要尝试所有可能的值组合,则此选项很有用。This option is useful when you don't know what the best parameter settings might be and want to try all possible combinations of values.

    • 随机扫描:如果选择此选项,模块将在系统定义的范围内随机选择参数值。Random sweep: When you select this option, the module will randomly select parameter values over a system-defined range. 必须指定要让模块执行的最大运行次数。You must specify the maximum number of runs that you want the module to execute. 如果你想要使用所选的指标提高模型性能,同时仍要节省计算资源,则此选项很有用。This option is useful when you want to increase model performance by using the metrics of your choice but still conserve computing resources.

  5. 对于“标签列”,打开列选择器以选择单个标签列。For Label column, open the column selector to choose a single label column.

  6. 选择运行次数:Choose the number of runs:

    1. 针对随机扫描的最大运行次数:如果选择随机扫描,则可以使用参数值的随机组合,指定要训练模型多少次。Maximum number of runs on random sweep: If you choose a random sweep, you can specify how many times the model should be trained, by using a random combination of parameter values.
  7. 对于“排名”,请选择用来为模型排名的单个指标。For Ranking, choose a single metric to use for ranking the models.

    运行参数扫描时,模块将计算模型类型的所有适用指标,并在“扫描结果”报告中返回这些指标。When you run a parameter sweep, the module calculates all applicable metrics for the model type and returns them in the Sweep results report. 模块对回归模型和分类模型使用不同的指标。The module uses separate metrics for regression and classification models.

    但是,所选的指标确定了模型的排名方式。However, the metric that you choose determines how the models are ranked. 只有按所选指标排名最高的模型才会输出为已训练的模型用于评分。Only the top model, as ranked by the chosen metric, is output as a trained model to use for scoring.

  8. 对于“随机种子”,请输入一个数字用于启动参数扫描。For Random seed, enter a number to use for starting the parameter sweep.

  9. 提交管道。Submit the pipeline.

超参数优化的结果Results of hyperparameter tuning

训练完成时:When training is complete:

  • 若要查看最佳模型的一组准确度指标,请右键单击该模块,然后选择“可视化”。To view a set of accuracy metrics for the best model, right-click the module, and then select Visualize.

    输出包括应用到模型类型的所有准确度指标,但选择用于排名的指标确定了哪个模型被视为“最佳”。The output includes all accuracy metrics that apply to the model type, but the metric that you selected for ranking determines which model is considered "best."

  • 若要保存已训练模型的快照,请选择“训练模型”模块右侧面板中的“输出”选项卡。To save a snapshot of the trained model, select the Outputs tab in the right panel of the Train model module. 选择“注册数据集”图标将模型保存为可重用模块。Select the Register dataset icon to save the model as a reusable module.

技术说明Technical notes

本部分包含实现详细信息和提示。This section contains implementation details and tips.

参数扫描的工作原理How a parameter sweep works

设置参数扫描时,需定义搜索范围。When you set up a parameter sweep, you define the scope of your search. 搜索可以使用随机选择的有限数量的参数。The search might use a finite number of parameters selected randomly. 或者,可以针对定义的参数空间执行穷尽式搜索。Or it might be an exhaustive search over a parameter space that you define.

  • 随机扫描:此选项使用设置的迭代次数训练模型。Random sweep: This option trains a model by using a set number of iterations.

    指定要迭代的值范围,模块将使用这些值的随机选择子集。You specify a range of values to iterate over, and the module uses a randomly chosen subset of those values. 值是使用替换法选择的,这意味着,以前随机选择的数字不会从可用数字池中删除。Values are chosen with replacement, meaning that numbers previously chosen at random are not removed from the pool of available numbers. 因此,选择任意值的机会在所有轮次中都保持相同。So the chance of any value being selected stays the same across all passes.

  • 整个网格:使用整个网格的选项意味着要测试每个组合。Entire grid: The option to use the entire grid means that every combination is tested. 此选项是最彻底的,但所需的时间最长。This option is the most thorough, but it requires the most time.

控制训练的长度和复杂性Controlling the length and complexity of training

迭代许多的设置组合可能会很耗时,因此,模块提供多种方式来限制该过程:Iterating over many combinations of settings can be time-consuming, so the module provides several ways to constrain the process:

  • 限制用于测试模型的迭代次数。Limit the number of iterations used to test a model.
  • 限制参数空间。Limit the parameter space.
  • 同时限制迭代次数和参数空间。Limit both the number of iterations and the parameter space.

建议通过管道来处理设置,以根据特定的数据集和模型确定最有效的训练方法。We recommend that you pipeline with the settings to determine the most efficient method of training on a particular dataset and model.

选择评估指标Choosing an evaluation metric

测试结束时,模型将提供一份包含每个模型的准确度的报告,使你可以查看指标结果:At the end of testing, the model presents a report that contains the accuracy for each model so that you can review the metric results:

  • 用于所有二元分类模型的一组统一指标。A uniform set of metrics is used for all binary classification models.
  • 准确度用于所有多类分类模型。Accuracy is used for all multi-class classification models.
  • 用于回归模型的一组不同指标。A different set of metrics is used for regression models.

但是,在训练过程中,必须选择单个指标,用来为优化过程中生成的模型排名。However, during training, you must choose a single metric to use in ranking the models that are generated during the tuning process. 你可能发现,最佳指标根据业务问题以及误报和漏报的代价而各不相同。You might find that the best metric varies, depending on your business problem and the cost of false positives and false negatives.

用于二元分类的指标Metrics used for binary classification

  • “准确度”是真实结果与案例总数之比。Accuracy is the proportion of true results to total cases.

  • “精准度”是真实结果与正面结果之比。Precision is the proportion of true results to positive results.

  • “召回率”是所有结果中所有正确结果的分数。Recall is the fraction of all correct results over all results.

  • “F 评分”是平衡精准度与召回率的指标。F-score is a measure that balances precision and recall.

  • “AUC”是一个值,表示在 x 轴上绘制误报、在 y 轴上绘制漏报时,曲线下的面积。AUC is a value that represents the area under the curve when false positives are plotted on the x-axis and true positives are plotted on the y-axis.

  • “平均对数损失”是以下两个概率分布之间的差:真实分布,以及模型中的分布。Average Log Loss is the difference between two probability distributions: the true one, and the one in the model.

用于回归的指标Metrics used for regression

  • “平均绝对误差”求模型中所有误差的平均值,其中,“误差”表示预测值与真实值之间的距离。Mean absolute error averages all the errors in the model, where error means the distance of the predicted value from the true value. 它通常缩写为 MAEIt's often abbreviated as MAE.

  • “均方根误差”度量误差平方的平均值,然后求该值的根。Root of mean squared error measures the average of the squares of the errors, and then takes the root of that value. 它通常缩写为 RMSEIt's often abbreviated as RMSE.

  • “相对绝对误差”以真实值的百分比表示误差。Relative absolute error represents the error as a percentage of the true value.

  • “相对平方误差”除以预测值的总平方误差,以此规范化总平方误差。Relative squared error normalizes the total squared error by dividing by the total squared error of the predicted values.

  • “决定系数”是用于指示数据在模型中的拟合度的单个数字。Coefficient of determination is a single number that indicates how well data fits a model. 如果值为 1,则表示模型与数据完全匹配。A value of one means that the model exactly matches the data. 如果值为 0,则表示数据是随机的,或者无法拟合模型。A value of zero means that the data is random or otherwise can't be fit to the model. 它通常称为 r2R2 或 r 平方。It's often called r2, R2, or r-squared.

不支持参数扫描的模块Modules that don't support a parameter sweep

在 Azure 机器学习中,几乎所有的学习器都支持使用集成式参数扫描进行交叉验证,可让你选择要通过管道处理的参数。Almost all learners in Azure Machine Learning support cross-validation with an integrated parameter sweep, which lets you choose the parameters to pipeline with. 如果学习器不支持设置值的范围,你仍可以在交叉验证中使用它。If the learner doesn't support setting a range of values, you can still use it in cross-validation. 在这种情况下,将为扫描选择允许值的范围。In this case, a range of allowed values is selected for the sweep.

后续步骤Next steps

请参阅 Azure 机器学习的可用模块集See the set of modules available to Azure Machine Learning.