MLflow 模型注册表示例MLflow Model Registry example

此示例演示如何使用 MLflow 模型注册表,以建立一个机器学习应用程序,预测风力发电场的每日电力输出。This example illustrates how to use MLflow Model Registry to build a machine learning application that forecasts the daily power output of a wind farm. 该示例演示如何:The example shows how to:

  • 使用 MLflow 跟踪和记录模型Track and log models with MLflow
  • 使用模型注册表注册模型Register models with the Model Registry
  • 描述模型并进行模型版本阶段转换Describe models and make model version stage transitions
  • 将注册模型与生产应用程序集成Integrate registered models with production applications
  • 在模型注册表中搜索和发现模型Search and discover models in the Model Registry
  • 存档和删除模型Archive and delete models

本文介绍如何使用 MLflow 跟踪和 MLflow 模型注册表 UI 和 API 来执行这些步骤。The article describes how to perform these steps using the MLflow Tracking and MLflow Model Registry UIs and APIs.

有关使用 MLflow 跟踪功能和注册表 API 来执行所有这些步骤的笔记本,请参阅模型注册表示例笔记本For a notebook that performs all these steps using the MLflow Tracking and Registry APIs, see the Model Registry example notebook.

使用 MLflow 跟踪功能加载数据集、训练模型和进行跟踪Load dataset, train model, and track with MLflow Tracking

需要先在试验运行期间训练并记录模型,才能在模型注册表中注册模型。Before you can register a model in the Model Registry, you must first train and log the model during an experiment run. 本部分介绍如何加载风电场数据集、训练模型并将训练运行记录到 MLflow。This section shows how to load the wind farm dataset, train a model, and log the training run to MLflow.

加载数据集Load dataset

以下代码加载一个数据集,其中包含美国风电场的天气数据和电力输出信息。The following code loads a dataset containing weather data and power output information for a wind farm in the United States. 数据集包含 wind directionwind speedair temperature 功能,每六小时采样一次(在 00:0008:0016:00 各采集一次),以及过去几年来的每日总电力输出 (power)。The dataset contains wind direction, wind speed, and air temperature features sampled every six hours (once at 00:00, once at 08:00, and once at 16:00), as well as daily aggregate power output (power), over several years.

import pandas as pd
wind_farm_data = pd.read_csv("https://github.com/dbczumar/model-registry-demo-notebook/raw/master/dataset/windfarm_data.csv", index_col=0)

def get_training_data():
  training_data = pd.DataFrame(wind_farm_data["2014-01-01":"2018-01-01"])
  X = training_data.drop(columns="power")
  y = training_data["power"]
  return X, y

def get_validation_data():
  validation_data = pd.DataFrame(wind_farm_data["2018-01-01":"2019-01-01"])
  X = validation_data.drop(columns="power")
  y = validation_data["power"]
  return X, y

def get_weather_and_forecast():
  format_date = lambda pd_date : pd_date.date().strftime("%Y-%m-%d")
  today = pd.Timestamp('today').normalize()
  week_ago = today - pd.Timedelta(days=5)
  week_later = today + pd.Timedelta(days=5)

  past_power_output = pd.DataFrame(wind_farm_data)[format_date(week_ago):format_date(today)]
  weather_and_forecast = pd.DataFrame(wind_farm_data)[format_date(week_ago):format_date(week_later)]
  if len(weather_and_forecast) < 10:
    past_power_output = pd.DataFrame(wind_farm_data).iloc[-10:-5]
    weather_and_forecast = pd.DataFrame(wind_farm_data).iloc[-10:]

  return weather_and_forecast.drop(columns="power"), past_power_output["power"]

定型模型Train model

下面的代码训练 Keras 中的神经网络,以便根据数据集中的天气功能预测电力输出。The following code trains a neural network in Keras to predict power output based on the weather features in the dataset. MLflow 用于跟踪模型的超参数、性能指标、源代码和项目。MLflow is used to track the model’s hyperparameters, performance metrics, source code, and artifacts.

def train_keras_model(X, y):
  import keras
  from keras.models import Sequential
  from keras.layers import Dense

  model = Sequential()
  model.add(Dense(100, input_shape=(X_train.shape[-1],), activation="relu", name="hidden_layer"))
  model.add(Dense(1))
  model.compile(loss="mse", optimizer="adam")

  model.fit(X_train, y_train, epochs=100, batch_size=64, validation_split=.2)
  return model

import mlflow
import mlflow.keras

X_train, y_train = get_training_data()

with mlflow.start_run():
  # Automatically capture the model's parameters, metrics, artifacts,
  # and source code with the `autolog()` function
  mlflow.keras.autolog()

  train_keras_model(X_train, y_train)
  run_id = mlflow.active_run().info.run_id

使用 MLflow UI 注册和管理模型Register and manage the model using the MLflow UI

本部分内容:In this section:

创建新的已注册模型Create a new registered model

  1. 通过单击 Azure Databricks 笔记本 UI 中的“试验”图标试验,导航到 MLflow 试验运行边栏。Navigate to the MLflow Experiment Runs sidebar by clicking the Experiment icon Experiment in the Azure Databricks notebook UI.

    运行边栏Runs sidebar

  2. 找到对应于 Keras 模型训练会话的 MLflow 运行,并通过单击“查看运行细节”图标在 MLflow Run UI 中打开它。Locate the MLflow Run corresponding to the Keras model training session, and open it in the MLflow Run UI by clicking the View Run Detail icon.

  3. 在 MLflow UI 中,向下滚动到“项目”部分,然后单击名为“模型”的目录 。In the MLflow UI, scroll down to the Artifacts section and click the directory named model. 单击显示的“注册模型”按钮。Click the Register Model button that appears.

    注册模型Register model

  4. 从下拉菜单中选择“创建新模型”,然后输入以下模型名称:power-forecasting-modelSelect Create New Model from the drop-down menu, and input the following model name: power-forecasting-model.

  5. 单击“注册”。Click Register. 这将注册一个名为 power-forecasting-model 的新模型,并创建一个新的模型版本:Version 1This registers a new model called power-forecasting-model and creates a new model version: Version 1.

    新模型版本New model version

    几分钟后,MLflow UI 显示一个指向新注册模型的链接。After a few moments, the MLflow UI displays a link to the new registered model. 按照此链接在 MLflow 模型注册表 UI 中打开新的模型版本。Follow this link to open the new model version in the MLflow Model Registry UI.

浏览模型注册表 UIExplore the Model Registry UI

MLflow 模型注册表 UI 中的模型版本页面提供了关于已注册的预测模型 Version 1 的信息,包括其作者、创建时间和当前阶段。The model version page in the MLflow Model Registry UI provides information about Version 1 of the registered forecasting model, including its author, creation time, and its current stage.

模型版本页Model version page

模型版本页面还提供了“源运行”链接,该链接将打开 MLflow 运行,该运行用于在 MLflow 运行 UI 中创建模型。The model version page also provides a Source Run link, which opens the MLflow Run that was used to create the model in the MLflow Run UI. 在 MLflow 运行 UI 中,你可以访问“源”笔记本链接,以查看用于训练模型的 Azure Databricks 笔记本的快照。From the MLflow Run UI, you can access the Source notebook link to view a snapshot of the Azure Databricks notebook that was used to train the model.

源运行Source run

源笔记本Source notebook

若要导航回 MLflow 模型注册表,请单击To navigate back to the MLflow Model Registry, click the 侧栏中的 “模型”图标。icon in sidebar.

生成的 MLflow 模型注册表主页显示了 Azure Databricks 工作区中所有注册模型的列表,包括它们的版本和阶段。The resulting MLflow Model Registry home page displays a list of all the registered models in your Azure Databricks Workspace, including their versions and stages.

单击“电力预测模型”链接打开注册模型页面,其中显示预测模型的所有版本。Click the power-forecasting-model link to open the registered model page, which displays all of the versions of the forecasting model.

添加模型说明Add model descriptions

可以向已注册的模型和模型版本中添加说明。You can add descriptions to registered models and model versions. 注册模型说明信息有助于记录适用于多个模型版本的信息(例如,建模问题和数据集的概述)。Registered model descriptions are useful for recording information that applies to multiple model versions (e.g., a general overview of the modeling problem and dataset). 模型版本说明信息有助于详细说明特定模型版本的独特属性(例如,用于开发模型的方法和算法)。Model version descriptions are useful for detailing the unique attributes of a particular model version (e.g., the methodology and algorithm used to develop the model).

  1. 向注册的电力预测模型添加概述性说明。Add a high-level description to the registered power forecasting model. 单击单元格底部的Click the 编辑图标 图标并输入以下说明:icon and enter the following description:

    This model forecasts the power output of a wind farm based on weather data. The weather data consists of three features: wind speed, wind direction, and air temperature.
    

    添加模型说明Add model description

  2. 单击“保存” 。Click Save.

  3. 从已注册的模型页面中单击“版本 1”链接返回到模型版本页面。Click the Version 1 link from the registered model page to navigate back to the model version page.

  4. 单击单元格底部的Click the 编辑图标 图标并输入以下说明:icon and enter the following description:

    This model version was built using Keras. It is a feed-forward neural network with one hidden layer.
    

    添加模型版本说明Add model version description

  5. 单击“保存” 。Click Save.

转换模型版本Transition a model version

MLflow 模型注册表定义了多个模型阶段:“无”、“暂存”、“生产”和“Archived” 。The MLflow Model Registry defines several model stages: None, Staging, Production, and Archived. 每个阶段都有唯一的含义。Each stage has a unique meaning. 例如,“暂存”指模型处于测试阶段,“生产”指模型已完成测试或审核流程,并已部署到应用程序 。For example, Staging is meant for model testing, while Production is for models that have completed the testing or review processes and have been deployed to applications.

  1. 单击“阶段”按钮以显示可用模型阶段和可用阶段转换选项的列表。Click the Stage button to display the list of available model stages and your available stage transition options.

  2. 选择“转换到 -> 生产”,然后在阶段转换确认窗口中按“确定”,将模型转换到“生产” 。Select Transition to -> Production and press OK in the stage transition confirmation window to transition the model to Production.

    转换至“生产”Transition to production

    在模型版本转换到“生产”之后,当前阶段将在 UI 中显示,并在活动日志中添加一个条目以反映转换。After the model version is transitioned to Production, the current stage is displayed in the UI, and an entry is added to the activity log to reflect the transition.

    生产阶段Production stage

    模型版本活动Model version activity

MLflow 模型注册表允许多个模型版本共享同一阶段。The MLflow Model Registry allows multiple model versions to share the same stage. 当按阶段引用模型时,模型注册表使用最新的模型版本(版本 ID 最大的模型版本)。When referencing a model by stage, the Model Registry uses the latest model version (the model version with the largest version ID). 注册模型页面显示特定模型的所有版本。The registered model page displays all of the versions of a particular model.

已注册模型页Registered model page

使用 MLflow API 注册和管理模型Register and manage the model using the MLflow API

本部分内容:In this section:

以编程方式定义模型的名称Define the model’s name programmatically

现在模型已经注册并转换为“生产”,你可以使用 MLflow 编程 API 来引用它。Now that the model has been registered and transitioned to Production, you can reference it using MLflow programmatic APIs. 定义注册模型的名称,如下所示:Define the registered model’s name as follows:

model_name = "power-forecasting-model"

注册模型Register the model

model_name = get_model_name()

import mlflow

# The default path where the MLflow autologging function stores the Keras model
artifact_path = "model"
model_uri = "runs:/{run_id}/{artifact_path}".format(run_id=run_id, artifact_path=artifact_path)

model_details = mlflow.register_model(model_uri=model_uri, name=model_name)

import time
from mlflow.tracking.client import MlflowClient
from mlflow.entities.model_registry.model_version_status import ModelVersionStatus

# Wait until the model is ready
def wait_until_ready(model_name, model_version):
  client = MlflowClient()
  for _ in range(10):
    model_version_details = client.get_model_version(
      name=model_name,
      version=model_version,
    )
    status = ModelVersionStatus.from_string(model_version_details.status)
    print("Model status: %s" % ModelVersionStatus.to_string(status))
    if status == ModelVersionStatus.READY:
      break
    time.sleep(1)

wait_until_ready(model_details.name, model_details.version)

使用 API 添加模型和模型版本描述Add model and model version descriptions using the API

from mlflow.tracking.client import MlflowClient

client = MlflowClient()
client.update_registered_model(
  name=model_details.name,
  description="This model forecasts the power output of a wind farm based on weather data. The weather data consists of three features: wind speed, wind direction, and air temperature."
)

client.update_model_version(
  name=model_details.name,
  version=model_details.version,
  description="This model version was built using Keras. It is a feed-forward neural network with one hidden layer."
)

使用 API 转换模型版本并检索详细信息Transition a model version and retrieve details using the API

client.transition_model_version_stage(
  name=model_details.name,
  version=model_details.version,
  stage='Production',
)
model_version_details = client.get_model_version(
  name=model_details.name,
  version=model_details.version,
)
print("The current model stage is: '{stage}'".format(stage=model_version_details.current_stage))

latest_version_info = client.get_latest_versions(model_name, stages=["Production"])
latest_production_version = latest_version_info[0].version
print("The latest production version of the model '%s' is '%s'." % (model_name, latest_production_version))

使用 API 加载已注册模型的版本Load versions of the registered model using the API

MLflow 模型组件定义从多个机器学习框架加载模型的函数。The MLflow Models component defines functions for loading models from several machine learning frameworks. 例如,mlflow.keras.load_model() 用于加载以 MLflow 格式保存的 Keras 模型,mlflow.sklearn.load_model() 用于加载以 MLflow 格式保存的 scikit-learn 模型。For example, mlflow.keras.load_model() is used to load Keras models that were saved in MLflow format, and mlflow.sklearn.load_model() is used to load scikit-learn models that were saved in MLflow format.

这些函数可以从 MLflow 模型注册表加载模型。These functions can load models from the MLflow Model Registry.

import mlflow.pyfunc

model_version_uri = "models:/{model_name}/1".format(model_name=model_name)

print("Loading registered model version from URI: '{model_uri}'".format(model_uri=model_version_uri))
model_version_1 = mlflow.pyfunc.load_model(model_version_uri)

model_production_uri = "models:/{model_name}/production".format(model_name=model_name)

print("Loading registered model version from URI: '{model_uri}'".format(model_uri=model_production_uri))
model_production = mlflow.pyfunc.load_model(model_production_uri)

用生产模型预测电力输出Forecast power output with the production model

在本部分中,生产模型用于评估风电场的天气预报数据。In this section, the production model is used to evaluate weather forecast data for the wind farm. forecast_power() 应用程序从指定的阶段加载预测模型的最新版本,并使用它来预测未来五天的发电量。The forecast_power() application loads the latest version of the forecasting model from the specified stage and uses it to forecast power production over the next five days.

def plot(model_name, model_stage, model_version, power_predictions, past_power_output):
  import pandas as pd
  import matplotlib.dates as mdates
  from matplotlib import pyplot as plt
  index = power_predictions.index
  fig = plt.figure(figsize=(11, 7))
  ax = fig.add_subplot(111)
  ax.set_xlabel("Date", size=20, labelpad=20)
  ax.set_ylabel("Power\noutput\n(MW)", size=20, labelpad=60, rotation=0)
  ax.tick_params(axis='both', which='major', labelsize=17)
  ax.xaxis.set_major_formatter(mdates.DateFormatter('%m/%d'))
  ax.plot(index[:len(past_power_output)], past_power_output, label="True", color="red", alpha=0.5, linewidth=4)
  ax.plot(index, power_predictions, "--", label="Predicted by '%s'\nin stage '%s' (Version %d)" % (model_name, model_stage, model_version), color="blue", linewidth=3)
  ax.set_ylim(ymin=0, ymax=max(3500, int(max(power_predictions.values) * 1.3)))
  ax.legend(fontsize=14)
  plt.title("Wind farm power output and projections", size=24, pad=20)
  plt.tight_layout()
  display(plt.show())

def forecast_power(model_name, model_stage):
  from mlflow.tracking.client import MlflowClient
  client = MlflowClient()
  model_version = client.get_latest_versions(model_name, stages=[model_stage])[0].version
  model_uri = "models:/{model_name}/{model_stage}".format(model_name=model_name, model_stage=model_stage)
  model = mlflow.pyfunc.load_model(model_uri)
  weather_data, past_power_output = get_weather_and_forecast()
  power_predictions = pd.DataFrame(model.predict(weather_data))
  power_predictions.index = pd.to_datetime(weather_data.index)
  print(power_predictions)
  plot(model_name, model_stage, int(model_version), power_predictions, past_power_output)

创建新模型版本Create a new model version

经典的机器学习技术对电力预测也是有效的。Classical machine learning techniques are also effective for power forecasting. 下面的代码使用 scikit learn 训练随机林模型,并通过 mlflow.sklearn.log_model() 函数将其注册到 MLflow 模型注册表中。The following code trains a random forest model using scikit-learn and registers it with the MLflow Model Registry via the mlflow.sklearn.log_model() function.

import mlflow.sklearn
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_squared_error

with mlflow.start_run():
  n_estimators = 300
  mlflow.log_param("n_estimators", n_estimators)

  rand_forest = RandomForestRegressor(n_estimators=n_estimators)
  rand_forest.fit(X_train, y_train)

  val_x, val_y = get_validation_data()
  mse = mean_squared_error(rand_forest.predict(val_x), val_y)
  print("Validation MSE: %d" % mse)
  mlflow.log_metric("mse", mse)

  # Specify the `registered_model_name` parameter of the `mlflow.sklearn.log_model()`
  # function to register the model with the MLflow Model Registry. This automatically
  # creates a new model version
  mlflow.sklearn.log_model(
    sk_model=rand_forest,
    artifact_path="sklearn-model",
    registered_model_name=model_name,
  )
from mlflow.tracking.client import MlflowClient
client = MlflowClient()

model_version_infos = client.search_model_versions("name = '%s'" % model_name)
new_model_version = max([model_version_info.version for model_version_info in model_version_infos])

wait_until_ready(model_name, new_model_version)

向新模型版本添加说明Add a description to the new model version

client.update_model_version(
  name=model_name,
  version=new_model_version,
  description="This model version is a random forest containing 100 decision trees that was trained in scikit-learn."
)

将新模型版本转换到“暂存”阶段并测试模型Transition the new model version to Staging and test the model

在将模型部署到生产应用程序之前,最好在暂存环境中对其进行测试。Before deploying a model to a production application, it is often best practice to test it in a staging environment. 下面的代码将新模型版本转换为“暂存”,并评估其性能。The following code transitions the new model version to Staging and evaluates its performance.

client.transition_model_version_stage(
  name=model_name,
  version=new_model_version,
  stage="Staging",
)

forecast_power(model_name, "Staging")

将新模型版本部署到生产环境Deploy the new model version to Production

验证新模型版本在暂存中的性能良好之后,下面的代码将模型转换到“生产”阶段,并使用与用生产模型预测电力输出部分完全相同的应用程序代码生成电力预测。After verifying that the new model version performs well in staging, the following code transitions the model to Production and uses the exact same application code from the Forecast power output with the production model section to produce a power forecast.

client.transition_model_version_stage(
  name=model_name,
  version=new_model_version,
  stage="Production",
)

forecast_power(model_name, "Production")

现在,“生产”阶段的预测模型有两个版本:在 Keras 模型中训练的模型版本和在 scikit-learn 中训练的版本。There are now two model versions of the forecasting model in the Production stage: the model version trained in Keras model and the version trained in scikit-learn.

产品模型版本Product model versions

备注

当按阶段引用模型时,MLflow 模型注册表会自动使用最新的生产版本。When referencing a model by stage, the MLflow Model Model Registry automatically uses the latest production version. 这使你能够在不更改任何应用程序代码的情况下更新生产模型。This enables you to update your production models without changing any application code.

存档和删除模型Archive and delete models

当不再使用模型版本时,可以将其存档或删除。When a model version is no longer being used, you can archive it or delete it. 还可以删除整个已注册模型;这将删除其所有关联的模型版本。You can also delete an entire registered model; this removes all of its associated model versions.

将电力预测模型的 Version 1 进行存档Archive Version 1 of the power forecasting model

将电力预测模型的 Version 1 进行存档,因为不再使用。Archive Version 1 of the power forecasting model because it is no longer being used. 你可以在 MLflow 模型注册表 UI 中或通过 MLflow API 对模型进行存档。You can archive models in the MLflow Model Registry UI or via the MLflow API.

在 MLflow UI 中将 Version 1 进行存档Archive Version 1 in the MLflow UI

若要对电力预测模型的 Version 1 进行存档:To archive Version 1 of the power forecasting model:

  1. 在 MLflow 模型注册表 UI 中打开其对应的模型版本页面:Open its corresponding model version page in the MLflow Model Registry UI:

    转换到存档Transition to archived

  2. 单击“阶段”按钮,选择“转换到 -> 存档” :Click the Stage button, select Transition To -> Archived:

    存档阶段Archived stage

  3. 在阶段转换确认窗口中按“确定”。Press OK in the stage transition confirmation window.

    存档模型版本Archived model version

使用 MLflow API 将 Version 1 进行存档Archive Version 1 using the MLflow API

下面的代码使用 MlflowClient.update_model_version() 函数将电力预测模型的 Version 1 进行存档。The following code uses the MlflowClient.update_model_version() function to archive Version 1 of the power forecasting model.

from mlflow.tracking.client import MlflowClient

client = MlflowClient()
client.transition_model_version_stage(
  name=model_name,
  version=1,
  stage="Archived",
)

删除电力预测模型的 Version 1Delete Version 1 of the power forecasting model

还可以使用 MLflow UI 或 MLflow API 删除模型版本。You can also use the MLflow UI or MLflow API to delete model versions.

警告

模型版本删除是永久性的,不能撤消。Model version deletion is permanent and cannot be undone.

删除 MLflow UI 中的 Version 1Delete Version 1 in the MLflow UI

删除电力预测模型的 Version 1To delete Version 1 of the power forecasting model:

  1. 在 MLflow 模型注册表 UI 中打开其对应的模型版本页面。Open its corresponding model version page in the MLflow Model Registry UI.

    删除模型版本Delete model version

  2. 选择版本标识符旁边的下拉箭头并单击“删除”。Select the drop-down arrow next to the version identifier and click Delete.

使用 MLflow API 删除 Version 1Delete Version 1 using the MLflow API
client.delete_model_version(
   name=model_name,
   version=1,
)
使用 MLflow API 删除模型Delete the model using the MLflow API

需要首先将所有剩余的模型版本阶段转换为“无”或“已存档” 。You must first transition all remaining model version stages to None or Archived.

from mlflow.tracking.client import MlflowClient

client = MlflowClient()
client.transition_model_version_stage(
  name=model_name,
  version=2,
  stage="Archived",
)
client.delete_registered_model(name=model_name)

笔记本Notebook

MLflow 模型注册表示例笔记本MLflow Model Registry example notebook

获取笔记本Get notebook