Manage Python 3 packages in Azure Automation

This article describes how to import, manage, and use Python 3 packages in Azure Automation running on the Azure sandbox environment and Hybrid Runbook Workers. Python packages should be downloaded on Hybrid Runbook workers for successful job execution. To help simplify runbooks, you can use Python packages to import the modules you need.

For information on managing Python 2 packages, see Manage Python 2 packages.

Default Python packages

To support Python 3.8 runbooks in the Automation service, some Python packages are installed by default and a list of these packages are here. The default version can be overridden by importing Python packages into your Automation account.

Preference is given to the imported version in your Automation account. To import a single package, see Import a package. To import a package with multiple packages, see Import a package with dependencies.

Packages as source files

Azure Automation supports only a Python package that only contains Python code and doesn't include other language extensions or code in other languages. However, the Azure Sandbox environment might not have the required compilers for C/C++ binaries, so it's recommended to use wheel files instead.

The Python Package Index (PyPI) is a repository of software for the Python programming language. When selecting a Python 3 package to import into your Automation account from PyPI, note the following filename parts:

Select a Python version:

Python 3.8(GA)

Filename part Description
cp38 Automation supports Python 3.8 for Cloud jobs.
amd64 Azure sandbox processes are Windows 64-bit architecture.

For example:

  • To import pandas - select a wheel file with a name similar as pandas-1.2.3-cp38-win_amd64.whl.

Some Python packages available on PyPI don't provide a wheel file. In this case, download the source (.zip or .tar.gz file) and generate the wheel file using pip.

Perform the following steps using a 64-bit Windows machine with Python 3.8.x and wheel package installed:

  1. Download the source file pandas-1.2.4.tar.gz.
  2. Run pip to get the wheel file with the following command: pip wheel --no-deps pandas-1.2.4.tar.gz

Import a package

  1. In your Automation account, select Python packages under Shared Resources. Then select + Add a Python package.

    Screenshot of the Python packages page shows Python packages in the left menu and Add a Python package highlighted.

  2. On the Add Python Package page, select a local package to upload. The package can be .whl or .tar.gz file for Python 3.8.

  3. Enter a name and select the Runtime Version as Python 3.8.

  4. Select Import.

    Screenshot shows the Add Python 3.8 Package page with an uploaded tar.gz file selected.

After a package has been imported, it's listed on the Python packages page in your Automation account. To remove a package, select the package and select Delete.

Screenshot shows the Python 3.8 packages page after a package has been imported.

Import a package with dependencies

You can import a Python 3.8 package and its dependencies by importing the following Python script into a Python 3.8 runbook. Ensure that Managed identity is enabled for your Automation account and has Automation Contributor access for successful import of package.

https://github.com/azureautomation/runbooks/blob/master/Utility/Python/import_py3package_from_pypi.py

Importing the script into a runbook

For information on importing the runbook, see Import a runbook from the Azure portal. Copy the file from GitHub to storage that the portal can access before you run the import.

The Import a runbook page defaults the runbook name to match the name of the script. If you have access to the field, you can change the name. Runbook type may default to Python 2.7. If it does, make sure to change it to Python 3.8.

Screenshot shows the Python 3 runbook import page.

Executing the runbook to import the package and dependencies

After creating and publishing the runbook, run it to import the package. See Start a runbook in Azure Automation for details on executing the runbook.

The script (import_py3package_from_pypi.py) requires the following parameters.

Parameter Description
subscription_id Subscription ID of the Automation account
resource_group Name of the resource group that the Automation account is defined in
automation_account Automation account name
module_name Name of the module to import from pypi.org
module_version Version of the module

Parameter value should be provided as a single string in the below format:

-s <subscription_id> -g <resource_group> -a<automation_account> -m <module_name> -v <module_version>

For more information on using parameters with runbooks, see Work with runbook parameters.

Use a package in a runbook

With the package imported, you can use it in a runbook. Add the following code to list all the resource groups in an Azure subscription.

#!/usr/bin/env python3 
import os 
import requests  
# printing environment variables 
endPoint = os.getenv('IDENTITY_ENDPOINT')+"?resource=https://management.chinacloudapi.cn/" 
identityHeader = os.getenv('IDENTITY_HEADER') 
payload={} 
headers = { 
  'X-IDENTITY-HEADER': identityHeader,
  'Metadata': 'True' 
} 
response = requests.request("GET", endPoint, headers=headers, data=payload) 
print(response.text)

Note

The Python automationassets package is not available on pypi.org, so it's not available for import on to a Windows hybrid runbook worker.

Identify available packages in sandbox

Use the following code to list the default installed modules:

#!/usr/bin/env python3

import pkg_resources
installed_packages = pkg_resources.working_set
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
   for i in installed_packages])

for package in installed_packages_list:
    print(package)

Python 3.8 PowerShell cmdlets

Add new Python 3.8 package

New-AzAutomationPython3Package -AutomationAccountName tarademo  -ResourceGroupName mahja -Name requires.io -ContentLinkUri https://files.pythonhosted.org/packages/7f/e2/85dfb9f7364cbd7a9213caea0e91fc948da3c912a2b222a3e43bc9cc6432/requires.io-0.2.6-py2.py3-none-any.whl 

Response  
ResourceGroupName     : mahja 
AutomationAccountName : tarademo 
Name                  : requires.io 
IsGlobal              : False 
Version               : 
SizeInBytes           : 0 
ActivityCount         : 0 
CreationTime          : 9/26/2022 1:37:13 PM +05:30 
LastModifiedTime      : 9/26/2022 1:37:13 PM +05:30 
ProvisioningState     : Creating 

List all Python 3.8 packages

Get-AzAutomationPython3Package -AutomationAccountName tarademo  -ResourceGroupName mahja 

Response : 
ResourceGroupName     : mahja 
AutomationAccountName : tarademo 
Name                  : cryptography 
IsGlobal              : False 
Version               : 
SizeInBytes           : 0 
ActivityCount         : 0 
CreationTime          : 9/26/2022 11:52:28 AM +05:30 
LastModifiedTime      : 9/26/2022 12:11:00 PM +05:30 
ProvisioningState     : Failed 
ResourceGroupName     : mahja 
AutomationAccountName : tarademo 
Name                  : requires.io 
IsGlobal              : False 
Version               : 
SizeInBytes           : 0 
ActivityCount         : 0 
CreationTime          : 9/26/2022 1:37:13 PM +05:30 
LastModifiedTime      : 9/26/2022 1:39:04 PM +05:30 
ProvisioningState     : ContentValidated 
ResourceGroupName     : mahja 
AutomationAccountName : tarademo 
Name                  : sockets 
IsGlobal              : False 
Version               : 1.0.0 
SizeInBytes           : 4495 
ActivityCount         : 0 
CreationTime          : 9/20/2022 12:46:28 PM +05:30 
LastModifiedTime      : 9/22/2022 5:03:42 PM +05:30 
ProvisioningState     : Succeeded 

Obtain details about specific package

Get-AzAutomationPython3Package -AutomationAccountName tarademo  -ResourceGroupName mahja -Name sockets 


Response  
ResourceGroupName     : mahja 
AutomationAccountName : tarademo 
Name                  : sockets 
IsGlobal              : False 
Version               : 1.0.0 
SizeInBytes           : 4495 
ActivityCount         : 0 
CreationTime          : 9/20/2022 12:46:28 PM +05:30 
LastModifiedTime      : 9/22/2022 5:03:42 PM +05:30 
ProvisioningState     : Succeeded 

Remove Python 3.8 package

Remove-AzAutomationPython3Package -AutomationAccountName tarademo  -ResourceGroupName mahja -Name sockets 

Update Python 3.8 package

Set-AzAutomationPython3Package -AutomationAccountName tarademo  -ResourceGroupName mahja -Name requires.io -ContentLinkUri https://files.pythonhosted.org/packages/7f/e2/85dfb9f7364cbd7a9213caea0e91fc948da3c912a2b222a3e43bc9cc6432/requires.io-0.2.6-py2.py3-none-any.whl 


ResourceGroupName     : mahja 
AutomationAccountName : tarademo 
Name                  : requires.io 
IsGlobal              : False 
Version               : 0.2.6 
SizeInBytes           : 10109 
ActivityCount         : 0 
CreationTime          : 9/26/2022 1:37:13 PM +05:30 
LastModifiedTime      : 9/26/2022 1:43:12 PM +05:30 
ProvisioningState     : Creating 

Next steps

To prepare a Python runbook, see Create a Python runbook.