Tutorial: Develop IoT Edge modules with Linux containers by using IoT Edge for Linux on Windows

Applies to: IoT Edge 1.4

This tutorial walks you through developing, debugging, and deploying your own code to an Azure IoT Edge device by using IoT Edge for Linux on Windows and Visual Studio 2022. You'll learn the most common developer scenario for IoT Edge solutions by deploying a C# module to a Linux device. You'll deploy and debug a custom IoT Edge module running in a Linux container on Windows. Even if you plan to use a different language or deploy an Azure service, this tutorial is still useful for learning about the development tools and concepts.

This tutorial includes steps for two IoT Edge development tools:

• The Azure IoT Edge Dev Tool CLI command-line interface (CLI), which is the preferred tool for development
• The Azure IoT Edge Tools extension for Visual Studio, which is in maintenance mode

Use the selector button at the beginning of this tutorial to select the tool version.

In this tutorial, you learn how to:

Prerequisites

This tutorial assumes that you use a machine running Windows as your development machine. On Windows computers, you can develop either Windows or Linux modules. This tutorial guides you through the development of Linux containers, by using IoT Edge for Linux on Windows for building and deploying the modules.

Before you begin:

• Install IoT Edge for Linux on Windows.

• Read the quickstart Deploy your first IoT Edge module to a Windows device.

• Download the .NET Core SDK.

• Install or modify Visual Studio 2022 on your development machine. Choose the Azure development and Desktop development with C++ workload options.

• After your Visual Studio 2022 installation is ready, download and install Azure IoT Edge Tools from Visual Studio Marketplace.

  You can use the Azure IoT Edge Tools extension to create and build your IoT Edge solution. The preferred development tool is the Azure IoT Edge Dev Tool CLI. The extension includes the Azure IoT Edge project templates that you use to create the Visual Studio project. Currently, you need to install the extension regardless of the development tool that you use.

  Tip

  If you're using Visual Studio 2019, download and install Azure IoT Edge Tools for Visual Studio 2019 from Visual Studio Marketplace.

• Create a free or standard-tier IoT hub in Azure as a cloud resource.

If you don't have an Azure trial subscription, create an Azure trial subscription before you begin.

Key concepts

This tutorial walks through the development of an IoT Edge module. An IoT Edge module is a container that has executable code. You can deploy one or more modules to an IoT Edge device. Modules perform specific tasks like ingesting data from sensors, cleaning and analyzing data, or sending messages to an IoT hub. For more information, see Understand Azure IoT Edge modules.

When you're developing IoT Edge modules, it's important to understand the difference between the development machine and the target IoT Edge device where the module will eventually be deployed. The container that you build to hold your module code must match the operating system (OS) of the target device.

For example, the most common scenario is someone developing a module on a Windows computer who intends to target a Linux device running IoT Edge. In that case, the container OS is Linux.

As you go through this tutorial, keep in mind the difference between the development machine OS and the container OS. For this tutorial, you'll use your Windows host for development and the IoT Edge for Linux on Windows virtual machine (VM) for building and deploying the modules.

This tutorial targets devices that are running IoT Edge with Linux containers. You can use your preferred operating system as long as your development machine runs Linux containers. We recommend using Visual Studio to develop with Linux containers, so that's what this tutorial uses. You can also use Visual Studio Code, although there are differences in support between the two tools. For more information, see Develop Azure IoT Edge modules using Visual Studio Code.

Set up the Docker CLI and Docker engine for remote connection

IoT Edge modules are packaged as containers, so you need a container engine on your development machine to build and manage them.

The IoT Edge for Linux on Windows VM already contains an instance of the Docker engine. This tutorial shows you how to remotely connect from the Windows developer machine to the IoT Edge for Linux on Windows VM Docker instance. By using this remote connection, you remove the dependency on Docker Desktop for Windows.

Configure the Docker CLI

The first step is to configure the Docker CLI on the Windows development machine to be able to connect to the remote Docker engine:

1. Download the precompiled docker.exe version of the Docker CLI from Chocolatey. You can also download the official cli project from GitHub and compile it by following the repo instructions.

2. Extract docker.exe to a directory in your development machine; for example, C:\Docker\bin.

3. Open About your PC > System Info > Advanced system settings.

4. Select Advanced > Environment variables. Under User variables, select Path.

5. Edit the Path variable and add the location of docker.exe.

6. Open an elevated PowerShell session.

7. Check that the Docker CLI is accessible by using this command:

   docker --version
   

   If you successfully configured everything, the output of the command should show the Docker version. It should look something like Docker version 20.10.12, build e91ed57.

Configure the Docker engine

The second step is to configure the IoT Edge for Linux on Windows VM Docker engine to accept external connections, and add the appropriate firewall rules.

1. Open an elevated PowerShell session and run the following commands:

   # Configure the IoT Edge for Linux on Windows VM Docker engine to accept external connections, and add the appropriate firewall rules.
   Invoke-EflowVmCommand "sudo iptables -A INPUT -p tcp --dport 2375 -j ACCEPT"
   
   # Create a copy of the IoT Edge for Linux on Windows VM _docker.service_ in the system folder.
   Invoke-EflowVmCommand "sudo cp /lib/systemd/system/docker.service /etc/systemd/system/docker.service"
   
   # Replace the service execution line to listen for external connections.
   Invoke-EflowVmCommand "sudo sed -i 's/-H fd:\/\// -H fd:\/\/ -H tcp:\/\/0.0.0.0:2375/g'  /etc/systemd/system/docker.service"
   
   # Reload the IoT Edge for Linux on Windows VM services configurations.
   Invoke-EflowVmCommand "sudo systemctl daemon-reload"
   
   # Reload the Docker engine service.
   Invoke-EflowVmCommand "sudo systemctl restart docker.service"
   
   # Check that the Docker engine is listening to external connections.
   Invoke-EflowVmCommand "sudo netstat -lntp | grep dockerd"
   

   Here's example output:

   PS C:\> # Configure the IoT Edge for Linux on Windows virtual machine Docker engine to accept external connections, and add the appropriate firewall rules.
   PS C:\> Invoke-EflowVmCommand "sudo iptables -A INPUT -p tcp --dport 2375 -j ACCEPT"
   PS C:\>
   PS C:\> # Create a copy of the IoT Edge for Linux on Windows VM docker.service in the system folder.
   PS C:\> Invoke-EflowVmCommand "sudo cp /lib/systemd/system/docker.service /etc/systemd/system/docker.service"
   PS C:\>
   PS C:\> # Replace the service execution line to listen for external connections.
   PS C:\> Invoke-EflowVmCommand "sudo sed -i 's/-H fd:\/\// -H fd:\/\/ -H tcp:\/\/0.0.0.0:2375/g' /etc/systemd/system/docker.service"
   PS C:\>
   PS C:\> # Reload the IoT Edge for Linux on Windows VM services configurations.
   PS C:\> Invoke-EflowVmCommand "sudo systemctl daemon-reload"
   PS C:\>
   PS C:\> # Reload the Docker engine service.
   PS C:\> Invoke-EflowVmCommand "sudo systemctl restart docker.service"
   PS C:\>
   PS C:\> # Check that the Docker engine is listening to external connections.
   PS C:\> Invoke-EflowVmCommand "sudo netstat -lntp | grep dockerd"
   tcp6       0      0 :::2375                 :::* LISTEN      2790/dockerd
   

Test the connection

The final setup step is to test the Docker connection to the IoT Edge for Linux on Windows VM Docker engine:

1. Get the IP address of the IoT Edge for Linux on Windows VM:

   Get-EflowVmAddr
   

   Tip

   If the IoT Edge for Linux on Windows VM was deployed without a static IP, the IP address might change across Windows host OS restarts or network changes. Make sure you're using the correct IP address for the IoT Edge for Linux on Windows VM every time you want to establish a remote connection to the Docker engine.

   Here's example output:

   PS C:\> Get-EflowVmAddr
   [03/15/2022 15:22:30] Querying IP and MAC addresses from virtual machine (DESKTOP-J1842A1-EFLOW)
    - Virtual machine MAC: 00:15:5d:6f:da:78
    - Virtual machine IP : 172.31.24.105 retrieved directly from virtual machine
   00:15:5d:6f:da:78
   172.31.24.105 
   

2. Connect to the IoT Edge for Linux on Windows VM Docker engine and run the hello-world sample container. Replace <EFLOW-VM-IP> with the IoT Edge for Linux on Windows VM IP address that you obtained in the previous step.

   docker -H tcp://<EFLOW-VM-IP>:2375 run --rm hello-world
   

   When the download of the container finishes, the container runs and produces this output:

   PS C:\> docker -H tcp://172.31.24.105:2375 run --rm hello-world
   Unable to find image 'hello-world:latest' locally
   latest: Pulling from library/hello-world
   2db29710123e: Pull complete
   Digest: sha256:4c5f3db4f8a54eb1e017c385f683a2de6e06f75be442dc32698c9bbe6c861edd
   Status: Downloaded newer image for hello-world:latest
   
   Hello from Docker!
   This message shows that your installation appears to be working correctly.
   
   To generate this message, Docker took the following steps:
    1. The Docker client contacted the Docker daemon.
    2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
       (amd64)
    3. The Docker daemon created a new container from that image which runs the
       executable that produces the output you're currently reading.
    4. The Docker daemon streamed that output to the Docker client, which sent it
       to your terminal.
   
   To try something more ambitious, you can run an Ubuntu container with:
    $ docker run -it ubuntu bash
   
   Share images, automate workflows, and more with a free Docker ID:
    https://hub.docker.com/
   
   For more examples and ideas, visit:
    https://docs.docker.com/get-started/
   

Create an Azure IoT Edge project

The IoT Edge project template in Visual Studio creates a solution that you can deploy to IoT Edge devices. Use the following steps to create an Azure IoT Edge solution and then generate the first module in that solution. Each IoT Edge solution can contain more than one module.

1. In Visual Studio, create a new project by selecting Create a new project on the start page or by selecting the New Project button on the toolbar.

2. On the Create a new project page, search for Azure IoT Edge. Select the project that matches the platform (Linux IoT Edge module) and architecture for your IoT Edge device, and then select Next.

3. On the Configure your new project page, enter a name for your project and specify the location, and then select Create.

4. In the Add Module dialog, select the type of module that you want to develop. You can also select Existing module to add an existing IoT Edge module to your deployment.

5. In Module Name, specify your module name.

6. In Repository Url, provide the name of the module's image repository. Visual Studio automatically populates the module name with localhost:5000/<your module name>. Replace it with your own registry information.

   Use localhost if you use a local Docker registry for testing. If you use Azure Container Registry, then use the login server from your registry's settings. The login server looks like <registry name>.azurecr.cn. Replace only the localhost:5000 part of the string, so that the final result looks like <registry name>.azurecr.cn/<your module name>.

7. Select Add to add your module to the project.

   

   Note

   If you have an existing IoT Edge project, you can change the repository URL by opening the module.json file. The repository URL is located in the repository property of the JSON file.

Now, you have an IoT Edge project and an IoT Edge module in your Visual Studio solution.

Project structure

Your solution has two project-level folders: a main project folder and a module folder. For example, you might have a main project folder named AzureIotEdgeApp1 and a module folder named IotEdgeModule1.

The main project folder contains your deployment manifest. The deployment manifest is a JSON document that describes the modules to be configured on the targeted IoT Edge device.

The module folder contains a file for your module code. It's named either Program.cs or main.c, depending on the language that you chose. This folder also contains a file named module.json that describes the metadata of your module. Various Docker files provide the necessary information to build your module as a Windows or Linux container.

Deployment manifest of your project

The deployment manifest that you edit is named deployment.debug.template.json. This file is a template of an IoT Edge deployment manifest that defines all the modules that run on a device. The file also defines how the modules communicate with each other. For more information about deployment manifests, see Learn how to deploy modules and establish routes.

The deployment template includes:

• The two runtime modules, edgeAgent and edgeHub.
• The custom module that you created in this Visual Studio project.
• A module named SimulatedTemperatureSensor. This default module generates simulated data that you can use to test your modules (or delete if it's not necessary). To see how the simulated temperature sensor works, view the SimulatedTemperatureSensor.csproj source code.

Set the IoT Edge runtime version

Currently, the latest stable runtime version is 1.4. Update the IoT Edge runtime version to the latest stable release or the version that you want to target for your devices:

Develop your module

When you add a new module, it comes with default code that's ready to be built and deployed to a device so that you can start testing without touching any code. The module code is located within the module folder in a file named Program.cs (for C#) or main.c (for C).

In the default solution, the simulated data from the SimulatedTemperatureSensor module is routed to your module. The module takes the input and then sends it to Azure IoT Hub.

When you're ready to customize the module template with your own code, use the Azure IoT Hub SDKs to build other modules that address the key needs for IoT solutions. These needs can include security, device management, and reliability.

Build and push a single module

Typically, you want to test and debug each module before running it within an entire solution with multiple modules. Because the solution will build or debug by using the Docker engine running inside the IoT Edge for Linux on Windows VM, the first step is building and publishing the module to enable remote debugging:

1. In Solution Explorer, select the module project folder (for example, myIotEdgeModule).

2. Set the custom module as the startup project. On the menu, select Project > Set as StartUp Project.

3. To debug the C# Linux module, you need to update the Dockerfile.amd64.debug file to enable the SSH service. Update the Dockerfile.amd64.debug file to use the following template: Dockerfile for Azure IoT Edge AMD64 C# Module with Remote Debug Support.

   Note

   When you select Debug, Visual Studio uses Dockerfile.(amd64|windows-amd64).debug to build Docker images. This file includes the .NET Core command-line debugger VSDBG in your container image while building it. For production-ready IoT Edge modules, we recommend that you use the Release configuration, which uses Dockerfile.(amd64|windows-amd64) without VSDBG.

   Make sure that in the last line of the template, ENTRYPOINT ["dotnet", "IotEdgeModule1.dll"], the name of the DLL matches the name of your IoT Edge module project.

4. To establish an SSH connection with the Linux module, you need to create an RSA key. Open an elevated PowerShell session and run the following commands to create a new RSA key. Save the RSA key under the same IoT Edge module folder, and make sure that the name of the key is id_rsa.

   ssh-keygen -t RSA -b 4096 -m PEM
   

   

5. If you're using a private registry like Azure Container Registry, use the following Docker command to sign in to it. You can get the username and password from the Access keys page of your registry in the Azure portal. If you're using a local registry, you can run a local registry.

   docker -H tcp://<EFLOW-VM-IP>:2375 login -u <Container Registry username> -p <Container Registry password> <Container Registry login server>
   

docker build --rm -f "<DockerFilePath>" -t <ImageNameAndTag> "<ContextPath>" 

# Build the image for the local registry

docker build --rm -f "./IotEdgeModule1/Dockerfile.amd64.debug" -t localhost:5000/iotedgemodule1:0.0.1-amd64 "./IotEdgeModule1"

# Or build the image for an Azure container registry

docker build --rm -f "./IotEdgeModule1/Dockerfile.amd64.debug" -t myacr.azurecr.cn/iotedgemodule1:0.0.1-amd64 "./IotEdgeModule1"

# Push the Docker image to the local registry

docker push localhost:5000/iotedgemodule1:0.0.1-amd64

# Or push the Docker image to an Azure container registry
az acr login --name myacr
docker push myacr.azurecr.cn/iotedgemodule1:0.0.1-amd64

Debug the solution

1. In an elevated PowerShell session, run the following commands:

   1. Get the moduleId value, based on the name for the Linux C# module. Replace the <iot-edge-module-name> placeholder with your module's name.

      $moduleId = Invoke-EflowVmCommand "sudo docker ps -aqf name=<iot-edge-module-name>"
      

   2. Check that $moduleId is correct. If the variable is empty, make sure you're using the correct module name.

   3. Start the SSH service inside the Linux container:

      Invoke-EflowVmCommand "sudo docker exec -it -d $moduleId service ssh start"
      

   4. Open the module's SSH port on the IoT Edge for Linux on Windows VM. (This tutorial uses port 10022.)

      Invoke-EflowVmCommand "sudo iptables -A INPUT -p tcp --dport 10022 -j ACCEPT"
      

   Warning

   For security reasons, every time the IoT Edge for Linux on Windows VM restarts, the IP table rule is deleted and goes back to the original settings. Also, you have to start the module's SSH service again manually.

2. After you successfully start the SSH service, select Debug > Attach to Process, set Connection Type to SSH, and set Connection target to the IP address of your IoT Edge for Linux on Windows VM. If you don't know the IP address of your IoT Edge for Linux on Windows VM, you can use the Get-EflowVmAddr PowerShell cmdlet.

   Type the IP and then select the Enter key. In the pop-up window, enter the following configurations:

   Field	Value
   Hostname	IP address for the IoT Edge for Linux on Windows VM
   Port	10022 (or the one that you used in your deployment configuration)
   Username	root
   Authentication type	Private Key
   Private Key File	Full path to the id_rsa value that you created in a previous step
   Passphrase	Passphrase used for the key that you created in a previous step

3. After you successfully connect to the module by using SSH, you can choose the process and select Attach. For the C# module, you need to choose process dotnet and Attach to Managed (CoreCLR). It might take 10 to 20 seconds the first time.

4. Set a breakpoint to inspect the module:

   • If you're developing in C#, set a breakpoint in the PipeMessage() function in ModuleBackgroundService.cs.
   • If you're using C, set a breakpoint in the InputQueue1Callback() function in main.c.

5. The output of SimulatedTemperatureSensor should be redirected to input1 of the custom Linux C# module. The breakpoint should be triggered. You can watch variables in the Visual Studio Locals window.

   

6. To stop debugging, select Ctrl+F5 or select the Stop button.

Clean up resources

If you plan to continue to the next recommended article, you can keep the resources and configurations that you created and reuse them. You can also keep using the same IoT Edge device as a test device.

Otherwise, delete the local configurations and the Azure resources that you used in this article to avoid charges.

Delete Azure resources

Deleting Azure resources and resource groups is irreversible. Make sure that you don't accidentally delete the wrong resource group or resources. If you created the IoT hub inside an existing resource group that has resources that you want to keep, delete only the IoT hub resource itself, not the resource group.

To delete the resources:

1. Sign in to the Azure portal, and then select Resource groups.

2. Select the name of the resource group that contains your IoT Edge test resources.

3. Review the list of resources that your resource group contains. If you want to delete all of them, you can select Delete resource group. If you want to delete only some of them, you can select each resource to delete them individually.

Next step

In this tutorial, you set up Visual Studio on your development machine, and you deployed and debugged your first IoT Edge module from it. Now that you know the basic concepts, try adding functionality to a module so that it can analyze the data passing through it: