[V2] modules cannot start after a hardware reboot

[Hammatt]

• V2 issue

• OS version and architecture used: Windows 10 Pro & Windows 10 IoT Core
  Intel x64

• Version used: agent: 1.0_preview 22, hub: 1.0_preview 21 (see [V2] edgeHub just stopped working? #558 for mismatched outdated hub version reason)

• V2 Docker version and containter type Docker version 18.03.1-ce, build 9ee9f40 Windows

Description of the issue:

After a hardware restart, something to do with the Docker Nat ID changes causing the containers to not start and to require a running the setup and login commands again before you can start them.

I've been this issue on Windows 10 IoT Core for a while now but recently it has started happening on Windows 10 Pro as well.

I understand that this is a known issue internally, but I haven't heard any updates and couldn't see anything tracking it here so I thought I would post for visibility & tracking purposes.

Console log of the issue:

PS C:\WINDOWS\system32> iotedgectl start
INFO: Using default IoT Edge configuration dir: C:\ProgramData\azure-iot-edge\config
INFO: Executing 'start'
Using configuration:

Schema Version:         1
Connection String:      HostName=<redacted>
Config Directory:       C:\ProgramData\azure-iot-edge\config
Home Directory:         C:\ProgramData\azure-iot-edge\data
Hostname:               desktop-5pdi0nj
Log Level:              info
Security Option:        selfSigned
Force No Passwords:     True
Certificate Subject:
                        countryCode: US, state: Washington, locality: Redmond
                        organization: Default Edge Organization, organizationUnit: Edge Unit, commonName: Edge Device CA
Deployment Type:        docker
Docker Engine URI:      npipe://./pipe/docker_engine
Edge Agent Image:       microsoft/azureiotedge-agent:1.0-preview
Registries:
                        <redacted>
Logging Driver:         json-file
                        max-size: 10m

INFO: Starting container: edgeAgent
ERROR: Could not start container: edgeAgent
404 Client Error: Not Found ("network 9d03b337f52a4878c7a82ddf6a1a62afb01c79d33249f434b3e7052e9bfaf986 not found")
ERROR: Exiting with errors. Return code: 1
ERROR: Exiting with errors. Return code: 1
PS C:\WINDOWS\system32> iotedgectl setup --connection-string <redacted> --auto-cert-gen-force-no-passwords
INFO: Using default IoT Edge configuration dir: C:\ProgramData\azure-iot-edge\config
INFO: Generating self signed certificates at: C:\ProgramData\azure-iot-edge\data\certs
INFO: The runtime configuration file C:\ProgramData\azure-iot-edge\config\config.json was updated with the setup options.
INFO: Executing 'setup'
INFO: Uninstalling all modules.
INFO: Stopping container: edgeAgent
INFO: Removing container: edgeAgent
INFO: Stopping containers by label: net.azure-devices.edge.owner=Microsoft.Azure.Devices.Edge.Agent
INFO: Removing containers by label: net.azure-devices.edge.owner=Microsoft.Azure.Devices.Edge.Agent
INFO: Removing volume: edgehub
INFO: Removing volume: edgemodule
Runtime setup successfully.


Using configuration:

Schema Version:         1
Connection String:      <redacted>
Config Directory:       C:\ProgramData\azure-iot-edge\config
Home Directory:         C:\ProgramData\azure-iot-edge\data
Hostname:               desktop-5pdi0nj
Log Level:              info
Security Option:        selfSigned
Force No Passwords:     True
Certificate Subject:
                        countryCode: US, state: Washington, locality: Redmond
                        organization: Default Edge Organization, organizationUnit: Edge Unit, commonName: Edge Device CA
Deployment Type:        docker
Docker Engine URI:      npipe://./pipe/docker_engine
Edge Agent Image:       microsoft/azureiotedge-agent:1.0-preview
Registries:
Logging Driver:         json-file
                        max-size: 10m

Use 'iotedgectl start' to start the runtime.
PS C:\WINDOWS\system32> iotedgectl login <redacted>
INFO: Using default IoT Edge configuration dir: C:\ProgramData\azure-iot-edge\config
INFO: The runtime configuration file C:\ProgramData\azure-iot-edge\config\config.json was updated with the credentials for registry: <redacted>.azurecr.io
INFO: Executing 'login'
INFO: Please use the start command to see changes take effect.
PS C:\WINDOWS\system32> iotedgectl start
INFO: Using default IoT Edge configuration dir: C:\ProgramData\azure-iot-edge\config
INFO: Executing 'start'
Using configuration:

Schema Version:         1
Connection String:      <redacted>
Config Directory:       C:\ProgramData\azure-iot-edge\config
Home Directory:         C:\ProgramData\azure-iot-edge\data
Hostname:               desktop-5pdi0nj
Log Level:              info
Security Option:        selfSigned
Force No Passwords:     True
Certificate Subject:
                        countryCode: US, state: Washington, locality: Redmond
                        organization: Default Edge Organization, organizationUnit: Edge Unit, commonName: Edge Device CA
Deployment Type:        docker
Docker Engine URI:      npipe://./pipe/docker_engine
Edge Agent Image:       microsoft/azureiotedge-agent:1.0-preview
Registries:
                        Address: <redacted>.azurecr.io, Username: <redacted>, Password:<redacted>
Logging Driver:         json-file
                        max-size: 10m

INFO: Starting container: edgeAgent
Runtime started.

[veyalla]

@Hammatt - this is a known issue we are working to address.

As a workaround, do docker rm -f $(docker ps -aq) from an admin powershell window before attempting to start the runtime after system reboot.

[Hammatt]

Still seeing this issue on Windows 10 IoT Enterprise too

[Hammatt]

@veyalla Is there a known workaround to get the modules running as a service? i.e. to have them start up without logging in on Windows 10 IoT Enterprise?

This was working at some point but stopped working for an unknown reason. The modules used to start themselves up on Windows 10 IoT Enterprise but now do not, and when I log into a session and try to start them manually I see the error saying that it cannot find the docker network.