How can I troubleshoot an EC2 Windows instance that is unreachable or can't reach the DNS after upgrading the drivers?

Last updated: 2021-07-23

I updated the PV, NVMe, or ENA drivers on my Amazon Elastic Compute Cloud (Amazon EC2) Windows instance. Now the instance is unreachable. Or, I upgraded the drivers and changed my instance type to a Nitro-based instance and now the instance can't reach the DNS. How do I troubleshoot this?

Short description

There are three types of AWS drivers used on EC2 Windows instances:

The type of AWS driver used by the instance depends on the instance type. Most instances in the Nitro-based instance family use the ENA driver for networking and the AWS NVMe driver for storage. Most instances in the Xen-based instance family (non-Nitro) use the AWS PV driver for both network and storage. Some instances use a combination of drivers for storage and networking. For information on which drivers an instance uses for networking and storage, see Summary of networking and storage features. In the Summary of networking and storage features table, EBS Only indicates that the volume uses AWS PV drivers. NVME EBS indicates that the instance uses NVMe drivers.

Note: AWS PV drivers are supported on Windows Server 2008 R2 and later. Windows Server 2008 and Windows Server 2003 use Citrix PV drivers. Note that Windows Server 2008 R2 supports AWS PV version 8.3.4 and earlier. For more information, see Paravirtual drivers for Windows instances.

When you're upgrading drivers on Windows instances or changing the instance type, make sure to install the latest version of the driver based on the instance type (Nitro/Xen).

Note: You can run the following PowerShell command to list the AWS drivers and driver versions running on the Windows instance:

Get-WmiObject Win32_PnpSignedDriver | Select-Object DeviceName, DriverVersion, InfName | Where-Object {$_.DeviceName -like "*AWS*" -OR $_.DeviceName -like "*Amazon*"}

Resolution

Troubleshoot an unreachable instance after upgrading Windows drivers

Review the instance screenshot or use EC2 Rescue for Windows to review system logs

Check the instance screenshot to determine the state of the instance. Then, review the troubleshooting steps in Troubleshoot an unreachable instance.

If the screenshot shows the Log on screen, but the instance status checks are failing, then the issue might be a corrupted network driver or undetectable driver.

Use the EC2Rescue tool to retrieve the unreachable instance's OS logs. Review the logs to get more information to troubleshoot the issue further. These logs can be Event Viewer logs, EC2Config logs, launch logs, and so on.

  • Check the setupapi.dev log located at %SystemRoot%\inf for information about device installation in plain text. You can use this information to verify the installation of a device with the timestamp and to troubleshoot device installation problems.
  • Check the AWSPVDriverMSI log file located at C:\Program Files\Amazon\XenTools. This log is specific lo AWS PV driver installation and reports any errors.

If the logs show that the drivers are corrupted or not installed properly, inject the drivers offline to the instance. You can use the AWSSupport-UpgradeWindowsAWSDrivers automation document to upgrade or repair storage and network AWS drivers on the instance. For more information, see Upgrade the AWS PV, ENA, and NVMe Drivers Using AWS Systems Manager.

Note: It's a best practice to backup your instance before upgrading the drivers or changing the instance type.

Use EC2Rescue to restore the instance

Use the EC2Rescue tool to restore the instance to last known correct configuration state. For more information, follow the Video walkthrough.

Use the latest available snapshot to replace the root volume of the instance

For more information, see Replace a root volume.

Use the EC2 Serial Console for Windows

If you’ve enabled EC2 Serial Console for Windows, then you can use it to troubleshoot supported Nitro-based instance types. The serial console helps you troubleshoot boot issues, network configuration, and SSH configuration issues. The serial console connects to your instance without the need for a working network connection. You can access the serial console using the Amazon EC2 console or the AWS Command Line Interface (AWS CLI).

Before using the serial console, grant access to the console at the account level. Then create AWS Identity and Access Management (IAM) policies granting access to your IAM users. Also, every instance using the serial console must include at least one password-based user. If your instance is unreachable and you haven’t configured access to the serial console, use one of the preceding methods to troubleshoot the instance. For information on configuring the EC2 Serial Console for Windows, see Configure access to the EC2 Serial Console.

Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.

The instance is reachable but can't access the DNS after upgrading the drivers and changing the instance to a Nitro-based instance type

When migrating the instance to the latest generation instance type, the static IP or custom DNS network settings on the existing elastic network interface might be lost. This is because the instance defaults to a new ENA device. To resolve this, reconfigure the elastic network interface settings. For more information, see Do I need to set a static private IP address for an Amazon EC2 Windows instance?