How can I troubleshoot an EC2 Windows instance that is unreachable or can't reach the DNS after upgrading the drivers?

Last updated: 2022-10-04

My instance is unreachable after I updated the paravirtual (PV), non-volatile memory express (NVMe), or Elastic Network Adapter (ENA) drivers on my Amazon Elastic Compute Cloud (Amazon EC2) Windows instance. Or, I upgraded the drivers and changed my instance type to a Nitro-based instance and the instance can't reach the Domain Name System (DNS). How do I troubleshoot this?

Short description

There are three types of AWS drivers used on EC2 Windows instances:

The type of AWS driver used by the instance depends on the instance type. Most instances in the Nitro-based instance family use the ENA driver for networking and the AWS NVMe driver for storage. Most instances in the Xen-based instance family (non-Nitro) use the AWS PV driver for both network and storage. Some instances use a combination of drivers for storage and networking. For more information on instances and which drivers they use for networking and storage, see Summary of networking and storage features. In the Summary of networking and storage features table, elastic block storage (EBS) Only indicates that the volume uses AWS PV drivers. NVME EBS indicates that the instance uses NVMe drivers.

Note: AWS PV drivers are supported on Windows Server 2008 R2 and later. Windows Server 2003 uses Citrix PV drivers. Windows Server 2008 R2 supports AWS PV version 8.3.4 and earlier and AWS ENA 2.2.3 and earlier. For more information, see Paravirtual drivers for Windows instances.

When you're upgrading drivers on Windows instances or changing the instance type, see Migrate to latest generation instance types (Nitro/Xen).

Note: You can run the following PowerShell command to list the AWS drivers and driver versions running on the Windows instance:

Get-WmiObject Win32_PnpSignedDriver | Select-Object DeviceName, DriverVersion, InfName | Where-Object {$_.DeviceName -like "*AWS*" -OR $_.DeviceName -like "*Amazon*"}

Resolution

Note: Changes to an instance might cause an instance to not boot. It's a best practice to create a snapshot of the root EBS volume before you modify the instance. For more information, see Create a snapshot.

To troubleshoot an unreachable instance after upgrading Windows drivers, do the following:

Review the instance screenshot

Review the instance screenshot or use EC2Rescue for Windows to review system logs.

  • Check the instance screenshot to determine the state of the instance. Then, review the steps in Troubleshoot an unreachable instance.
  • If the screenshot shows the Log on screen, but the instance status checks are failing, then the issue might be a corrupted or undetectable driver.

Review system logs using EC2Rescue for Windows

If the instance type was changed, first attempt to revert the instance type or revert to a T2 instance type, and then review the logs listed below. If this doesn't work, then use the EC2Rescue tool to retrieve the unreachable instance's operating system (OS) logs. These logs can be Event Viewer logs, EC2Config logs, launch logs, and so on.

  • Check the setupapi.dev log located at %SystemRoot%\inf for information about device installation in plaintext. You can use this information to verify the installation of a device with the timestamp and to troubleshoot device installation problems.
  • Check the AWSPVDriverMSI log file located at C:\Program Files\Amazon\XenTools. This log is specific to AWS PV driver installation and reports any errors.

If the logs show that the drivers are corrupted or aren't installed properly, then inject the drivers offline to the instance. You can use the AWSSupport-UpgradeWindowsAWSDrivers automation document to upgrade or repair storage and network AWS drivers on the instance. For more information, see Upgrade the AWS PV, ENA, and NVMe drivers using AWS Systems Manager.

Note: It's a best practice to back up your instance by taking snapshots before upgrading the drivers or changing the instance type.

Restore the instance using EC2Rescue for Windows

Use the EC2Rescue tool to restore the instance to the last known correct configuration state. For more information, see the Video walkthrough.

Replace the root volume using the latest available snapshot

For more information, see Replace a volume using a previous snapshot.

Troubleshoot using the EC2 Serial Console for Windows

If you’ve turned on EC2 Serial Console for Windows, then you can use it to troubleshoot supported Nitro-based instance types. The serial console helps you troubleshoot boot issues, network configuration, and secure shell protocol (SSH) configuration issues. The serial console connects to your instance without requiring a working network connection. You can access the serial console using the Amazon EC2 console or the AWS Command Line Interface (AWS CLI).

Before using the serial console, grant access to the console at the account level. Then create AWS Identity and Access Management (IAM) policies granting access to your IAM users. Also, every instance using the serial console must include at least one password-based user. If your instance is unreachable and you haven’t configured access to the serial console, use one of the preceding methods to troubleshoot the instance. For more information on configuring the EC2 Serial Console for Windows, see Configure access to the EC2 Serial Console.

Note: If you receive errors when running AWS CLI commands, Confirm that you’re running a recent version of the AWS CLI.

To troubleshoot instances that are reachable but can't access the DNS after upgrading Windows drivers, do the following:

Reconfigure the Elastic Network Interface settings

When migrating the instance to the latest generation instance type, the static internet protocol (IP) on the existing elastic network interface might be lost. Custom DNS network settings might be lost during this migration too. This is because the instance defaults to a new ENA device. To resolve this, reconfigure the elastic network interface settings. You will need local credentials to reconfigure these settings. For more information, see How can I assign a static private IP address to my Amazon EC2 Windows instance?