Wireless LAN (802.11) Testing Prerequisites
This topic describes the process used to test a Wireless LAN (WLAN) device to make sure that it functions correctly with Windows. These procedures use the Microsoft Windows Driver Kit (WDK) and Windows Hardware Lab Kit (Windows HLK). To comply with the WLAN Windows Hardware Certification Program, you must run all of the tests that Windows HLK identifies as being required for the device. We also recommend that you work through the tests in order by the following levels: "Basic", "Functional", "Reliability", and "Certification."
Note
For a driver to pass testing and obtain Windows Hardware Certification, you must use the latest version of the WDK to compile the driver.
Note
WLAN Windows HLK tests that use software-based access points are exclusively supported by using specific Atheros WLAN NICs. Windows HLK customers who test WLAN drivers must use a WLAN adapter installed in the AP Controller Machine to complete their submissions. Only two devices have been specifically tested at the time of Windows 8.1 release: Dlink DWA-552 and Dlink DWA-556. Additional models might have worked in the past and might continue to work, but cannot be guaranteed. If you have questions about this, please contact us at wlanndt@microsoft.com.
The tests to run depend on the capabilities of the device or system that you are testing. The tests take approximately 36-40 hours to run. These times represent fully passing results. Failures in any test or reboots due to crashes add additional time to the tests. These times can vary by platform and performance of individual machines that are used in the tests.
Windows HLK Setup for WLAN Device Testing
A Windows HLK setup for WLAN device testing consists of the following components:
A computer that is running Windows HLK Studio. Through this application, you can control and configure the Windows HLK Controller to send jobs to the Windows HLK clients in the Windows HLK system pool. All requirements for a WLAN Windows HLK submission are enforced when the Windows HLK test execution job is configured.
A Windows HLK Controller computer. This computer must run Windows Server. For more information see Windows HLK Prerequisites. (In many cases, this computer can be the same computer that runs Windows HLK Studio.) This component reserves Windows HLK clients in the system pool for a test job. In Figure 1 WLAN Setup, the Windows HLK Controller and Windows HLK Studio machine are represented as one machine.
A Device Under Test (DUT), which is a computer that is running the desired operating system and architecture for which you are seeking certification, and on which the device for certification is installed. You must install the Windows HLK Client on this machine.
A Support Device Under Test (SUT), which is a computer that is running the same version and processor architecture of Windows as the DUT. The SUT must also have the same WLAN device installed and be running the same driver version as the DUT.
An AP Controller, which is a machine that is running the same processor architecture of Windows as the DUT and SUT. This machine must have a compatible WLAN adapters installed (Dlink DWA-552 is recommended) and two Ethernet adapters installed, one of which must be capable of reaching 1Gbps. You must install the Windows HLK Client on this machine.
Two ASUS RT-N66U and/or ASUS RT-AC66U routers or two Aruba 515 or 535 APs. See Figure 1 WLAN Setup for configuration details and physical arrangement of the routers.
An 802.11AC capable router/access point. See Figure 1 WLAN Setup for configuration details and physical arrangements. The 802.11AC router must support at least the same number of antennas, spatial streams, and max throughput as the DUT. Using an 802.11AC router that does not support enough throughput will fail the 802.11 performance tests. You can use the 2.4Ghz side of the 802.11AC router as the AP for Wlan L1 and Device Fundamental tests. You should configure this with WPA2 Personal and an AES cipher. The defaults for these tests use an ssid of kitstestssid and a password of password.
An 802.11w-capable access point. This item is unchanged from previous WLAN Windows HLK releases and the previously used access point can be used.
Note
Windows Vista and earlier versions of Windows are deprecated and are not supported for any role in the test configuration.
Figure 1 WLAN Setup shows the WLAN setup:
Prepare for Wireless LAN (802.11) Testing
RT-N66U or RT-AC66U (AP1 & AP2) Preparation:
These devices do not currently support 802.11ax, WPA3 or Enhanced Open. Use the Aruba devices described below for testing these features.
Make sure that you are using 3.0.0.4.376 or earlier version firmware. New units might have a compatible firmware version directly out of the box. (We recommend that you keep a copy of working firmware available.)
You must pre-configure AP1 and AP2 before you connect them. You can use the AP Controller or a separate machine to perform the configuration. During this process, make sure that the computer is connected to the router's LAN ports by using the Ethernet network only.
When you set up a new AP for the first time, we recommend you skip the Configuration wizard on the APs by navigating directly to: http://192.168.1.1/index.asp .
The WAN/Internet ports on both AP1 and AP2 must be configured in the WAN section of Advanced Settings using a connection type of Static IP. The static IP used must be in the subnet of gigabit switch 2. (For example: 192.168.100.0/255.255.255.0) Switch 1 and switch 2 must use separate and non-overlapping IP subnets.
The WAN settings page requires the Default Gateway, DNS Server1, and DNS Server2 fields are filled out. For ease of configuration, we recommend you enter the static IP address of Nic 2 on the AP Controller (which must be in the same subnet range as the WAN IP Setting).
You must disable the firewall in the Firewall section of Advanced Settings.
You must enable Telnet on the System tab in the Administration section of Advanced Settings on both AP1 and AP2.
For debugging purposes from the AP Controller, we highly recommend that you enable web access from WAN on the System tab in the Administration section of Advanced Settings on both AP1 and AP2.
Important
When the tests run for the first time, they disable all LAN ports on both ASUS routers. This is expected. If you want to connect to the APs to verify settings during the testing, use the AP controller and connect over the WAN port by using the provisioned static IP. The tests enable the LAN ports of each AP as needed during testing. If you need to connect to a LAN port for testing, stop the tests from running and manually power off and on the AP. This re-enables the LAN ports until the next test starts.
You can troubleshoot the tests' ability to connect to the APs using Telnet.exe to connect to both routers on the WAN IP address from the AP-Controller. If you cannot connect to either AP from the AP-Controller using the WAN IP address, the username of admin, and password that you configured (default password is admin), the tests cannot connect and will fail.
These preparation steps do not apply to the 802.11 AC router in the setup. For the 802.11AC router, configure it to match the maximum bandwidth and spatial streams that your device supports.
Aruba (AP1 & AP2) Preparation:
The currently supported Aruba APs are 515 and 535 running ArubaOS Instant 8.5.0.0.
In a normal installation, Aruba APs are set up in a "swarm" with one AP designated the Controller and the rest clients. In this case, AP configuration changes are applied on the Controller and (eventually) exported to the appropriate clients. As the AP Controller system configures each AP independently, this Controller/client setup will not work for WLAN HLK testing. Instead, the APs must be converted from Managed to Standalone mode.
Assign the AP a static IP
To set up a new Standalone Aruba AP, unplug it from the network, plug it's USB cable into a PC and...
Use Device Manager to determine which COM port the AP is plugged into
Start PuTTY, select Serial and enter the COM port
Power-cycle the AP
Before the boot starts there is a 2-second pause to press Return and stop the boot
Once the boot has stopped you are in a special command-mode terminal session. Use it to assign a static IP address:
apboot> factory_reset (optional)
apboot> setenv ipaddr 192.168.s.n (use Gigbit Switch 2's subnet)
apboot> setenv netmask 255.255.255.0
apboot> setenv gatewayip 192.168.s.cc (the IP of the AP Controller)
apboot> saveenv
apboot> printenv
apboot> boot (takes about 3 minutes)
Convert to Aruba Instant OS
Standalone mode is only supported by the Aruba Instant OS. When you log in the web on a non-Instant OS you will see the Access Point Provisioning page to force the upgrade:
Log into the Web interface. The default login is admin/admin or admin/serial#.
Browse to the new OS image. (Must be on the local machine.)
Select Save and Upgrade Now.
Immediately after the Upgrade Successful the AP will reboot.
Turn on Telnet and Convert to Standalone mode
Once the AP is running Instant do the final setup steps:
Log in on the Web interface. The default login is either admin/admin or, if the AP was just upgraded, admin/serial#.
If the AP was just upgraded you will see a Reset default password page. Enter the new password, click OK then log in again with your new password.
Enable Terminal access (telnet):
Configuration > System > Show Advanced Options > General > Terminal Server
Set static IP (only necessary if IP is changing)
Configuration > Access Points > Select AP's MAC > Edit (pencil) > Specify Statically
Convert to Standalone:
Maintenance > Convert
Convert one or more Access Points to > Standalone AP
Access Point to convert > Select AP's MAC from drop-down list (it's usually the only one)
Convert Do you want to Continue? > OK
Topology Configuration and Preparation:
Each line in Figure 1. WLAN Setup represents a connected Ethernet cable.
(ASUS only) AP1 and AP2 must be connected to Gigabit Switch 2 using the WAN/Internet port.
(ASUS only) The Ethernet cable between the LAN ports on AP1 and AP2 is required. The connection can be between any LAN port on AP1 to any LAN port on AP2.
The AP Controller requires two LAN adapters. One LAN adapter (Nic 1) connects to Gigabit Switch 1, and the second lan adapter (Nic 2) connects to Gigabit Switch 2. Nic 2 MUST be a gigabit adapter and cannot be a USB Ethernet adapter. This adapter is used to measure 802.11AC performance. The IP address on Nic 1 must be assigned using DHCP. The IP address on Nic 2 must be statically assigned. This assignment method enables the tests to identify the adapters.
Do not use the same IP address space for Gigabit switch 1 and Gigabit Switch 2. They must contain IP/subnet masks that do not overlap. Further, you must use DHCP for address assignment on Gigabit Switch 1 (this can be the same network you typically use for other HLK testing) and the IP range for Gigabit Switch 1 must NOT be in the range 192.168.x.x. We suggest that you use 192.168.100.* for the addressing on gigabit switch 2.
The DUT, SUT, AP Controller, and HLK Controller need hard-wired LAN connections to gigabit switch 1. This cannot be the same switch as gigabit switch 2. A DHCP service must provide IP addresses on gigabit switch 1. If Nic 1 on the AP controller is not assigned an IP address using DHCP, tests will fail.
Each static IP connected to gigabit switch 2 must be unique.
Figure 1. WLAN Setup includes some notes to keep regarding IP addresses, SSIDs, passwords, etc. You are prompted for these values during test runs.
Both wireless adapters that are installed in the DUT and SUT must fully support Wifi-Direct. We recommend that you use the same adapter and driver in both DUT and SUT.
For devices that support 802.11AC, both the DUT and SUT must contain the same wireless adapter and driver.
AP Controller Configuration:
The AP Controller is required to have one WLAN adapter in order to pass all WLAN HLK tests.
The AP Controller requires 2 Ethernet adapters. One of these must support gigabit speeds.
The Ethernet adapter that is connected to switch 2 must be a gigabit Ethernet adapter. This requirement ensures unrestricted performance testing.
The architecture of the AP controller operating system must match the architecture of the DUT operating system (32bit for x86, 64bit for AMD64).
The AP Controller does not require a single-processor architecture. This detail is adjusted by the test automation.
HLK Pool Configuration:
After you configure the test topology, prepare the test systems for WLAN device testing using the following steps:
Set up the following machine configurations by using the previous configuration instructions:
One test device on the DUT
One test device on the SUT
One WLAN adapter on the AP Controller
Two Ethernet adapters on the AP Controller
Windows HLK Studio and Windows HLK Controller
Based on the operating system for which you are testing the device, install operating system builds on the Windows HLK clients (three test computers namely DUT, SUT and AP Controller machine) as described in the previous section.
Install HLK controller and HLK studio machine on Windows Server machine. For more information see Step 2: Install Client on the test system(s).
Install HLK clients on three test computers DUT, SUT and AP Controller. For more information see the Windows Hardware Certification Step-by-Step Guide.
On the Windows HLK Controller click Start, click All Programs, click Windows Kits, click Hardware Certification Kit, and click Windows HLK Studio.
Click Configuration and then click Machine Management.
Click Create Machine Pool.
Type the machine pool name in the New Pool field and then press Enter. The new machine pool should appear as a node under $ (Root).
Select the Default pool. The Windows HLK Client machines should be listed in the Machines list.
Press and hold the CTRL key and click each of the three defined machines to select all three machines.
Drag the three selected machines to the newly created machine pool.
In the new machine pool, right-click the three machines. Click Change Machine Status and then click Reset. The machines will change from a Not Ready state to an Initializing state and then to a Ready state. During the initialization process, devices on the machine are enumerated and cataloged. It is important for all devices to have been installed by this point.
You can configure additional pools on one Windows HLK Controller by replicating the entire topology and restarting with step 4
If you are using a debugger machine with each topology (in the case of multiple topologies), you can install separate copies of the Windows HLK Studio on additional machines so that more than one tester can share the same controller. Use separate projects for each topology/pool and only use one topology per pool.
Run Wireless LAN (802.11) tests for Certification
The following procedure demonstrates how to run the WLAN device tests:
Open the Windows HLK Studio.
On the Project tab, click Create Project and name the project.
Click the Selection tab.
Select your new machine pool in the dropdown in the upper left and click Device Manager. The list populates with the names of all the devices installed on the machines in that pool.
In the list, locate the WLAN driver under test and check the box next to it.
Note
There might be more than one WLAN driver listed. Make sure you check the one that is on the DUT. The machine name is listed in the right column.
Click the Tests tab - by default all the available certification tests applicable to the selected device will display.
You can filter the displayed list by clicking View by and then selecting additional options, such as Basic, Functional, or Reliability.
Only Certification tests are required for the certification process; however, running Basic and Functional tests provides a specific set of functionally important tests.
Select the test to run by checking the box on the left next to it. Click Run Selected. You are prompted to add any additional parameters. (You can also check the box next to multiple tests.)
Not all parameters are used for all tests. Only the parameters that are necessary for the selected test(s) are shown. You can view detailed parameter descriptions by hovering the mouse pointer over the parameter name. Many of the parameters will be derived from the topology notes mentioned above. If you press the F1 key while a test is selected, detailed information on that test and its parameters is shown.
Machine Set section is always present for Functional and Reliability tests. This section represents the machine resources that are necessary to run the tests. You must address all "!" symbols before you click OK.
Select the first test name from the list (if there is more than one).
From the Role menu, select Client - the DUT machine should already be selected and grayed out.
From the Role menu, select Support and check the box next to the SUT Machine.
From the Role menu, select AP and check the box next to the AP Machine.
Click OK to schedule. You can monitor the detailed progress of the tests in Windows HLK Manager, or simply watch as each test result displays in the Status column.
View Results and Log Files
Results and logs can be viewed through Windows HLK Manager or Windows HLK Studio.
In Windows HLK Manager, click Explorers and then click Job Monitor.
When the client machines complete test jobs, you can gather information from the Job Execution Status frame by right-clicking a job and viewing errors, job reports, or result reports. You can also click Browse Job Logs to access test results.
In HLK Studio, click the Results tab. View the Status column to monitor the results of each test. You can choose each column to sort the results. If a test passes, you will see a green checkmark; if it fails, you will see a red X. For more detailed information on viewing logs, see Step 7: View test results and log files.
Create a Package
After passing all of the necessary tests, you are ready to create an .hlkx submission package for certification.
Windows HLK Studio supports package creation, so you don't have to use a separate submission tool. The package creation feature lets you add necessary resource files to complete the certification. For detailed instructions on how to create a package, see Step 8: Create a submission package.
Notes and Troubleshooting for RT-N66U firmware
The tests are designed to work against RT-N66U vB1 APs that have at least firmware version 3.0.0.4.376. Purchases of new RT-N66U devices might meet this requirement with no firmware upgrades required. If your purchased devices meet this requirement, you can start testing as soon as you complete the configuration instructions described in this topic. Devices that come with an later version of firmware might be compatible. Downgrade to the version 3.0.0.4.376 only in the case of confirmed problems or communication from Microsoft.
We highly recommend that you save a backup of the required software to recover the APs, as well as a few versions of firmware. Make sure you have backup copies of the following:
ASUS Firmware Restoration Utility
ASUS RT-N66U B1 Firmware Version 3.0.0.4.376 (if available)
Current and/or latest version of ASUS RT-N66U B1 Firmware
You can download these items as follows:
Go to Asus Global Official Support and click Enter Download Center.
Type RT-N66U and then click Search.
Click RT-N66U (Ver.B1) (or applicable version that matches your version).
Next to OS:, select Windows 10 from the OS dropdown menu. Match the architecture to that of the computer you are using for any updates to the APs.
Click + next to Utilities, and download the latest available Firmware Restoration utility.
Click + next to Firmware, and download ASUS RT-N66U B1 Firmware Version 3.0.0.4.376.
You can also download the latest available firmware for future use.
AP Firmware Troubleshooting
Warning
Applying new firmware to consumer devices can sometimes, although rarely, cause permanent damage. You must read the device manual that came with your router and be careful when you flash new firmware on a device. Detailed steps, especially around the 30-30-30 reset, are required before flashing.
The most reliable way to install firmware on an RT-N66U device is by using the Firmware Restoration Utility. You might also be able to upgrade firmware by using the web interface. Consult the device manual for instructions. To upgrade firmware by using the restoration tool, the following steps are required. (These steps can also be found in your manual under Firmware Restoration.
Remove power from the device.
Press and hold the reset button on the device.
Turn on/plug in the device keeping the reset button held for 8 seconds.
Release the reset button when the power LED has started to slowly flash on and off.
Connect one computer to one of the LAN ports on the device (do not connect any other ports during this process).
Change the IP address on the computer to 192.168.1.4 with a subnet mask of 255.255.255.0 and no gateway address.
Start the Asus firmware restoration utility.
In the firmware restoration utility, browse for the firmware image you downloaded (version 3.0.0.4.376 or earlier).
Click the Upload button.
If the upload is successful, multiple progress bars appear. The first progress bar shows the upload of the image to the device; additional progress bars show the overall upgrade process.
Do not use the device, press any buttons, or remove power while the device is being upgraded. We recommend that you leave it alone for 15 minutes.
After the upgrade is successful, you can remove the static IP address from the computer and access the devices web-server at http://192.168.1.1/index.asp to confirm a successful upgrade.
Important
Every time that you upgrade firmware, after the device has finished the upgrade and has restarted, you should reset the NVRAM settings by performing a 30-30-30 reset. The steps for this are as follows:
When the device is ON and functioning, hold down the WPS button for 30 seconds.
While continuing to hold the WPS button, unplug the device and leave it unplugged for 30 seconds.
While continuing to hold the WPS button, plug the device back in and wait for an additional 30 seconds (90 seconds total).
Release the WPS button. This will cause all LEDs to flash on and off again. The device reboots and goes through a prolonged internal setup process again it becomes functional again.
After the router has restarted to respond to web requests, you can continue the preparation steps documented above starting here: http://192.168.1.1/index.asp .
If a device is no longer functional for any reason, or specifically because of a previous attempt to install firmware that was not compatible, try the following instructions first before you replace the device:
With the device ON, perform the 30-30-30 reset instructions listed above with one modification:
After the 90 seconds of holding the WPS button and removing/reapplying power, as soon as all leds on the device turn on and then off, immediately remove power as quickly as possible.
Before plugging the device back in, hold in the reset button (not wps), then continue to plug in the device. Keep the reset button held for at least 8 seconds, but no more than 10.
The power LED should be slowly flashing. If it is not, repeat the instructions above.
Perform the Firmware Restoration instructions that are listed above to restore either the default firmware or newly downloaded firmware.