If you’re anything like me, it seems like the steps for setting up automated testing for a Windows Store Application are convoluted and esoteric. While we wait on Microsoft to release CodedUI for Windows Store applications, many of us still need start testing. While there is plenty of good advice on manual testing of applications and automating Desktop applications, I’m going to focus on automated testing of Windows Store applications. It took some time to understand how to get my automation application to work with a Windows 8 Store application. In an effort to spare you my headaches, I want to share my knowledge of how to approach testing of these apps.
In these posts, I will build up a sample client Automation Client which targets a Windows 8 Store app. This first post will focus on the steps to install your Windows Store application so that it is accessible to your Automation Client. The second post will deal with setting up your Automation Client with the correct accessibility options and digitally signing your client. The third post will go through steps to debug your elevated code, the code you need in your client for it to always run on top, and how to get all the controls in a window.
For the actual coding of my Automation Client, I’m using one of the accessibility libraries provided with Windows 8 called UIAutomationCore.dll. This is a COM library which is provided for developers to create applications for disability programs. These applications include screen readers, magnifiers, and other assistive technologies. Thankfully for those who need to set up automated testing, we can use these same libraries to drive our testing. You can read up on these libraries on Microsoft’s MSDN at UI Automation. I suggest you spend lots of time getting to know these. After we get through what you need to do to get access to your application the main part of getting your automation testing running is going to come from these libraries.
The second piece of background information for you is this article: Automating the testing of Windows 8 apps. This article explains how to install your application, activate it, manage the life cycle, and uninstall your application. It is a good article, and the basis for much of this first post. I found some holes that took me some time to figure out how to do it correctly. I’ll try to fill those in as much as possible.
The third and final piece of background information is about your test environment. Everything in this post assumes you are using Visual Studio 2012 for development of your Automation Client. I’m also going to assume that you want to test your own or a trusted colleague’s Windows Store application. IF you are testing something you aren’t familiar with for some unknown reason, then you need to take the precautions you usually would when dealing with unknown software from the web. Make sure you have administrative access to the test machine. There will be several points along the way to (and through) automated testing where you will need it. As we come to those points, I’ll do my best to explain what’s going on and as much as I understand of why you need administrative access to do it.
There are a few steps you need to take to get your application ready for automated testing. They need to be performed at least once per application, and a few will need to happen per testing version of the Windows Store application. I will go over the first set of those steps in this post. Happy Automating!
Install Windows Store Application On Your Test System
First, get a development license.
If you’ve been doing any Windows Store Application development, you already know that you have to get a developer license. If you are new to Windows Store Applications and mainly focused on the testing aspects, you will still need the developer license. Microsoft makes it easy to get one of these and you don’t have to pay for it. The license allows you to install your application on your machine without going through all the steps required for publication to the Windows store. It should be obvious that it is intended for development and testing purposes. You will need one developer license per user and you’ll need a certificate per application. You don’t have to worry about the certificate for testing purposes however. It will be generated for you. The developer license wizard will also start off automatically. There are several possible points where the license is generated. If you already have a valid one, you don’t need to do it again until it expires.
The first way to get a developer license is to start Visual Studio 2012 for the first time. See this MSDN entry. This same entry explains how to get the developer license from the command line in PowerShell.
If you still don’t have a developer license when you install the Windows 8 Store application to your system, you will be prompted to get one. See the blog post about automating Windows 8 applications.
In order to run your tests on your Windows Store application, you have to actually install it to your system. The following steps tell you how. Don’t worry; you’ll get really used to doing these. You’ll have to follow them again for each version of the Windows Store application you want to test.
Second, generate the application packages for your Windows Store application.
I’m running Visual Studio 2012 Ultimate and using screenshots and steps from that version of Visual Studio.
1. Open Visual Studio.
2. Open the .sln which contains your Windows Store Application
3. Run it in debug mode at least once to make sure the thing is working. You’ll waste a lot of time otherwise.
4. Stop the Windows Store Application
5. Click Project -> Store -> Create App Packages
6. Choose No
7. Click Next
8. Now, I usually leave the path alone. It’s up to you where you put your app packages.
9. I also allow the package builder to increment the version automatically. Depending on your environment you may need to change this.
10. I set the CPU to neutral.
11. Since I am testing, I want as much crash information as I can get my hands on. I leave the last option selected
12. Click Create
13. Your output location in the screenshot above should match the output location on the previous screenshot.
14. Click OK
Now you have an application package with the appropriate certificate to install to your local machine.
Third, set up your PowerShell terminal.
Start a PowerShell terminal. The first time you run PowerShell you’ll need to enable scripts. After you’ve enabled scripts once, you won’t have to do it again. Run PowerShell terminal with Administrative privileges so you can change the way PowerShell handles scripts by default and so you can install your application. PowerShell is now included by default in Windows 8.
1. Click the Search Charm
2. Click Apps and Type PowerShell
You should see the following:
3. Right Click on Windows PowerShell
4. You will likely be using this application a lot so I recommend pinning to the taskbar.
5. Click Run as Administrator
In the blog post from Microsoft, they recommend only enabling signed scripts to be run. Since I am using some unsigned scripts, I enabled all, and I will go through how to do that. If you are going to use my directions when I include a script or reference one from the MSDN, you will want to enable all the scripts.
Note: this is a security hole so make sure you know what you are doing before you do it. If you only intend to run the installation script for your Windows Store application, then you may only want to enable running of Signed scripts. See the MSDN blog post for the command for only running signed scripts.
6. For running unrestricted scripts, on the command line type Set-ExecutionPolicy Unrestricted and enter.
7. Type y and enter
Now you can run any PowerShell scripts.
Fourth, uninstall the debug version.
Before you try to install the application, uninstall the debug version. If you don’t do this, your attempt to install your application will fail.
1. Go to Start and right click on the Windows Store App you created and ran in debug mode.
2. Select Uninstall
Fifth, install your Windows Store application.
1. In PowerShell, navigate to the folder your app package for your Windows Store app is stored
2. Type .\Add-AppDevPackage
3. If the installation fails, then make sure you’ve uninstalled the old application
4. If the installation succeeds, you should see a message that the install was a success.
The whole point of doing the install in this manner rather than using the debug option in Visual Studio is that we want to be able to launch our Windows Store Applications using our Automation Client. The Windows Store application has to have some kind of unique ID that doesn’t change between installs/debug runs of the Windows Store Application. This is done because you won’t be able to use System.Diagnostics.Process.Start to launch your application the way you would with a Desktop application. If you read through the MSDN blog post, you’ll see there is a way to launch your application using its unique ID and an executable written in C++. You can run the C++ executable using System.Diagnostics.Process.Start. I’ll go over how to get the unique ID and how to use the executable in the next post. After I finish with that, I discuss a topic that isn’t covered in the blog post. I’ll explain how to set up the elevated permissions you need to run a desktop application in the Protected UI in which the Windows Store Applications run.
You want your Automation Client to have elevated privileges because it has to be a desktop application with the ability to do two things. First, the Automation Client has to be able to run at the same time as your Windows Store Application. If it doesn’t, then it can’t invoke the controls and read the UI of your Windows Store application. If your Windows Store application has been launched but isn’t the top application, it goes into suspended mode. This means that your Automation Client can’t be a Windows Store Application itself because the application you are testing would be suspended while your Automation Client is running. Since Windows Store applications are barred from running other Windows Store applications, your Automation Client has to be a desktop application. Second, your Automation client needs to have access to both the Protected UI and the desktop. You will want access to the desktop so that you can use the accessibility COM which will allow you to automate your testing of a Windows Store application. Setting up the elevated privileges is a little more complex than running your Automation Client as an Administrator.
There’s still some more work to do to get your Automation Client ready to test a Windows Store Application. Check out Part Two of this guide to continue to set up your Automation client.
If you have any questions about automated testing or need help with automated testing of your project then feel free to contact us. You can also connect with us on Twitter, Facebook, or LinkedIn. Learn more at http://frogslayer.com/.
Note: This particular post fills in the holes that I had to figure out when reading the first part of Automating the testing of Windows 8 apps. The first quarter of the next post will also cover information in that blog post, and then I will diverge into the security requirements for your Automation Client.
BlogSlayer is the official blog of FrogSlayer, a custom software product development shop in Bryan/College Station, Texas. Our specialty is getting your product to market in 90 days or less. If you would like a free consultation for your project or big idea, email us at firstname.lastname@example.org. You can also connect with us on Twitter, Facebook, or LinkedIn.