Friday 10 May 2013

Metro Apps Automation in C#

After having done heck of research on how do I go about automating my Windows8 app, from among lot many third party tools to Windows UIA itself, I realized UIA was the way to go.

But then there are hardly any samples, that would help you get started with this. There are a couple available for Windows7, but Metro as we all know is a new world of its own.

This post would help you get started with Windows UI Automation for a Windows 8 (aka Metro) app, in C#, in just a few steps. All this information is actually gathered from a variety of tutorials, blogs, msdn links, forums, scattered all over the web and put together in this post.

You can create a Unit Test Project from
 Templates>Other Languages>Visual C# > Test > Unit Test Project
and your test cases would need all the below workflow, after which you can 'assert' all that you want to.
(I'm working on Visual Studio 2012, Ultimate )

1. Auto-launching of your app

This post I must say has been an amazing source of information. It also mentions how do you go about launching your app in C++.
The process and all the content stays true, but there is this below piece of code in C#, doing the same. "<appID>" here is the app ID of the application you want to launch.

 public static void launchApp()
 {
      ApplicationActivationManager appActiveManager =                                       
                              new ApplicationActivationManager();
      uint pid;
      appActiveManager.ActivateApplication("<appID>", 
                            null, ActivateOptions.None, out pid);
 }

public enum ActivateOptions
{
        None = 0x00000000,  
        DesignMode = 0x00000001,  
        NoErrorUI = 0x00000002,             
        NoSplashScreen = 0x00000004,  
}

[ComImport,Guid("2e941141-7f97-4756-ba1d-9decde894a3d"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
interface IApplicationActivationManager
{
       IntPtr ActivateApplication([In] String appUserModelId, 
                                  [In] String arguments, 
                                  [In] ActivateOptions options, 
                                  [Out] out UInt32 processId);
       IntPtr ActivateForFile([In] String appUserModelId,
                     [In] IntPtr /*IShellItemArray* */ itemArray, 
                     [In] String verb, 
                     [Out] out UInt32 processId);
       IntPtr ActivateForProtocol([In] String appUserModelId, 
                     [In] IntPtr /* IShellItemArray* */itemArray,                      
                     [Out] out UInt32 processId);
 }


[ComImport, Guid("45BA127D-10A8-46EA-8AB7-56EA9078943C")]
//Application Activation Manager
class ApplicationActivationManager:IApplicationActivationManager
{
[MethodImpl(MethodImplOptions.InternalCall, MethodCodeType =  
                    MethodCodeType.Runtime)/*, PreserveSig*/]
public extern IntPtr ActivateApplication(
                           [In] String appUserModelId,
                           [In] String arguments,
                           [In] ActivateOptions options,              
                           [Out] out UInt32 processId);


[MethodImpl(MethodImplOptions.InternalCall, MethodCodeType =
                                     MethodCodeType.Runtime)]
public extern IntPtr ActivateForFile(
                      [In] String appUserModelId,
                      [In] IntPtr /*IShellItemArray* */itemArray, 
                      [In] String verb, 
                      [Out] out UInt32 processId);
       


[MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = 
                                      MethodCodeType.Runtime)]      
public extern IntPtr ActivateForProtocol(
                     [In] String appUserModelId, 
                     [In] IntPtr /* IShellItemArray* */itemArray, 
                     [Out] out UInt32 processId);
 }



2. Accessing the controls :

You can inspect about the controls present in your app, using "Inspect.exe", available with Windows Kits. For me it is present at :
"C:\Program Files\x86)\Windows Kits\8.0\bin\x64".

Now for the below code to get working. Make sure to include these dll's beside the default ones : 
> PresentationCore.dll
> PresentationFramework.dll
> UIAComWrapper.dll [Most important]
> WpfUtilities.dll

I'm attaching them all here, for easy access or you can go ahead and search them in your Visual Studio installed location.

Below piece of code is just a sample to get you started to access any type of controls in your app. Here I'm just trying to find a button and click it.
(Note: All Windows UIA code would work just fine as seen in the msdn docs.)

//Gives you the root element i.e 'Desktop'
AutomationElement aeDesktop = AutomationElement.RootElement;

//Make a condition to find your app by 'NameProperty'
Condition appCondition = 
new PropertyCondition(AutomationElement.NameProperty, 
                    "MyApp", PropertyConditionFlags.IgnoreCase);

AutomationElement aeAppWindow =
                          aeDesktop.FindFirst(TreeScope.Children, 
                          appCondition);


AutomationElement aePane =
     aeAppWindow.FindFirst(TreeScope.Descendants,                                
     new PropertyCondition(AutomationElement.ControlTypeProperty,  
         ControlType.Pane));
            
AutomationElementCollection aeButtons = 
     aePane.FindAll(TreeScope.Descendants,
     new PropertyCondition(AutomationElement.ControlTypeProperty,
         ControlType.Button));

//Click on sample button
System.Drawing.Point p = aeButtons[3].GetClickablePoint();
Mouse.MoveTo(p);
Mouse.Click(MouseButton.Left);
Thread.Sleep(2000);


3. Closing your app

Hitting an 'Alt + F4' closes your app. You can do the same by this below line of code :

SendKeys.SendWait("%{F4}");

["SendKeys" is available in Systems.Windows.Forms dll]


That is all. You have now successfully automated a simple workflow of your Metro app. Good luck automating your next Windows app ! 

Post in your comments if you have any doubts or you find a better way of doing the same thing or better still, you find the post useful ! :)

19 comments:

  1. First link under the first point seems to be broken.

    Good article Guriya, thanks.

    ReplyDelete
  2. Hey.. this was really helpful! Thanks!!

    ReplyDelete
  3. I tried your code, but I'm unable to find the AutomationElement for my app window. I also tried printing all the Apps using FindAll function, but It only prints desktop app not the Metro Apps. I tried same code in c++ it worked fine. Do you any idea?

    ReplyDelete
  4. Add UIAUtomationCLient.DLL. AutomationELement class is present in that.

    ReplyDelete
  5. Hi Prashanth,
    Have you launched your application successfully?
    Only after the launch, your app window would be active and will be recognised by Windows UIA.

    ReplyDelete
  6. yes.. I'm able to launch application successfully.

    ReplyDelete
  7. Prashanth, do you see your App window being detected via "Inspect.exe" ?
    This should be present in your Windows Kits installation folder. For me it is in "C:\Program Files (x86)\Windows Kits\8.0\bin\x64\inspect.exe"
    Launch 'inpect' first, and then your app, you should be able to view your app in the left tree somewhere.
    It's strange because, for me all desktops as well as metro apps are getting detected.

    ReplyDelete
  8. Now I am able to detect my app. I was using UIAutomationClient.dll instead of UIAComWrapper.dll. I am not sure why it didn't get worked with UIAutomationClient.dll. Let me know if you know the answer.

    Thanks for your help.

    ReplyDelete
  9. Thank you SO much Guriya, this post is pure gold!
    On a side note, I was having some troubles initially because sometimes I would be able to find my App's descendant AutomationElements, but other times I wouldn't.
    I added "Thread.Sleep(1000);" right after launching and that fixed the problem.

    ReplyDelete
  10. Also, before 'hitting' Alt+F4, it would be wise to test if your app is still on the foreground. It might have crashed as part of the test process, in which case hitting Alt+F4 would close Visual Studio instead.

    http://stackoverflow.com/questions/115868/how-do-i-get-the-title-of-the-current-active-window-using-c

    ReplyDelete
  11. I tried your code but couldn't find ApplicationActivationManager.

    Error:
    The type or namespace name 'ApplicationActivationManager' could not be found (are you missing a using directive or an assembly reference?)

    ReplyDelete
  12. @Prashanth, why UIAComWrapper works and UIAutomationClient doesn't because, the former is Unmanaged API while the latter is a managed one. You can get more details about that here : http://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/f3bbec4d-f7c6-4e66-8d74-9d3db562a6c3/ui-automation-cannot-find-an-element-inspect-object-can


    @Diogo, Thanks :)
    I agree it would be wise to test if the app is still on the foreground, before attempting to close it.
    My aim here was to just give an idea how we can go about sending keystrokes in Automation.

    @Anonymous, Have you taken the code entirely? 'ApplicationActivationManager' class is present in the 'Auto Launching of App' code section. Let me know if this helps

    ReplyDelete
  13. Ok, I did not understand a thing :D But I do appreciate the language.

    On a serious note, i did get something (a little)

    ReplyDelete
  14. Very helpful info. When i try to open a Windows 8 app (Camera\Control Panel), i get the message : The app can t open while the file explorer is running in administrator privileges

    Any help!

    Prem.

    ReplyDelete
  15. Hi Guriya,

    'm able to launch the app but controls are not being identified and appWindow is returning a null value. 'm using all the dlls mentioned.

    Please help me out in this, in comments it is mentioned that app shud b in foreground how can I get that..

    Thanks in advance.

    ReplyDelete
    Replies
    1. Hi Anonymous,
      how are you able to launch the app with this code ,
      it's telling that the output class library cannot be started directly
      Please reply

      Delete
  16. Condition appCondition =
    new PropertyCondition(AutomationElement.NameProperty,
    "MyApp", PropertyConditionFlags.IgnoreCase);

    AutomationElement aeAppWindow =
    aeDesktop.FindFirst(TreeScope.Children,
    appCondition);


    Ican't run this code. It always return null... Please help me......
    AutomationElement aePane =
    aeAppWindow.FindFirst(TreeScope.Descendants,
    new PropertyCondition(AutomationElement.ControlTypeProperty,
    ControlType.Pane));

    ReplyDelete
  17. Thank you for the great article, Do you have a way to pin the modern app to the taskbar via this api or another api? I have to test the scenario of pinning it to the startscreen and taskbar in win10.

    ReplyDelete