Search

GUI Automation with PowerShell - Part 2

In the last post, we discussed some tools we have available that we may use for identifying GUI elements of desktop applications and the PowerShell module that we may use to work with those elements.


Here, I'll be showing you what we need as an input to the UiAutomation module of PowerShell to uniquely perform actions on UI elements once we identify them. And yes, the identification part will be covered moving ahead once we're aware of what we need to look for as an input to the module.


Let's start.


 

You would first need to download the module. Click here.


This would take you to the FileShare section where you'll find three folders with different names. I could not upload all the releases due to size limitations but have uploaded the latest release 0.8.7 Beta 3. Depending on your setup version you may need to select the module file you would work with.


If you want it simple, download the zip file named: UIAutomation.0.8.7B3.NET40.zip along with .net version 4 from here.


Complete the .net installation - it shouldn't be difficult. Now, extract the contents of the UIAutomation.0.8.7B3.NET40.zip file anywhere on your disk. For now, let's consider that you have extracted it to D:\UIAutomation.0.8.7B3.NET40


Now, every time you need to use this module, you'll need to explicitly import the module using the command:

Import-Module D:\UIAutomation.0.8.7B3.NET40\UiAutomation.dll

There is an alternative to this though if you do not want to import the module every time and just have it auto-imported. But, that's the story for another time. I will link it here if/when it completes. For now, let's get into some more details of this module.


 

As you must know (or not - who cares), PowerShell has a naming convention wherein verbs such as Get, Set, Read, Register, Show, New, Wait or Invoke are used to perform relevant functions. The same is applicable even here - UiAutomation module has commands with these prefixes.


One more thing to note is, all (the ones we're interested in for this post) commands under this module starts as either Get-Uia.., Set-Uia.., Invoke-Uia.. - that is, Uia is added after the verb.

You can go about exploring the commands with the help of Windows PowerShell ISE as in the screenshot below. Do note that you need to first have the module imported using the Import-Module [Path] command as mentioned above.

Content of UiAutomtion Module
Content of UiAutomation Module

Now it goes without saying that if you need to uniquely identify a window, you need to provide its unique identity. Let's take an example as the Notepad window that we used in the first part.



Using UiAutomationSpy


This would be one of the two methods of doing it, with half of the task, i.e., of getting the selector, automated using the UiAutomationSpy.exe tool included in the UiAutomation module folder by default. Note that you would need to run this tool as an administrator for optimal performance.


In the image below, we have selectors for each menu item under Notepad. If you're not sure how we reached here, do refer to the first part of this tutorial.




Now, to click on the File menu item, we need to further invoke the Invoke-UiaMenuBarExpand cmdlet on the output of the generated commands. Again, we have 5 commands to uniquely identify each menu item.






You can run the following command on PowerShell window where you have imported UiAutomation module:


Get-UiaWindow -Class 'Notepad' -Name 'Untitled - Notepad' | `

Get-UiaMenuBar -AutomationId 'MenuBar' -Name 'Application' | `

Get-UiaMenuItem -AutomationId 'Item 1' -Name 'File' | `

Invoke-UiaMenuBarExpand


As you should have noticed, we have included the generated script for File menu item to Invoke-UiaMenuBarExpand which would do its task of expanding the Menu item (File in our case). Need help with understanding the command a little better? Let me try.


Using Get-UiaWindow -Class 'Notepad' -Name 'Untitled - Notepad', we're creating an object for the window of class type 'Notepad' with the name 'Untitled - Notepad' which is being passed as an input to Get-UiaMenuBar cmdlet since before we could point to the MenuBar, we first need to specify which window is under consideration. The parameters passed on to Get-UiaMenuBar are further made specific with the parameters specified. Then, we further pass on the output of these commands to Get-UiaMenuItem and finally expand the menu using Invoke-UiaMenuBarExpand.


Confused about where the parameter values are coming from? We will discuss how to make the parameter values specific when using the manual method - UIAVerify - mentioned in the first part.



Using UIAVerify

Using this tool isn't that difficult though there is a little more to do, some may feel. Once you open the UIAVerify window, you will be presented with all the open application's components in a hierarchical view. For now, we're interested in doing the same task as we did above with UiAutomationSpy.


Pulling back the command generated earlier, let's relate it to the details from UIAVerify, line by line:


Get-UiaWindow -Class 'Notepad' -Name 'Untitled - Notepad' | `

We decide that we have to use Get-UiaWindow because we have the Notepad's window as: "window" "Untitled - Notepad" - See the relation?


"Untitled - Notepad" is the Name parameter's value.



Get-UiaMenuBar -AutomationId 'MenuBar' -Name 'Application' | `

Similarly, the next level in the hierarchy is: "menu bar" "Application" "MenuBar" hence, we use Get-UiaMenuBar.


(no, we're not deciding as per the value under the third quote but the first. The third quote contains the AutomationId whose value is "MenuBar")


Get-UiaMenuItem -AutomationId 'Item 1' -Name 'File' | `

Finally, we have: "menu item" "File" thereby making us use Get-UiaMenuItem with the Name parameter's value as "File"


Up till this point, we've been able to point down to the exact UI element but, now we need to perform some action on top of the referenced element. In our case, we need to expand the Menu (menu bar's) Item (File is an item in the menu bar).


Invoke-UiaMenuBarExpand

This is pretty much self-explanatory.


Hereon, you can go and check various Get, Set and Invoke commands available.


Go. Go and experiment now. Everything is not spoon-fed.

155 views0 comments

Recent Posts

See All