GUI Automation with PowerShell - Part 1
Updated: Nov 19, 2019
Note: I'm just going to talk about the tools I've used or could recommend and have working knowledge of. If you have something to add, please feel free to use the comments' section.
With all the emerging (or established) tools available for GUI based automation like:
- UiPath - Freeware and license based
- Blue Prism - License based
you might wonder why should you even consider writing code from scratch provided that UiPath's recording option let's you do half the task by just pointing and stating what needs to be done:
So, if you're one among the people who have that concern, sorry, the rest of this post isn't going to excite you. I'll not be listing the advantages or disadvantages of either but yes, would definitely help you go about with a detailed approach into this topic. I would be posting it along multiple posts and here, we'll be covering the introduction to the tools we're going to be using.
Let's get started.
The approach we're going to take for our automation using PowerShell is similar to what UiPath also works on - selector based item recognition. Selectors are nothing but the components of any application that forms the whole GUI. For example, in notepad window, we have the menu bar as one component (under which comes each menu item and more, as per the hierarchy):
The selector we get for this Menu bar from Notepad using UiPath is in the image below.
If you've ever worked with Selenium based web automation, you must be aware of XPath (XML Path Language), which basically lets you get down to the element at the smallest section transversing through the top sections. That is similar to what is happening here. Just that, here we're working with application level and narrowing down with VB expressions instead of XML or HTML.
Now, since we're not going to be using UiPath, we need to be able to narrow down to the UI components, uniquely identify each and be able to manipulate the same. To manipulate or use the components we identify, we'll need to have a PowerShell based module which we'll be importing.
To summarize, we need:
UiAutomation Module - This module contains the cmdlets for GUI element manipulation.
UiAutomationSpy or UIAVerify - Tools to inspect the UI elements' hierarchy. Now, both of these have a slight difference in the way you may use them. UiAutomationSpy is included with the UiAutomation module and could help you generate the selector based commands which require slight modification before it could be used. The latter is good when you want to go about manually inspecting elements and use them further.
Personally, I prefer using UIAVerify and here is a screenshot of the same for Notepad's window:
As you might see, we have the hierarchical view of the Notepad's window's UI elements. Information on how to use the details from this tool for automation with UiAutomation module is a story for the next post. Let's just get our tools and decide what we need as per our requirements now.
Doing justice to the other tool mentioned - UiAutomationSpy, here is a screenshot and more details on what happens here:
Once we use Start and point on UI elements (be it File under Notepad or close button on Mozilla Firefox), the Script section starts getting selectors for each and can be fetched once you Stop.
For this example, I had pointed to all the 5 menu items on Notepad window: File, Edit, Format, View and Help. If you look closely, you may notice that there is only a slight difference in the selector values for each. And, as I had mentioned earlier, we do need to modify the code to do anything at all after having got the selector.
For example, if you need to click on the selector you got, you will need pipe the output to Invoke-UiaClickMenuItem.
Now that we've discussed the tools, more details will be posted with the upcoming posts. Stay tuned.