flaui / flaui

UI automation library for .Net

Badges

What	Badge
Chat
Build
Tests
Libraries (NuGet)
CI Artefacts	FlaUI CI

Introduction

FlaUI is a .NET library which helps with automated UI testing of Windows applications (Win32, WinForms, WPF, Store Apps, ...).
It is based on native UI Automation libraries from Microsoft and therefore kind of a wrapper around them.
FlaUI wraps almost everything from the UI Automation libraries but also provides the native objects in case someone has a special need which is not covered (yet) by FlaUI.
Some ideas are copied from the UIAComWrapper project or TestStack.White but rewritten from scratch to have a clean codebase.

Sponsoring

If you appreciate my work, feel free to support me by sponsoring on github or with a one-time payment over at PayPal.

Why another library?

There are quite some automation solutions out there. Commercial ones like TestComplete, Ranorex, CodedUI just to name a few. And also free ones which are mainly TestStack.White.
All of them are based on what Microsoft provides. These are the UI Automation libraries. There are three versions of it:

• MSAA
  • MSAA is very obsolete and we'll skip this here (some like CodedUI still use it)
• UIA2: Managed Library for native UI Automation API
  • UIA2 is managed only, which would be good for C# but it does not support newer features (like touch) and it also does not work well with WPF or even worse with Windows Store Apps.
• UIA3: Com Library for native UI Automation API
  • UIA3 is the newest of them all and works great for WPF / Windows Store Apps but unfortunately, it can have some bugs with WinForms applications (see FAQ) which are not existent in UIA2.

So, the commercial solutions are mostly based on multiple of those and/or implement a lot of workaround code to fix those issues. TestStack.White has two versions, one for UIA2 and one for UIA3 but because of the old codebase, it's fairly hard to bring UIA3 to work. For this, it also uses an additional library, the UIAComWrapper which uses the same naming as the managed UIA2 and wraps the UIA3 com interop with them (one more source for errors). FlaUI now tries to provide an interface for UIA2 and UIA3 where the developer can choose, which version he wants to use. It should also provide a very clean and modern codebase so that collaboration and further development is as easy as possible.

Usage

Installation

To use FlaUI, you need to reference the appropriate assemblies. So you should decide, if you want to use UIA2 or UIA3 and install the appropriate library from NuGet. You can of course always download the source and compile it yourself.

Usage in Code

The entry point is usually an application or the desktop so you get an automation element (like a the main window of the application). On this, you can then search sub-elements and interact with them. There is a helper class to launch, attach or close applications. Since the application is not related to any UIA library, you need to create the automation you want and use it to get your first element, which then is your entry point.

using FlaUI.UIA3;

var app = FlaUI.Core.Application.Launch("notepad.exe");
using (var automation = new UIA3Automation())
{
	var window = app.GetMainWindow(automation);
	Console.WriteLine(window.Title);
	...
}

using FlaUI.Core.AutomationElements;
using FlaUI.UIA3;

// Note: Works only pre-Windows 8 with the legacy calculator
var app = FlaUI.Core.Application.Launch("calc.exe");
using (var automation = new UIA3Automation())
{
	var window = app.GetMainWindow(automation);
	var button1 = window.FindFirstDescendant(cf => cf.ByText("1"))?.AsButton();
	button1?.Invoke();
	...
}

Further Resources

YouTube Tutorials

Have a look at H Y R Tutorials. This channel provides some videos to get you started with FlaUI.

FlaUI UITests

FlaUI itself contains quite some UI tests itself. Browse to the code of them here and look how they work.

Chat

Head over to the chat to ask your specific questions.

Contribution

Feel free to fork FlaUI and send pull requests of your modifications.
You can also create issues if you find problems or have ideas on how to further improve FlaUI.

Donors and Sponsors

• Thank you Gehtsoft USA LLC for the generous donation

Acknowledgements

JetBrains

Thanks to JetBrains for providing a free license of ReSharper.

AppVeyor

Thanks to AppVeyor for providing a free CI build system for FlaUI.

TestStack.White

Thanks to the creators and maintainers (especially to @JakeGinnivan and @petmongrels) for their work and inspiration for this project.

Microsoft

Thanks to Microsoft for providing great tools which made developing this project possible.

flaui's Issues

Is it still alive?

Is the development dead?
Are you going to create a nuget package?

Implement ContextMenu

Implement the ContextMenu on Window.
Should handle at least WinForms, WPF and Win32 ContextMenues.

question to get value

Hi. Thank you for your work with flaui, I am starting to use it and looks better alternative.

But I can't get the actual value of a control like a textbox or label, how can I achieve it?

Maybe some more samples will help us.

That you.

Enabled properties are not available via wrapper classes

I'm migrating code from TestStack.White to this library.
There are lost of things missing especially Enabled property on various wrapper classes.
Namely the Button class lacks such ability meaning existing tests cannot be written using FlaUI

Create Win32 test application

Is this needed? There are some things which are different so it could make sense to have one.

LegacyIAccessible, DataGrids

Hey, all. Battling with some old Delphi apps and teaching myself UIA. I have some DevExpress data grids that I am trying to get into. Can anyone explain to me how to use "LegacyIAccessible" in context of FlaUI and to get data to and from the controls?

Cheers.

Registering for Events

How do I register for events and synchronously wait for those events to occur?
There is no documentation on this and I see barely any tests for event handling, while there seem to be multiple types of events registered.

My scenario is that when I click on the first list it changes the second and third list.

When I loop through the list and select it

foreach (var family in settingsDB.Fams())
{
	family.ScrollIntoView();
	family.Select();
	
	foreach (var model in settingsDB.Mods())
	{
		model.ScrollIntoView();
		model.Select();
		foreach (var version in settingsDB.Vers())
		{
			version.ScrollIntoView();
			version.Select();
		}
	}
}

The selection is too fast and it clicks on the second list before the first list has caused the second list to change. What is the best way to detect that the second list has changed before selecting it?
I was looking into registerStructureChangedEvent but found no good examples of how to use this.
UI.Wait is not helpful here.

Window.ModalWindows does not report all modal windows

In our application we have a progress bar dialog that is displayed before a file is opened, and the first action the user needs to do is to pick the file in a Open File dialog. Both dialogs are displayed with ShowDialog so they are both modal, but the Window.ModalWindows only reports the first progress dialog.

As you can see the file dialog is a child of the progress bar dialog:

Code using ModalWindows:

Window[] windows = App.MainWindow.ModalWindows;
// windows.Length == 1

Progress dialog has the following pattern:

IsTopMost=False
CanMinimize=False
CanMaximize=False
WindowVisualState=Normal
WindowInteractionState=BlockedByAModalWindow

File dialog has the following pattern:

IsTopMost=False
CanMinimize=False
CanMaximize=False
WindowVisualState=Normal
WindowInteractionState=ReadyForUserInteraction

Can not get or set value of Spinner inputs

FlaUI appears to be unable of getting or setting the value of a numeric spinner control in a windows forms application.

In UIA2 note that the edit control inside of the combo-box does not show up:

In UIA3 it shows up in the inspect tool:

However, the edit control which contains the value does not seem to be target-able by code

branch with spinner on win forms app

My attempts to select the current value:

        [Test]
        public void Spinner()
        {
            var comboBox = App.GetMainWindow(Automation).FindFirstDescendant(cf => cf.ByAutomationId("numericUpDown1"));
            Assert.IsNotNull(comboBox);

            var edit = comboBox.FindFirstDescendant(cf => cf.ByControlType(ControlType.Edit));
            Assert.IsNotNull(edit);

            var value = comboBox.AsComboBox().EditableText;
            Assert.AreEqual("42", value);

            var value2 = edit.AsTextBox().Text;
            Assert.AreEqual("42", value);
        }

Allow Raw-Walking with FlaUInspect

Currently, FlaUInspect supports only viewing control elements. Rewrite it so that it uses the TreeWalker and allow switching the TreeWalker to Raw or Content.

Implement missing patterns

There are still some patterns missing.
They are all (in an old format) in FlaUI.UIA3.Patterns (excluded from project). To implement them, following things need to be done:

• Interfaces created and put in FlaUI.Core.Patterns
• Implementation in FlaUI.UIA3.Patterns
• Implementation in FlaUI.UIA2.Patterns (can be copied from UIA3 and slightly adjusted)
• Add the patterns in IPatternFactory and it's implementation for UIA2/UIA3

Following patterns are missing:

• AnnotationPattern
• DockPattern
• DragPattern
• DropTargetPattern
• ItemContainerPattern
• LegacyIAccessiblePattern
• MultipleViewPattern
• ObjectModelPattern
• ScrollItemPattern
• ScrollPattern
• SelectionPattern
• SpreadsheetItemPattern
• SpreadsheetPattern
• StylesPattern
• SynchronizedInputPattern
• TableItemPattern
• TablePattern
• Text2Pattern
• TextChildPattern
• TextEditPattern
• TextPattern

Please provide demo app with sources.

I install package from nuget, but my code not work. Please provide demo source of app.
http://joxi.ru/l2ZvLlBU1qvE2J.jpg

Searching in UIA2 versus UA3

var screen = App.GetMainWindow(Automation);

var titleBar = screen.TitleBar; // Does not work.
Assert.That(titleBar, Is.Not.Null, @"Could not locate TitleBar as property of screen.");

titleBar = screen.FindFirstByXPath(@"/*[@AutomationId='TitleBar']")?.AsTitleBar();
Assert.That(titleBar, Is.Not.Null, @"Error locating TitleBar by given XPath.");

titleBar = screen.FindFirstByXPath(@"/*[@LegacyIAccessible.Role = 'title bar']")?.AsTitleBar();
Assert.That(titleBar, Is.Null, @"Error locating TitleBar by given XPath.");

//var propertyCondition = CacheRequest .PropertyLibrary.LegacyIAccessible.Role, 0x1;
//titleBar = screen.FindFirst(TreeScope.Children, propertyCondition)?.AsTitleBar();
//Assert.That(titleBar, Is.Null, @"Could not locate by LegacyIAcccessibility.Role");

// Can locate by ILegacy Pass through
//titleBar = screen.FindFirstDescendant(fn => fn.ByControlType(ControlType.TitleBar))?.AsTitleBar();
//Assert.That(titleBar, Is.Null, @"Could not locate TitleBar by ControlType");

// Can locate by manual Automation Tree search and condition creation
//titleBar = screen.FindFirst(TreeScope.Children, new PropertyCondition(Automation.PropertyLibrary.Element.AutomationId, "TitleBar"))?.AsTitleBar();
//Assert.That(titleBar, Is.Null, @"Could not locate TitleBar AutomationId");

//titleBar = App.GetMainWindow(Automation).FindFirstDescendant(ctrl => ctrl.ByLocalizedControlType("TitleBar"))?.AsTitleBar();
//Assert.That(titleBar, Is.Null, @"Could not locate TitleBar using Localized Control Type of 'TitleBar'");

Testing an old Delphi app.

I am varying results between UIA2 and UIA3. Everyone one of these tests yields null in UIA3. Can anyone provide a little more insight with regards to the caveats in searching? How do I correctly grab something by legacy accessibility?

Cheers.

Create a RemoteWebDriver for Selenium

Like
https://github.com/Microsoft/WinAppDriver
or
https://github.com/2gis/Winium.Desktop

Create nuget packages

Investigate in the best method to automatically create nuget packages (and releases) with AppVeyor and add implement this.

Implement base version of inspect

Should allow browsing the elements and switching between UIA2 and UIA3.

can find loginname textbox but cannot set value

use flaui can find loginname textbox, but can not set value for this textbox. raise exception

Implement ErrorCodes

Throw better exceptions on known error codes on known locations (like in FromHandle with an invalid handle)).
https://msdn.microsoft.com/en-us/library/dd757489(v=VS.85).aspx

ComboBox doesn't have method Select(string itemText)

It would be nice if comboboxes had a method to select items by text instead of only by index.
Ofcourse it's possible to do so yourself as user, but it kind of feels like basic functionality for a combobox.

I'm currently in the process of migrating a small project where I'm using TestStack.White to FlaUI.
I'll post issues here for situations where I think White's API is superior. This is one of those cases ;)

Implement List control

New Window after login.. Is there a way to find a root element(desktop)?

My application launches a new window after login which is not the child window.
If I have the root element, then I can find the child(new window) under the root element.

How can I find the root element?

The below code that I used to launch my application and login cannot find the new window.
var app = Application.Launch("CC.exe");
var automation = new UIA3Automation();
var window = app.GetMainWindow(automation);

         //Code for login and wait for new window to launch..         

        var _ccMain= window.FindFirstDescendant(cf => cf.ByAutomationId("Shell"));

Create UniversalWindowApp test application

Since they should be fully supported, should we add such an application?

How to enable UI actions logging?

Cannot access buttons under a tabular toolbar

I'm using FlaUI to test an application that has a 5 tab toolbar with unique buttons under each. The application starts with the first tab open and it's child buttons visible. Clicking these buttons works as intended.

However, when I use FlaUI to click another tab, it still only has access to the child buttons of the first tab. Is there some way to reset the current window or application so that the new tab's buttons are visible. Thank you.

Missing classes like AutomationElementCollection, AutomationElementIdentifiers

I am planning to upgrade from UIAComwrapper to Fla UI. But seems Fla UI is missing few of the classes I am currently using with an old library like AutomationElementCollection, AutomationElementIdentifiers.

Please confirm if this is an issue or I am missing some configuration

Add(ByIndex(0))

I'm investigating using FLaUI instead of White and was looking for the equivalent of AndIndex in White

For Example in White I have
mainWindow.Get(SearchCriteria.ByControlType(ControlType.Edit).AndIndex(2));

What's the equivalent in FLaUI to the above line, to use And.Index(2)?

Grid has no methods Select(string itemName), Exists(string itemName) or SelectedItem property

Grids are often used with a single column dimension as lists.
It would be nice if there was a way to select items/rows based on their value/text instead of only by index.
It would also be nice if there was an Exists(string itemName) method that identified if an item with specified value exists in the Grid. It might be valuable to implement this method for all AutomationElements that contain multiple items (ComboBox, Grid, GridRow, ...)

Something that's definitely missing is a way of returning the selected rows (for example a SelectedItems property, like the ComboBox has). Unless I'm somehow overlooking this functionality?

Add more complex examples

It would be nice to have more complex examples.
https://gist.github.com/vasily-v-ryabov
https://github.com/mmarquee/ui-automation/wiki#real-world-examples

Don't pollute stdout

At present classes in FlaUI.Core write data to the console this limit's the ability of consumers to use this library in CLI applications as the library ends up polluting stdout. Please consider allowing consumers of this library to pass alternate loggers, or defaulting to Trace or Debug for log information.

Implement Grid control

Create TreeWalker for UIA2 / UIA3

Implement SetTransparency

Some other ui automation wrappers/frameworks have this feature:

• https://github.com/mmarquee/ui-automation/blob/master/src/main/java/mmarquee/automation/controls/AutomationWindow.java#L230
• https://github.com/pywinauto/pywinauto/blob/master/pywinauto/controls/HwndWrapper.py#L1251

FindChildren / FindDescendants not seeing elements the Spy tools see

I am able to use flauinspect.exe and UISpy.exe to view the Win32 Tree and Controls but I'm not able to Find Children or Descendants of the Window. The length is always 0. How do I enable the code version to see everything the Spy tools see?

Not possible to handle window that is displayed when selecting combobox item

I have a combo box item that if the user chooses it, it will reset some other values in the same UI. The application then displays a message box asking the user to confirm that the other values are changed. But the ComboBoxItem.Select() freezes the code execution so it is not possible to handle the message box that is displayed.

Im porting my project from White and this worked there somehow..

Implement Caching

UIA supports caching. We need a concept on how to support it.

Sign the assembly

As it is now it cannot be used for testing signed assemblies.

Sceanrio: the tested assembly is signed and has InternalsVisibleTo the test assembly. I think this means the test assembly must also be signed.

Unable to access label text in Legacy Win 32 app

Can you help me figure out how to access the Driver Info in this screen shot?
This is the same application you previously downloaded.
File > New will bring up this screen.

Typical Patterns are not available and only the parent pane is available. The labels do not appear in spy trees.

It does not appear to be Content Element using UISpy > Query

Signed NuGet package

I've noticed that there is a signing key in the source tree, but NuGet package does not seem to be signed.
I get the following errors

>CSC : error CS8002: Referenced assembly 'FlaUI.Core, Version=1.2.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name.
>CSC : error CS8002: Referenced assembly 'FlaUI.UIA3, Version=1.2.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name.

Could you please provide a signed version as NuGet package as well?

Add information from WPF VisualTreeHelper

In WPF, there is the VisualTreeHelper which allows you to browse the whole visual tree and get information out of it.
It is possible to attach to an application and get the visual tree out with information like text styles.
See https://wpfinspector.codeplex.com or https://snoopwpf.codeplex.com for examples.
We should add this as well to provide additional information for WPF applications.

Custom wait hooks

Is there currently any mechanism in FlaUI to make a custom wait hook? Possibly similar to test stack's IWaitHook (http://teststackwhite.readthedocs.io/en/latest/AdvancedTopics/Waiting/)

How click to control without mouse moving?

I use demo code from wiki. All work, but my mouse) Its manually moving it. I think this library use winapi and its can click on button without mouse moving...

'Value does not fall within the expected range' when accessing cached IsOffScreen

Im getting the below stack trace when I try to use caching of the IsOffScreen property.

My cache request looks like this:

var cacheRequest = new CacheRequest();
cacheRequest.Add(MainWindow.Automation.PropertyLibrary.Element.AutomationId);
cacheRequest.Add(MainWindow.Automation.PropertyLibrary.Element.IsOffscreen);
cacheRequest.Add(MainWindow.Automation.PatternLibrary.WindowPattern);
cacheRequest.TreeScope = TreeScope.Descendants;
using (cacheRequest.Activate())
{
    Window[] modalWindows = MainWindow.FindAllDescendants(cf => cf.ByControlType(ControlType.Window)).Select(ae => ae.AsWindow())
        .Where(w => w.Properties.IsOffscreen.ValueOrDefault).ToArray();

and the exception raised is:

Result StackTrace:	
at interop.UIAutomationCore.IUIAutomationElement.GetCachedPropertyValueEx(Int32 propertyId, Int32 ignoreDefaultValue)
   at FlaUI.Core.BasicAutomationElementBase.TryGetPropertyValue[T](PropertyId property, T& value)
   at FlaUI.Core.AutomationProperty`1.get_ValueOrDefault()
   at App.FunctionalTesting.App.<>c.<CheckForCrash>b__63_2(Window w) in C:\App.FunctionalTesting\App.cs:line 249
   at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at App.FunctionalTesting.App.CheckForCrash() in C:\App.FunctionalTesting\App.cs:line 248
   at App.FunctionalTestingTests.SampleTest.Test() in C:\App.FunctionalTestingTests\SampleTests.cs:line 28
Result Message:	System.ArgumentException : Value does not fall within the expected range.

Add GitLink support

See https://github.com/GitTools/GitLink#source-stepping-returns-html
and https://github.com/TestStack/White/commits/UIAComWrapper/appveyor.yml (changes from 31.12.2015)
https://oren.codes/2015/09/23/enabling-source-code-debugging-for-your-nuget-packages-with-gitlink/

Error on Windows XP SP3

Hi, first to many thanks for this work.

I am trying to automate a Visual Foxpro aplication, and It works well in windows 10 an 8, 8.1, but when I try it in windows XP SP3, I get an error:

• When using UIA3Automation, a get: Unregistered class for component with CSLID {FF48DBA4-60EF-4201-AA87-54103EEF594E}.

• And When using UIA2Automation, the process to get the previously started aplication main windows is too slow in all Windows Operating System tested, and in windows XP I get this error : exception in type initializer "FlaUI.UIA2.Indentifiers.AutomationObjectIds".

Any idea, I need help.

Thank you

Where can I get help from a document?

Where can I get help from a document?
Thanks!

Support searching by XPath

As the title says, allow easy searching by XPath.

Child Windows not shown in FlaUInspect-1.0.0 but same is visible in VisualUIAVerify.exe

Hi,

I am getting a pop up window in my desktop application which i am not able to get a handle using FlaUI framework.
If i inspect the same from FlaUInspect-1.0.0 also not able to see the window but same is visible from VisualUIAVerify.exe.

The Window Object marked in yellow in VisualUIAVerify1.png is not present in FlaUInspect1.png.

Tried by app.GetMainWindow(automation); and app.GetAllTopLevelWindows(automation); but still no luck.
I am about to complete Automation POC on my 7 Desktop Enterprise Desktop applications and this is the only situation which is stopping to development on Smartbear Test Complete and jump on to FlaUI.

Tried with Winium tool also before and got struck at the same point.
Please let me know if any more info required.

The application is developed in Progress.

Attached the screenshot of both FlaUIInspect and VisualUIAVerify after greying out the sensitive data.
The Window Object marked in yellow in VisualUIAVerify1.png is not present in FlaUInspect1.png.

Support Custom Patterns

Support UIA2 and UIA3

As the title says, user interfaces / base classes to support UIA2 and UIA3 since UIA2 performs way better on WinForms and UIA3 way better on WPF / Store apps.

Popup support

Popup support is missing from FlaUI, similar to Whites Window.Popup and also Window.Popupmenu.

Snippet from our particular use case:

<UserControl x:Class="Herpa.AppBase.BarWithButtons"
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
             xmlns:local="clr-namespace:Herpa.AppBase"
             xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
             Height="30"
             Background="{DynamicResource DarkBackgroundBrush}"
             d:DesignHeight="15"
             d:DesignWidth="60"
             mc:Ignorable="d">
	<UserControl.Resources>
		<ResourceDictionary>
			<Menu x:Key="m_viewsMenu"
			      MouseLeave="m_viewsMenu_OnMouseLeave"
			      Style="{StaticResource VerticalMenu}">
				<MenuItem Name="m_mainMenuItem"
				          Margin="0"
				          Background="Transparent"
				          Opacity="0"
				          Padding="0"
				          StaysOpenOnClick="True" />
			</Menu>
		</ResourceDictionary>
	</UserControl.Resources>

	<Grid VerticalAlignment="Center" MouseDown="UIElement_OnMouseDown">
		<StackPanel HorizontalAlignment="Right"
		            VerticalAlignment="Center"
		            Orientation="Horizontal">

			<Button Name="m_viewsButton"
			        Width="24"
			        Height="24"
			        Margin="1,1,1,1"
			        HorizontalAlignment="Right"
			        Background="Transparent"
			        Click="ViewsButton_OnClick"
			        ToolTip="Open a view" />

			<Popup Name="m_viewsPopup"
			       Child="{DynamicResource m_viewsMenu}"
			       Placement="Top"
			       PlacementTarget="{Binding ElementName=m_viewsButton}"
			       StaysOpen="False" />

		</StackPanel>
	</Grid>
</UserControl>

Usage in a test in this particulare case (Using TestStack.White)

MainWindow.Get<Button>( SearchCriteria.ByAutomationId( "m_viewsButton" ) ).Click();
Assert.That( MainWindow.HasPopup() );
Menus menus = MainWindow.Popup.Items;