I use SpecFlow with Coded UI to create some automated functional tests for a WPF application. Test case execution is performed using MsTest and Visual Studio Premium 2012.
I have a lot of test cases. If I execute them one by one everything is OK. If I put them all in an ordered test I receive the following error:
Microsoft.VisualStudio.TestTools.UITest.Extension.UITestControlNotAvailableException: The following element is no longer available: Name [], ControlType [Custom], AutomationId [reags:LoadView_1], RuntimeId [7,1620,64780193] ---> System.Windows.Automation.ElementNotAvailableException: The following element is no longer available: Name [], ControlType [Window], AutomationId [UnitializedCB3702D1-14B6-4001-8BC7-CD4C22C18BE1], RuntimeId [42,1770052]
at Microsoft.VisualStudio.TestTools.UITest.Extension.Uia.UiaUtility.MapAndThrowException(SystemException e, IUITechnologyElement element)
at Microsoft.VisualStudio.TestTools.UITest.Extension.Uia.UiaElement.get_AutomationId()
at Microsoft.VisualStudio.TestTools.UITest.Extension.Uia.UiaElement.HasValidAutomationId()
at Microsoft.VisualStudio.TestTools.UITest.Extension.Uia.UiaElement.get_FriendlyName()
at Microsoft.VisualStudio.TestTools.UITest.Common.UIMap.UIMapUtil.FillPropertyFromUIElement(UIObject obj, IUITechnologyElement element)
at Microsoft.VisualStudio.TestTools.UITest.Common.UIMap.UIMapUtil.FillPropertyOfTopLevelElementFromUIElement(UIObject obj, IUITechnologyElement element)
at Microsoft.VisualStudio.TestTools.UITest.Common.UIMap.UIMapUtil.FillTopLevelElementFromUIElement(IUITechnologyElement element, TopLevelElement obj, Boolean stripBrowserWindowTitleSuffix)
at Microsoft.VisualStudio.TestTools.UITest.Common.UIMap.UIMapUtil.GetCompleteQueryId(UITechnologyElement pluginNode)
at Microsoft.VisualStudio.TestTools.UITesting.UITestControl.GetQueryIdForCaching()
at Microsoft.VisualStudio.TestTools.UITesting.UITestControl.<>c__DisplayClass6.<CacheQueryId>b__5()
at Microsoft.VisualStudio.TestTools.UITesting.CodedUITestMethodInvoker.InvokeMethod[T](Func`1 function, UITestControl control, Boolean firePlaybackErrorEvent, Boolean logAsAction)
at Microsoft.VisualStudio.TestTools.UITesting.UITestControl.CacheQueryId(String queryId)
at Microsoft.VisualStudio.TestTools.UITesting.UITestControl..ctor(IUITechnologyElement element, UITestControl searchContainer, String queryIdForRefetch)
at Microsoft.VisualStudio.TestTools.UITesting.TechnologyElementPropertyProvider.GetPropertyValue(UITestControl uiControl, String propertyName)
at Microsoft.VisualStudio.TestTools.UITesting.UITestPropertyProvider.TryGetPropertyFromTechnologyElement(UITestControl uiControl, String propertyName, Object& value)
at Microsoft.VisualStudio.TestTools.UITesting.PropertyProviderBase.GetPropertyValue(UITestControl uiControl, String propertyName)
at Microsoft.VisualStudio.TestTools.UITesting.UITestPropertyProvider.GetPropertyValueWrapper(UITestControl uiControl, String propertyName)
at Microsoft.VisualStudio.TestTools.UITesting.UITestControl.GetPropertyValuePrivate(String propertyName)
The first couple of errors were fixed using this hint, but I have some auto-generated steps and in order to re-search the controls I have to move the code and... a lot of unnecessary and annoying work.
Could you suggest some another solution to fix this? Is there some trick with the ordered tests? Or some nice clean-up methods for problems like this?
Thanks!
Here's what I did with a recent project.
First I created some CodedUI test methods as if SpecFlow didn't exist so I could keep those layers separate. Then I created step definition classes in C# that delegate to the coded UI test methods I created.
In a before scenario hook I created my UIMap instances (the classes generated by the CodedUI test generator) so each scenario had a fresh instance of my UIMap classes. You need this because object references in these classes are cached. Each new screen in your app is a whole new object tree that CodedUI must traverse.
Many times my step definitions just dive right into the CodedUI API to create custom searches, and I used the auto generated methods in my UIMap classes as a point of reference.
A little elaboration on how I set up my test project.
I created a new "Test" project in Visual Studio 2010, which references the following libraries:
Microsoft (probably comes with default Test project template)
NuGet Packages
This was my first stab at CodedUI Tests. I came from a Ruby on Rails background, and did a fair amount of reading online about implementing CodedUI Tests and SpecFlow tests. It's not a perfect setup, but it seems to be pretty maintainable for us.
Tests (Test project)
Features/
Bar.feature
Foo.feature
Regression/
Screen1/
TestsA.feature
TestsB.feature
StepDefinitions/
CommonHooks.cs
DataAssertionSteps.cs
DataSteps.cs
FormSteps.cs
GeneralSteps.cs
PresentationAssertionSteps.cs
Screen1Steps.cs
Screen2Steps.cs
UI/
FormMaps/
Screen1FormMap.cs
Screen2FormMap.cs
UIMapLoader/
User.cs
UIMap.uitest (created by CodedUI test framework)
Models (C# Class Library Project)
Entities/
Blog.cs
Comment.cs
Post.cs
Repositories/
BlogRepository.cs
CommentRepository.cs
PostRepository.cs
ViewModels/
Screen1ViewModel.cs
Screen2ViewModel.cs
This folder contains all the SpecFlow feature files implementing the basic business rules, or acceptance tests. Simple screens got their own feature file, whereas screens with more complex business logic were broken into multiple feature files. I tried to keep these features friendly to read for both Business and Developers.
Because our Web Application was not architected in a manor allowing unit testing, all of our testing must be done through the UI. The Tests/Regressions folder contains all the SpecFlow feature files for our full regression of the application. This includes the really granular tests, like typing too many characters into form fields, etc. These features weren't really meant as business documenation. They are only meant to prevent us from being woken up at 3 a.m. because of production problems. Why do these problems always happen at 3 a.m.? ...
The Test/StepDefinitions folder contains all the SpecFlow Step Definition files. I broke these files down first into common steps, and then steps pertaining to a particular screen in my application.
CommonHooks.cs -- Created by SpecFlow
[Binding]
public class CommonHooks
{
[BeforeTestRun]
public static void BeforeTestRun()
{
...
}
[BeforeScenario]
public void BeforeScenario()
{
User.General.OpenLauncher();
}
[AfterScenario]
public void AfterScenario()
{
User.General.CloseBrowser();
User.General = null;
}
}
The BeforeScenario and AfterScenario methods are where I create and/or destroy instances of the CodedUI UIMap classes (More on that further down)
DataAssertionSteps.cs -- Step definitions asserting that data shows up, or doesn't show up in the database. These are all Then ... step definitions.
Scenario: Foo
Then a Foo should exist
In DataAssertionSteps.cs:
[Then(@"a Foo should exist")]
public void ThenAFooShouldExist()
{
// query the database for a record
// assert the record exists
}
DataSteps.cs -- Steps to seed the database with data, or remove data. These are all Given ... step definitions used to set up a scenario.
FormSteps.cs -- Step definitions for interacting with forms. These all tend to be When I ... steps
GeneralSteps.cs -- Realy generic step definitions. Things like When I click the "Foo" link go here.
PresentationAssertionSteps.cs -- Generic steps asserting that the UI is behaving properly. Things like Then I should see the text "Foo" go here.
Screen1Steps.cs -- When I needed steps for a particular screen, I created a step definition file for that screen. For example, if I needed steps for the "Blog Post" screen, then I created a file call BlogPostSteps.cs, which contained all those step definitions.
The Tests/UI folder contains a bunch of custom written C# classes that we used to map label text found in our *.feature files to the names of form controls. You might not need this layer, but we did. This makes it easier to refactor your test project if form control names change, and especially for Web Projects because the HTML form field names change based on the <asp /> containers in our ascx files.
Example class:
namespace Tests.UI.FormMaps.Screen1FormMap
{
public static IDictionary<string, string> Fields = new Dictionary<string, string>()
{
{ "First Name", "UserControlA_PanelB_txtFirstName" },
{ ... },
...
};
}
Example Step:
When I enter "Joe" in the "First Name" textbox in the "Screen 1" form
Example Step Definition:
[When(@"I enter ""(.*)"" in the ""(.*)"" textbox in the ""(.*)"" form")]
public void WhenIEnterInTheTextboxInTheForm(string text, string labelText, string formName)
{
if (formName == "Screen 1")
{
// form control name: Screen1FormMap.Fields[labelText]
}
...
}
The step definition then used the Tests.UI.FormMaps.Screen1FormMap.Fields property to retrieve the form control name based on the label text in the *.feature files.
Tests.UI.FormMaps.Screen1FormMap.Fields["First Name"]
Tests/UI/UIMapLoader/User.cs
The other thing inside this folder is the UI/UIMapLoader/User.cs file. This class is a custom written class providing easy access to all the UIMap classes generated by the CodedUI Test framework.
namespace Tests.UI.UIMapLoader
{
public static class User
{
private static UIMap _general;
public static UIMap General
{
get { return _general ?? (_general = new UIMap()); }
set { _general = value; }
}
}
}
That way the Step Definition classes can easily access the UI maps via:
User.General.SomeCodedUITestRecordedMethod(...);
You saw a reference to this class in the BeforeScenario and AfterScenario methods in the CommonHooks.cs file referenced above.
This is just a class lib to encompass the entities and repositories allowing the test project to access the database. Nothing special here except the ViewModels directory. Some of the screens have complex relationships with data in the database, so I created a ViewModel class to allow my SpecFlow step definitions to easily seed the database with data for these screens.