Behavioural testing in .Net with SpecFlow and Selenium (Part 2)

In this series of posts I am going to run through using Selenium and SpecFlow to test .Net applications via the user interface. I will talk about the situations this type if testing can be used, and how we can create effective, non-brittle test with the help of the Page Objects Pattern.

The series in full:

In the first post in this series I looked at creating a scenario with SpecFlow and generating some empty step definitions. In this post I will implement the steps using Selenium in order to make the scenario test pass.

Selenium

Selenium is a web automation framework. In simple terms it allows you to drive a browser via code. Selenium can be used in many browser automation situations, but one of  it’s main uses is for automated testing of web applications.

When it comes to testing the behaviours of web pages there has traditionally been a problem where the testing code becomes tied to the implementation details of the page. This can make the tests brittle as a change to the page implementation details, such as a change to a field name or Id, will break the test even if the behaviour of the page has not changed.

In order to protect against this it is desireable to encapsulate the implementation details of a page under test in a single place, so that if there is an implementation change there is only a single place to change, instead of having to fix all the separate bits which rely on those details.

Separating the behaviour of a page from its implementation details is the purpose of the Page Object pattern. A page object is simply a class that encapsulates the implementation details for a web page, allowing the automation to focus only on behaviour.

Implementing the first scenario

The first test is all about the Search Page so it is the Search Page page object that is first to be created. I find it easiest to work in a TDD style workflow, using the SpecFlow steps to guide me towards the implementation one step at a time. Lets have a look at the implementation:

public class CrazyEventsSearchPage
{
    [FindsBy(How = How.Id, Using = "region")]
    private IWebElement regionDropdown;

    [FindsBy(How = How.Name, Using = "Submit")]
    private IWebElement submitButton;

    private static IWebDriver driver;

    public static CrazyEventsSearchPage NavigateTo(IWebDriver webDriver)
    {
        driver = webDriver;
        driver.Navigate().GoToUrl("http://localhost:2418/event/Search");
        var searchPage = new CrazyEventsSearchPage();
        PageFactory.InitElements(driver, searchPage);
        return searchPage;
    }

    public void SelectRegionDropdown()
    {
        regionDropdown.Click();
    }

    public IList<string> GetAllPossibleRegions()
    {
        var selectList = new SelectElement(regionDropdown);
        var options = selectList.Options;

        return options.Select(webElement => webElement.Text).ToList();
    }
}

Remember that the job of a page object is to encapsulate the implementation details of a web page. Selenium provided some helper for working with page objects. To use them make sure you have installed the Selenium Webdriver Support Classes into your test project via NuGet. PageFactory.InitElements(driver, searchPage) take a page object, in this case searchPage and ensures that all the elements are populated with the details from the web page. In this case it will populate the regionDropdown and submitButton web elements ready to be used.

It is the methods and fields of the page object that encapsulate the implementation details. If the id of the region needs to be changed, only the WebDriver FindsBy attribute that would need to be changed. If submitting the form needed to do something else, only a single change would be required.

The NavigateTo method is used as the entry point into the web site. The other methods are used by the SpecFlow steps.

[Binding]
public class EventSearchSteps
{
    private CrazyEventsSearchPage searchPage;
    private CrazyEventsSearchResultsPage searchResultsPage;
    private IWebDriver driver;

    [BeforeScenario()]
    public void Setup()
    {
        driver = new FirefoxDriver();
    }

    [AfterScenario()]
    public void TearDown()
    {
        driver.Quit();
    }

    [Given(@"that I want to search for an event by region")]
    public void GivenThatIWantToSearchForAnEventByRegion()
    {
        searchPage = CrazyEventsSearchPage.NavigateTo(driver);
    }

    [When(@"a list of possible regions is presented")]
    public void WhenAListOfPossibleRegionsIsPresented()
    {
        searchPage.SelectRegionDropdown();
    }

    [Then(@"the list should contain ""(.*)"", ""(.*)"", ""(.*)"" and ""(.*)""")]
    public void ThenTheListShouldContainAnd(string p0, string p1, string p2, string p3)
    {
        var regions = searchPage.GetAllPossibleRegions();
        Assert.IsTrue(regions.Contains(p0));
        Assert.IsTrue(regions.Contains(p1));
        Assert.IsTrue(regions.Contains(p1));
        Assert.IsTrue(regions.Contains(p1));
    }

    [Then(@"""(.*)"" should be the default value")]
    public void ThenShouldBeTheDefaultValue(string p0)
    {
        var selectedRegion = searchPage.GetSelectedRegion();
        Assert.IsTrue(selectedRegion.Contains(p0));
    }
}

I have added some setup and teardown methods to ensure that the web driver is initialised and shut down correctly. With all the implementation in place the scenario now passes.

Another scenario

Its time to add another scenario:

Scenario: Search for events in a region and display the results
	Given that I want to search for an event by region
	When I select the "London" region
	And perform a search
	Then The search results page is displayed
	And the following results are displayed
	| Event Code | Event Name                        | Region | Description                           |
	| CH/3001    | Cat Herding                       | London | A starter session for the uninitiated |
	| CH/302     | Cat Herding - Advanced techniques | London | Taking it to the next level           |

This scenario uses a SpecFlow table to define the expected results. Gherkin has a lot of powerful features to make the most of defining scenarios. The steps definitions were generated in the same way as the first scenario. I created another page object for the search results:

public class CrazyEventsSearchResultsPage
{
    private static IWebDriver driver;

    [FindsBy(How = How.Id, Using = "searchResults")] private IWebElement searchResults;

    public CrazyEventsSearchResultsPage(IWebDriver webDriver)
    {
        driver = webDriver;
    }

    public bool IsCurrentPage()
    {
        return driver.Title == "Crazy Events - Search Results";
    }

    public List<List<string>> GetResults()
    {
        var results = new List<List<string>>();
        var allRows = searchResults.FindElements(By.TagName("tr"));
        foreach (var row in allRows)
        {
            var cells = row.FindElements(By.TagName("td"));
            var result = cells.Select(cell => cell.Text).ToList();
            results.Add(result);
        }

        return results;
    }
}

This page object doesn’t have a direct entry point, as this page cannot be reached directly without a search being performed. Instead it is reached from the search page object. I have added some extra functionality for performing the search and initialising the search results page object. We will see these in a moment.

And the step definitions for the new steps. Remember that any identical steps share an implementation.

[When(@"I select the ""(.*)"" region")]
public void WhenISelectTheRegion(string region)
{
    searchPage.SelectRegion(region);
}

[When(@"perform a search")]
public void WhenPerformASearch()
{
    searchResultsPage = searchPage.SubmitSearch();
}

[Then(@"The search results page is displayed")]
public void ThenTheSearchResultsPageIsDisplayed()
{
    Assert.IsTrue(searchResultsPage.IsCurrentPage());
}

[Then(@"the following results are displayed")]
public void ThenTheFollowingResultsAreDisplayed(Table table)
{
    var results = searchResultsPage.GetResults();
    var i = 1; //ignore the header

    foreach (var row in table.Rows)
    {
        Assert.IsTrue(results[i].Contains(row["Event Code"]));
        Assert.IsTrue(results[i].Contains(row["Event Name"]));
        Assert.IsTrue(results[i].Contains(row["Region"]));
        Assert.IsTrue(results[i].Contains(row["Description"]));
        i++;
    }
}

The additional methods for the search page object are:

public string GetSelectedRegion()
        {
            var regionElement = new SelectElement(regionDropdown);
            return regionElement.SelectedOption.Text;
        }

        public void SelectRegion(string region)
        {
            var regionElement = new SelectElement(regionDropdown);
            regionElement.SelectByText(region);
        }

        public CrazyEventsSearchResultsPage SubmitSearch()
        {
            submitButton.Click();

            var searchResultsPage = new CrazyEventsSearchResultsPage(driver);
            PageFactory.InitElements(driver, searchResultsPage);

            return searchResultsPage;
        }

The SubmitSearch() method initialises the search results page object ready to be used in the scenario steps.

In conclusion…

Page objects are a powerful way to overcome the traditional problems with automated browser tests. When Selenium is coupled with SpecFlow they allow for a TDD style workflow that can give true outside-in development.

Another useful side effect of using SpecFlow scenarios is that the steps create an API for the application. Once the steps have been created it is possible to create new scenarios without any coding at all, which means that it is accessible to members of the team who might have otherwise found implementing tests a bit too developer centric.

Even the implementation of the page objects is not generally too complicated once the pattern has been established.  I appreciate that there is a bit more of a learning curve but I think this type of testing offers far more that the record and playback type behavioural tests and would be a useful skill for all team members to have.

Finally, as with all code, do not hesitate to refactor the scenarios, steps and page objects. There are lots of things that could be refactored in this simple example. A couple of the obvious things are the hard coded URL and logic that is used to get the data from the search results page. I will leave these as an exercise for the reader.

The series in full:

Advertisements

20 Responses to Behavioural testing in .Net with SpecFlow and Selenium (Part 2)

  1. Pingback: A Smattering of Selenium #124 « Official Selenium Blog

  2. Pingback: The easy way to runSelenium and SpecFlow tests in a TeamCity build with IIS Express « James Heppinstall: On Development

  3. Pingback: Behavioural testing in .Net with SpecFlow and Selenium (Part 1) « James Heppinstall: On Development

  4. Pingback: Managing SpecFlow acceptance test data with Entity Framework « James Heppinstall: On Development

  5. Really nice posts here James!

    First one I see that both manage to introduce SpecFlow in a nice way as well as promoting a maintainable structure for the automation code. Well done, sir!

    There’s really nothing that I would improve on this post. If you wanted to you could use the ScenarioContext.Current-dictionary to store your Page objects in (hence free yourself from private fields, see http://www.youtube.com/watch?v=IGvxMPX55vE ).
    That’s just a detail, that i like to use. No biggie

    Thanks for a great post!

    • jheppinstall says:

      Hi Marcus,

      Thanks for your comment.

      It is interesting that you mention using ScenarioContext, as it is something that my team and I are looking into as a way of increasing the reuse of our step definitions.

      As our suite of test grows we are finding that our current use of private fields doesn’t give us the flexibility we would like, and we are relying a lot on scoped steps.

  6. Pingback: Getting started with automated acceptance tests in an Agile project | James Heppinstall: On Development

  7. Chandana says:

    Hi James,

    Can you tell me how to execute scenario.context for multiple drivers?

    For example i want to execute same scenario with multiple Browsers. Is there any way that we can handle it?

    • jheppinstall says:

      Hi Chandana,

      There is no nice way to run tests against multiple browsers using out of the box SpecFlow functionality. Some people have suggested a few ways to do it but I don’t think it is really sustainable on a large project.

      The way I go about testing on multiple browsers is to use Selenium Server and browser clients and have the tests run as part of TeamCity builds against the different browsers. I am in the process of writing about how to use Selenium Server with TeamCity and it should be ready in the next few weeks so be sure to check back.

      Regards

      James

  8. Greg Ferreri says:

    Thanks for the informative post. What would you suggest for setting up the test data? For instance, in your search scenario, where would I write the code that sets up my database such that the call to the search page produces the expected results? How does that get incorporated into SpecFlow?

  9. Keith says:

    I found your article really interesting – i’m new to gui based testing, and was wondering why specflow and selenium offers so much more than simply using nunit and selenium.

    i can code for multiple browser support by using

    [TestFixture(typeof (ChromeDriver))]
    [TestFixture(typeof(InternetExplorerDriver))]
    public class TestMultipleBrowsers where TWebDriver : IWebDriver, new()
    {}

    and i can simply use your page class examples with test cases like this.

    ///
    /// A test case making use of page class.
    ///
    [Test, Description(“GIVEN I am on the google search page ” +
    “AND i have entered a search for ‘bread’ in the text box, ” +
    “WHEN I press enter key ” +
    “THEN i will get a list of matching pages and the page title will start with ‘bread’ “)]
    public void PageClassExample()
    {
    GoogleSearchPage searchPage;
    searchPage = GoogleSearchPage.NavigateTo(driver);
    searchPage.SendTextToBox();
    searchPage.SubmitTextInBox();
    var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
    wait.Until((d) => d.Title.ToLower().StartsWith(“bread”));

    Assert.AreEqual(“bread – Google Search”, driver.Title);
    }

    By using nunit directly, i can make use of row tests, ie TestCase, and i can reference input / output values by using the dynamic TestcaseData, and this appears to be much so more powerful than using specflow.

    What have i missed, why is the use of specflow and selenium simpler?

    Any advice will be appreciated.
    Keith

    • jheppinstall says:

      Hi Keith,

      Thanks for your comments. At the end of the day NUnit, SpecFlow and Selenium are just tools and its important to use the right tools for your situation. Whilst your NUnit example may have some advantages, SpecFlow offers me more in my particular project environment. I will try to outline a few reasons why it works for me:

      1) I don’t think that it is the responsibility of my test to specify the browser it will be running on. In my setup my SpecFlow test don’t know anything about browsers. In Fact they don’t know that Selenium even exists.

      2) I can get re-use from my test steps. In your example you cannot re-use the individual steps, so each test would require the full implementation each time. As my suite of tests grow it is possible to add new tests just by creating the SpecFlow scenario as each step already has implementation.

      3) SpecFlow is more than just wiring up tests. It is about capturing the behaviour of the system in such a way that it can be understood by the entire project team. NUnit lacks some of that readability. On my project testers as well as developers write the SpecFlow feature files and add scenarios for upcoming functionality (as well as wiring up the steps)

      It is possible to get similar functionality from NUnit (after all, SpecFlow uses NUnit underneath) but you will have to work for it. SpecFlow gives you a lot to work with and is easy to use, but it is not just about running tests.

      For me, if you are working on a large enterprise project with lots of acceptance tests you will get a lot further with the structure that SpecFlow provides. But as with most things, your mileage may vary,

      • Keith says:

        James,
        Really interesting points. I am still struggling with it – i need to make a choice for the framework approach my test team use, (and maybe my devs) and am trying to weigh up the procs and cons of both approaches. I’ve typically worked on api tests – so tend to end up with many 1000’s of tests via nunit.

        In answer to your points.

        1) Using the overload on the test fixture will simply provide me with a way of running both sets of browser tests (ie and chrome) via resharper or nunit gui. Disconnecting the knowledge of the browser from the test cases seems worthwhile, but your implementation code will still need to know which browser or tool its hooking into. In the nunit case i have the ability to use the category labels, test fixturesetup, or testsetup as a means of disconnecting the test cases from the actual browser knowledge.

        2) step re-use is interesting – its really very easy to do this via code isn’ it? In my example “GIVEN I am on the google search page ” then i’d use the test fixturesetup to ensure all tests in the class started in the right place.

        3) I partially agree with you here – but by making use of the nunit Description anyone in the project team can see what the test intent is. By making use of the category flags, and projects / assemblies the nunit gui can end up providing pretty clear information about what you are testing where, and the given, and when, then details of the tests themselves are visible via the properties display.

        I can see that spec flow makes it simple for non technical minded people to see the test intent, (as well as the test data) but is that enough to justify the extra process steps? If they wish to modify test input or output data – then i suppose it provides an easy way to do this.

        However, I think i could fairly easily provide dynamic data in / out (testcasedata) which can be pulled from config files or sql tables or anywhere else – so i can allow users to specify scenarios, alongside data, and allow them access to update input / result data as required.

        At the moment i’m still leaning towards the nunit approach, as it seems to be potentially so much more powerful / flexible than the specflow framework and less steps involved in the longer term, and doesn’t create the same volume of template code behind the scenes.

        I can see that the rest of the test fraternity love specflow / cucumber, but i can’t still quite see the killer benefit it will provide.

        Many thanks for your articles – very useful and interesting.
        Keith

      • jheppinstall says:

        Hi Keith,

        I am not sure there is a single killer benefit, just several small benefits that add up to a better experience, especially as the number of tests increases.

        All you points are valid, and you could use NUnit’s various features to create an approximation of what Specflow does. But that is what SpecFlow does anyway so why not let it, and make use of its other features? This is what SpecFlow exists to do.

        Back to the points

        1) The tests know about an injected driver instance but they do not know what type of driver it it. This is set in config so we can run Firefox locally and Firefox and IE on the CI box.

        When you have a few tests, running them from you own development environment is no big deal but you will soon reach a point where it is not sensible to run all the tests as they take several hours to run. It would be sensible to run them as part of you CI routine but you don’t really want your CI machine (for example your TeamCity Agent or TFS box) to churn through hours of tests so you will need to do something about that using Selenium Server and Client machines. You will need to think about think whichever approach you decide to take.

        2) As before, you could make it work with NUnit, but SpecFlow has this covered. Just writing the scenario in the feature file would get you the re-use. I am really interested in the ability to create new tests without having to write any code at all other than adding a new scenario.

        3) The natural language feature files are not just about test intent. They are about decoupling the scenario from the implementation. In my team the features are written before the development starts. We use the scenarios to drive the development from the outside in. It is part of the wide BDD practices.

        One thing I think is worth considering is that scaling acceptance tests is hard and required a bit more process to get the best from it. We are at about 500 UI driven acceptance tests and we have had to work hard. Often the initial resistance kills this kind of testing before it starts to deliver the benefits.

        An other thing is that you absolutely must use the page object pattern (a grand name for a simple thing) to encapsulate your pages whichever approach you do. You simple will not succeed without doing this. Do not the Selenium leak into your test code.

        It can be a big commitment but it is worth the effort in the long run.

        I would try SpecFlow on some real scenarios and see if it help you make up your mind. The usual small examples are not generally realistic enough for you to get a real feel for things.

        Cheers

        James

  10. David says:

    Hi James,

    Nice post. Very informative and easy to understand.

    At present I use SpecFlow and WatiN. My decision to use WatiN is much like you’ve already mentioned, it seemed to suit my situation.

    When writing behavioural tests I prefer reusable steps.

    e.g. [When(@”I click the “”(.*)”” button”)] can be handled using WatiN by browser.Button(Find.ByText(buttonText)).Click();

    This suits me as it’s not tightly coupled to any page and I don’t have to profile pages as shown in your example i.e.

    [FindsBy(How = How.Name, Using = “Submit”)]
    private IWebElement submitButton;

    I’ve barely scratched the surface with BDD so my question is if the same can be achieved using Selenium? Others in our field do tend to lean more towards using selenium.

    Thanks

    • jheppinstall says:

      Hi David,

      As far as I know Selenium and WatiN have very similar features so pretty much anything that is possible in WatiN should be possible in Selenium. I have’t used WatiN much so I can’t make a direct comparison, but I find Selenium to be very capable especially in an ‘enterprisey’ environment (e.g. running large test suites across different browsers with Selenium hub and clients).

      The reason to use the page object model is not to make the coupling tighter, in fact quite the opposite. It is about encapsulating change. Having a reusable step at the level of a button press will start out fine, but you may find it to be a bit too finely grained. For anything but a simple scenario you are going to end up with dozens of steps:

      Given I click the username field
      And I enter a value “username” in the username field
      And I click in the password field
      And I enter the value “password” in the password field
      And I click the “login” button
      Then …

      It is better to have a Method on the Page that encapsulated the login process so that the steps becomes:

      Given I have logged in with as “Username” with password “password”
      Then…

      Also if you change the text of a button (in reality it is better to use Ids that text) you will have to change all the tests that reference the button. This makes a test suite very brittle, which is one of the main arguments used against automated acceptance testing. With the Page objects you only make the change in one place.

      I will go as far as to say your automated acceptance testing will not work for anything other than trivial examples unless you use the Page object model.

      If you have any more questions give me a shout.

      Cheers

      James

      • David says:

        Hi James,

        Thanks for your reply. Some interesting points to ponder over.

        I think I may have oversold myself as wanting everything reusable. My main aim is to remove developer intervention where possible, hence my reluctance to profile each page.

        I agree with the steps becoming too finally grained. When it comes to repeatable steps like login I tend to use the following approach

        [Given(@”I am logged in”)]
        public void GivenIAmLoggedIn()
        {
        Given(“I am on the index page”);
        When(“I enter my username andd password”);
        And(“I click the login button”);
        }

        Username and password are defined in my app config so they can be changed depending on the environment i.e. local, test, ua etc.
        If I’m honest I’m not hugely fond of that approach however 🙂

        Regarding using ids rather than button text. I agree absolutely when using your approach that this would be best.

        However personally, if the button text changes, I think I want my tests to break. I appreciate that maybe I’m not able to appreciate the impact on testing an enterprise level application (many tests) but using BDD from the start of the project and not just as a retrospective action (as I see a lot) the scripts become (I think as intended) a form of documentation. They tell the business we understand how each feature should behave. Certainly we are using some tests to ensure the correct legal error message appears when appropriate.

        I guess the fear is the tests could become confusing to what’s actually happening on the screen when being reliant on ids.

        I see a lot of test steps as being trivial i.e. click this element, enter this text, check for this message. Things that should not require a developer to get involved every time. A lot of tests can be written therefore without the need of a developer. For everything else the developer gets involved. Maybe it’s just a question of available resource! 🙂

        I appreciate the discussion James.

  11. Pingback: Confluence: Team Service Manager

  12. Pingback: Testes de interface com PageObjects e Selenium ‹ 100loop.com

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: