We are moving to an Acceptance Test Driven Development approach for defining features. It seems to be working well, but we're starting to run into issues with test management. At the moment, we use SharePoint/Excel to track the acceptance tests. This is because non-technical customers, QA and dev all update the tests. The problem is that the tests don't live with the code, so they aren't branched/versioned along with the code, and it is all very manual. We're looking at full on test case management software (e.g., Zephyr, TestRail, etc), but that feels a little heavy, and ultimately the test data still doesn't live with the code.
Is there a test management application that is friendly to non-devs, but stores data in a way that will work with git? Is trying to keep the tests alongside the code a fool's errand?
Thanks, Erick
Yes, I like your idea to keep the tests together with the code.
One framework that you could use to achieve this goal is FitNesse. It is a well known test management tool that supports collaboration between business guys, developers, and testers. With the Git Plugin for FitNesse you can keep your tests within a Git repository.
We maintain our acceptance tests, which are based on another framework named Concordion, also in our source code repository. Concordion specifications consists of two parts: a well-formed HTML document describing the functionality, and fixture code written the programming language of your application such as Java or C# that finds concrete examples in the document and uses them to validate the system under test.
Our executable specifications are written with a WYSIWYG HTML editor such as Microsoft WebExpression or BlueGriffon by the product owner or testers. We keep them in our repository and access them via TortoiseGit, which works very well as all members of our team have technical background. To involve non-technical you would probably need to write a small script that pulls the latest updates into the local repository, start the editor of your choice, and push the changes after editing back into the central repository.
Please, note that there is a Concordion extension for Excel available, which enables to specify your test data (i.e. the input and expected output values) in an Excel spread sheet and use this to test your application.
IMHO testing is often some sort of detailed analysis in software projects. As many details are not discussed and specified, testers do often the job of thinking about the details of a feature. Unfortunately, this happens after developers did their work, which causes a lot of rework and additional cost. Thus, base on your ATDD effort you should try to improve the collaboration of your team on specifications before you head for test automation.
Please, have a look at a talk of Gojko about the benefits to invest 5-10% of a sprint to define details collaboratively. The specification work in our team has improved a lot since we do specification workshops based on the ideas of specification-by-example.