There are three types of test suites used in Microsoft Team Foundation Server (TFS) for organizing test cases: regular, query-based, and requirements-based suites. Each has its advantages and disadvantages, and though none of them are perfect, using requirements-based suites as your primary means of test case organization allows for the best combination of traceability and usability.
Regular test suites are essentially folders; they can contain either test cases or additional test suites. They’re the most straightforward way to organize your tests, since their contents are not dependent on queries or on the test cases being linked to work items, and you can create the test cases directly within the suite. However, there’s nothing automatic with regular test suites. Any test cases created outside of the suite must be manually added, and test cases created in the suite must be manually linked to the work items that they test. While certainly good for basic organization, they have limited integration with the rest of TFS and no automatic organization or insertion of test cases.
To create a query-based test suite, you define a TFS query, and the resulting suite will contain any matching test cases. These are great for finding a subset of test cases that you want to run on a regular basis – for instance, finding all test cases in a given sprint. However, when using query-based suites, you cannot create new tests directly within the suite, nor can you remove test cases from the suite without changing the query or test cases. If you want to create a new test case for the suite, you must ensure it is created in a way that allows it to be captured by the query, which can be problematic if multiple people are creating test cases. Furthermore, if a test case is covered by multiple queries in the same plan, it will show up multiple times when you run tests, and will affect the test results accordingly.
Requirements-based suites are a good middle ground between regular test suites and query-based test suites, though they’re not without their flaws. When creating a requirements-based suite, you select a work item, and the suite contains all test cases that are linked to it. Unlike a query-based suite, you can create a test case directly within the suite, and unlike a regular suite, the test case will automatically be linked to what it tests. Additionally, any linked test cases created elsewhere in TFS will automatically display here, making it easy to ensure all work items are being tested. However, since you have a test suite for each item, it’s easy to end up with a large number of suites, each with only one or two test cases – though this can be mitigated somewhat by using regular suites as folders to organize them. Additionally, you’re likely to end up with duplicate test cases in your test plan, as any test case that covers more than one work item in your test plan will appear in multiple suites.
Even with their flaws, however, I find requirements-based suites to be the most useful. While regular suites are good for basic organization and query-based suites are good for finding a specific set of test cases to run, requirements-based suites provide a good mix of both to allow you to manage and execute your test cases. When organized into folders using regular suites, it’s not too difficult to manage a large number of requirements-based suites. As for duplicate test cases, they can be managed by marking the duplicate cases to not be run and by ensuring that your test cases are divided appropriately among your testers. Ultimately, if having a high level of traceability is a priority for your team in cases such as custom application development, using requirements-based suites is the best way to achieve that goal while keeping your test cases easy to manage.