There Will Be Bugs! | Application Development Blog Series

Best practices for minimizing the impact and mitigating costs associated with fixing software bugs

First of all, let’s clear something up: what is a bug? A bug is a flaw in software that results in the application not functioning as intended, and is something that is theoretically preventable.

There will be bugs in all of the software applications that you use in your business, whether you custom develop them or buy them “off the shelf.”

Just let that sink in for a moment.

Changes, enhancements, or new features will appear periodically due to changes in the needs of the business, but they are not bugs.  A change (or enhancement) is a result of a new requirement that was not known up-front.  Frequently, owners of custom applications will feel like they have a “bug” because certain business rules are not applied in the desired manner, however these are often changes or undocumented requirements (i.e., not bugs).

Bugs are an inevitable aspect of any software development project, as all software is created by imperfect humans.  There are a variety of techniques that development firms like Entrance use in the application development practice to detect and eliminate bugs before they go to the production environment which helps minimize and mitigate the impact both to end users and the project timeline:

1. Well-written user stories with acceptance criteria – The most important step in preventing bugs is understanding what the application should do before any code is written.  At Entrance, we create a “user story” format to capture detailed requirements as part of our agile project methodology.  Additionally, it is crucial to capture acceptance criteria so that there is no ambiguity about the desired outcome of the feature.
2. Automated testing – Automated testing is one piece of software testing another piece of software.  It increases the efficiency of the quality assurance process while mitigating the likelihood of regression bugs.  In the same way that business software can make your office more efficient, automated testing allows basic component testing to be performed frequently and very quickly.
3. Quality assurance review – The most fundamental aspect of assuring quality is testing the software by a specifically trained individual who is dedicated to finding bugs before your users do.
4. User acceptance testing (UAT) – The final checkpoint for quality assurance is testing by actual users.  Entrance helps our clients through the UAT process through the creation of the user acceptance criteria and by facilitating the execution of the user testing process.

Entrance uses industry-standard tools such as Microsoft’s Team Foundation Server (TFS) to track and manage bugs found in applications that we develop. By tracking bugs in a detailed manner, we can calculate quality metrics and monitor our portfolio of application development projects.  Quality metrics allow us to identify trends, norms, and abnormalities so that we are able to keep all of our projects on track.

The cost of remediating bugs is addressed differently between off-the-shelf and custom applications.  In the Commercial Off-the-Shelf (COTS) environment, you either get “no warranty” (as-is) or you pay an annual “maintenance fee” (usually in the neighborhood of 20% of the software’s cost).  If you’re paying a “maintenance fee” then you’re pre-paying for bug fixes (think of it as bug insurance).  In the custom development world, as the application owner you pay for bugs but the cost is typically recognized incrementally as the bug appear over time.

There are different ways to manage the cost of remediating bugs that make it through to production.

1. Warranty – Warranties are not common in the software world due to the complex nature of business applications.  Custom software may be warrantied in certain situations, most commonly in a fixed-fee project where the vendor agrees to implement specific functionally for a firm price.  A warranty might also be offered for a separate fee as an option.  If a warranty is offered, expect the vendor to be very particular about what is or is not covered by the agreement.
2. Time and materials – In a time and materials scenario, the software owner will engage the vendor after a defect is identified and will be subject to the availability of the vendor at the time service is requested.  This option exposes the software owner to the most risk and is generally only advisable for software that is plainly not mission critical.
3. Retainer – Retainers tend to offer the best balance of cost and risk mitigation for most software owners.  A retainer relationship with a vendor guarantees the availability of the vendor up to a certain number of hours (typically per month) for a fixed fee and may provide a discount over base pricing when pre-committing to a sizeable volume.  Additional hours beyond the monthly hours included in the retainer fee are typically offered on a time and materials basis subject to the availability of the vendor.  The main advantage of a retainer is that you can be assured that the vendor will be available to address any business critical issues that may arise in a given month.  Depending on the structure of the retainer, hours not used for bug fixes or support may be available for other efforts such as enhancements.  Prepaid hours do not roll-over to the next month because the vendor has already committed resources in advance.