Part 1 | Part 2 | Part 3 | Part 4 |
Testing has become an important topic in recent years thanks to the explosion of testing technologies and continuous integration (CI) approaches but also due to the need for an ever-widening range of tests for a variety of use cases. For many developers, understanding how to incorporate testing into their development workflows can be daunting due to the many terms involved and, worse yet, the many tools available both in software-as-a-service (SaaS) companies and in open-source ecosystems like Drupal.
Yuriy Gerasimov (Senior Back-End Engineer at Tag1 Consulting) presented a session at DrupalCon New Orleans about modern testing approaches and how to decide on the correct suite of tests for your software development workflows. In this four-part blog series, we analyze the concepts in contemporary testing approaches that you need to know in your day-to-day and why they can not only protect but also accelerate your project progress. In this first installment, we take a look at how to sell testing as an important component of client (your stakeholders) projects, as well as why automated testing is an essential component of any web implementation.
Why testing?
Many people around the web development landscape have heard of testing, but when you ask about real-world examples on real-life projects, the same developers admit that their testing infrastructures are sorely lacking. The most important reason for this is that while testing is a compelling value-add for developers, it can be difficult to incorporate testing as a required line item in projects, especially when business stakeholders are looking to save as much money as possible. How to sell testing to clients continues to be a challenging problem, especially because requesting that a client pay for an additional 100 hours on top of the 500 already incurred can be anathema to their sense of frugality.
After all, many customers will respond by arguing that by choosing you as an agency or developer, they already trust you to foster a high-quality outcome. As such, many clients will ask, “Why do we need to spend extra money on testing?” Without the overt benefit that project components like architectural workshops and actual implementation provide, testing is often the forgotten and most easily abandoned stage of a build.
A real-world example of the need for testing
In his talk at DrupalCon New Orleans, Yuriy describes a large project (prior to his time at Tag1) on which he collaborated with many other people on a development team tasked with finishing the implementation in six months. The project was for a local municipality, with many integration points and features. Every feature needed to work perfectly, including critical features for civic life such as applying for permits, and tolerance for dysfunction was low.
By the end of the project, originally slated for six months, Yuriy’s development team ultimately spent six months developing and an additional six months fixing issues and testing functionality. Fortunately for his team, the municipality had already been through a project whose timeline ballooned out of control, and the team was able to deliver the project within a year as opposed to the previous partner, who spent two years on the same project.
One of the most alarming aspects of the project at the time was that all of the testing the team had done until that moment consisted of manual testing sessions. A meeting was convened, and every developer stood up, responsible for describing the rationale for each feature they had built and demonstrating the feature. Every team member would then test each constituent feature and fix issues on the spot, in the moment.
Learning from past mistakes
As one can imagine, this manual testing approach is highly unsustainable for projects that require tight timelines with a high degree of confidence. Yuriy learned many lessons from the project, and in a subsequent implementation six months later, in which he and his collaborators built an application for people with hearing and speech difficulties, he made considerable changes. The project was complex, with several servers communicating with the application through REST APIs and a high expectation for a user experience that would allow users to click icons representing elocutions that would speak in their place.
From the beginning, Yuriy and his team baked in automated testing up front to test communication with the REST APIs and ensure all requests were functioning properly. They built the project to be scalable because they knew that many users would be using the application simultaneously. In the end, the quality assurance (QA) overhead was minimal, because developers on the team could simply run automated tests and show the result to the client. Even though the size of the project was roughly the same, having built-in automated testing with acceptance criteria was a benefit difficult to overstate.
Defending quality: Selling testing to customers
When testing aficionados attempt to sell testing to customers, they must frame the investment in terms of quality and long term vs. short term costs (failing to deal with this in the short term will actually cost you more in the long term). However, it is admittedly difficult to sell something when its success cannot be measured. After all, from the client perspective, a buyer is selecting a vendor based on the quality with which they implement projects. But there are only anecdotal metrics that indicate whether an organization performs better than another with high-quality projects. For this reason, it is essential that developers interested in selling testing as part of their contracts offer metrics that are comprehensible to the customer.
In the end, the sole concern of customers is that software is delivered without bugs. While ease of maintenance is also important, this is generally considered table-stakes among stakeholders (or a problem for the future). In order to provide a high degree of confidence for issue-free builds, we need metrics for traits like performance and code quality (like adherence to Drupal’s coding standards). Thus, when a customer asks about the justification of a metric such as code quality, we can show the results of tools like code audits, which in Drupal consist of a single Drush command that generates a full report. By performing a code audit on a codebase written by a less experienced team, for example, clients can be sold immediately on the value of your team by seeing the results of a highly critical code audit—and will seldom be opposed to your team winning the contract.
Automated testing
For many developers who are new to the concept of automated testing, the term can unleash a torrent of anxiety and concern about the massive amount of work required. This is why Yuriy recommends, first and foremost, building a testing infrastructure and workflow that demands minimum effort while yielding maximum dividends. Nonetheless, successful automated testing requires a robust testing infrastructure and a healthy development culture that is supportive. Without these elements, success is seldom guaranteed.
Fortunately, the up-front cost of automated testing is low owing to the “one and done” nature of automated testing. Though it’s likely you’ll spend a few weeks building out the infrastructure, there is no need to repeat the same process over and over again. Nevertheless, Yuriy recommends exploring the approaches that other software companies and industries undertake to understand how they tackle similar requirements. For example, automated testing for the C language has been around for many years. Moreover, there is no need to write our own continuous integration (CI) server, thanks to the wide variety of services available on the market, including software-as-a-service (SaaS) solutions that charge as little as $50 per month.
Even if you have written a large number of tests for your project, one of the most important aspects of automated testing may seem particularly obvious. If you don’t run automated tests regularly, you won’t receive any benefits. For instance, it is certainly not adequate to inform your customer that you have implemented automated testing unless you are running said tests weekly or monthly based on the project requirements. Otherwise, the value of the time you have spent implementing automated tests becomes questionable.
Conclusion
As you can see, testing is frequently the most easily forgotten component of web projects due to the extent to which clients question its value. However, armed with the right approaches to selling tests, you too can cultivate a culture of quality assurance (QA) not only within your development team but also for your business’s customers. With the help of automated testing, you can reduce the headaches for your team down the road and justify additional time that means extra money in your pocket.
In this blog post, we covered some of the important aspects of modern testing approaches and why customers are beginning to take a second look at the importance of quality in their projects. In the second installment of this four-part blog series, we’ll turn our attention to implementing testing infrastructures and fostering a development culture favorable to testing within your existing projects. We’ll discuss some of the maintenance costs associated with implementing automated testing and begin to look at two of the most prominent areas of testing: code checks and unit testing.
Special thanks to Yuriy Gerasimov, Jeremy Andrews, and Michael Meyers for their feedback during the writing process.