In the life of a project when is time most precious? The first day? Or the last day?
Writing unit tests makes sense, but are you disciplined enough to write them? I understood the value of writing unit tests and had used them on numerous projects. I knew you should write them but I had a hard time articulating their value. I no longer have that problem. Here is an experience that will forever have me not just writing unit tests, but using Test Driven Development (TDD).
Then it happened – a moment that still causes my heart to race when I think about it. A serious, show stopping performance issue was uncovered.
The project had been running for months and now it was the Thursday before Thanksgiving. The scheduled release for the web site was the 1st of December, just days away. A release before the end of the year was required as hundreds of thousands of marketing dollars were committed for that timeframe. Then it happened – a moment that still causes my heart to race when I think about it. A serious, show stopping performance issue was uncovered. The project manager said he would have to tell senior management that we’d have to slip the release until after the middle of January. The performance issue appeared to be in the data access layer. Touching that layer would invalidate over a month of exhaustive testing. Merely performing all the tests again after a fix, assuming we could fix it, would take the project beyond the end of the year. I pleaded with the project manager to give us a few days to analyze the problem. He agreed to give us until the following Wednesday – the day before Thanksgiving. The work began…
The performance problem was perplexing. We ran tests early in the project to prove the approach was viable. But we clearly saw performance concerns when running significant load tests. We went to work with profilers to identify the bottlenecks. After days we finally thought we had a fix. We made the change and ran our unit tests. At this time we had over 94{f073afa9b3cad59b43edffc8236236232bb532d50165f68f2787a3c583ed137f} code coverage. After the first change over 80 percent of the tests failed. But we discovered this in less than 10 minutes! After two more changes we felt we had the issue solved. Before the deadline we were able to show significant performance improvements. But what about all the testing that needed to take place? I reviewed our unit testing strategy with the project manager. That boosted confidence in the changes. But how could we be sure. After many discussions we agreed on a plan. We would have as many testers as we could spare hit the site and test for two days. If we encountered any issues we would then tell senior management that we would need to slip the release past the end of the year. Not a message I wanted to deliver.
After two full days of exhaustive testing by dozens of people the results were in… Not one bug was found!
After two full days of exhaustive testing by dozens of people the results were in… Not one bug was found! Due to some other changes the release was pushed until December 15th, but on that day the site went live. (Sidebar: By the way… The site was actually fast enough before we discovered the issue. As a matter of fact, the site was so fast that the load testing software caused wild fluctuations on the server that are impossible to achieve in real life scenarios. However we made the site even faster and improved memory usage on the server.)
This series of events convinced me. I was glad I had taken unit testing on faith. I know with certainty that if we did not have those unit tests we would not have shipped the product on time. Since then I’ve taken unit testing much more seriously. I now use TDD. I now target 100{f073afa9b3cad59b43edffc8236236232bb532d50165f68f2787a3c583ed137f} code coverage (for the non-presentation tier) and I work to minimize code required for the presentation tier.
So now back to the question… When is time most precious? It was clear in my example that when we got down to the final days of the project we did not have time to spare. But my answer now will surprise you. With test driven development I no longer fear the final days of a project, but instead approach the end with great confidence. Now EVERY day of the project is precious and moves me one day closer to a successful conclusion. An effective test strategy allows me to equalize the days.