Headline: The only real way to be Agile is if you have healthy automated testing. (See Unicorn Series Overview for other topics.)
Results
Let’s start with some results of the practices described here:
- Quality: Customer reported bugs range from 1 to 10 per year, despite thousands of users
- Cost: There is no dedicated testing team
- Speed: Our objective (usually met) is to fix a bug within 1 day
Yes – better, faster, cheaper.
Overview
The most agile teams seem to have short sprints. Ours are 1 week. Yes – 1 week. On any given week we could release a new version of the software fully tested and ready to go.
I once worked on a team that required 2 months of testing after the last line of code was changed. (Ok… we all know that at least a few lines of code change each week…)
Why did it take so long to perform the tests? They were mostly manual tests.
Automated Testing Is Required
So the only way to make sure you can release is to complete your test plans which is only fast when they are automated.
Our Automated Tests
We have several layers of automated tests:
- C# Unit Tests
- Jasmine (via Chutzpah) TypeScript tests for AngularJS code.
- Web User Interface Tests (via Selenium)
All of the C# and TypeScript tests run with every CI (Continues Integration) build. After each successful build, we run a CD (Continuous Deployment) release we run all the Web tests before swapping the slots in the Azure environment. After every commit a developer makes, all tests are run from these three layers so we catch failures almost immediately.
After every commit a developer makes, all tests are run…
These automated tests (C# and TypeScript) cover more than 90{f073afa9b3cad59b43edffc8236236232bb532d50165f68f2787a3c583ed137f} of the code base.
Other Benefits
Other real benefits from this strategy include:
- It is fairly easy to refactor code as requirements change with this level of test coverage. (See A Strategic Case for Unit Testing)
- It is faster and easier for new developers to the team to contribute because 1) because the unit tests document the code, and 2) they can code with limited fear of breaking things.
I’ll work to be a bit more active on the Unicorn Series in the months ahead.