distributedlife

On this topic I am specifically talking about functional test automation as achieved by a tool like Selenium. Furthermore this could apply to any software development process. It’s just a bit more pertinent in the Agile world.

Now don’t get me wrong, I love automation. Automation when used appropriately is smart development. What I don’t like is throwaway statements made by test managers with a grand vision to automate everything.

Firstly you can’t. Automation tools do not come with the “did we build the correct system?” assertion. Validation is one half of what we are testing and this extends into the qualitative testing scenarios like usability testing.

When we talk about verification then we have a host of different problems. The second problem with automation is that it breaks under change. Agile is all about being able to handle change and if you have to reengineer your automated tests every time you want to refactor then it is painful for the automation developer.

The next problem is that automation scripts take time to write and suitably skilled personnel are required. This is a bigger problem than it looks. Most automation developers I have encountered are not overly fussed with software development concepts like code reuse, good coding practices or management concepts like source control and versioning. This last point is more about automation in general than directly related to Agile.

Automation tests do have value and does have a place in the software development lifecycle. They are very useful for evaluating your non-functional performance testing requirements. They help provide assurance that nothing as regressed in your application due to some other change. They help repeating tests across multiple platforms.

Where should automation developers sit in the lifecycle? In Agile it is my feeling that automation developers should sit a few sprints behind so that the likelihood of change is reduced. Manual testers can then provide feedback to the automation developers about which requirements are done and the work can commence on automating those scenarios.

The bottom line is that automation is good but is not a silver bullet.