When I have the time, I’ll be using this space for all my non-testing adventures.

One of the key points about the learning objectives and the road maps is that they don’t instruct you in how to go about testing. The idea is to define what needs to be learned and to leave the how to be defined by context.

Here is an example learning objective from the automation discipline road map:

Understands how change impacts automation tests

This learning objective does not tell you how change impacts automation tests. That is outside the scope of what you need to learn. There is more than one way to write an automated test and each of these ways have pros and cons that impact the severity with which change will impact an automated test.

As the learning objective cannot document each and every way that change could impact an automation test this information is left for the tester to research and learn. How is that to be done? Research, books, blogs, peers, mentors and trail, error and practice. The same way we learn anything else.

The main goal of each road map is to help you identify what you can learn rather than telling you how to test. Start here if you want to learn more about testing.

The fundamentals description is: a set of learning objectives that support all disciplines of software testing. It’s the common techniques, responsibilities and concepts that we use as testers.

Here is the link to the current fundamentals information. In future the fundamentals will just be mentioned within the nine disciplines.

These are the eight starting disciplines of a learning framework that I hope will go a long way towards solving our three problems: a lack of direction, an information deficit and retention.

• Behaviour & Functionality
• User Interaction
• Business Domain Knowledge
• Management
• Performance
• Automation
• Infrastructure & Integration
• Security

I’ve covered these many times since first proposing my learning framework. For full details please go to this link. I’ll briefly summerise the disciplines for those don’t want to break the flow.

Behaviour & Functionality embodies traditional testing scope with an emphasis on the behaviour of the system. Behaviour is defined as the response of a system to interaction. User interaction is not covered here.

User Interaction covers testing the user interface, usability and the user experience. The user interaction tester knows that the success of a system extends beyond features sets and functional correctness and a system that helps a user achieve their goal can be more successful.

Business Domain Knowledge is about applying the testing skills set to the business domain. We ask questions like “what is the impact of software failure on business or on society as a whole?”.

Management is where the test lead and test manager exist and the skills they bring to the profession.

Performance is about looking at the behaviour of the system under a variety of controlled circumstances. Performance testing is not just looking at the implementation but covers architecture design, network topologies and infrastructure provisioning. A performance tester should be able to identify performance bottlenecks when they are designed rather than after they are built.

Automation covers both the facilitation of testing and the verification of behaviour. The verification of behaviour is limited to what can be easily automated; services, APIs, batch processes and web interfaces.

Infrastructure & Integration looks at the interactions between software components and the interactions between software and hardware.

Security weighs up the competing concerns of protecting against external vulnerabilities and allowing the users to be effective in the system.

The first four disciplines are non-technical while the latter for are technical. By technical I mean that a deeper understanding of the implementation if required rather than knowledge of a programming language. An important distinction to make as I know testers who are interested in technical testing but are not interested in learning programming, a common assumption.

In time these disciplines may split into more refined specialisations and they are not mutually exclusive. The more a tester knows about each discipline the more effective they can be. Each discipline is a way of focussing our thoughts into related topics. Next I’ll cover what is within each discipline.

[to be continued...]

Each discipline consists a set of learning objectives that a tester should eventually learn. The goal is that each learning objective helps guide the tester in their studies.

The learning objectives cover five things:

Concepts – theoretical constructs relating to the discipline. For example the performance discipline has “understands what CPU starvation is and how it impacts performance.”

Responsibilities – cover what, as a tester, I am responsible for. What I should be covering when I test a system. The business domain knowledge discipline has such responsibilities as “the identification of systems that require failure evident capabilities but do not feature them”

Techniques – are different ways of approaching a problem. For example the user interaction discipline has ‘hallway testing’ technique which can be used to identify user interaction issues; however hallways may not be available, as such the ‘think aloud’ technique can be applied for the same goal.

Artefacts – are the manifestations of the work performed by testers; this includes test cases and defects and test plans and performance test plans.

Lenses – are different ways of looking at a system within the bounds of discipline; this help the tester narrow their focus of the system. Each lens is applied one at a time. The behaviour and functionality discipline has such lenses as: Behaviour, Functionality, Compliance and Data Integrity

[to be continued...]

I’m going to talk about the lack of direction a little bit more.

At present no certification, nor the Association of Software Testers currently define what we do when we test.

What I mean by this is; when I am asked to test an API, there is nothing that tells me that I need to consider the command/query separation principle and that violations of it should be raised with the developer.

In terms of what they do provide the certifications only have vague notions of specialisation. The ISTQB has test analyst, technical tester and test manager but as you will soon see these categories are too broad to be usable and the still don’t provide the kind of guidance I mentioned before.

Now, this lack of guidance is not necessarily the fault of the certifications; they have an emphasis on the process of testing. Not what to test.

In terms of advancement certifications only provide simplistic notions. The ISTQB has foundation, advanced and expert levels while the QAI has associate, practitioner and manager. This last advancement tier is tied into specialisation. These rudimentary advancement models and the lack of emphasis on career development showed up in my research: 64% of those with a certification felt that even with a certification they still have insufficient information to achieve their career goals.

The companies that hire testers also use simplistic notions of advancement with terms such as Junior and Senior. These are not appropriate ways of indicating competency. They are based on the needs of an organisation at a given point of time; not some measure of competency as defined by the tester and the profession.

To overcome the issues of a lack of direction, retention and an information deficit we need three things:

1. A structure that provides clarity about what we do when we test

2. A set of specialisation that reflect that diversity that exists within the field

3. A framework that realistically allows us to assess our competency

I’ll cover these tomorrow.

[to be continued...]

Naturally she didn’t appear to be enjoying herself.

So I went around to her desk one day to find out how she was going. I asked her: how are you going in testing, do you like it? hate it? are they aspects you like or interested in?

“I hate it” she said; “I hate it and I am going to leave.”

So I offered to talk to her about testing, what options I felt were available in the hope that we could find a new direction for her and a better suited mentor.

I showed her a diagram of each of the different disciplines I consider in software testing; it’s like this linked imaged and I asked her are you interested in being involved in the process of creating applications that help the user achieve their goals? Or are you interested in the performance or security aspects of a system? Perhaps your interested in the interactions between software components or the interaction between software and the hardware that it runs on? Or does considering the business implications of failing software or the implications of software failure on society as a whole? Or would you just like to focus your energy on just making sure the software works?

“I didn’t even know there were different specialisations in software testing. Nobody told me that.” She said. “And I’m interested in security testing.”

She was the first person in the organisation that had shown any interest in security testing.

And she wanted to quit because she didn’t even know it was an option.

And she wasn’t the first person I have met who had the same problem. So I undertook some research into how people view themselves in software testing and how they viewed their introduction into the profession. The results are linked here; but I’ll summarise the important parts below.

70% of software testers were unaware of the different specialisations that exist at the start of the their career. This is a lack of direction.

The majority of software testers surveyed felt they have insufficient information to achieve their career goals. This is an information deficit.

One in five testers who are leaving the profession are doing so because of this lack of information. This problem is retention.

[continued...]

It’s been a hectic seven weeks since the end of my recent travels; I’ve moved cities, found a place to live, spoken at two conferences and delivered a couple of workshops. One a dedicated session for the ThoughtWorkers in China regarding career development and my learning framework and the second on the User Interaction discipline; if I can I’ll post what was interesting about the session. I’d never given a workshop on such activities before. It was great fun.

The language barrier was tough and I fumbled a few times; it is difficult to determine if someone with English as a second language is understanding you. In the end they did and I should not of worried.

For those that were at the conference; this post is to push the topic onto the first page. It seems the side bar doesn’t show up on the iPad and therefore finding the links difficult.

The paper.

The best place to start.

Just to make this clear. The goal of the tester is to help build software; we do this by providing information to stakeholders about the software. In some cases the people paying for or building the software may not give a shit about the end user. Do you think testers working on a gambling system are really looking out for the end user as the user’s food money is being taken from them? Any tester on such a project should probably look up ethics at this point.

If you’re interested in the impact of software on people then look at all the ways that the software can interact with people. The obvious; the person at the terminal is user interaction. You also need to consider the people that manage the systems that run the application.

Take a step back and consider the impact of software on the business that it runs in. If a policing organisation implements a system that allows their operatives to store case information digitally there are many benefits. But if they have to use it at their desks when previously they could have filled some of it out on the road then those hours at the office are either going to come out of their time on the beat or they’re going to be working longer hours.

Take another step back and consider the impact of the software on society. What is the impact on society if the police were on the beat for an hour less each day? What is the impact on the families of those officers if they have to work an additional hour?

Software systems aren’t just users and terminals anymore, they haven’t been for a while; they’re integration with society can have non-obvious effects.

Identifying the benefit or impact of failure of a system is easy in comparison to the unexpected side effects of software.

There is more than one end user. Thinking otherwise is like wearing blinkers.