Issue 48

The Testing Map – Processes and standards

Claudiu Draghia
Quality Manager @Capgemini

The Testing Map aims at covering, in visual form, the most important information a software tester should know. There is one area which tackles processes in particular. I will talk about this area in the lines that follow.

You are not entitled to your opinion

Nobody likes "processes". They are boring and it is very difficult to see any value in following them (I hate to say it, but it is very often the case). They have the flavor of "just things we have to do" because someone wants this to happen. Processes somehow seem not to fit software development properly. Developers and testers are craftsmen. How can you add a "process" to a craft? Although the answer might seem challenging, it is actually very simple: no one wants a bug-free code (except perhaps developers and testers), but everyone wants a working product. In order to have a working product, you need to put some rules on the table. There are a lot of disciplines, beside development and testing that make or break a product. Working as a team can be done only by knowing what is expected of you, what we have to deliver. It is true that, in software development, there are a lot of useless rules that come from a postindustrial mindset, and that cannot perceive testing and software development are crafts.

We are to blame for accepting the situation, in many cases. If we want to change the situation, we should no longer be entitled to our opinions. Yes … "You are not entitled to your opinion. You are entitled to your informed opinion. No one is entitled to be ignorant." (Harlan Ellison)

If I said that the waterfall model "is risky and invites failure", most of you would agree with me. But what if I tell you that this statement comes from the paper "Managing the development of large software system" by Winston Royce - the father of the waterfall model. It is on the second page of the paper!!! It is the first statement after the classic waterfall drawing:

What if I told you that Jeff Sutherland, one of the fathers of SCRUM, states, in his book "Scrum: The Art of Doing Twice the Work in Half the Time", that he did not intended for "SCRUM to become another process". What if I told you that you can listen the free book on Amazon Audible? Would you listen to it? Do you know that when the Agile Manifesto was written (2001) "…Scrum was pretty much unknown at the time. It was only with the advent of Ken Schwaber's 2-day Certified Scrummaster Certificate, several years later, that Scrum grew in popularity …" (Alistar Cockburn).


Processes become annoying and useless when the purpose is not clear. When people do not understand the reasons, do not search for the reason and refuse to fight the status quo. It reached a point where standards stepped in, to make sure you do a good job as a developer, tester, analyst…. A standard dictating what you should do is an affront to anyone that values their individuality.

There are good and bad standards, depending on the context. But why do we need standards? In Wikipedia's article on the History of software engineering, the period ranging between 1965 and 1985, is entitled "The software crisis ". Many projects ran over budget and over schedule. Some projects caused property damage. A few projects caused loss of life. Loss of life is not an overstatement: the Therac-25 was a radiation therapy machine. It was involved in at least six accidents between 1985 and 1987. Patients were given massive overdoses of radiation because of concurrent programming errors.

In 1980 we got ITIL (IT Infrastructure Library) and in 1989 we got CMM (Capability Maturity Model). The standards were built on the 2 ways of selling a software program: as a service - ITIL or as the source code- CMM.

It might seem strange but both standards were required and were funded by governments: ITIL in the UK (through the Central Computing and Telecommunications Agency) and CMM in the USA (the main sponsors were the Office of the Secretary of Defense (OSD) and the National Defense Industrial Association).These standards came out of the need of software buyers to have a better chance of getting what they paid for.

Standards are not that bad. What is really bad is the way they are understood or applied. You might see them as a collection of best practices (what someone thought is best), but there is no such thing as a best practice: there are only good practices in context. In order to use a standard, you need to know the context (of the company or project) and you need to have a solid understanding of the standard in order to make sure a practice is implemented properly. We have to leave it to another article to clarify the difference between knowing and understanding.

You should stay away from folklore. In order to understand a standard, you need to explore it, find reliable people that can give you valuable information. You need to do research, not to rely on folklore.

One thing to know about standards is that they are built to be used and applied in different contexts. Their language might not be easy to understand or apply. Let's take an example from ISO 9001:2008.

The chapter Control of monitoring and measuring equipment includes the following statement: "...necessary to ensure valid results, measuring equipment shall: … be calibrated or verified, or both, at specified intervals, or prior to use, against measurement standards …". The first thing that I thought about was that this does not apply to software development. This is something for factories. I could not have been more wrong. Think about how you monitor production system: there is a calibration or verification step that you do. Think about the moment when you run performance tests: you evaluate the performance based on a standard you define (a benchmark). There are plenty of ways we do this in software development, but not using these words: measuring equipment, calibration, measurement standards. Again, it takes context knowledge and standard knowledge to do a proper implementation.

Since I have put the ISO subject on the table, have you heard about ISO 29119? I can bet you have not heard of it!!! It is an ISO standard for testing. It costs around 160Euros just to see it. The feedback from the testing community was really bad. There is even a petition called "Stop 29119", signed by some big names form the testing world.

I like standards. I believe in some of them. I believe they can help you if you understand them. Most of the problems they cause are due to misunderstanding and different, or wrong, interpretations. Most of the problems are caused by folklore. This is way I strongly believe, that when it comes to standards and processes: "You are not entitled to your opinion. You are entitled to your informed opinion." (Harlan Ellison)

P.S. No standard requires that you do all the practices. There can always be exceptions if a clear motive is given.


  • ntt data
  • 3PillarGlobal
  • Betfair
  • Telenav
  • Accenture
  • Siemens
  • Bosch
  • FlowTraders
  • MHP
  • BCR
  • Itiviti
  • Connatix
  • UIPatj
  • MicroFocus
  • Colors in projects