There's a lot of talk about standards with 29119 morphing out of the darkness like the T-1000 (and, if you're opposed to it, I recommend that you sign http://www.ipetitions.com/petition/stop29119). So I tweetered this:

Which was my way of saying that we all certify ourselves to do the right thing for our own contexts without imposition of structure from outside of that context. There's a couple of ways to go from here. One is to say "we do not need standards" which is true. Let's say, though, that we had a competing "standard" (with the commercial advantage of being completely free and built and designed by top-quality ethical testers. That's you, by the way) what could we put in it? Ethical principles to guide our application of processes? Heuristic suggestions? Anti-standard policies? I'm interested in what you think, but I'm not sure I know what I think yet.

Some reading materials*

http://clarotesting.wordpress.com/testing-standards-can-we-do-better/

http://clarotesting.wordpress.com/2012/06/06/iso-29119-the-new-soft...

http://www.uploads.pnsqc.org/2011/slides/Bolton_StandardsAndDeviati...

http://blog.johanjonasson.com/?p=774

http://pro-testing.arabuusimiehet.com/archives/354

http://pastebin.com/jtYbV2MN

* Because of the world I move in, and my attitude to current standards, most people I read on the subject are against it. If anyone has any good-quality pro-standards links they'd like me to include here please let me know, or include them in a reply.

Views: 993

Reply to This

Replies to This Discussion

Hi Chris,

The only problem with the ISO-29119 is that it's called a standard. Call it a heuristic and I suspect we'd all be fine.
Instead of throwing the 'standard' away and creating a new one, let's just ADD a 1000 different heuristics, approaches, templates, strategies, processes, models,... to it?

I'm guessing, however, that this is not how ISO operates.

In any case, I don't think we can come up with a standard that doesn't banalize our craft.
Take "Keyword driven testing" for example. It's a useful tool, but a horrible standard. It also sounds utterly ridiculous.

I thought this might come up, it's because I've phrased things badly, and that's my fault.

I agree, it's because it's a standard that's the problem, although not the only problem with it. I don't actually want to create a new standard at all, but I'm interested in what a non-standard "standard" would look like. A pseudo-standard, if you like. The trappings of a standard - an official looking code, a certification to make it seem official, the usual nonsense surrounding a standard, but with context-insensitive heuristics to help defend against the type of inflexibility found in standards. With that in mind I don't want to have a usable standard at the end, but I'm interested in what people would have in such a thing. I'm interested in what sorts of policies defend against seductive, dogmatic thinking and if there's a place for such a defence that might be taken up by companies who would otherwise want a standard certification. I also want people to think about standards, their weaknesses and strengths, and to stir up some interest and awareness in 29119 and whatever comes after it.

Your example, keyword-driven testing, is a particular technique/approach, whereas an ethical code can steer the choice of, and application of, techniques and approaches. 

So yes, not really a standard in the same way that 29119 is a standard. Let's call it a collection of personal policies or disclaimer-laden advice for the sake of the discussion. Or maybe reclaim "standard" in it's use as an industry norm, or value based off of a set of principles like humanism (in the way "gold standard" is used for currency).

Hi,

I'm definitely following your reasoning and intent. It has occupied my thinking as well. Not because of ISO29119, but because of other such instances before.

I'm afraid that it's not only testing that suffers from this dilemma. "How can we, as goodguy-sheep, save our customers from the big bad sales-talk wolf?"

What I gather from your previous post is that you'd want to illicit some form of "defence" against ISO29119 (and other such practices) so that companies who offer an alternative approach can still convince customers they should choose the sensible method.

I'm unsure that we can come up with short, clear, generalized standards/anti-standards or any other "thing" that has the same impact on the customer's choice while still staying truthful to our professionalism and craft.

We could, however, remind our prospects and customers of the first four principles of CDT.
We, the team of software developers, testers, analysts, stakeholders, managers and everyone involved are professional intellectual workers who are building towards a solution to a problem. This problem is subject to change, the solution is subject to change, so is the team, so is the customer. No one practice or standard can offer you the key to success, but working together as a team of professionals goes a long way.

Furthermore, I feel sorry for the less eloquent tester who have to counter the hardheaded person yelling "But it's a universal standard, why wouldn't we do it the same way?!".

PS: no testing is keyword driven. We should call it "Keyword driven scripting". We could also invent the "Mouse driven testing" in which you only use your mouse to test. I'll start the wiki page a.s.a.p. ;)

Agreed. And yes, there's really no defence against bad thinking except for good thinking. It seems likely that the only way forward is to promote science, humanism, philosophy, passion, craftsmanship and testing skills... which is a much harder than sell writing down "I am correct: do these things" and selling copies.

...and agreed with Keyword-Driven Testing, but I limit myself to one rant at a time :)

A decision to adopt 29119 is likely to centre on value added vs cost of adoption.

It's all a bit academic in my view as most enterprise-level software QA Departments already incorporate much of the material covered in 29119 as a matter of routine anyway - and have done so for decades. There's little in there that's new, although I'm pleased to see risk-based approaches finally taking a more prominent role.

I really don't see where and what the problem is. It's not like anyone is going to be forced into adopting it, so unless customers start demanding that suppliers are ISO-29119 complient, it's going to be optional. As always, some suppliers will make the jump first just to gain a perceived marketing advantage over those that don't - "Hey, buy from us we're now fully ISO-29119 complient". 

I've included some links in the OP for some background to my question. I'm more interested in what's best for the testing industry, test clients and product users than what's good for certification salespeople.

Peter - I couldn't agree more. The only real fault I have with ISO-29119 is that there's nothing new in it. As 29119 was published I was starting a job where I would have to build test processes from scratch. I read 29119 and realized that it echoed my approach to testing and implementing a process based on it would save me a little time and, possibly, help us in the pursuit of higher levels of CMMi accreditation. As you say, if you work for a large company you will already be using everything in 29119.

If the standard went ahead, who would it effect?  Anyone here need to comply to standards and will a new one make a difference to their work?

Standards have never played much of a role in my testing career mostly because of companies I choose to work with.  It would be interesting to hear from people that these standards actually effect.

If one believes that standards have a detrimental affect on the industry, then one could conclude that it would affect all testers. If one believes that imposed processes has a detrimental affect on testing, then one could conclude software companies would be impacted, as well as software, and the users of that software. We can all assume that standards proponents and certification companies would benefit.

We are affected but that's more by implication than directly. I work for a supplier of Financial applications, so our customers demand complience with a plethora of national and international statutory legislation. That's achieved through regular independent external auditing of our development and QA procedures and processes.

Many years ago we adopted those parts of the ye olde IEEE standards that made sense for the way we develop and test our apps - and it has resulted in making it much easier for auditors to confirm that we have a tightly managed, carefully controlled and consistent approach to delivering products for this market sector.

Obviously 29119 won't suit everyone, if only because (I assume) they either don't operate under similar constraints to us or maybe have a different approach to the how-to-demonstrate-complience problem. I fully expect that if and when ratified, we will review 29119 just like we did for IEEE and cherry-pick those that are sensible for us, and ignore those that are not.

In short, if adds sufficient value to both what you do and what you deliver, then consider adoption, otherwise ignore.

As an ex-auditor I am deeply sceptical of the idea that the convenience of auditors should be a factor influencing the way that testing and software development are managed and performed.

In fact I'd go as far as to say I'm hostile to the idea. It very much puts the cart before the horse. Documentation should never be required "for the auditors". If it is required it is because it is needed to manage the project, or it is a requirement of the project that has to be justified like any other requirement. That obviously applies to regulated environments. Even then the real need is to comply with the law, not to give the auditors an easier job.

That might seem like nit-picking, but once you lose sight of that you can start doing pointless work at the expense of stuff that would have been genuinely useful.

The easiest audits to perform are those where you can run down a checklist, asking questions that allow only binary, yes or no, answers. "Have you done X to Y? Yes or no?"

That's so easy you can give the job to anyone who's literate. The trouble is that it attempts to redefine the problem area as a neat and ordered space, where binary questions and answers are appropriate. Software development and testing are far messier, (Complicated or Complex rather then Obvious, if you're familiar with Cynefin).

Taking a simplistic approach to auditing a complex problem area not only fails to provide useful information, it becomes genuinely damaging when auditees start to do things that distract from the real work, but are designed to ensure they pass the audit.

I used to despair of auditees who would proudly show off useless shelfware documentation, or pointless processes, that existed only because they assumed "the auditors want it". I would point out that it was unnecessary and wasteful, and that while they had been wasting time on that they had failed to do anything about genuine risks and weaknesses.

I have seen good auditing, and dreadful, dysfunctional auditing. I have nothing but contempt for the auditors who put their convenience ahead of their professional responsibilities. Neither auditing nor testing are meant to be easy; they're meant to be valuable.

Any auditors who grab copies of ISO 29119 and think it will give them a valid benchmark against which to audit are taking money under false pretences. Auditors have a duty to understand the context of their problem area, and to inform senior management about the risks. In principle it's not so very different from testing.

ISO-29119 encourages cherry-picking in a way that IEEE-829 did not. I would argue that the later standard is also better written, is more coherent and is much easier to make work.

RSS

Adverts

© 2017   Created by Rosie Sherry.   Powered by

Badges  |  Report an Issue  |  Terms of Service