"What are you going to do about the information that testing will provide you with ?"This is the kind of question that really shoves lots of stakeholders out of their comfort zone. I've seen more than one with flickering eyebrows, when I've asked this question.
I think, with such reactions, that this is truly an important question to ask:
- If there's no answer to this question, is the testing we do going to be useful ? - to anyone ?
- If there is one answer to this question, we might forget to debate it to figure whether it's the right answer (one could be reminded of Jerry Weinberg's Rule of Three here..)
- If there are more answers, they are bound to be in conflict with each other. Which are more important for now ? - and later ? (oh yes, priorities do change over time ;-)
I find this question to be very valuable to ask, though. It gives some insight into why testers are called onboard a project. It might be, that a project manager just realised that the project manual called for testing to be done (in which case the project manager would probably be surprised and perhaps even angry when you present her/him with the bug list after testing... this was supposed to be the last activity in the project, right ?). Or that a customer have huge problems with a delivery and wants some input to throw in the face of the developers. Or that the scrum master needs to get his demos to actually not crash - or wants to feed his team with tasks that stabilize the code.
Testing comes in multiple varieties - so does the usage of test results.
The best advice I can give you is: ask the question and debate the answer.
In a way, this is where testing really starts...
you have described really a very useful information to us which is really a very useful to me.Thanks for asking such great question to all tester.
ReplyDeleteGood point, Carsten, but (and I know I may sound like the devil's advocate here)... isn't this always the most important question before you set out to do any kind of research for anyone? "What do you expect from me and what will you do about what I'm going to tell you?"
ReplyDeleteAnd to counter you entirely... I actually find that if there's any discipline where we as consultants can help customers WITHOUT even knowing what they'll do about what we find, it's in testing.
Why?
Because neither we nor our customers know what kind of information we'll find. We know that we'll learn a lot about the system under test (part of what can be characterised as bugs) THEREBY allowing the customer to improve the system based on FACTS rather than assumptions and presumptions, but that's up to the customer (project manager, development team etc) to decide.
The question is excellent for reflection of course, but I wouldn't mind if my customer said: "I don't know. You tell me?"
Cheers,
Anders
A manager once said: Just test, don't find all those bugs. I guess his primary reason for having a test, was to present that it had been tested - not to present the test results.
ReplyDeleteSometimes the primary impact isn't the actual work but the impression of work being done
Customers most often want assurance of quality from testers. We can't give them that - we can give them knowledge about the quality (in specific areas).
ReplyDeleteI think we can do a great job without knowing the answer to this question... after all: Who knows what we'll be finding? Essentially, I'm gettin ghired here to find out what's in there because the customer doesn't know.
So yes... great question. Especially for reflection. But often, I'd just get started and produce info first, then help the customer use the information I've digged up. He's in a learning process - and so am I! :-)
Soa Testing: Thank you. Your comment is much appreciated!
ReplyDeleteJesper: I've seen managers planning with bumping off (the expensive) developers once the test phase begun.. Needless to say, that didn't happen, and it ruined in part the business case. I think your example supports the message that there are many incentives for testing :-)
Anders: Oh - I didn't try to imply that you only ask this question once and for all.. I think you can ask it as much and as often as you feel like..
All - thanks for sharing your thoughts!
Oh, ok :) still, I'm a little puzzled by the unconditional title of your post here. Can I suggest a rephrasing? "The most important question in test management". Or even put a "One of the..." in front. Because in actual testing, I feel there are more important questions to ask than questioning how and when the test results will be used. E.g. who are the users, how are they using the system, how is the system built etc.
ReplyDeleteAll that said, I agree completely on your statement: "I find this question to be very valuable to ask." Indeed it is!
Thanks for response!
There's another, more charitable way to interpret the instruction that Jesper's manager gave, perhaps: "Don't believe that you can find all those bugs (I don't believe you can do that anyway.) Just do your best to reveal whatever information you can." That would seem perfectly okay to me.
ReplyDeleteNow, Jesper was there. Maybe he can tell us if the manager meant, "I want you to pretend to test; make sure you don't find any bugs." To which my answer would be "Pretending to do my job is not a service that I offer."
---Michael B.
@Anders Yes; I try to prefer "one of the" to "the" on its own.
ReplyDeleteI like to think of Carsten's question as the bass line of the testing music.
---Michael B.
@Anders - I partially agree with you: 'One of' should be part of the title.
ReplyDeleteAs for the other comment, as I see it, in testing we're supposed to find information. If we already know (or suspect) that this information is not going to be used anyway, we will probably not do a very good job (why should we ?) (feel free to substitute 'Carsten' for 'we' if you wish...)
So I don't think this question is restricted to management; the answers impact the entire test process.
I'd like to expand a bit here. Years ago (on a conference) I was on a tutorial where one of the exercises was to lay out a plan. One guy (dutch, as far as my memory goes) took the lead and said: 'I've grown accustomed to always start by asking Why'. The rest of us looked at him in silence, then this statement was dismissed with a mumbled comment of 'this was the assignment the instructor gave us, that's why - now what do we need to do first ?'.
I gave this a lot of thought afterwards, since I too have come to realise that this is an (one of several) important question. Lots of times we plunge into an assignment without understanding it, and though 'why' holds an acusive element which could be loosened up by asking 'how..' instead, it's valuable and
.. eventually lead me to this blog post.
Very useful post. This is my first time i visit here. I found so many interesting stuff in your blog especially its discussion. Really its great article. Keep it up. Security doors & Window
ReplyDelete