PanelSoft

User Interfaces and Usability for Embedded Systems

Feedback to "A Question of Quality"- Murphy's Law, February 2002

Read the original artilce at A Question Of Quality

return to Murphy's Law

Ron Bauerle quotes his favorite bits from the article.

I could cut and paste a dozen excerpts, but will let you read it for yourselves...

Actually, here are the two best:

"If no one uses a document to produce something of value to the company, it should not be written. Auditing alone is not reason enough to create documents.

[...]

the ISO 9000 industry can't produce any measurements to show that applying ISO 9000 has improved a company in any way. Something - productivity, share price, or customer satisfaction - should increase if the movement's claims are true, but no measurements exist to prove that this is the case. "

Too bad this didn't come out 5 years ago when it was getting shoved down our throats...

Ron

Darrell Diem

Niall:

Just finished your article in Feb ESP. Excellent points. I used to work for a company who promoted the person who got the ISO 9000 certification to be vice president, and later on the company went bankrupt, why? You right on the point of documentation. If a company has ISO9000 certification does not mean it produces quality products. If more than a quarter of a million companies are ISO9000 registered and if they are all buying and selling products to eachother, can you imagine the quality of the products based on that certification? You are also right about auditors. I am sure there are bogus auditing companies, too. And, who audits these auditors? and what standard do they follow? The ISO9000 sounded like a fad to me when it spread all over the world and waiting for some company to come and proclaim that they do not need ISO certification, that their process and products beat the ISO documentation requirements.

Hopefully, enough people will voice their opionion about this useless standard that does not have any effect on the quality and everybody will start saving money that they have spend on audits and keeping the certificaiton alive. Keep up the good work.

Dinesh Lad

I read your article for embedded sytems regarding ISO-9000.
I thought it was spot-on.
Please continue the flow of great ideas.

Gus S Calabrese

You've touched --- no, cut --- a nerve with your insightful comments about the bloated, self-ingratiating bureaucracy that has grown up around ISO. What started as, I believe, a thinly-veiled form of protectionism, has become entrenched in modern business. The sad thing is, everyone participating KNOWS that the ISO badge is meaningless, and yet we keep playing along. The system adds nothing to productivity, is prohibitively expensive to smaller companies, and, in spite of its ostensible aim, leads to mediocrity becoming entrenched when the paperwork becomes easier than the real work of progress.

I believe in continuous, demonstrable improvement, but ISO is not the answer. Thanks for the courage to speak out.

Vince Owens Ohio, USA

Cheers,
William Carroll

I made a brief study of quality methods while being retreaded in graduate school, and this was followed by employment in the telecom industry in an ISO 9000 shop. I can only say that Niall Murphy was, if anything, too kind to ISO in his February column. The major fallacy of today's "quality" efforts in large electronics/software shops is the assumption that engineering is a production activity. Engineering is by definition the solving of problems, of application of new principles, or new application of principles. Engineering activities are by definition either done once, or unnecessarily duplicated. Process efforts such as TQM are aimed at improving quality by reducing variance in PRODUCTION activities. The idea is to bring a process into statistical control by removing special causes of variance, and then to reduce the variance. If the product then is unsuitable, it is at least conformant to design, so the design can then be adjusted. This all makes perfect sense, and reduces costs as well, thus supporting Philip Crosby's assertion that "quality is free".

ISO 9000 attempts a much lower bar: that of documenting whatever process is to be used. The biggest REAL value a firm can get from ISO 9000 is frequently simply to record and know what it is actually doing, as some firms don't even know THAT. The idea is that the process may be bad, but at least it's written down so it can be reviewed and analyzed. And, presumably, has measurements specified so the benefits/losses of any change may be quantified. The best engineering managers, when forced to do ISO 9000, will use the opportunity to learn more, and to modify practices to better accommodate their engineers. This is too infrequent, of course. In all cases, ISO 9000 imposes overhead - IMHO, about 20%.

The biggest COMMERCIAL value a firm gets from ISO 9000, however, is the claim that it is certified. This allows it access to business from customers who wrongly believe that the certification proves something about the quality of product or service. Now we're doen to the nitty-gritty. The goal is certification. Who cares what we do, or even how we do it, as long as we're certified? And we know all about the audits (we buy them from a vendor), so we can usually fake a process sufficiently to get the certification.

This whole approach works against an effective engineering organization, because it is required to document processes that probably shouldn't exist in the first place. Now, granted, there are certain "engineering" activities (perhaps, e.g., the archiving of blueprints) that admit of a high degree of repeatability, and these might be a proper subject of certain ISO procedures. Similarly, perhaps, a checklist of certain regulatory requirements. But the general case is that the important stuff either isn't repeatable, or doesn't need to be.

Larry Brunelle

Mr. Murphy: I most resonantly enjoyed your ISO9000 diatribe published in the February 2K2 Embedded Systems Programming. My poor wife has been subjected to almost a decade of my rantings on the subject; she is convinced I am seeing plots. Perhaps there is one: an insidious plan to prevent creative people from competing with slugs and nitwits. In nature, remember, the reproductive capacity of the nitwit exceeds the survival skills of the clever.

My first exposure to this insipid scam came without warning. Being employee number three of a subsidiary which grew to over 300 in five years, I was most astonished and affronted when two persons whom I had never before encountered sauntered into my lab and announced in a huff that I had entirely too many data books on my shelves, and must remove all but 12 by the end of the day. It seems our new ISO9000 program (started as a '"quality improvement" enhancement by the owner-president's idiot brother, who, because he looked good in a polo shirt, wound up running our division) included interior decour. My several dozen data books were in violation of the new ISO documentation, which also dictated plants and carpets. Within 3 months every project was put on hold pending ISO evaluation; a year later, we still hadn't been able to work on anything, and were roundly scolded for not getting projects done. Any attempt to actually do anything was punished. Failure to achieve deadlines was punished. By the end of a year, every competent engineer had quit. I beat the crowd by 6 months.

Some time later (after an episode of actually enjoying my job, an infraction for which engineers are inevitably punished) I found myself in the service of a well known environmental instrumentation manufacturer. For the first month I had nothing to do, as my in-processing paperwork was trapped in the ISO9000 mill, and I was not allowed to do more than count my (and I am NOT making this up) seven -count them, seven- ISO process mandated paperclips. There I sat, making almost a hundred large per year, without a phone and counting paperclips. My intranet connection would of course not be activated until the paperwork was ground into pulp.

Now the nutty thing about this company's ISO nonsense was that there were several levels of documentation. Persons on lower levels were not allowed (for security reasons! God, they measure poop... what security reasons?) to view the ISO9000 documents for higher levels. Yet the lower levels had to FOLLOW the higher levels' documentation. But we could not know what the documents said. It was up to the department managers to 'guide' the workers; workers were not allowed to know what the documents required, but one was 'informed' when one violated a higher-level procedure. Then one was forced to guess what the proper procedure was. Wrong guesses were punished.

Once I was official, I still had nothing to do, as the project I was hired to work on was stuck in another ISO-process-required committee trying to decide (again, I'm not making this up) whether the device would have four or seven buttons. Now, I could have been working on the innards of the thing, but nooo... engineering work was forbidden until all specifications were signed off. That took almost 8 months from the date I was hired. Eventually, marketing's gestation period ended and I was presented with a final spec which made no sense, defied the laws of physics, and granted engineering 4 months. They began the button argument 6 months before I hired on. A full 14 months were spent in ISO-process mandated wrangling over the number of buttons on the damned thing, and then they wanted it finished in 4 months. I insisted this time table was absurd, and was informed that our ISO documentation called for an 18 month development cycle, there were still 4 months left, and it would be engineering's fault if it didn't get done on time. During the proceedings five other engineers quit, leaving 2 of us to pull this off. This did, of course, not affect the schedule. The Nazi accountant who ran the place viewed the reduction in salaries paid out to engineers as a GOOD THING. No replacements were recruited. The vacated engineering space was taken over by marketing.

But I digress... After counting my allotted paperclips often enough to ensure myself that there were, in fact, seven, I amused myself by looking at the schematics for a new process monitoring probe they had recently introduced. After some pondering, I came to the conclusion that the thing simply couldn't work. This device was supposed to measure dissolved oxygen in water. The basic sensor was ingenious and the system worked great on the bench. Unfortunately, my analysis indicated that submersion in a real world earthed tank would produce a ground loop of such biblical proportions that smoke could well ensue. I pointed this out, and was summarily punished.

Two months later, there were rumblings from MARKETING (I must use full caps as these entities consider themselves dieties and will broach no disrespect) that customers were complaining, and these complaints were not being submitted in our ISO9000 certified method. Indeed, instead of filling out the proper forms, they were (gasp) bitching... Bitching, I say... that the sensors worked OK in a bucket but went nuts in the process tank. Following several weeks of hand-wringing, a contingent of managers descended upon the corner-of-the-hall which was my office and demanded that I explain why I thought the sensor was flawed. I outlined the current design's deviations from accepted engineering practice. I was informed that the sensor had passed all its ISO9000 tests, and was punished.

A month later, as the complaints mounted and as I continued to await the final specification necessary before beginning work on my project, I was confronted by the director of engineering, the principal engineer, and the head of marketing. They all wanted to know why I had been speaking (internally, of course) negatively regarding the aforementioned sensor. I was shown the result of the recent ISO9000 audit, which found no problems with the suspect sensor's development, and I was roundly chastised for my bad attitude. At this point I had already set up, in the lab, one of the sensors; mounted on a chemists' tripod; it could be immersed in either an isolated beaker or a grounded metal sink. Suffering some derision for setting up an experiment which was not predicated upon established processes, I convinced my antagonists to accompany me to the lab. In short order, I demonstrated that whilst the sensor performed splendidly when bobbing in an isolated beaker, it went berserk when immersed in the grounded lab sink. Upon some discussion, it was acknowledged that 100 percent of the sensors shipped had been returned as non-functional. We, of course, were merrily shipping non-functional replacements. I was given 2 weeks to correct any design deficiencies, and admonished to follow the ISO procedures in future.

At the end of two weeks, I had corrected the design flaw, prototyped the solution (I got in trouble for that too... prototypes were not part of the design process) and proved, via grounded sink, that the solution worked. Another month was spent in getting the changes thru the manufacturing process.

Eventually I was ensconced in the project for which I was hired. I will not go into the frustration involved; suffice it so say that gin was the only respite.

Five months later, I got a call from one of the few cogent marketing guys, who was attempting to resolve an installation issue. (This company doesn't do field technicians... because their ISO process is so tight, they have convinced themselves that they don't have field failures...all problems are due to the customers' ineptness, and if the customer is truly pissed, they dispatch a marketing guy to buy more gin.) This guy had been sent out to a site which had about a dozen of these sensors; they all worked in a bucket, but went nuts when put in the grounded tank. [wasn't this where I came in?] I was sure my correction was valid, but the guy in the field assured me he had picked up a new batch of sensors from manufacturing before heading out; these were fresh off the press, and they were failing on contact with the grounded process water. After about an hour of over-the-phone diagnosing, I asked him for the model number embossed on the side of those new sensors. Sure enough, it was the model number of the non-functional version. I stomped out to manufacturing and examined the latest crop.... as predicted, they were continuing to produce the non-functional version. Turns out that our ISO9000 process requires us to use up all existing circuit board stock before employing any new designs. Never mind that the existing boards didn't work. ISO is ISO, and God forbid we would not follow the ordained path.

Irritated beyond normal levels, I hijacked a lift and tossed the crate of defective boards into the dumpster. Now, this is wierd... I got in a world of trouble for taking the defective boards 'out of the process'; but once they WERE out of the process, we couldn't use them. They stayed in the dumpster, and the corrected design (in the real world a normal ECO would have sufficed) was used on subsequent production runs. No problems since. I was reprimand for finding a problem. I was reprimanded for finding a solution. I quit a few months later, when (again, I'm not making this up) I was chastised for pointing out that their new humidity sensor would only work in a vacuum.

I took interviews at several concerns, but generally lounged about the house. Eventually the wife forbade further remodeling activities, so I accepted the position of lead hardware engineer at a power conversion concern. I already had some 'alone time' scheduled, for which they were most accommodating. After a few weeks spent wandering in the desert (Death Valley and Mojave... OK... I'm enough American Indian to getcha into the woods; not enuf to getcha out...) I reported for work. I was astonished to find that the company I was now working for was not the same company with which I had, on several occasions, interviewed. In the interim they had gone ISO.

Regardless of the fact that the company had just pulled off a 200 million dollar IPO, my cube was sans chair. I had a computer but no monitor. Two weeks later, the chair appeared; I could figure out how to sit in it myself, but the monitor was another story. I could not 'use' the monitor until signed off on it via ISO paperwork, after which the monitor may be scheduled for delivery. Now, this company was not yet fully into the ISO thing; they were just trying to function in an "ISO FASHION" in preparation to the actual event. Oh-oh. I sensed trouble brewing.

I was tasked with developing CPLD code for a new power converter. The specification required us to use a particular tool in a specified manner using ISO documented methods. Unfortunately we had not actually bought that tool, so, as specified in another document, we were not permitted to even think about HOW we would use it if in fact we had access to it. I spent months judiciously not using the required tool to work on the project we were not authorized to work on which, according to the schedule we were not allowed to see, we were behind.

I began to suspect that a John Cleese character was running the place. Total Industrialized Schizophrenia. Corporate dementia.

On a Monday our group was told to have a subsystem prototype working by Friday. On Wednesday the design was done and the parts reqs were submitted, with a note that the parts had to be in next-day. We immediately got a memo from purchasing that the procurement paperwork took 7 to 10 days to process, after which the actual parts may be ordered; no overnight shipments would be processed without the express permission of the GM, and everyone should please go away. I asked the GM for overnight authorization, but he said the ISO process didn't require that, and refused.

Of course on Friday there was nothing to show. The GM went ballistic; "where's the prototype?" "No parts, I answered" "I don't want any excuses" he scolded. "We need parts NOW if we're going to prototype NOW" I retorted. "I'm tired of excuses" he snorted, and stormed out of the room. I received a memo on the importance of learning to follow the ISO procedures. (We were not ISO yet, but we apparently needed to practice...) and another memo on the importance of meeting schedules.

Still not quite ISO, but practicing, some really nifty procedures were put in place. One of them mandated that any samples from vendors be processed thru purchasing, that an internal part number be assigned and that part number be ordered from the vender using a zero-price PO, that the part be entered into the inventory control system and assigned a stock location. Receiving could not receive any parts without an accompanying PO number stamped on the box, and once received under the null price PO that part would not be delivered to an engineer without a Bill of Materials number, which wasn't assigned until the engineer actually designed the part into something and the formal design review had been completed. Which pretty much shoots the concept of a 'sample'. So we took to requesting sample parts be delivered to our homes. Then they came out with a rule that no parts may be used (even for prototyping) which didn't go thru the null-value purchasing process. No parts were allowed in the lab unless they had a Part Number. Now you just try giving a null-value PO to something like Maxim's web-sample site. They can't process such a thing. No such a field as 'null value PO". So we couldn't get samples from MAXIM, or TI, or DALLAS, or anyone. And even if they did come in, we couldn't get them out of the inventory system unless they were going into a product. It cost about 50 dollars per PO to process the paperwork, so not ordering samples for prototyping saved money. Recognise please that this new company didn't actually HAVE a product yet. They are still planning for the day when they have one.

So there we are, in the normal monday morning beating, being berated because the tests on the prototype we don't have parts for yet are not on schedule. After three hours, the powers that be allow us the remainder of the week to finish the tests; these runs are vital, as they will be presented to the board (which comprises doctors and lawyers and famous astronauts) who are expecting splendid results. I am allowed, as a special dispensation, to bring in parts from my private stash in order to commence these tests. This is a once-only allowance, and is being granted only after I have begged forgiveness for not having specified the components two months before being appraised of the design requirements. I am most grateful for the leniency, and we engineers grovel our way out of the meeting intent on performing this suite of tests.

Upon entering the lab, we are amazed to find that every single bit of test equipment is missing. Every 'scope, every analyzer, every multimeter... nowhere to be found. After some hunting, we learn that everything has been sent out for calibration... some stuff will be back by friday; some in another week.

Our (not that we're certified yet) ISO9000 plan, which no engineer has ever seen, calls for all instruments to be calibrated this week. Doesn't matter what other schedules are paramount. So in I go to the management gods to try to make sense of this; "We don't want any excuses... these tests must be successfully completed by the end of the week", says they; "All the test equipment is gone, says I"; "We're tired of your excuses", says they.

Late Friday some of the equipment returned. I spent Saturday and Sunday trying to figure out why my CPLD code was not producing the waveforms I had designed with the obsolete free demo version of the package we were required to use but not allowed to purchase. Late Sunday evening I began to suspect the freshly calibrated analyzer. I broke protocol, drove 30 miles home and fetched my own (The company had others back from cal, but I was not authorized to use them; they were locked away, and only the QC guy could release them. We were not allowed to use our own stuff, but I was desperate.) and found that the freshly calibrated analyzer had holes in its trace buffer which made it look like my logic was flawed. I was maximally pissed. I'd spent 20 weekend hours figuring out that our freshly calibrated instrument was defective. I resolved to stop being so nice and really have a fit.

Monday morning, before I could vent, we engineers were informed via corporate lecture that anyone entering the labs without an approved antistatic foot strap would be summarily dismissed. This was the first edict of our new corporate ISO quality wonk, and was presented with much fanfare. Recognize please that this company didn't actually have a product yet... but it was important that if ever we did manage to develop a product, everyone involved would be static proof. Engineering was then chastised for not completing the aforementioned specified tests.

So we engineers approached the labs, and were greeted by a security person who demanded to know where our antistatic foot straps where. I said that this was all new to us, and we would be glad to put them on. Where could we get them? "Oh, we don't have any right now, they're still on order", he responded. We were not allowed into the lab. Off to the GM I stormed. "Now let me get this straight... we have to have antistatic foot straps to enter the lab, we will be fired if we don't have them, but they're still on order? We can't finish these tests without test equipment, and we certainly can't do them without going into the lab but we'll get fired if we go into the lab to test circuits we don't have parts for using test equipment we don't have access to without the antistatic straps we don't have... that makes no sense". "All I ever get from engineers is excuses", he huffed. "I don't understand what you have against ISO", he barked, and stormed out. My chest began to hurt, and I was hauled away on a gurney.

Enough is enough. It's now been several months, and I have no intention of going back. If ever I return to engineering, I shall avoid any company which is or makes noises about becoming ISO. I shall negotiate a clause into my contract stipulating that should the company choose in future to go ISO, I get a year's pay severance. But I probably won't go back to engineering. I'll just putter around my gardens; if I keep the hedges high enough, maybe the ISO squads won't find me.

Regards,
BruceR

Steve Hanka, Qwest

Niall,

I could not help but respond to your diatribe against ISO 9000. Although there are some points in your article I would heartily agree with there are also some statements that I strongly disagree with. I have been a Lead TickIT auditor and worked for Lloyd's Register Quality Assurance (LRQA) from August 1995 to January 2001. I have audited hundreds of companies: small companies, large companies, privately held enterprises and DOD subcontractors and everything in between. Everything from shrink-wrap products, to database apps, to sophisticated embedded systems have come under my review. And yes, I have audited things far afield from software like machine shops, in vitro medical products, chemical plants, etc. I have a degree in Chemistry, have done grad work in Electrical Engineering and Bioengineering, worked as an electrical engineer, but most of my career has been taken up doing software development and software test (20+ years). I only bring up my qualifications so you know that the background of a TickIT auditor is typically broad and significant experience in software development and/or software test is required. (I can't speak for the typical background of a non-TickIT auditor).

My disagreements:

1) I will start at the end of your article and hit what I think is the most important point: incorrect implementation. This is indeed the reason for the kind of irritating and counter-productive practices you have cited. However, to answer your question of "how to know when it (ISO 9000) is being used the wrong way" I think I can offer some guidance. The answer is really the answer to this question: "What do you need to produce a good quality product?" I would suggest this "minimum set" of activities: a plan/schedule, a requirements statement, some design work, decent coding practices, decent reviewing practices for your designs, adequately designed tests, tracking of test results, a good source control system and some way to keep track of the most recent versions of your documents. In the area of design, ISO 9000 doesn't ask you to do too much more than this. I don't think any of these are too much and I have been on many projects where the absence of one or more of these led to disaster. If you can think of some way to efficiently encapsulate the way you will use these things then you have a good set of practices. If you can think of a way that will make these hard to use or creates a nightmare of overhead then you have a bad set of practices, i.e. an incorrect implementation.

I hear many objections whenever I bring up this rebuttal. One is: "the procedures don't allow us to do so-and-so the way we want to, we are stuck with this stupid way of doing things". Sometimes this is truly the case, in which case I think that someone in the quality department has created some kingdom that he/she is protecting and won't allow constructive change to occur. The majority of the time, though, I find that the developers themselves have not been involved in the process of creating the practices/procedures. Another objection is: "when we do those things the project just bogs down in paperwork." If the system is that hard to manipulate (like your example of the ponderous method of applying for a document number and the duplication of effort that followed) then the system needs to have some fat trimmed from it. However, more often than not, I get this from people that don't like to document anything at all, that just want to "get the job done." I find it hard to work with people that don't document things or keep up their documentation because, in a team environment, inevitably I will write some code that is supposed to interface with theirs and do so depending on information from their document, only to find out that they have changed their code but not updated their document. Their "getting the job done" has impeded my progress and slowed down the team as a whole.

2) You state that "the motivation behind ISO 9000 is to produce documentation for audit purposes." I would counter that documents are the products of those "minimum set" of activities that I identified in my first point above. If you don't have a written set of requirements then you don't know what you are producing and you have no basis for test. If you don't document your design you have insufficient detail to proceed to coding. I could say similar things about the documents produced for each of the elements in that "minimum set". If you are producing documentation beyond what you need to do your job (refer to your comment on "who reads it all"), then you might have a case. However, if you think there is too much documentation required in your system then you should change your system. The ISO 9000 requirement for activities to be auditable (i.e. produce documentation) dovetails quite nicely with an expectation on the part of many "stakeholders" in the company that developers will take the time to practice due diligence to adequately document their design and development activities.

3) You state that "audits performed after the fact based on paperwork generated for auditing purposes achieves little" and that "processes should be audited by watching what actually happens". This is patently untrue and a rather fanciful wish and, ultimately, a cheap shot. If I come into a company and review a particular development program and I find that there is no schedule available, no requirements available, no source code control system in use, no tests being written, no test results available, etc, I am not going to think that these people are efficient and super-productive; I am going to wonder what the hell these people are doing. You HAVE TO produce documents of the type that are auditable in the normal course of running a project. If you don't then the developers and testers don't know what "truth" is on the project and the whole thing spins out of control, especially in a team environment. For instance, in one of the worst projects that I was ever on we had multiple versions of the software requirements statement and never did produce half of the originally planned design documents. This was so because there were a large number of developers (and some management) that just wanted to "get the job done." You can imagine what kind of chaos existed on this team and, of course, the project ended in failure. On the other hand, given that you run a project well, then you will be producing the documentation of the type that ISO 9000 requires anyway, so what's the big deal? As to "watching what actually happens" if this isn't what "actually happens" then I don't know what is.

4) You state that "If an auditor flunks a company, they lose a customer. So it's not surprising that few companies ever lose their certification." Maybe this might be true of other certification bodies, but it wasn't true at LRQA. Personally, I "flunked" one out of every three or four companies on their first certification audit. I raised major non-compliances at certified companies a number of times, and these are threats to keeping a certification. I know of more than one certified company that LRQA had to begin proceedings for withdrawal of certification because they did not resolve these major non-compliances.

Now, as for what I agree with:

5) Companies do indeed create ISO 9000 quality systems that are administrative monsters; unwieldy and ultimately counterproductive turkeys that are millstones around everyone's neck. Why they do so is beyond me because it is not required by the standard, they cost too much to maintain, they don't contribute to quality and, (from a personal point of view) they are harder to audit. Every time I saw a procedure that was outdated, a form that was unnecessary or a process step that did not add much value I would at least recommend that they get rid of the offender and sometimes I'd raise a non-compliance. It is quality systems of this type that generate all the myths of ISO 9000 being inflexible and stifling of innovation.

6) Companies do indeed miss the point often, i.e. "by setting certification as their goal, businesses neglect more important objectives." I can't tell you how many companies that I was forced to recommend for certification because they operated a quality system that was compliant to the ISO 9000 standard but did nothing to improve quality, much less improve their bottom line. In my opinion these companies wasted their money AND missed a real opportunity for improvement. These companies don't really care if there is any kind of quality system in place, they just want a piece of paper on the wall.

7) Quality staff that understand ISO 9000 but don't understand software/technology only add a superficial layer of quality. To really contribute to the quality of a software project you must understand the processes that are used in the development of software. This ultimately means that you must have a software professional that is trained in quality for these positions, not a quality professional that is trained in software.

8) Although you are a bit more cynical than I am about this, I will have to agree that ISO 9000 has become big business. I think this has led to slight increase in overall quality in other industries, mainly because of the emphasis on process. It has not led to an increase in overall quality in the software world mainly because of reasons cited in (6) and (7) above. Two illustrations of this: a) Companies will shop around and find the consultant that will write their quality systems for them (rather than write it themselves) at the lowest price and will find a certification body that has requirements lax enough to allow them to pass with the least amount of hassle. How can you say that a company like this has changed anything, much less improved their quality by being ISO 9000 certified? b) The companies that I know of that were at risk of losing their certification by LRQA if they didn't address their non-compliances responded by simply moving to another certification body. This is a failing of the ISO 9000 world which you did not point out and probably didn't know about.

Well, I could write a lot more on this subject but I'll let you go with this: the problem is really not ISO 9000, it is the way that we deal with software quality and process. No one wants to pay the real price of quality, which is to change the way we do things. Consider how many quality processes, systems, approaches and gurus we have against the truly rare occurrence of high quality software.

Cheers,
Larry Bannister

Hi,

I liked your article and have some comments, but I don't want to look like a fool or for you to take me as an idiot, so please try to bear with me.

I feel that your problems with ISO 9000 are more with how it has been implemented, and the management employee relationship rather than with the intent of ISO. A "learning organization" (I use quotes around learning organization because I read the term somewhere and can't remember where) is one that is supposed to listen to its employees and learn from its mistakes. That is, if a process is stupid, then the process should change. Often, management believes it does that when in fact the relationship is backwards, that is, if the boss makes the rules, then the rule cannot be stupid. That could be what you meant when said when you wrote that if a document does not add quality, then the document should not be written. Aha! If the process says the document should be written, then automatically and logically by the boss's definition, that document MUST add quality.

Am I brilliant or idiotic? (I hope somewhere in-between) I don't believe that ISO 9000 says that stupid documents should be written. Finding out what's stupid is the job of the person who documents the process (oops, that could be your boss).

I am curious as to what documents that you dislike exactly are the ones you wrote about?

I will also point out (you can't stop me unless you delete this right now) that your example of writing test cases points to stupidities in the process, not that the idea of documenting the process itself is stupid. My rationalization is that there is nothing wrong for one document to point to another rather than to need to write the same table, paragraph, whatever twice -- just like using an html pointer if the documents are on-line. The process could have been modified to say this. Also, the random number? Anyone could have provided a tool that created random numbers, and the process could say "Run the Random number generator, and apply the data to the test input". The problem is not the process, but the person who writes the steps in the process. Similarly, applying for unique document ID's can be automated. If your company does not listen to its workers as to the best way for work to be done, then the company deserves what it gets! (And that explains an awful lot of crap going on, doesn't it)

I also think you are missing the big picture, that if something fails in the field miserably, and someone takes the effort to diagnose the problem, and it points to testing, the tests can be exactly rerun to see if the problem is in improper testing, or manufacture, or whatever, but only if there is enough documentation to rerun the test. If no one will ever need to re-test in the future, or will ever want proof that something was ever tested, then why not throw out testing altogether?

I hope this wasn't all a dreadful waste of electrons. I am a programmer, not a boss, and I have to work under process, and some of it yes, sucks, but I have also seen quality go up when done right. But noooo, there are no good metrics that measure it. We are not manufacturers we are artisans and everything we do is unique. Think of process as the net and court lines in tennis.

I like my letter so much, I think I will send myself a copy.

bob schaefer