Examples of inherent safety by design

Ronen E said:

To me the most important question about this (as a QA professional, not as a SW professional, which I'm not) is how can you guarantee the effectiveness and completeness of SW testing? IMO, you can't, because regardless of field and type of objects being tested, testing can only be indicative, never a proof.

Click to expand...

I have no disagreement with the above statement, but I want to comment on:

Ronen E said:

This is why you can never achieve AFAP through tetsing - one could always argue that you could have tested more, achieved a higher level of certainty and thus lowered the risk a bit more. I'm not arguing that this is effective, or that you should (I think that the AFAF concept, which ignores context, is ridiculous in any technical field), just trying to highlight that relying only, or mostly, on testing, for mitigating high risks, is problematic.

Click to expand...

I'm unconvinced that there is a more objective form of demonstrating effectiveness of engineering solutions than testing. Diverse test methods have different levels of effectiveness, but even the least powerful test method can be repeated, unlike relying on 'professional judgement'.

Specific to software testing, there are a variety of test methodologies which are recognized as being more likely to expose defects than a simple 'test to the requirements'. For example: a low-hanging fruit for external reviewers to question when it comes to testing software is "boundary testing". A mature ME software development program will apply a variety of applicable test methods when challenging a software solution. I will concede that the individual effectiveness (or if you prefer, "power") of test methods for software is generally impossible to calculate. Having written that, I should note that I have only very rarely seen the mathematical power of test methods (and the results of such test methods) be formally used to rationalize a specific risk reduction in controls outside of software.

Tidge said:

I'm unconvinced that there is a more objective form of demonstrating effectiveness of engineering solutions than testing.

Click to expand...

Sure. Testing is a great way to demonstrate something (if we ignore costs for a moment), and if done correctly it may also be objective. But demonstration is not proof or validation (the latter being concerned with high levels of consistency and reliabilty, over time and across the entire spectrum of circumstances).

Tidge said:

even the least powerful test method can be repeated, unlike relying on 'professional judgement'.

Click to expand...

I'm not arguing here for "professional judgement" as a better alternative for testing, not at all. Coming from the HW world (not necessarily electrical) my first preference is always calculations, or another method of theoretical, robust engineering analysis. It may not always be possible for SW applications, and thus we might have to fall back to testing; that's okay, as long as we don't lose sight of the fact that this is indeed a fall-back, and don't delude ourselves that through testing we fully proofed a proposed solution.

Tidge said:

Specific to software testing, there are a variety of test methodologies which are recognized as being more likely to expose defects than a simple 'test to the requirements'.

Click to expand...

Likely. So there's a probabilistic element here. I am convinced that this is related to (if not proportional with) the resources put into the testing.

Tidge said:

Having written that, I should note that I have only very rarely seen the mathematical power of test methods (and the results of such test methods) be formally used to rationalize a specific risk reduction in controls outside of software.

Click to expand...

In my opinion the attempt to make risk management, in general, appear quantitative and objective is futile and ridiculous. IMO it is highly subjective (making it really objective would be impractical). This is not to say it's pointless or useless. On the contrary, it can be highly enlightening and beneficial. But most of the value is in the process which makes us think hard and explore all relevant aspects, and which also makes us accountable, not in the number crunching. I wasn't commenting on this from a "narrow" risk control perspective but from a broader perspective - how do we actually gain upfront confidence in our designs? Be they SW, HW or a combination.

As mentioned several times above, testing is never really proof (this applies not just in software, but all fields) because it's impossible to cover all the permutations. I think it's worth to point out that at heart, modern regulation relies on trust with traceable responsibility, rather than hard proof.

This "trust with traceable responsibility" can be found in ISO 17050 (declaration of conformity), also EU regulations and the EU Blue guide. It's also interesting to see the evolution in the definition of "objective evidence":

ISO 14971:2000:
information which can be proven true, based on facts obtained through observation, measurement, test or other means

ISO 14971:2007
data supporting the existence or verity of something

It's a dramatic tone down and proof (pun intended) that regulators are aware that asking for "proof" is the proverbial pot of gold at the end of the rainbow. I think a lot of this thread seems to dancing around this subject.

Rather than looking for some impartial metric, mystical holy grail of right and wrong, the trust/responsibility model means you just have to keep asking yourself: am I prepared to take responsibility for this if something goes wrong? Can I stand up in court and say that the neatness of the code, methods and libraries used, testing performed and records kept were reasonable for the situation? If yes, then sign off and move on to the next design project.

I think IEC 62304 and ISO 14971 have been carefully written to fit with the trust/traceable responsibility model and avoid too much regulatory burden. Most of the issues are with auditors that read more into a standard than actually exists. For IEC 62304, the only weak point is the SWF100 in the classification section which can be misread to create Class C when that class is not warranted. I think this a huge error so bad that it should be challenged as being suitable for a harmonized standard in the EU, and if there is a manufacturer out there willing to take this though the formal channels I'd be happy to support!

Just a brief comment - testing serves only to create data. You need to analysis the data and conclude on something (and you usually need to have predefined criteria/justification/etc/for the analysis), but then again, results of the testing itself is only the data.

Marcelo Antunes said:

Just a brief comment - testing serves only to create data. You need to analysis the data and conclude on something (and you usually need to have predefined criteria/justification/etc/for the analysis), but then again, results of the testing itself is only the data.

Click to expand...

I certainly haven't argued for testing (software, hardware) in the absence of of criteria. My advice to teams working through a medical device RM process is (once risks have been identified) is to establish/identify risk control measure appropriate for the risks, and then, at an appropriate level, establish a requirement which is testable (and correct, non-conflicting, etc.). The requirement can then help to establish the "null hypothesis" (for a frequentist analysis) or can inform prior selection (for a Bayesian approach). Either way, test data doesn't exist in a vacuum. After performing the tests, there is some new state of information about the effectiveness of the risk control. I am not under the delusion that a majority of ME design professionals are consciously applying 14971 in this specific manner, but I do believe that this is a mathematical description of what is happening in a 14971 risk management process.

Ronen E said:

Coming from the HW world (not necessarily electrical) my first preference is always calculations, or another method of theoretical, robust engineering analysis. It may not always be possible for SW applications, and thus we might have to fall back to testing; that's okay, as long as we don't lose sight of the fact that this is indeed a fall-back, and don't delude ourselves that through testing we fully proofed a proposed solution.

Click to expand...

To my way of thinking, 'robust engineering analysis' is another method of testing: If prior to the analysis there were pre-defined criteria and the implemented design was reviewed specifically against the pre-defined criteria then the analysis is in some ways repeatable. That is to say, the same hardware design and review criteria could be sent to two different (but equally capable) engineering reviewers and the outcomes would hopefully be aligned.

Ronen E said:

I wasn't commenting on this from a "narrow" risk control perspective but from a broader perspective - how do we actually gain upfront confidence in our designs? Be they SW, HW or a combination.

Click to expand...

(emphasis mine)

I read the word "upfront" as implying "prior to the implementation of risk controls". Is this what you mean? Or do you perhaps refer to the identification of a risk control that is implemented with something you wish to identify as IBD? If it is the latter, my personal expectation for an IBD risk control is one of the following (keep in mind that I personally do not like to use IBD to identify 'implementations' that are not actually in the design to rationalize the non-existence of certain hazards, which I have witnessed from some teams):

1) The implemented risk control is uniformly recognized as prima facie effective for the risk, e.g. for fire risks, materials that do not burn below a certain temperature that is never reached.
2) An analysis of the implementation is performed against appropriate criteria for the identified risk.

If case (1) is true, then I would not expect to see very much in the way of 'supporting evidence'... with the caveat that external/unfamiliar reviewers may have a different baseline and may want to see more.
For case (2), I would expect the analysis to be discoverable in the risk management files.

Tidge said:

I read the word "upfront" as implying "prior to the implementation of risk controls". Is this what you mean? Or do you perhaps refer to the identification of a risk control that is implemented with something you wish to identify as IBD?

Click to expand...

Neither. By "upfront" I meant before the device is released for use (on the market).

Ronen E said:

... how do we actually gain upfront confidence in our designs? Be they SW, HW or a combination.

Click to expand...

Tidge said:

I read the word "upfront" as implying "prior to the implementation of risk controls". Is this what you mean? Or do you perhaps refer to the identification of a risk control that is implemented with something you wish to identify as IBD?

Click to expand...

Ronen E said:

Neither. By "upfront" I meant before the device is released for use (on the market).

Click to expand...

I would not use the word "upfront" to describe this situation, as it implies a rather complete design.

For finished ME devices: if such a device receives certification to 60601-1 via an accredited testing laboratory, then the manufacturer will have received third-party assurances that the device has met the requirements of clause 4.2. For ME devices, the 'confidence' you seek is achieved through application of 14971.

Tidge said:

For finished ME devices: if such a device receives certification to 60601-1 via an accredited testing laboratory, then the manufacturer will have received third-party assurances that the device has met the requirements of clause 4.2. For ME devices, the 'confidence' you seek is achieved through application of ISO 14971.

Click to expand...

That's a technical (high-level) description of the process.

The really difficult question (which I was implying) is how to implement ISO 14971, or more generally risk management, in an effective and efficient way (see my signature).

IEC 60601-1 certifying 3rd parties have no super powers. They are humans just like you and me. At best, they implement ISO 14971 (or scrutinize its implementation) rationally and consistently, typically based on broadly-accepted industry conventions (that are sometimes wrong or inconsistent). At worst, they just tick formal boxes. Technically, that might be enough to meet legal requirements for placing on the market, but it hardly provides the kind of confidence I'm talking about. The result? Lots of recalls on a daily basis - if you subscribe to the FDA's email notifications you'll get your daily supply, right into your inbox.

Ronen E said:

The really difficult question (which I was implying) is how to implement ISO 14971, or more generally risk management, in an effective and efficient way.

Click to expand...

I don't believe there is any secret, most efficient way to implement 14971; it is essentially a Plan-Do-Check-Act methodology, albeit one that proceeds from achieving and maintaining safety through the minimization of unacceptable risks. This cycle is relevant at every stage of the product lifecycle, even as the focus shifts from establishment of high-level product requirements through the testing of low-level implementations up though customer feedback into corrective actions.

Organizations need to drive excellence at all stages of the PDCA cycle. In my experience independence among the actors in the PDCA cycle is key: a simple model is the management->development->test group->quality review one. As an example of a potential breakdown: In such a model if the development team is acting in more than one role (small or less mature organizations often have developers doing both design, i.e. "doing", and testing, i.e. "checking") it is more likely in my experience that defects begin escaping. Independence among actors has at least the potential do bring different perspectives which can expose gaps, often this is simply due to different experiences.

More generally to 14971, the concept of including different categories of experts is crucial. My experience has been that a dedicated clinical team is necessary for identifying and ranking harms, and that such folks bring importance expertise relevant to use cases and inform the "P1" and "P2" probabilities in a Hazard Analysis. Even so, just as being an ME device engineer doesn't transform a person into a medical professional, clinical expertise doesn't magically transform a person into an avatar of ME device safety: it is possible to work with great clinicians who have very little appreciation for basic electrical safety.

Tidge said:

I don't believe there is any secret, most efficient way to implement 14971; it is essentially a Plan-Do-Check-Act methodology, albeit one that proceeds from achieving and maintaining safety through the minimization of unacceptable risks. This cycle is relevant at every stage of the product lifecycle, even as the focus shifts from establishment of high-level product requirements through the testing of low-level implementations up though customer feedback into corrective actions.

Organizations need to drive excellence at all stages of the PDCA cycle. In my experience independence among the actors in the PDCA cycle is key: a simple model is the management->development->test group->quality review one. As an example of a potential breakdown: In such a model if the development team is acting in more than one role (small or less mature organizations often have developers doing both design, i.e. "doing", and testing, i.e. "checking") it is more likely in my experience that defects begin escaping. Independence among actors has at least the potential do bring different perspectives which can expose gaps, often this is simply due to different experiences.

More generally to 14971, the concept of including different categories of experts is crucial. My experience has been that a dedicated clinical team is necessary for identifying and ranking harms, and that such folks bring importance expertise relevant to use cases and inform the "P1" and "P2" probabilities in a Hazard Analysis. Even so, just as being an ME device engineer doesn't transform a person into a medical professional, clinical expertise doesn't magically transform a person into an avatar of ME device safety: it is possible to work with great clinicians who have very little appreciation for basic electrical safety.

Click to expand...

I almost fully agree with all that (almost - except the part where it seems you place importance on numbers - P1, P2 etc.), but it's all quite besides the point I was trying to make, that number crunching can be very satisfying and can create an appearance of rationality, but it doesn't make a lot of difference in terms of actually making a device safer. If anything, it's many times used as a fig leaf to cover up hazards and hazardous potential that weren't properly mitigated (typically due to resources load or clashes with other requirements).