Examples of inherent safety by design

Let me go on record here as saying that I do agree that SWF100 is incorrect but it is how the regulations are interpreted. I am looking for a good way to deal with this deplorable situation with an actual medical device where some of the risks are not negligible, and could be dealt with by taking steps to reduce their likelihood. I think that's the best way to deal with them, but on paper in an audit this leaves us vulnerable to not having subscribed to SWF100.

The arguments for thinking SWF100 is required are as follows:

1) FDA Guidance (Premarket Submissions for Software Contained in Medical Devices, 2005):
this document has two sections on this topic. level of concern is to be determined prior to any mitigation, and a hazard analysis is to be included in the submission for all levels of concern. in table 3, the hazard analysis is described as "Tabular description of identified hardware and software hazards, including severity assessment and mitigations." Later, on page 20, this document states, ".. software failures are systemic in nature and therefore the probability of occurrence cannot be determined using traditional statistical methods. Therefore, we recommend that you based your estimation of risk for your Software Device on the severity of the hazard resulting from failure, assuming failure will occur."

Comment: this is an unambiguous statement about how software risk should be assessed. It has nothing to do with prototyping or level of concern. An almost identical statement is made in the 1999 Guidance document about Off-the-shelf software, so this guidance is not an aberration. The 1999 document uses the word systematic instead of systemic but the point is made clearly that "engineering risk management for medical device software should focus on the severity of the harm ..."

2) IEC 62304 1.1:
this document contradicts itself in section 4.3, where it states in a sidebar to figure 3 that "probability of a software failure shall be assumed to be 1." a NOTE in this sidebar says "Such RISK CONTROL measures may reduce the probability that a software failure will cause HARM and/or the severity of the HARM."

Comment: this is consistent with not using mitigation to determine level of concern in the first sentence, but the NOTE contradicts this.
Section B.4.3 states, " when software is present ... the probability of the software failure cannot be considered in estimating the RISK ... In such cases, ... the probability for the software failure to occurring should be set to 1."

Confusion: section 4.3 and B.4.3 are about software safety classification rather than risk assessment. I agree that this is different than mandating this approach to be the actual risk assessment. This is a flaw in the standard, where it is not stated directly but is implied that the software safety classification be used as starting point. However, in section 7 of the standard, mitigation is not discussed. One is left with the impression that using probability as a mitigation in the actual risk assessment is therefore OK. It's not OK for FDA. Their documents, cited above, are clear about this. And a company wants one way to document this in the DHF.

3) ISO 14971:2012:
It's great to point out that annex ZA is "informative." That's on paper. In practice NB auditors treat it as a statement of requirements and enforce it that way. We can always get a lawyer who will take any position we want and defend it. Auditors are not lawyers and they rarely have enough time to do a thorough review so nuances get thrown out the window pretty quickly. Try using "information for safety" and see what happens in an MDSAP or 13485 audit. My question is about following SWF100 even though I think it's wrong, then justifying the resulting higher-than-real risk based on 'as far as possible' (AFAP) from 14971, and requiring a risk-benefit analysis for each risk item. This gives me a clear traceability from 14971 to 62304 and FDA and even clause 14 of 60601-1.

Final Questions: I hoping someone else gone through an audit with this or has a better idea. Does anyone have any information about how MDSAP auditors are trained on this topic?

Marcelo Antunes said:

As I mentioned somewhere else, I've tried some times to remove the classification from IEC 62304 because, in my opinion, it creates more problems (these discussions) than solves problems (the "level" activities and paperwork required for "low risk" software). In my opinion, the best practice is, forget the classification and do everything (and for "low risk" software, you would certainly need to do a little "more").

Click to expand...

I see there is a redline version of IEC ISO 62304. the redlines cross out some of the text related to classification, but don't cross out the note about probability is 1. I don't have a problem with the classification scheme, only with the probability statement.

robert.beck said:

Final Questions: I hoping someone else gone through an audit with this or has a better idea. Does anyone have any information about how MDSAP auditors are trained on this topic?

Click to expand...

I have experience both with FDA submissions and MDSAP audits of process/files related to European submissions. My teams have used "Software Hazard Analysis (SWHA)" to support a master Hazard Analysis for ME devices. A SWHA is similar to a FMEA, with some important differences: Prior to implementation of risk controls it is assumed that the software (in a particular context related to a risk detailed in the master Hazard Analysis) "doesn't work" (alternatively, that the software "doesn't implement the risk control"). Risk controls (typically "specified software requirements") are established. The onus on the development team is to develop software which is then demonstrated to meet the requirements, i.e. "the sotfware is tested against the requirements". Once objective evidence exists that the risk control is implemented, there is the usual round of checking for any new contributions to risk, and the control is fed back into the master HA to see how the final level of risk is affected (for whatever context the software was allocated to play a part).

To bluntly answer the question about MDSAP auditors in order to speak to Software Risk Analysis: They are directed to apply the requirements of 14971 to all elements of a risk management file. Specifically: It doesn't matter it your RM Plan has a master Hazard Analysis supported by FMEA, FMECA, Software HA, they will insist that all elements of 14971 are covered by each document in the file. I believe that they have landed on this approach primarily because they are trying to avoid applying "critical thinking" in an effort to treat all of their clients the same.

How this relates to the (recent, quoted) question: Practically speaking, this has meant for my teams that we were required (by MDSAP audit, but not by the FDA) to perform a Risk Control Option Analysis for every line of FMEA and Software HA (documents subordinate to a master HA in our world). You may not have expected this. We certainly did not.

Mini-rant: This is in my opinion an exercise that is at least a waste of time, but also leads to confusion about how risks from ME devices (to patients, users, the environment) are actually controlled. Failure Modes are analyzed & mitigated at the FMEA level; Software Risk controls are verified to be implemented at the SWHA level. The overall effectiveness of Risks is only possible at a single (in the case of my teams, the HA) level. My teams have complied with the MDSAP auditor's request, but in the case of Software it has proven to be particularly meaningless to try to apply the concept of 'as low as possible' to software development and testing.

For example: Those of us with software experience can generally agree that 'sloppy code' can be 'written better' for many reasons such as maintainability, but in practical terms a subroutine that is demonstrated to meet its requirements is no less effective because it has a poorly constructed case statement than an alternate which follows a more stringent coding standard. No one can seriously argue that applying a CamelCase variable naming convention in source code is going to lead to a different risk profile than source code that unevenly applies variable names, or a functions return statement that fits on one line is different than one that takes three. It would be rather absurd to see an abundance of software risk controls relating to things like coding standards and compiler switch options in place of actual testing.

Apologies Robert, I did get the wrong end of the stick from post #45.

Yes, unfortunately it's common for auditors to use well meaning but poorly considered guidance as audit criteria.

My recommendation is to be well versed on ISO 17021-1:2015 which the auditing agencies need to follow. It's good to have a copy of this standard on hand. Some key clauses for this case (there may be others):

8.5.1 a) and 9.2.1.4: the audit criteria has to be from "normative" documents (i.e. not guidance/informative)
9.4.5.3 non-conformities must be against specific requirements (i.e. the auditor has to reference the specific clause of the normative document)
8.5.1 f), 9.8: Complaint handling (in case they insist on using guidance documents as audit criteria).

Since the auditor is not allowed to refer to a guidance document, this clears out a lot of references to SWF100 (including the FDA's guide). The only normative reference I know of is IEC 62304 Clause 4.3 but as discussed this is the context of classification. You might have an auditor that does not know or care about the rules, in which case it is good to make it clear (politely, but with conviction) that you will not as a matter of principle accept any non-conformity that refers to a guidance document, and if they do it will result in a formal complaint. And if they insist, make sure they write it up as a formal non-conformity so that you now have objective evidence of the agency breaching the rules, which can be then used in the complaints process (including if necessary to their accreditation agency, which will have a similar complaints procedure).

Separately when determining if an interpretation like SWF100 makes sense or not, it's good to have test cases from other situations, not just your particular case. For example, dialysis systems have around ten different ways to kill a patient, so it's clear they need to have independent control and protection systems. Dialysis needs to be adapted for each patient, and the complexity is such that practically the protection system also has to be software based. Under SWF100, it is impossible to declare a dialysis system as safe. Even for lower risk devices like a diagnostic ECG, SWF100 can't work as there is no alternative but to rely on software for the diagnosis. Even for a simple electronic thermometer SWF100 does not work. There are no viable solutions that avoid relying on software, and SWF100 means you can't rely on the software.

So ti's good to have these examples on hand and ask the auditor to explain how SWF100 could work in practice. Couple that with good awareness of the ISO 17021 structure, the illegality of using guidance in audits as criteria and you should be good to go.

Annex ZA is similar. It's well established that companies overuse warnings and cautions. But the other extreme - to declare all "information for use" as being banned as a risk control is so far out of whack with reality that it becomes obvious that this was just a bit of a rant by the author, which was never screened by the EU lawyers because it just an informative annex. I considered writing an article on this for my website about the "Annex ZA misdirection" but I found it was already well covered by several sources (check out the BSI presentation), so I presumed the situation was clear. If there are still auditors misreading it ...

thanks for you interesting reply, Peter. I'm reacting to audits in which non-conformities were issued because during an iterative software development process, the auditor thought there should be a CAPA for every software change. This took a long discussion of the V model to get through this one. in another audit, the auditor issued a non conformity based on the transportation clause of ISO 13485 because the company's software was downloadable from their web site, and there was no documentation of CRC checking of the downloaded file vs. the file on the website. the response in this case, too late to avoid the written non-conformity, was that CRC is built-in to modern browsers.
In another two instance, during separate audit with separate auditors, the iterative nature of software development with built-in regression testing at suitable intervals was cited for not document this properly as 'rework.'

these kinds of situations make me realize that it has to be kept simple for the auditors. it has to be simple for them to approve something and slightest red flag can lead to an audit non-conformity. Audits are generally very time constrained, and if the auditor is a decent person you may be allowed to explain, or you may not. one must be prepared for both eventualities. If I do get a chance to explain, the explanation has to be short and succinct.

I can see an FDA inspector bristling as i bring out ISO 17021 to justify why FDA's guidance documents can be ignored. in my experience, these are treated and enforced as extension of the rules. a good example is software validation. the rule states that "software must be validated for it's intended purpose." From that starting point, there are least two good guidance documents that expand this and those documents are used as enforcement guidelines. the point is not to be right, the point is to get the product approved so it can be place on the market and hopefully sold. Audit non-conformities get in the way, as do product problems. Nonetheless, I will take a good look at ISO 17021 to see if I can make use of it.

I just want to point out what I perceive as the defect on 62304-2015 (w/ Amendment 1)

robert.beck said:

2) IEC 62304 1.1:
this document contradicts itself in section 4.3, where it states in a sidebar to figure 3 that "probability of a software failure shall be assumed to be 1." a NOTE in this sidebar says "Such RISK CONTROL measures may reduce the probability that a software failure will cause HARM and/or the severity of the HARM."

Comment: this is consistent with not using mitigation to determine level of concern in the first sentence, but the NOTE contradicts this.
Section B.4.3 states, " when software is present ... the probability of the software failure cannot be considered in estimating the RISK ... In such cases, ... the probability for the software failure to occurring should be set to 1."

Confusion: section 4.3 and B.4.3 are about software safety classification rather than risk assessment. I agree that this is different than mandating this approach to be the actual risk assessment. This is a flaw in the standard, where it is not stated directly but is implied that the software safety classification be used as starting point. However, in section 7 of the standard, mitigation is not discussed. One is left with the impression that using probability as a mitigation in the actual risk assessment is therefore OK. It's not OK for FDA. Their documents, cited above, are clear about this. And a company wants one way to document this in the DHF.

Click to expand...

I also believe that the text of the standard has a defect by confusing the concept of altering the design of the system (i.e. the system is re-architected in a way that software is not responsible for the most serious risks) with implementing (non-software) risk controls once the software has been identified as playing a role in the most serious risks. If the ME device design does not have software responsible for implementing risk controls or contributing to risks, a lower level of software system safety classification is obviously appropriate.

In my mind re-architecting an ME system (that uses software) is akin to this hypothetical hardware redesign: An electrically powered device can draw power from the mains; in such cases folks familiar with 60601-1 will be familiar with the specific risks associated with mains power. If the system is re-engineered to only run from battery power, those specific risks no longer apply.

I believe what you are saying is known as 'inherent safety by design' only you are the bringing up the case where the design is altered in order to achieve inherent safety, after it was determined that the original design did not have this.

If this kind of engineering process, whether software or hardware, is prohibited by regulation, that is a serious problem. devices will be less safe because of this prohibition. I don't think this is what is meant by the thrust of either FDA's guidances nor IEC 62304. Both of these set of regulations support 'inherent safety by design' as the first step for risk reduction. Obviously, with software, there are design reviews, and if these are only to rubber-stamp the existing design, they are pointless. good design reviews will result in improvements in the form of design changes. regulatory bodies and agencies know that they should stay out of this because their only interest is in the safety and usability of the end-product. it's only the later parts of the development cycle that are visible to regulators, and they don't care how may napkins you used to figure something out as long as eventually you get to a good SDLC.

robert.beck said:

I believe what you are saying is known as 'inherent safety by design' only you are the bringing up the case where the design is altered in order to achieve inherent safety, after it was determined that the original design did not have this.

Click to expand...

Yes. I have a reluctance to reference an architecture redesign as an IBD risk control in software because the FDA guidance allows for Level of Concern classification based on the role of the software, but it disallows classification based on risk controls.

Stepping away from software: My teams have struggled with when explicitly listing IBD would be called for. We typically avoid the use of "obvious" (or "no value added") IBD controls by leveraging the design concept document and an early-concept hazards checklist. For example, if our concept does not include a nuclear power source, we would never bother listing IBD risk controls for risks relating to certain types of radiation hazards.

I have a personal distaste for trying to claim "something" (a risk control) is Inherent By Design when a "thing" is not actually "in" the "design". I am aware of difference of opinion on this particular matter, and I agree with Robert's assessment that it is the documented safety of the end-product which is paramount.

Tidge said:

For example: Those of us with software experience can generally agree that 'sloppy code' can be 'written better' for many reasons such as maintainability, but in practical terms a subroutine that is demonstrated to meet its requirements is no less effective because it has a poorly constructed case statement than an alternate which follows a more stringent coding standard. No one can seriously argue that applying a CamelCase variable naming convention in source code is going to lead to a different risk profile than source code that unevenly applies variable names, or a functions return statement that fits on one line is different than one that takes three. It would be rather absurd to see an abundance of software risk controls relating to things like coding standards and compiler switch options in place of actual testing.

Click to expand...

Possibly unintentionally, you are actually arguing for less coding QA and more coding QC. To me the most important question about this (as a QA professional, not as a SW professional, which I'm not) is how can you guarantee the effectiveness and completeness of SW testing? IMO, you can't, because regardless of field and type of objects being tested, testing can only be indicative, never a proof. This is why you can never achieve AFAP through tetsing - one could always argue that you could have tested more, achieved a higher level of certainty and thus lowered the risk a bit more. I'm not arguing that this is effective, or that you should (I think that the AFAF concept, which ignores context, is ridiculous in any technical field), just trying to highlight that relying only, or mostly, on testing, for mitigating high risks, is problematic.

Peter Selvey said:

So ti's good to have these examples on hand and ask the auditor to explain how SWF100 could work in practice.

Click to expand...

While this may be correct in principle, I think it's unlikely to work in practice. I believe most auditors operate based on some policy dictated to them from above, be it through internal (auditing body) policies, procedures and documents. through their training or through highlighting external documents that the auditing body "swears by" (and yes, in the absence of something better they might be "just" guidances, even this is technically illegal - see the EU's MEDDEVs). Therefore I predict that most auditors would insist on following such a line rather than dedicate significant audit time to arguing their own personal conviction / logic in the matter at hand, which is likely to be aligned with the auditing body's anyway. Especially when the topic is as complex and controversial as this.

Peter Selvey said:

Annex ZA is similar. It's well established that companies overuse warnings and cautions. But the other extreme - to declare all "information for use" as being banned as a risk control is so far out of whack with reality that it becomes obvious that this was just a bit of a rant by the author, which was never screened by the EU lawyers because it just an informative annex. I considered writing an article on this for my website about the "Annex ZA misdirection" but I found it was already well covered by several sources (check out the BSI presentation), so I presumed the situation was clear. If there are still auditors misreading it ...

Click to expand...

You keep coming back to the Annex Z issue so I think that a broader clarification is important here.

The "Annex Z practice" goes across many harmonized standards (I believe that now it's included in every standard being harmonized). These Annexes are "informative" in the sense that no one is obliged to comply with them in order to comply with the published standard itself. They merely "inform" (and in that, clarify), compliance with which normative parts of the published standard should provide the Presumption of Conformity with the associated EU Directives/Regulations, and with which regulatory clauses. In this context it's obvious why they're "informative" - compliance with harmonized standards is in principle not mandatory for the purpose of EU regulatory compliance. A manufacturer is at perfect liberty not to comply with them; but then they'd still have to comply with the applicable regulations themselves, and the question would be how, and do they actually comply. What the "Annex Z system" is trying to provide is some sort of clarification of (or interpretation of, or guidance for) the regulatory requirements, which are otherwise sometimes quite vague or in dispute. You could argue that the EN ISO 14971:2012 Annexes Z did a bad job at that (and I'd wholeheartedly agree), but that was the intention, and it's not unique to ISO 14971. So I think that clinging to the fact that Annexes Z are Informal is missing the point. Yes, the Annex is Informal but the regulations it's connected to are mandatory. The real question is how the word of the regulations should be interpreted.