Update on IEC 62304 revision - May 2012 - 2013 (TBD)

Re: Update on IEC 62304 revision - 4 May 2012 - 2013 (TBD)

Some changes from my recent update due to discussions and decisions at the IEC SC 62A plenary meeting.

The revised classification scheme and legacy annex will now be dealt with in an Amendment to IEC 62304 Ed. 1. This will be developed together with the continuing work on Ed. 2.

IEC 62304 Ed. 2 will have a scope expansion. It will be applied for health software, instead of only medical device software, to align with the development of IEC 82304.

Re: Update on IEC 62304 revision - 4 May 2012

It is difficult to make my head around this standard in general and it is also true for this classification stuff as well. More quite often looks for me like an academic level white paper rather than the real world driven best practise guide.

Before going any further, can I ask a clarification what actually "software failure" and "acceptable level" refer to in the context of this classification please?

Many thanks.
Cheers!

Before going any further, can I ask a clarification what actually "software failure" and "acceptable level" refer to in the context of this classification please?

Click to expand...

Software failure - software not behaving as specified.

Acceptable level - result of a risk assessment per ISO 14971. Meaning, for the related hazardous situations, is the risk acceptable (or not)?

Thanks, I have more or less similar recollection about these terminologies.

Than the question ...

Does it make a medical device software safe in any means if it behaves as specified? ... or at least if we have a reasonable confidence that it's very likely that it behaves as specified ...

And only my side note for the "acceptable level" would be that there are manufacturers are not using at all this concept and considering any remaining level is not acceptable. The wondering remains the risk benefit analysis responsibility.

The more general thing I am not really getting with this standard is to how (on earth) could anyone determine the safety classification of the device at any time earlier than the end of the development. Seriously, in reality there is a fussy direction where to go to for a long long time.
Or a similar question from the different angle, if we had determined initially that it is the lowest safety class but at the end of the development we see that actually it is the highest, than ... what ... do we retrospectively expected to create all the expected deliverables? What value is that retrospective provides?

Many thanks.

Cheers!

Does it make a medical device software safe in any means if it behaves as specified? ... or at least if we have a reasonable confidence that it's very likely that it behaves as specified ...

Click to expand...

Only if the specified behavior does not lead to a non-acceptable risk - but that's why there's a risk management requirement on the standard. In the case of confidence, the idea is that applying the standards does give you an increased level of confidence (it's a kind of risk control measure).

And only my side note for the "acceptable level" would be that there are manufacturers are not using at all this concept and considering any remaining level is not acceptable. The wondering remains the risk benefit analysis responsibility.

Click to expand...

Sure, and that's part of the confusion in understanding the classification scheme, that's why it's getting modified, so people can understand that they do not need to what you said they are doing (hopefully )

The more general thing I am not really getting with this standard is to how (on earth) could anyone determine the safety classification of the device at any time earlier than the end of the development. Seriously, in reality there is a fussy direction where to go to for a long long time.
Or a similar question from the different angle, if we had determined initially that it is the lowest safety class but at the end of the development we see that actually it is the highest, than ... what ... do we retrospectively expected to create all the expected deliverables? What value is that retrospective provides?

Click to expand...

In fact you are expected to defined the safety classification for the first time in the final design...

Although what you said is possible (determine a lower safety class but then evaluate as highest) this is not really good practice. If you are not sure of the safety class, you need to use the highest one (there's even a requirement on the standard for this). So the problem would in principle be that you first determine a higher class, and then find out that it's in fact lower (maybe because you implemented risk control measures that make the software less riskier).

I think there is a difference between reliability and safety. Reliability is a kind of subset to safety.

If we want to make sure that the software behaves according to its specification than that is merely reliability. Nevertheless we assume the specification is accurate, that's another kind of instability in this system.
Safety for me has more or less nothing to do with defective behaviour. That is the goal of reliability.

Let me have a simple example.
Let's have a central data collection unit storing wireless data from sensors attached to patients.
There is the hazard, that actually some wireless data can be lost/falsified due to interferences and it can lead to patient harm due to miss treatment based on the data stored etc.
Regardless how many time this scenario is thoroughly tested to see if it works according to the specification, the hazard is there. That's I have referred to as a reliability aspect.
Whereas if there is an implemented safety intelligence, for example specific/encrypted/redundant/etc. protocol that ensures a higher level of confidence that data can not be undetectably corrupted during wireless transfer, than there is something that we can call as inherent safety.

Going back to the origin of this to the classification 1- .
I think that's neither good enough nor a bare minimum to say,

Identify hazardous situation where software failure can be part of the sequence of events

Click to expand...

because software can go wrong infinite number of ways and infinite number of ways can cause harm on patient regardless how thoroughly tested and how carefully designed. It is the paradigm of software engineering.
Moreover it stipulates that the software failure has anything to do with safety.
I believe to limit the software to fail is for reliability, whereas safety is an additional intelligence built in the software to eliminate the potential caused of the hazardous situation leads to patient harm.


And for this classification stuff.
Please consider this.
Would you spend 3o% more on R&D cost to be prepared even for a not likely to be product at all kind of prototypes or you would just ignore the voluntary standard that says that?
(I really apology, but for me it is an utter nonsense to expect that anyone can determine a safety level of a planned or ongoing development prior having the output of the development in place)
That's the reason I am saying at this stage it is not really considering real life challenges.

Many thanks.
Cheers!

I think there are a couple of good points here from Sagai.

In hardware, we assume that a failure can occur, so protection is put in place. The same approach should be taken in software, rather than relying on increasing levels of design controls, if there is significant risk, protection should be used like data checking as Sagai suggested.

It's often said that software cannot be 100% tested. Great, true, but so what? Nothing can be 100% tested. I have been recently working with humidifiers and found the output temperature and humidity are highly sensitive to many factors, like supply voltage, room temperature, flow rate, room air flow, gas inlet temperature, chamber water level, initial conditions and even in some cases how long you held the sensor in your hand while assembling the set up. The results are not linear so you can't assume highest / lowest combinations will be the worst case. Like software, it would be impossible to test all conditions. But we live with it and get on with it anyway. And you put in reliable protection that operates well before any serious harm can occur.

For classification, the current 62304 makes in effect two simplifications, (1) it assumes the same failure rate for all types of software; (2) it assumes that if there is any possibility of harm, it assumes a 100% probability of harm irrespective of the actual situation. With these two assumptions in place, we end up with a simple home use thermometer being Class C, because if the thermometer software fails, there is a clear possibility of serious harm. This is despite that the software is relatively simple and confident validated even with simple Class A controls; and that the potential for harm is remote even if the software did fail.

Of course we know Class C for a thermometer is unreasonable, but I was recently involved in a case where the FDA rejected a 510(k) application and stated in the letter they believed the software should be Class C according to IEC 62304. The device was diagnostic only, but with a long, remote, improbable sequence of events it was possible the patient could be seriously harmed. And I've heard of NBs also asking for Class C in unreasonable situations.

Marcelo, I'm interested to know roughly where the committee is going for this? Have they fixed the basic problem of these two over-simplistic assumptions? I suspect not, because it gets really complicated (consider Classification in Europe, 6 pages of questions, coupled with a 52 page MEDDEV guide). If the committee wants to keep it simple they will still end up with the same problems.

Going back to the origin of this to the classification 1- .
I think that's neither good enough nor a bare minimum to say,
Quote:
Identify hazardous situation where software failure can be part of the sequence of events
because software can go wrong infinite number of ways and infinite number of ways can cause harm on patient regardless how thoroughly tested and how carefully designed. It is the paradigm of software engineering.

Click to expand...

Yes, and in this case the software failure has to have a probability 100 % (the software failure, not the risk - this is part of the misunderstanding).

Would you spend 3o% more on R&D cost to be prepared even for a not likely to be product at all kind of prototypes or you would just ignore the voluntary standard that says that?
(I really apology, but for me it is an utter nonsense to expect that anyone can determine a safety level of a planned or ongoing development prior having the output of the development in place)
That's the reason I am saying at this stage it is not really considering real life challenges.

Click to expand...

You don't need to spend more if you clearly know your software and how it can lead to problems. But if you don't, your are required to perform the most stringent process.

For classification, the current 62304 makes in effect two simplifications, (1) it assumes the same failure rate for all types of software; (2) it assumes that if there is any possibility of harm, it assumes a 100% probability of harm irrespective of the actual situation.

Click to expand...

Your first assumption is correct, but the second is not - this is part of the misunderstanding with classification that the group is trying to clarify in the second edition/amendment/

The standard requires that the software failure probability must be 100%, meaning, the software will always fail. This means that the software failure, which is a part of the sequence of events leading to harm, has to be 1. This does not mean that P1, or P2, or the probability of occurrence of harm is 1. It only means that part of P1 (which is the software failure) is 1.

Of course we know Class C for a thermometer is unreasonable, but I was recently involved in a case where the FDA rejected a 510(k) application and stated in the letter they believed the software should be Class C according to IEC 62304. The device was diagnostic only, but with a long, remote, improbable sequence of events it was possible the patient could be seriously harmed. And I've heard of NBs also asking for Class C in unreasonable situations.

Click to expand...

This does not seem reasonable, but as I said, this probably comes from the misunderstandings about the standard requirements.

Marcelo, I'm interested to know roughly where the committee is going for this? Have they fixed the basic problem of these two over-simplistic assumptions? I suspect not, because it gets really complicated (consider Classification in Europe, 6 pages of questions, coupled with a 52 page MEDDEV guide). If the committee wants to keep it simple they will still end up with the same problems.

Click to expand...

As I said, the groups is trying to correct the perceived misunderstanding in the second one.

The exact wording in the current standard is:

Class C: Death or SERIOUS INJURY is possible

Click to expand...

Under plain English interpretation, the words "is possible" effectively sets P2 as 100% irrespective of the actual probability. It's not a matter of what the committee thinks or intended, it's the words on the paper in the normative section that matter. It's a global regulatory standard, it should be written in a way that which is interpreted consistently independent from the committee.

Also, if you want to consider P2 as being flexible, then you cannot avoid defining acceptable probabilities for each class. For example, at what probability of death is Class C necessary? 0.1%? 1%? 10%?

In fact, you cannot make a decision unless you consider a realistic value for P1, the critical failure of software. If you use P1 of 100%, then you end up with a Class C limit for P2 at around one in a million, which again makes a thermometer Class C. However, more realistic values for P1 are in the range of 0.01 to 0.0001 depending on the nature and complexity of the software. A simple thermometer is towards the 0.0001 side, and the probability of serious harm is around 0.01 or less, making the overall probability OK without Class C controls.

There is a wide variation in the types of software, and the actual critical failure rate of software is very small considering the volume of software on the market. So allocating a single figure of 100% is a serious departure from reality, so obviously it will result in problems in classification decisions.