Defining user groups for summative evaluation

Hello,

I'm solliciting your advice regarding the characterization of our user groups for the summative evaluation.

Context:
We're developing a Class I medical device that aims to collect motion signals. We defined 4 intended user profiles and 2 intended use environments:

• Profile 1: Subject/patient
• Profile 2: Researcher
• Profile 3: Healthcare professional
• Profile 4: Home use helper

For each profile we defined characteristics regarding age, prevalent physical/mental condition (for subject/patient profile), educational level, minimum knowledge required and minimum experience required for proper product usage.

• Environment 1: hospitals, rehabilitation centers
• Environment 2: patient's home

Profile 1 should use the device in environment 1 & 2, profiles 2 & 3 in environment 1 and profile 4 in environment 2.

According to 62366-1 and FDA guidance, user groups should be representative of the user profiles, and the test environement should be representative of the intended use environement.
We already intend to have one user group representative of profiles 2, 3 as they have the same characteristics & requirements except for their educational level. All hazard-related use scenarios are performed by these 2 profiles, and their educational level has no consequence on their execution.

However a few questions arise:

We would like to include profile 4 into this group as they perform the same hazard-related use scenarios as profile 2 and 3 and have the same characteritics as profile 3. However unlike profile 2 & 3 they would intervene in patient's home environement only.
In the use scope of our device, environments 1 and 2 have no relevant difference (a rationale can be provided): the patient will only be asked to move around with the device.
Is it far-fetched to still have profile 4 represented in this group? Or should they defacto be in a separate group as they don't have the same use environment?

In theory, profile 1 can perform all hazard-related use scenarios, and as explained beforehand we don't require any type of experience or previous knowledge from any of the profiles, so there is no relevant difference between the 4 of them.
If they aren't physically/mentally able to, profile 4 will perform these scenarios for them. The only use scenario that is specific with profile 1 and doesn't require any help from profile 4 is moving with the device. This scenario doesn't lead to any hazard.
Considering this information, is it relevant to have another user group representative of profile 1 only?


I tried to be consise but still precise, so let me know if you need more clarification.

Thanks.

With 'lay users' you can have no expectations for training, education, etc. If your profile 4 folks have some training or education level then that might move them to a different user group. There may be other factors as well. For example, if the caregiver has multiple patients, there could be a potential for mixing up information.

yodon said:

With 'lay users' you can have no expectations for training, education, etc. If your profile 4 folks have some training or education level then that might move them to a different user group. There may be other factors as well. For example, if the caregiver has multiple patients, there could be a potential for mixing up information.

Click to expand...

No training will be provided to any of the profile, they are required howeved to have read the IFU.
Our only exception is profile 1 from which we require to understand the explanations provided by a user from profile 2/3/4, as to cover the possibility that they might be not able to read and understand the IFU.

That's why as of today we're thinking about having :

• 1 user group that covers profile 2/3/4
• 1 or 2 that cover profile 1. The subject or patient's condition might have an influence on their interaction with the device, so grouping them with the others profiles is pushing the limit. However we don't yet if it would be better to group all types of patients into one group, or have one group with supervision (taht cannot perform all tasks) and one without (that can perform all tasks).

Is profile 4 a professional with healthcare experience or a lay user? If you can explain how the difference in use environment would not be expected to impact the user interface with the device, you may be able to rationalize that your test environment is representative of both home and clinical use. However, if you are calling out different user groups, I think it will be a challenge to convince FDA that combining them is appropriate.

In my experience, FDA is kind of fussy about user groups and will want a very thorough rationale to explain why user groups can be combined. Even then, you run the risk that they disagree and want 15 users for each group you've identified, full stop. Their POV tends to be, if your users are different enough to be considered distinct user groups, you need to evaluate them separately. Even if they are performing the same tasks, their job and experience level may impact how they interpret your labeling and execute the tasks. It's important to note that even if you don't require a certain level of expertise, users will be coming in with different levels of experience so you need to consider that in your analysis.

Solyneu said:

• 1 or 2 that cover profile 1. The subject or patient's condition might have an influence on their interaction with the device, so grouping them with the others profiles is pushing the limit. However we don't yet if it would be better to group all types of patients into one group, or have one group with supervision (taht cannot perform all tasks) and one without (that can perform all tasks).

Click to expand...

You should have 2 groups to cover "patients that cannot perform all tasks" and "patients that can perform all tasks" since clearly, these patient populations have different ability levels. You also need to explain how the users are determined to be "capable" or not by the HCP.
Another note on this: If you have a user group that requires supervision for the study, FDA will not let you evaluate the adult participant for 2 user groups (eg, if you have an adult who attends with their child, they can't be part of Child Participant #1 and also Adult Participant #1).

Thanks for your answer

EmiliaBedelia said:

Is profile 4 a professional with healthcare experience or a lay user?

Click to expand...

They would be lay users indeed, most of the times a family member.

Then no, you will not be able to combine them because their education and experience will be different. You should have separate groups for lay users and professionals.