This is the fifth in a series of articles about MSA. The focus of this article will be on measurement repeatability and reproducibility commonly referred to as a gage R&R study.
This article will deal solely with the AIAG MSA methodology. The AIAG methodology is the methodology required by many customers, particularly in the automotive industry. Whether you agree with it or not, it is a standard approach and has widespread acceptance. Most suppliers have no option other than to comply. I will deal with other approaches in a later article.
This is where calibration completely separates from MSA. There is no equivalent in calibration to R&R. Calibration, bias studies, linearity studies and stability studies have all focused on measurement bias. R&R studies focus on measurement variation. Let’s first start with definitions. What is Repeatability? What is Reproducibility? What is an Operator by Part interaction?
Repeatability is the measurement variation observed when a single operator measures one part multiple times.
Reproducibility is the measurement variation observed when multiple operators measure one part multiple times. Depending on the measurement system, AND how the MSA study is designed Reproducibility may also be the measurement variation observed when multiple measurement stations or devices measure one part multiple times. For example, a measurement device may consist of a fully automated measurement device comprised of multiple stations. Each station is dimensionally unique and the difference contributes to measurement variation. Another example is multiple, fully automated measurement devices that measure the same characteristic. Each device has a slightly different measurement bias and contributes to the measurement variation. Yet another example is a semi-automated measurement device that is manually loaded. The resulting measurement is influenced by the manner in which the product is loaded into the fixture. Each operator that loads the product has a slightly different technique for loading that influences the measurement variation.
Operator by Part Interaction is a situation where the result of an operator’s measurement technique is influenced by the part itself. For example, two operators measure a shaft diameter using techniques that are identical in all respects except one. Operator A takes measurements at the midpoint of the shaft length. Operator B measures at one end of the shaft. Two shafts out of ten have burrs on the ends. Operator A’s measurements are not affected by the burr. Operator B’s measurements are affected by the burr. This will result in an interaction between the operator and the part itself.
Part Selection
The first step in an effective R&R study is to determine the use of the gage itself. Will it be used for part inspection to a tolerance, for process control, for statistical studies (e.g., a hypothesis test, capability study, DOE, etc.), or for a combination of these? This is very important because it influences the selection and quantity of parts needed for the R&R study.
If the gage is used solely for part inspection, the selection of parts is not critical because the part variation is not included in the calculation of the R&R metric, %Tolerance (i.e., P/T Ratio). Some will recommend that parts representing the full spread of the tolerance be used. While this does not hurt, it is not really necessary. If a gage linearity study has been performed, the change in bias over the tolerance spread is known. If a gage linearity study has not been performed and there is a linearity issue an R&R study will not detect it.
If the gage is used for process control or for statistical tests, the selection of parts is critical because the part variation is part of the calculation of the R&R metric, % Study Variation (i.e., %GRR). It is vital that the parts selected for the study reflect the actual variation of the process. That is, the StdDev of the parts equals the StdDev of the process. Some statistical packages, such as Minitab, allow the entry of the historical StdDev of the process. If your software has this option, use it, entering the process StdDev from a capability study or calculated from SPC charts. If the software does not have the feature, manual calculations using the historical value are still possible as follows"
% Study Variation = 100 * [StdDevR&R / StdDevTotal Variation]
StdDevTotal Variation = SQRT[StdDevR&R^2 + StdDevPart Variation^2]
Manually substitute the StdDev from a capability study for StdDevPart Variation
How many operators, trials and parts do I use?
The recommended standard is to have three operators measure ten parts three times each. Is this always the best approach? What flexibility do we have in modifying this? To answer this question, we need to look at how the data are used by the ANOVA calculations.
Source of Variation degrees of freedom (n-1)
Reproducibility (3 operators) 2
Parts (10 parts) 9
Pure Error (Repeatability) 78
Total Variation (90 measurements) 89
The 10/3/3 approach provides very good estimates of the total variation and the repeatability. The least reliable estimate of variation will be the Reproducibility because it has the smallest degrees of freedom. If concessions must be made, it is better to run fewer trials in order to maintain or increase the number of operators. The total number of measurements should be maintained near 90. The number of parts may be reduced, if (and only if) an independent estimate of part variation (such as from a capability study) is available and used as described in the previous section.
Selection of Operators
Always use the actual operators that will perform the measurement. Do not use personnel that will not perform the measurement task. Select the operators randomly. Do not handpick the best operators. If only one operator performs the measurement task (e.g., complex analytical equipment), perform the study with that operator only. There is no Reproducibility component in that situation
Measurement of Parts
Parts should be introduced randomly to each operator by an independent party that is not involved in the actual measurements. This is to prevent potential measurement bias caused by an operator remembering a previous measurement and consciously or unconsciously adjusting the next measurement to match.
Parts should be measured using the same method that will normally be used. If Reproducibility is adversely affected by the use of different methods, you need to know that. If there is significant within-part variation in form that adversely affects Repeatability, you need to know that also.
What method do I use?
In the MSA manual, there are two optional methods: the Range method and the ANOVA method. Both methods will provide very similar results. The Range method uses simpler math, but the ANOVA method can detect a potential Operator x Part interaction. If you have software available, use the ANOVA method. It provides additional information. The only compelling reason for using the Range method is if you must perform manual calculations.
continued in next blog entry
This article will deal solely with the AIAG MSA methodology. The AIAG methodology is the methodology required by many customers, particularly in the automotive industry. Whether you agree with it or not, it is a standard approach and has widespread acceptance. Most suppliers have no option other than to comply. I will deal with other approaches in a later article.
This is where calibration completely separates from MSA. There is no equivalent in calibration to R&R. Calibration, bias studies, linearity studies and stability studies have all focused on measurement bias. R&R studies focus on measurement variation. Let’s first start with definitions. What is Repeatability? What is Reproducibility? What is an Operator by Part interaction?
Repeatability is the measurement variation observed when a single operator measures one part multiple times.
Reproducibility is the measurement variation observed when multiple operators measure one part multiple times. Depending on the measurement system, AND how the MSA study is designed Reproducibility may also be the measurement variation observed when multiple measurement stations or devices measure one part multiple times. For example, a measurement device may consist of a fully automated measurement device comprised of multiple stations. Each station is dimensionally unique and the difference contributes to measurement variation. Another example is multiple, fully automated measurement devices that measure the same characteristic. Each device has a slightly different measurement bias and contributes to the measurement variation. Yet another example is a semi-automated measurement device that is manually loaded. The resulting measurement is influenced by the manner in which the product is loaded into the fixture. Each operator that loads the product has a slightly different technique for loading that influences the measurement variation.
Operator by Part Interaction is a situation where the result of an operator’s measurement technique is influenced by the part itself. For example, two operators measure a shaft diameter using techniques that are identical in all respects except one. Operator A takes measurements at the midpoint of the shaft length. Operator B measures at one end of the shaft. Two shafts out of ten have burrs on the ends. Operator A’s measurements are not affected by the burr. Operator B’s measurements are affected by the burr. This will result in an interaction between the operator and the part itself.
Part Selection
The first step in an effective R&R study is to determine the use of the gage itself. Will it be used for part inspection to a tolerance, for process control, for statistical studies (e.g., a hypothesis test, capability study, DOE, etc.), or for a combination of these? This is very important because it influences the selection and quantity of parts needed for the R&R study.
If the gage is used solely for part inspection, the selection of parts is not critical because the part variation is not included in the calculation of the R&R metric, %Tolerance (i.e., P/T Ratio). Some will recommend that parts representing the full spread of the tolerance be used. While this does not hurt, it is not really necessary. If a gage linearity study has been performed, the change in bias over the tolerance spread is known. If a gage linearity study has not been performed and there is a linearity issue an R&R study will not detect it.
If the gage is used for process control or for statistical tests, the selection of parts is critical because the part variation is part of the calculation of the R&R metric, % Study Variation (i.e., %GRR). It is vital that the parts selected for the study reflect the actual variation of the process. That is, the StdDev of the parts equals the StdDev of the process. Some statistical packages, such as Minitab, allow the entry of the historical StdDev of the process. If your software has this option, use it, entering the process StdDev from a capability study or calculated from SPC charts. If the software does not have the feature, manual calculations using the historical value are still possible as follows"
% Study Variation = 100 * [StdDevR&R / StdDevTotal Variation]
StdDevTotal Variation = SQRT[StdDevR&R^2 + StdDevPart Variation^2]
Manually substitute the StdDev from a capability study for StdDevPart Variation
How many operators, trials and parts do I use?
The recommended standard is to have three operators measure ten parts three times each. Is this always the best approach? What flexibility do we have in modifying this? To answer this question, we need to look at how the data are used by the ANOVA calculations.
Source of Variation degrees of freedom (n-1)
Reproducibility (3 operators) 2
Parts (10 parts) 9
Pure Error (Repeatability) 78
Total Variation (90 measurements) 89
The 10/3/3 approach provides very good estimates of the total variation and the repeatability. The least reliable estimate of variation will be the Reproducibility because it has the smallest degrees of freedom. If concessions must be made, it is better to run fewer trials in order to maintain or increase the number of operators. The total number of measurements should be maintained near 90. The number of parts may be reduced, if (and only if) an independent estimate of part variation (such as from a capability study) is available and used as described in the previous section.
Selection of Operators
Always use the actual operators that will perform the measurement. Do not use personnel that will not perform the measurement task. Select the operators randomly. Do not handpick the best operators. If only one operator performs the measurement task (e.g., complex analytical equipment), perform the study with that operator only. There is no Reproducibility component in that situation
Measurement of Parts
Parts should be introduced randomly to each operator by an independent party that is not involved in the actual measurements. This is to prevent potential measurement bias caused by an operator remembering a previous measurement and consciously or unconsciously adjusting the next measurement to match.
Parts should be measured using the same method that will normally be used. If Reproducibility is adversely affected by the use of different methods, you need to know that. If there is significant within-part variation in form that adversely affects Repeatability, you need to know that also.
What method do I use?
In the MSA manual, there are two optional methods: the Range method and the ANOVA method. Both methods will provide very similar results. The Range method uses simpler math, but the ANOVA method can detect a potential Operator x Part interaction. If you have software available, use the ANOVA method. It provides additional information. The only compelling reason for using the Range method is if you must perform manual calculations.
continued in next blog entry
Last edited:
Not really. Thank you for catching that. You are correct and I fixed it.
