Capabiliy Analysis comparison (Binomial vs Nonnormal Weibull)

I am looking to provide a performance/capability report for a process. The process is determined by sampling a GO/NO GO attribute. The target is a one-sided specification with the lower spec being just 0. In the example below I have 25 subgroups with a constant sample size of 373. I am using Minitab to perform the calculations. Outside of two points on the UCL line, the process is stable.

I have performed a Binomial Capability Analysis initially to evaluate the process. See screenshot below. My concern in the screenshot below is that the binomial may not be the best fit given the Plot.




In questioning the binomial capability analysis, I selected a better fit with the 3 parameter Weibull with a p value of > .5 (normal works too in this case but the data will not always show normal as it improves, so I excluded it). I selected capability analysis -> Normal/Nonnormal -> set column and subgroup size -> set lower spec to 0 as boundary and upper spec. The normal capability showed overall performance ppm > USL to be (3789). The nonnormal Weibull capability showed performance ppm > USL to be (9018). See screenshot below of nonnormal.





“My confusion is in which capability is the best representation of the process as far as ppm is concerned, given there is such a large difference between the two.” I’m probably missing something simple here. Any help is appreciated.

Every time-independent binary random variable follows a binomial distribution. Thus, if you had a stable process, the data would follow the binomial distribution. However, this is not the case in your dataset -- look at the SPC chart. Hence, the I would argue that your process is not sufficiently stable to use the capability index as a good forecast. Hence, I would optimise the process -- or maybe its a measurement problem.

In addition, I would also argue that the Weibull distribution does not describe the dataset. You might want to calculate some "quality parameter" fr the fit to obtain a quantitative result, but remember: Using a variable/loose fit distribution usually increased the bias. This is the reason why flexible distributions should be used with care.

The reason that it does not fit a binomial distribution is because:

• It is not in control. Note the cumulative % defective is still increasing.
• It may exhibit overdispersion. The Binomial plot also indicates over-dispersion. Run the P-chart Diagnostic in Minitab. You may need to use a Laney P-chart to correct for over-dispersion.

I would be very wary of the binomial fit for this, as the variation seems too high. I would (per Dr. Don Wheeler) plot this on a x-individuals and moving range chart. Likely you have a trend, and/or something is happening that is making the results NOT independent from each other.

Dr. Deming tells a story of a shoe factory that was making too many defective product, at about 9.5 percent. But when he plotted the data, the variability was actually TOO LOW for the binomial distribution. That led him to start asking questions. Come to find out, the QA inspector, when the day's count was approaching what would give a 10 percent rate, EVERYTHING PASSED. This was due to it was "common knowledge" that if the rate of defects exceeded 10 percent, then the factory woudl be shut down, and everyone out of the job. HMMMM.

With all due respect to Steve and Miner: so much math, so little insight. The purpose of a ‘capability study’ is to understand the process capability. And then to initiate improvement if it isn’t capable enough. It is not to determine some theoretical distributional fit.

I would first say that the process is NOT capable of meeting the specifications as it is a chart of defects. And the average defect rate is ~2% going as high as 4%.

Secondly there is an out of control LOW set of points at 14 and 15 (2 of 3 sequential points near the lower limit) and we don’t need limits to see that. As Ott so famously said: plot your data and THINK about your data. Reams of statistical manipulation are not thoughts about the data. What question are really asking? This OOC low condition is a clue as to how to improve the process. Why are we ignoring it? Sure the process isn’t ‘stable’ but that won’t stop us from improving it. Maybe we can find a chart that will have this process ‘look stable’ what ever that is, but why? To what end? Back to point 1. (A Laney p chart or I, MR chart would be better choices if the only goal here is to monitor and alarm the process although the defect rate is fairly high for a Laney chart)

The third thought I have is why wouldn’t you use continuous data? I get that a go-no go inspection is more efficient for production but for process control there is far better insight to be gained from a continuous data measurement. The sample size can be fairly small, much smaller than 373.

My take on Ott: “the manipulation of mathematical formulas is no substitute for thinking.”

And the succinct summary: The process is not capable, it is not stable and it is getting worse.
Is that OK? What is your role in this organization? Why are you performing a capability study? What is the end goal?

Bev D said:

With all due respect to Steve and Miner: so much math, so little insight. The purpose of a ‘capability study’ is to understand the process capability. And then to initiate improvement if it isn’t capable enough. It is not to determine some theoretical distributional fit.

I would first say that the process is NOT capable of meeting the specifications as it is a chart of defects. And the average defect rate is ~2% going as high as 4%.

Click to expand...

Sadly, you are correct. I got focused on the specific question asked rather than the underlying issue.

Miner said:

Sadly, you are correct. I got focused on the specific question asked rather than the underlying issue.

Click to expand...

Yeah - happens to me too sometimes. The math is seductive. Easy to get lost in it.

Bev D said:

And the succinct summary: The process is not capable, it is not stable and it is getting worse.
Is that OK? What is your role in this organization? Why are you performing a capability study? What is the end goal?

Click to expand...

First, let me say thanks for the supportive responses. I came into the organization as a QE to assist with their manufacturing process. This is a homegrown continuous flow molding process. Prior to me the manufacturing group did not have any interaction with Quality. The need for better quality has come with the growth of the organization. No one has ever provided information to the production group to better understand the quality yield of the process. So, my end goal is to be able to give them some level of insight into how the process is performing. I agree it is not stable or capable.

The customer specification right now is 5%, with our internal control being 3%. Prior to me coming in they would run on average a defect rate of 9-10%. There are still times where we push the line over 3%, but generally we can stay below 5%. The customer would like better than 5% though and I believe we can most definitely get it under 1%.

I have worked with them on identifying causes that contribute to better or worse control (i.e., proper changeover/maintenance at the floor level) and some process design changes. There has been a lot of resistance from production and process engineering unfortunately. We finally have some design changes that we believe will help continue to push us in the right direction.

As far as continuous data, there is a weight that is recorded, and it should fall within a certain tolerance. However, it falling in this window doesn’t always mean that the product will pass visual inspection. Hopefully with some of the process changes coming up it will make the weight a more viable indicator of the product’s visual appearance. It is used to ensure the product is of a certain weight, so it’s not entirely useless data.

OK now we understand enough o provide some good advice. Keep it simple. Trend control charts and Paretos of defect types (not causes). Use Don’t even think about statistical distribution fitting or p values or stuff like that. Your production people won’t understand it, won’t gain any actionable insight and will just think of you as a voodoo, woo-woo math geek. Not someone who is out to help them. And while I might find the math interesting I have never calculated a p value or performed an ANOVA, t-test or other fancy stats test but I have solved hundreds if not thousands of complex problems over my 40+ year career.

As already stated control charts do’t rely on any theoretical distribution. Just calculate the limits and use them to provide insight. The limits themselves are not the goal, insight to drive improvement action is the goal. Start with a look at my resources and MIner’s resources. read Donald Wheeler (SPCPress.com) and Bob Emiliani’s blog…help your production people solve the problems