Defining Internal Audit Process Goals & Objectives

Phew, I was begining to wonder..........

Bill Pflanz said:

Just so you don't feel lonely Andy, I totally agree with you. The problem is that Kreco wants to know how to put a 5th wheel on a wagon. The fact that it is not needed is being lost in all of the ideas on how it should look and work. Even worse it smells of setting a performance objective which can then be used to beat the quality manager over the head.

Bill Pflanz

Click to expand...

Thanks Bill! My primary concern relates to counting the number of non-conformances which tells you zero/nothing/nada/zilch about the effectiveness of the audit - because there are too many variables to control! I've seen companies who want to track the number of audits performed to schedule:mg: The number of external audit findings is just a bogus number too

I'm beginning to think that there are some folks managing an audit program who have too much time on their hands

Andy

You can measure the value of an audit program!

To all,

Being a professional TS 16949 / ISO 9001 / ISO 17025 3rd party auditor and also an instructor of the IRCA 5 day QMS lead auditor course I have thought much about this subject and thoroughly enjoyed this discussion thread.

I start my audit of the audit program with the same question every time, "Can you quantify the value that the audit program has provided to the organization over the past year?".

Think about it... It's not that you are just performing internal audits, it's how well you are performing them that matters. Thank you Helmut, "Gee, the process approach is a cool thing".

Many of the measures that have been proposed are just measures of conformance, not of effectiveness.

So, as a consultant I have created the following forms and tools for internal audits that provide measurements of the auditee and the auditor / audit program. When the auditor writes an OFI (see problem solver and improvement form), the auditee ranks the value 1-10 (see ranking criteria). This is a measure of the audit program effectiveness. When the auditor writes a NCR, it gets a severity value 1-10 (does anybody else love FMEAs?) so you can look at total / average severity of the audit findings instead of the total quantity, which is a very subjetive and misleading measure. The auditee gets a feedback survey on auditor performance which is also used to measure the audit program. All of these indicators are summarized and presented at management review in the audit scoring matrix.

You can find more free stuff like this on my web site (see my profile). I know that there has to be even better ways of doing this and was looking to find some ideas here. Please send your improvements to me through my web site and I will add them. I would especially appreciate it if someone would improve the ranking criteria and audit feedback survey questions.

Enjoy and Good luck!

PS- I know that my forum name can be misinterpreted (I am not In Search Of a boy!). That's not what I was going for and it's too late to change now. I'll think about it more next time I choose a name...

Additional thoughts on auditor competence

An audit program is only as effective as the competence of the auditors. Too often we send internal auditors to a 2-3 day course (if they are lucky) and call them trained and competent. This is hardly the case. How many audit program managers are identifying additional competencies for internal auditors and continually improving their auditor competencies? Here is a matrix to identify, manage, analyze and measure auditor competencies. This could also be used as a measure of audit program effectiveness.

Again, any improvements would be appreciated.

People are a good place to start.......

isoboy said:

An audit program is only as effective as the competence of the auditors. Too often we send internal auditors to a 2-3 day course (if they are lucky) and call them trained and competent. This is hardly the case. How many audit program managers are identifying additional competencies for internal auditors and continually improving their auditor competencies? Here is a matrix to identify, manage, analyze and measure auditor competencies. This could also be used as a measure of audit program effectiveness.

Again, any improvements would be appreciated.

Click to expand...

But hardly the only issue of an effective audit program. I agree totally about the training. In fact I believe the whole auditor training requirements have remained stagnant for nearly 20 years. Just check out the requirements for the 'old' IRCA auditor candidates and compare them to the current (now obsolete) RAB requirements. The RABQSA program goes some way, but it's way complex for most, and who follows through with full certification - especially at nearly $600 a time? Managment wouldn't support it like they do 6 Sigma.

6 Sigma, on the other hand demands better training/competencies in my view, especially if you equate a Green Belt to an auditor and a Black Belt to a Lead auditor..........:mg:

Having a complete audit program that management support - because they understand it - with competent auditors - now that's world class........

Andy

Re: You know that I have a different view........

AndyN said:

well IMHO internal audits have little or nothing to do with improvement, as currrently practiced Well, c'mon now, how many organization would have an internal audit program if 'ISO' didn't 'Sayso'??

So, here's my take on audits. They're to validate that the process was being followed, when the process performance results are reviewed by management, who can then decide what to improve and correct based on whether their people were following the process - after all, just getting good results isn't the whole story.

I have yet to see an organization where the internal audits are used by management to 'go after' improvements. Furthermore, what auditors have been equiped, like a 6 Sigma Green Belt, with the tools to identify/evaluate/measure/report improvement?

Seriously folks, who are we really kidding?

When we see an organization putting the same kinds of resources behind an internal audit as they put behind 6 Sigma, I'll believe it is about improvement, until then it's just a 'feel good' thing to discuss here

Andy

Click to expand...

Thank you. I was beginning to think that I was alone in a world where none of these high level theories apply. I am now tasked to come up with Goals and Objectives for my position of Document and Audit Control ... I have had 5 different Managers with different ideas on a QMS. I have been at one extreme where I have been left all to myself to determine what the audit goal is or the other extreme have been contradicted at each step of the way. Some Managers never answer a CAR as they don't consider anything in their department as deficient..despite external and internal audit results to the contrary. I can only assume their Goals and Objectives do not include supporting the QMS but in dollars and cents. How do you set goals and objectives for Document/Records Control and Audits in such an environment?

Re: You know that I have a different view........

xfngrs said:

Thank you. I was beginning to think that I was alone in a world where none of these high level theories apply. I am now tasked to come up with Goals and Objectives for my position of Document and Audit Control ... I have had 5 different Managers with different ideas on a QMS. I have been at one extreme where I have been left all to myself to determine what the audit goal is or the other extreme have been contradicted at each step of the way. Some Managers never answer a CAR as they don't consider anything in their department as deficient..despite external and internal audit results to the contrary. I can only assume their Goals and Objectives do not include supporting the QMS but in dollars and cents. How do you set goals and objectives for Document/Records Control and Audits in such an environment?

Click to expand...

I think my friend Andy may be a wee bit pessimistic. Many companies do in fact try to get improvements out of their internal audits. However, many times, the audit programs are not set up to produce improvements - only compliance.

1. So, having a program designed for improvements is an important step. (I have developed software designed to do that.)

2. Many managers want to "improve." But many do not realize that the ISO based intenral audits "can" do that, if they are done correctly. Thus, the internal audits become percieved as a "non-value-added" exercise. I have that conversation almost every week. But, done correctly, they can provide much value. And, if you use a well developed program, they become a very useful exercise embraced by management.

3. If top mgt does not support and impose prompt responses to corrective actions, that should be flagged as a nonconformity. I wrote that several time in recent weeks. They are "obligated" to support and promote the corrective action process, if they are certified to ISO. You can "steer" your auditor to this issue, if he does not "see" it.

It is a problem, but it is a solvable problem.

RCBeyette said:

While the Auditor may not be quite on the same level as the Devil, there may be a reason that you're receiving as many findings are you are. To have a goal to reduce the number of external findings is not beneficial in my opinion. If there is no change to your management system (i.e., if you do not plan to improve your system), you can't expect the number of findings to go down...at least, that's how I see things.

Click to expand...

RCBeyette is right in this statement above; As Deming said, if your going to improve xyz by 5% this year why did'nt you do it last year? If your going to improve xyz by 5% this year why not 10%?

If you could not do better last year, by what method are you going to do better this year? I.e what has management done to improve the capability of the process to deliver 5 or 10% this year? (I say management because they own the system, not the workers.)

This is the problem with numberised targets, which management tend to set because its quick and easy to do and because they have no understanding of what Deming is on about here.

Wee bit pessimistic, Helmut? Sadly, I've not seen any of your clients!

Re: You know that I have a different view........

xfngrs said:

Thank you. I was beginning to think that I was alone in a world where none of these high level theories apply. I am now tasked to come up with Goals and Objectives for my position of Document and Audit Control ... I have had 5 different Managers with different ideas on a QMS. I have been at one extreme where I have been left all to myself to determine what the audit goal is or the other extreme have been contradicted at each step of the way. Some Managers never answer a CAR as they don't consider anything in their department as deficient..despite external and internal audit results to the contrary. I can only assume their Goals and Objectives do not include supporting the QMS but in dollars and cents. How do you set goals and objectives for Document/Records Control and Audits in such an environment?

Click to expand...

It's really, really important for an organization to hold people accountable for resolving nonconformances. In my place we send it up to the next level of management if there is no response, no fix or the fix is ineffective. But this punitive-looking response to problems has benefited from a temperate approach overall. Over the years of doing internal audits I have learned some things that have really helped:

1) Allow "push back." I have taken to sending out a "draft" audit report and giving my people about a week to rebut my findings. I word the offer to say something like "There may be some new information" or some such. I say that if I don't hear from them on xx/xx/xxxx, I will issue as-is. This has helped for two reasons: it gets them to rise to the surface and think about quality, and it makes it clear I'm not trying to dictate things to them.

2) In cases of unresolvable dispute I "punt" to the next management level, explaining that I'm not the end of the story here. Other people make policy, I just interpret the standard and point out where we don't meet it. I make a meeting and present the issue as matter-of-factly as can possibly be done. Either the next manager up makes a command decision to direct the middle manager to behave, or that the organization does not wish to address the issue. If that happens, you can document as such and close the nonconformance. After all, every horse can reserve the right to die of thirst.

With these two soft touch approaches I have made a good deal of progress establishing myself as an auditor who is reasonable and keeps the organization's best interests at heart. Interestingly, while one might think I set myself up to be identified as a wuss, the opposite has happened. I have lasted in my position while others have moved around; When a new manager gets in place I can "get him up to speed" by sending him past audit reports and describing what happened. Of course I don't miss a chance to make my recommendation. I got to do that earlier today, with the SPC program. Next week I'll do it with MSA, with a different new manager.

It's possible to have a meaningful effect but it can take years. It's taken me almost six years, but when I am of the right mind I can reflect on it and think "I have accomplished X, Y and Z."

It is difficult to set out with an expectation to accomplish X, Y and Z. We work with what we have, and sometimes that isn't a lot. But my boss and I set out to turn our audit program from a compliance-based program to risk based. Early on he asked for me to write a proposal and I gave him Beyond Compliance. Later on we developed Risk Based Auditing. Eventually I gave his boss a plan for Global Performance Audit Management.

I also did some work on my document control process. I attached the project report.

Maybe these will give you some ideas. But I urge you to consider them carefully against your organization's dynamics, culture and goals. I mean overall, not the most difficult individuals; they are always among us.

I hope this helps!

Re: You know that I have a different view........

Jennifer Kirley said:

Either the next manager up makes a command decision to direct the middle manager to behave, or that the organization does not wish to address the issue. If that happens, you can document as such and close the nonconformance.

Click to expand...

I reckon that’s an excellent approach Jenifer, QA people should not be expected to be the 'cause champion' and be percieved as the villain at the same time. You have found a way out of this dilemma.

I would just like to know how you write up the closure when management does not want to address the issue?

What happens when the CB auditor comes along and sees this approach in action. I presume you disguise it somehow is that right?

thanks