Recently, one of the authors was in a conversation with a food processing client. The client stated that they used a third party audit firm to conduct sanitation audits of suppliers on their behalf. The audit report gave a glowing account of the operational effectiveness of the supplier; however, when the client visited the supplier several weeks after the audit, they encountered numerous audit findings. So, why didn’t the audit report agree with the observed performance? The fact is that criticisms often made about third party audits may provide the answer. These criticisms point out that most of the auditor’s time on-site is spent reviewing procedures, checking for signatures on records and talking with managers, with only limited amounts of time spent on the plant floor observing manufacturing and talking with the operators.

So, can audits be made more effective? The traditional third party sanitation audit is really an inspection. The auditor uses a checklist to determine what elements are to be addressed in the audit. This checklist may require that the auditor evaluate a certain number of procedures, plans and documents. An inspection checklist typically includes some of the following questions for the auditor:

1. Is there a documented sanitation program?
2. Are the specifics of the sanitation program followed?
3. Are there sanitation standard operating procedures (SSOPs)?
4. Is a pre-operational audit conducted prior to production?
5. Are corrective actions taken in response to pre-op findings?
6. What are the general conditions in the plant? Are the floors, walls and ceilings in good condition? Are the exterior structure and grounds maintained in a condition to prevent contamination of food?

Typically, a week before the audit, quality assurance (QA) personnel scurry around the plant to prepare for the auditor’s visit. Hazard Analysis and Critical Control Point (HACCP) plans are reviewed, records are checked for signatures, and the corrective action log and other documents are brought up to date. Some training may be done to ensure employees can answer audit questions correctly. QA works with operations to ensure the plant has a thorough cleaning the night before the audit. Then, just before the auditor arrives, there is a final pre-audit audit to ensure that the plant is looking good. Experienced QA personnel know just what to do to ensure that the plant receives a good audit score for that snapshot in time of the auditor’s visit.

An audit’s true function is to provide the client with an accurate and detailed picture of the plant’s performance. The client wants to know what is really happening in the plant day after day, and whether actions need to be taken to improve the plant’s performance. Unfortunately, this function is not achieved when an auditor walks into a facility and conducts an inspection. The inspection does not dig deep enough into a company’s activities to determine how the business normally runs from day to day. This surface level auditing has been the norm for the majority of sanitation audits conducted in the U.S. Customers ask, if the traditional audit process does not work, what will work? How can a client get an accurate assessment of a supplier’s performance? The key is to have processes in place to allow the auditor to understand how the organization’s food safety management system works. In addition, the audit process needs to assess if critical processes are not present or if critical procedures are weak.

A Turtle in the House
Evaluation of written documents only provides a start to answering these questions. The written documents provide the formal description of the management practices and the records serve as evidence those practices are being applied (or not applied). The auditor must determine if the informal management process or culture is in harmony with the documented procedures. The informal management process describes how a plant actually operates. For example, the authors have visited a number of organizations that publish very strong quality policies. However, when the authors visit the plant floors to observe production and question line employees, they often discover that the actual production practices are not in alignment with the stated quality policies.
Work is conducted within an organization in a system or a set of interacting activities or processes. No activities are conducted in isolation. They feed into each other through a series of inputs and outputs. The process turtle is one way to depict a process and the linkages to other processes (Figure 1). The tail, body and head of the turtle is the familiar SIPOC (Suppliers, Inputs, Process, Output, Customers) model from Six Sigma. The SIPOC model provides a description of the flow of a product through a manufacturing system. The process approach adds the enabling technologies and support processes that allow the system to function properly. A process audit uses the process turtle model to evaluate the way an organization identifies and manages the individual processes and the linkages between the processes. In addition, the audit determines if these processes are effective and efficient.

A process audit is conducted by reviewing procedures and documents, and asking questions to individuals directly involved with the process who perform work that is linked to the process. Next, the auditor determines if the responses are consistent and in alignment with documented policies, objectives, procedures and records. If the responses are consistent, then there is evidence that the system is functioning properly. If the responses are not consistent, then the auditor will have to continue to search to determine the reason for the inconsistency and to collect evidence which support the inconsistency. This is known as creating and following audit trails. The auditor seeks to find the reason for the inconsistency and then link it back to the management system and a standard, so that the supplier can identify and address the inconsistency. With this system, the detailed audit checklist is substituted for a less detailed checklist (a memory aid) that focuses on evaluating multiple elements of a food safety management system at the same time.

For example, an auditor is to determine if a HACCP plan is being properly followed. The auditor may start by reviewing the HACCP plan. Next the auditor may observe a QA technician performing a temperature check at a critical control point (CCP). The individual uses a digital thermometer to record the temperature of a piece of cooked meat and records the results on a glove. The results are then transferred to a QA form. At the end of the day, a clerk enters the data into the QA database. An auditor conducting a process audit may ask the QA tech some of the following questions at this point:

• Would you please explain to me what you are doing?
• Why is this job critical?
• What is the critical limit for this test?
• If a test result falls below the critical limit, what do you do?
• When was the last time you found a result below the critical limit?
• Why are you recording your results on a glove?
• What happens to the glove after you record the results?
• How do you know that the thermometer is reading the correct temperature?

Of course, the ultimate auditor follow-up to these questions and answers is: “Show me.” The answers can be verified by comparing performance and results to procedures and records. The auditor would likely then ask the QA tech’s supervisor:

• Would you please explain the calibration process for thermometers?
• What do you do if a product temperature is below the critical limit?
• What is the process for handling potentially unsafe product and recalling the product?
• How do you test these processes?
• What happens if a QA tech or a QA clerk makes mistakes when entering or recording data?
• How do you use this data to ensure food safety?
• How to you use this data to update the food safety management system?

These answers can be verified by comparing performance and results to procedures and records. The auditor could likely then ask the QA clerk:

• Would you please show me how you enter data into the computer?
• How do you verify that the data are entered correctly?
• What happens if you make a mistake in entering data?

The answers can be verified by comparing performance and results to procedures and records. The auditor would then approach other individuals that link to this process. For example, the auditor can ask the person responsible for calibrating the thermometer about the calibration process, or the person responsible for maintaining the computer database about the format and maintaining the integrity of the data.

This series of questions provides documented evidence for a large number of elements required by the International Organization for Standardization (ISO) 22000 food safety management system standard, including control of documents and records, identified responsibility and authority, communications, establishing the HACCP plan, verification planning, traceability, control of nonconformity, control of monitoring and measuring, food safety management system verification, and improvement.

When open-ended questions are asked and audit trails are followed, it is very difficult for a supplier to distract an auditor away from the execution of a thorough, in-depth audit with artificially “sanitized” answers. The only way to prepare for this type of audit is to do the right thing day after day. Most of the time line employees are willing to tell you what they do, even if they are not doing their job correctly. In addition, by investigating the linkages, one can determine if the processes are operating in a manner that is consistent with stated policies and objectives.

The Power of Improvement
The process audit is a powerful tool. It allows the auditor to go beyond inspecting the cosmetic issues in a plant. The process audit is designed to understand how the plant functions day to day. Cosmetic issues, such as determining if HACCP plans are signed and up to date, are still audited. However, these issues become part of the audit not the primary focus. When an auditor sees a potential finding in a cosmetic issue, this issue sends a signal to the auditor. The auditor responds by digging deep into the process and other supporting processes. This technique forms an audit trail that allows the auditor to determine whether the organization’s system is effective to manufacture safe food. The audit report then becomes a record of the supplier’s performance on that day with an added indication of how that supplier operates on a daily basis.

John G. Surak, Ph.D., is the principal of Surak and Associates and provides consulting for food safety and quality management systems, auditing management systems, designing and implementing process control systems, and implementing Six Sigma and business analytics systems. He led the U.S. delegation to ISO committee meetings that developed ISO 22000. Surak can be reached at jgsurak@yahoo.com.

Tatiana A. Lorca, Ph.D., is the food technology specialist for BSI Management Systems Americas, Inc., where she is in charge of developing and coordinating food safety training and certification programs for the Americas. Lorca can be reached via e-mail at tatiana.lorca@bsi-global.com.