One of the greatest challenges a laboratory faces is defending its analytical results. Laboratories whose results are used for process control may be questioned when the results reported do not have the expected outcome on the process. This is often the case with microbial analysis since microbes do not, rarely in fact, behave in the expected way. Whether the results are for indicator organisms used to help identify possible pathogenic contamination or general microbiological results used as an indicator of product quality, the results obtained and the product outcome are not always the same.

Laboratories that produce results used to ensure compliance to product specifications may receive even more scrutiny. These laboratories are sometimes challenged by production management who are striving to maximize efficiency, or may be limited by product availability and quality. This is another situation in which the raw materials going into a process may not correspond to the finished product coming out of the process. For this reason, laboratory results may again be questioned.

For this reason it is critical that a laboratory has programs in place to help assure the confidence of the laboratory results being produced. Sharing such programs with the customer of the laboratory will help build confidence in the results. There are several ways that a laboratory can do this. Basic quality assurance and quality control programs can by used to build the foundation for accurate and precise results. Comprehensive and ongoing training of laboratory personnel ensure that laboratory technicians and management have the knowledge and skills necessary for laboratory analysis. Laboratory audits can be used as a tool for ensuring compliance to the programs established. All of these will provide the laboratory with a means to support their results.

Proficiency testing is another effective tool that can be used to ensure laboratory results. It is also a necessary component for laboratories that wish to become accredited to the International Organization for Standardization (ISO) 17025 laboratory management standard, which provides the basis of laboratory accreditation in more than 40 countries, or other recognized laboratory quality assurance standards—all of which require that a proficiency testing program is in place to checks actual laboratory performance of a process.

There are several different ways that proficiency testing can be designed; however, all are derived from the same basic concept. According to Klesta in the AOAC International publication, “Quality Assurance Principles for Analytical Laboratories 2000,” proficiency testing is “the use of interlaboratory comparisons of results from a number of laboratories to determine laboratory testing performance.”1 In other words, individual laboratories are provided with a sample originating from the same source and the reported results are statistically analyzed to determine proficiency. The samples type and preparation, and statistical analysis employed can be tailored to fit the needs of the laboratory organization.

Participating in a laboratory proficiency testing or check sample program to establish and confirm the accuracy and precision of the lab’s results not only raises the confidence level of the lab and its customers, but answers the larger question, “What good are laboratory results to achieving food safety goals or informing food safety decision-making if they are not reliable?”

Raising Confidence
The most common means of assessing proficiency for microbial analysis is the use of an outside organization that specializes in preparing these types of samples. Although technology has been developed that can enable individual organizations to utilize samples that may be purchased by private companies, this resource is not widely in use in the United States. Organizations such as AOAC International and the American Proficiency Institute (API) offer packages where samples are sent to the participating laboratories several times throughout the year. These programs focus on analytes that can be either pathogenic or non-pathogenic in nature. The samples are usually spiked dry powdered milk products that are rehydrated and analyzed by whatever means the laboratory currently employs. The laboratory results are then sent back to the proficiency testing organization, statistical analysis is performed, and the results of proficiency returned to the laboratory. While the hard statistics are included with the results, most also provide a simple “acceptable/unacceptable” report on the lab’s performance that can be easily viewed by parties that may not have adequate statistical knowledge.

Another application for proficiency testing is assessing the accuracy of the laboratory’s chemical analysis. These laboratories may be conducting analyses on meat, dairy or rendered products, as well as a number of grain-based and multi-ingredient products. In many cases, products are tested during production using a rapid methodology and verified in the laboratory with a more traditional method. In both cases, it is beneficial to include all methods employed in the proficiency program.

Consideration must be given, however, for the differences in the methodologies, as this one means of variability that will affect the outcome of proficiency testing results.The other two major causes of variability are sample preparation by the organization preparing the proficiency sample for distribution, and variability by the laboratory personnel conducting the analysis.

Sample preparation variability may be lessened by purchasing the sample from an organization specializing in this sort of work. AOAC International and the National Institute of Standards and Technology (NIST) both provide samples for a variety of applications. One benefit of these samples is that they often have a longer shelf life than those prepared by individual laboratory organizations. One disadvantage is that they are often more expensive than those done “in-house.”

The choice of whether to purchase samples or prepare them in-house should be determined based on the way the laboratory results are used. A laboratory with ISO 17025 certification will most likely purchase samples, while a laboratory analyzing a low volume of samples to get a general idea of how a process or product is performing may choose to prepare its own. In either case, proper preparation of the proficiency sample should be evaluated.

If an organization chooses to prepare its own samples there are a few factors that should be considered. Sample homogeneity and storage, for example, can easily affect variability. To help ensure homogeneity, an organization may select products that are similar in nature, yet whose matrices are inherently more homogeneous. For example, a finished ground beef product is relatively nonhomogenous, while a ground turkey or bologna-type meat may be very homogenous. If a facility regularly analyzes ground beef, and has procedures that call for the laboratory to process the sample further, the person preparing the proficiency sample may choose to prepare the sample by mixing a large portion to create a semi-homogenous sample. This sample may then be sent to the laboratory to assess an individual technician’s sample preparation. If the laboratory normally analyzes samples as they are received and does no further homogenization, a product that is very homogenous may be used. This information can be used to evaluate the technician’s analytical technique, as well as the method itself.

Improper storage is another factor that can easily influence the accuracy of results. Samples should be hermetically sealed to avoid loss/gain of moisture or contamination by an outside source. Vacuum sealing is the preferred method of preservation; however, vacuuming may have a tendency to draw out moisture from the sample. To avoid this, the sample should be massaged to allow the moisture back in. Foil-lined vacuum bags will further prevent moisture loss through the membrane of the package and should be used for samples which will be stored for a significant amount of time. In most cases, liquid samples will be stored in high-density polyethylene bottles to obtain moisture retention. Most lab procedures will specify the means of storage and these should also be followed for proficiency samples.

The amount of sample sent to the laboratory is another consideration that should be addressed. The premise behind any proficiency testing program should be to treat the proficiency sample in the same fashion as any routine sample. The amount of sample sent to the laboratory should be limited to avoid a situation in which the sample is run numerous times and an average result is given. If routine samples are analyzed one time, the proficiency sample should be analyzed one time.

Another way to avoid this central tendency scenario is to send the sample in a blind or double-blind fashion. The laboratory technician will know the sample is a proficiency sample, but will be unaware of the matrix with a blind sample. The technician will not know that the sample is a proficiency sample or the matrix with a double-blind sample. In many cases, sending the sample blind or double-blind is difficult because the sample may not be easily disguised. There is also debate as to the value of either technique.

Debate Over “True” Results
Another source of debate among scientists is the method that should be used to determine the “true” result. This is the result that is used in comparison to the results reported by the participating laboratories. There are several options, but we’ll address a few of the more popular ones. One option is to have an “expert” laboratory’s result be the true result. An “expert” laboratory would be defined as one that has some form of formal accreditation and that routinely conducts analysis on the same product as the proficiency sample. Another option would be to have the average of all results from the participating laboratories be used to determine the true result. If this method is used, consideration must be given for differences in laboratories and methodologies, as well as results that may be considered outliers.

Another approach when developing a program would be to include the results from an “expert” laboratory, and see how they relate to the results of the participating labs. From that, the determination of which method to use can be made. In either case, the laboratory preparing the proficiency sample should run a series of analyses to ensure proper preparation of the proficiency sample. If it cannot be established that the sample is homogenous and of a probable value, then the sample should not be sent.

Once the samples have been sent, analyzed and the results reported, statistical analysis is used to assess proficiency. The most common practice used to determine proficiency is the use of z-scores. Z-score statistics is a method in which a value is given to a laboratory result based on how it compares to the “true” result. The greater the difference between the reported laboratory result and the true result, the greater the z-score. In general, a z-score greater than 2 is considered questionable, while a z-score greater than 3 is considered unacceptable. These are generalities and a laboratory organization may choose to tighten its requirements if it is analyzing very homogenous samples and extremely confident in the quality of its results. In either case, a questionable or unacceptable result should be investigated and corrective action documented.

Attaining proficiency in any field or profession is something to which we should all strive. For food testing laboratories, obtaining science-based evidence that confirms that laboratory methods, processes and personnel protocols are in control and are providing accurate, defensible results is an effective way to assure confidence for the lab’s customers. Proficiency testing is an excellent way to accomplish this goal.

Read siebar "Proficiency Testing Programs Web Guide"

Jeremy Lawrence, Analytical Services Manager, has been with Wichita, KS-based Cargill Meat Solutions, formerly Excel, for nine years. He is currently the corporate director for the laboratories at Cargill Meat Solutions’ 20 locations.

Proficiency Testing Programs Web Guide

Interested in finding out more about organizations offering standardized proficiency testing programs for food laboratories? Here’s a few websites to visit for more information. Also, remember to check with the food industry trade associations with which your company is affiliated since several offer information on proficiency testing programs to their members.

A2LA • www.a2la.org
A2LA, the nonprofit laboratory accreditation body, offers an excellent resource listing several of the available proficiency testing programs, including those that are accredited by A2LA. Download the PDF from the website.

American Industrial Hygiene Association (AIHA) • www.aiha.org
Offers proficiency testing and accreditation programs for food laboratories that perform microbiology, chemistry and residue chemistry tests.

AOAC International • www.aoac.org
Offers comprehensive laboratory proficiency testing programs for microbiology, nutrition labeling, and pesticides. Accredited by A2LA.

American Proficiency Testing Institute (API) • www.foodpt.com
Offers PT programs for labs performing microbiology and chemistry analyses on food products. Participants can select from a variety of programs and matrices to match the testing performed at their facility.