Advertisement
Articles
Advertisement

Liquid Handling Instruments: Defining Accuracy and Precision

Tue, 02/13/2007 - 12:21pm
By George Rodrigues, Ph.D., Senior Scientific Manager, ARTEL

The 2007 Perspectives on Instrumentation column will offer guidance and insight into the use, maintenance and application of liquid handling instruments. Liquid delivery is a common laboratory process, and this critical function is often overlooked. As a result, routine research and tests results can be in error based on simple misunderstanding or misapplication of liquid delivery instruments. The first article in this series focuses on defining and exploring accuracy and precision, which are fundamental elements of regulatory compliance.

As an example of the importance of accuracy and precision, in January of this year, the US FDA sent out over 1,000 letters to drug product owners regarding a “lack of assay reproducibility between original and repeat results,” and in a related warning letter stated that because of lack of reproducibility “the reported concentration results cannot be considered accurate.” The laboratories involved have spent many thousands of hours and tens of millions of dollars responding to this situation. These sorts of events grab headlines and create excitement, but most of us prefer a less thrilling work experience.1
What are accuracy and precision?

click the image to enlarge
No measurement is perfect and all measurements have some error associated with them. In the laboratory, two terms often used somewhat interchangeably to describe measurement errors are accuracy and precision. While general usage may be lax at times, in statistical calculations and in regulations, these two terms have different and very distinct meanings. Knowing the “precise” definitions and importance of each are critical to the development of a sound laboratory quality control program.

Accuracy refers to the deviation of a measurement from a standard or true value of the quantity being measured. We can talk about the accuracy of a single measurement. For example, if a pipette is set to dispense 100 microliters but actually delivers 99 microliters, the accuracy of that particular dispense is off (or the pipette is inaccurate) by -1 microliter. Notice that in this case we know what happened during that last dispense (it was 1 microliter too low), but we don’t have much knowledge about what is likely to happen the next time this pipette is used.

We can also talk about the accuracy of a group of repeated (replicate) measurements. In these cases, we must first determine the mean of the group, and then compare that average value with the standard or true value. Accuracy for a group of measurements refers to the deviation of the group’s mean value from the standard or true value. But knowing accuracy alone is of limited use. For example, if we know that three replicate measurements averaged exactly 100 microliters, we still can’t predict how likely it is that the next dispense will be within some limits. One pipette might deliver 99, 100 and 101 microliters (pretty good), while a second delivers 80, 100 and 120 (pretty bad). The averages of both sets of data are exactly 100 and both are perfectly accurate, but which one would you use if your life depended on the next volume measurement being very close to 100?

Precision tells us how close a group of measurements are to one another. The closer the data replicates, the more likely the results will be similar in the future. For this reason, good precision has predictive value; it gives us confidence in future results. Precision is usually calculated and discussed in terms of standard deviation (s) and coefficient of variation (CV). A precise or closely clustered data set has a smaller CV and is generally more reliable than one that is widely scattered.

Because precision is concerned with the closeness of two or more measurements to each other rather than to a standard value, it is possible for a group of values to be precise without being accurate, or to be accurate without being precise (see Figure 1).
When should accuracy and precision be investigated?
A laboratory should investigate the accuracy and precision of a method when the method is new, the method is questioned because of external quality control data, or the validity of the results is questionable. It is also a common practice to check the accuracy and precision of laboratory equipment whenever a new instrument is brought into the lab, when equipment is suspected of being damaged, and on a periodic schedule thereafter. In addition, accuracy and precision checks are frequently used as part of the qualification process of a new laboratory technician. Table 1 shows some examples of errors that influence accuracy and precision.
How should accuracy and precision requirements be determined?
To determine the importance of accuracy and precision of liquid handling and other equipment in a method, you will have to take a close look at the intended use of results, sources of error, and how those errors will affect results. A three step analysis is involved: 1. Establish the limits of acceptable error in final results; 2. Determine each predominant source of error in the method; and 3. Do a statistical analysis of the impact of these errors on your results. This analysis will help you to focus efforts on controlling and reducing the largest sources of error in your results.
Why is it necessary to test for both accuracy and precision in liquid delivery?
Both accuracy and precision are necessary in order to ensure that results are valid. In general, failure to achieve either accuracy requirement or precision requirements is sufficient to constitute a failed test or calibration.
What is required when equipment accuracy and/or precision requirements are not met?
Table 1: Some Errors Influencing Accuracy and Precision
Type of error Example Corrective Action
Personal Error Excessive thumb pressure or pipetting too quickly Training and competence testing for accuracy and precision.
Method Error Poor precision from using a variable volume pipette at the bottom of its range Use a pipette sized so the set volume is in the top half of it's range
Instrumental Error Delivering inaccurate volume because of a corroded pipette piston Clean and check pipettes regularly
The US FDA sets high standards for ensuring that calibrated equipment meets pre-established accuracy and precision limits:

“When accuracy and precision limits are not met, there shall be provisions for remedial action to evaluate whether there was any adverse effect on product quality.” 21 CFR 820.72 (b)

Remedial action requirements include repair and/or recalibration of the equipment and consideration of potential quality impacts in three particular areas. According to FDA Quality System Requirements Manual Part 7: Equipment and Calibration, this includes:
•     on the product design or process validation parameters or data;
•     on the quality of existing components, in-process, or finished products; and
•     appropriate corrective action.

This regulation means that something as simple as a failed pipette could call into question and place at risk any work done with that device since the last prior calibration, and both validation data and products (in-process or finished) can be held suspect. What constitutes appropriate corrective action can vary depending on the situation. An isolated mechanical failure might be corrected with a simple repair, while a pattern of recurring failures in a particular laboratory may require process reengineering to prevent recurrences or to mitigate the associated risks. Either way, it is desirable to avoid remedial action. Therefore, regular verification of accuracy and precision are essential.

Lack of accuracy or precision is like a roller coaster ride. Results can be up one minute and down the next. Attention to the accuracy and precision of pipettes and other liquid handling devices improves confidence and quality in laboratory analytical testing. When tests are correct and reliable, production is smooth and laboratories are less likely to be troubled by regulatory compliance concerns. A precise and accurate QC laboratory is a pleasant place to work – and a smooth running operation leaves us more time for thrill-seeking outside of work.

References1http://www.fda.gov/cder/news/pharmaco_studies/default.htm


About the Author: George Rodrigues, Ph.D., is Senior Scientific Manager at ARTEL. Rodrigues earned his BS in Chemical Engineering at the U.C. Berkeley, and a Ph.D. in Chemical Engineering at the University of Wisconsin. Dr. Rodrigues can be contacted at (207) 854-0860 or grodrigues@artel-usa.com.

Look for these upcoming articles in the Perspectives on Instrumentation column:
April - The Impact of One Microliter: The trend toward smaller volumes has reached the point where a variation of one microliter in one component of a test may cause the test to fail outright or to give an erroneous result. Looking at common assay methods and test applications, we will review possible impacts of a one microliter variation in a sample.
September - Calibration vs. Verification: New regulations, rules, guidelines and methods may set new standards for how and when each of these two related concepts apply.
November- FAQs on Liquid Handling: Here, we will answer your questions about liquid delivery methods, instruments, applications and techniques. Email your questions to grodrigues@artel-usa.com.

Advertisement
Advertisement

Share this Story

X
You may login with either your assigned username or your e-mail address.
The password field is case sensitive.
Loading