Liquid Handling Instruments: Defining Accuracy and Precision
By George Rodrigues, Ph.D., Senior Scientific Manager, ARTEL
The 2007 Perspectives on Instrumentation column will offer guidance and insight
into the use, maintenance and application of liquid handling instruments. Liquid
delivery is a common laboratory process, and this critical function is often overlooked.
As a result, routine research and tests results can be in error based on simple
misunderstanding or misapplication of liquid delivery instruments. The first article
in this series focuses on defining and exploring accuracy and precision, which
are fundamental elements of regulatory compliance.
As an example of the importance of accuracy and precision, in January of this
year, the US FDA sent out over 1,000 letters to drug product owners regarding
a “lack of assay reproducibility between original and repeat results,” and in
a related warning letter stated that because of lack of reproducibility “the reported
concentration results cannot be considered accurate.” The laboratories involved
have spent many thousands of hours and tens of millions of dollars responding
to this situation. These sorts of events grab headlines and create excitement,
but most of us prefer a less thrilling work experience.1
What are accuracy and precision?
click the image to enlarge |
No measurement is perfect and all measurements have some error associated with
them. In the laboratory, two terms often used somewhat interchangeably to describe
measurement errors are accuracy and precision. While general usage may be lax
at times, in statistical calculations and in regulations, these two terms have
different and very distinct meanings. Knowing the “precise” definitions and importance
of each are critical to the development of a sound laboratory quality control
program.
Accuracy refers to the deviation of a measurement from a standard or true value
of the quantity being measured. We can talk about the accuracy of a single measurement.
For example, if a pipette is set to dispense 100 microliters but actually delivers
99 microliters, the accuracy of that particular dispense is off (or the pipette
is inaccurate) by -1 microliter. Notice that in this case we know what happened
during that last dispense (it was 1 microliter too low), but we don’t have much
knowledge about what is likely to happen the next time this pipette is used.
We can also talk about the accuracy of a group of repeated (replicate) measurements.
In these cases, we must first determine the mean of the group, and then compare
that average value with the standard or true value. Accuracy for a group of measurements
refers to the deviation of the group’s mean value from the standard or true value.
But knowing accuracy alone is of limited use. For example, if we know that three
replicate measurements averaged exactly 100 microliters, we still can’t predict
how likely it is that the next dispense will be within some limits. One pipette
might deliver 99, 100 and 101 microliters (pretty good), while a second delivers
80, 100 and 120 (pretty bad). The averages of both sets of data are exactly 100
and both are perfectly accurate, but which one would you use if your life depended
on the next volume measurement being very close to 100?
Precision tells us how close a group of measurements are to one another. The closer
the data replicates, the more likely the results will be similar in the future.
For this reason, good precision has predictive value; it gives us confidence in
future results. Precision is usually calculated and discussed in terms of standard
deviation (s) and coefficient of variation (CV). A precise or closely clustered
data set has a smaller CV and is generally more reliable than one that is widely
scattered.
Because precision is concerned with the closeness of two or more measurements
to each other rather than to a standard value, it is possible for a group of values
to be precise without being accurate, or to be accurate without being precise
(see Figure 1).
When should accuracy and precision be investigated?
A laboratory should investigate the accuracy and precision of a method when the
method is new, the method is questioned because of external quality control data,
or the validity of the results is questionable. It is also a common practice to
check the accuracy and precision of laboratory equipment whenever a new instrument
is brought into the lab, when equipment is suspected of being damaged, and on
a periodic schedule thereafter. In addition, accuracy and precision checks are
frequently used as part of the qualification process of a new laboratory technician.
Table 1 shows some examples of errors that influence accuracy and precision.
How should accuracy and precision requirements be determined?
To determine the importance of accuracy and precision of liquid handling and other
equipment in a method, you will have to take a close look at the intended use
of results, sources of error, and how those errors will affect results. A three
step analysis is involved: 1. Establish the limits of acceptable error in final
results; 2. Determine each predominant source of error in the method; and 3. Do
a statistical analysis of the impact of these errors on your results. This analysis
will help you to focus efforts on controlling and reducing the largest sources
of error in your results.
Why is it necessary to test for both accuracy and precision in liquid delivery?
Both accuracy and precision are necessary in order to ensure that results are
valid. In general, failure to achieve either accuracy requirement or precision
requirements is sufficient to constitute a failed test or calibration.
What is required when equipment accuracy and/or precision requirements are not met?
| Table 1: Some Errors Influencing
Accuracy and Precision |
| Type of error |
Example |
Corrective Action |
| Personal Error |
Excessive thumb pressure or pipetting too quickly |
Training and competence testing for accuracy and precision. |
| Method Error |
Poor precision from using a variable volume pipette
at the bottom of its range |
Use a pipette sized so the set volume is in the top
half of it's range |
| Instrumental Error |
Delivering inaccurate volume because of a corroded pipette
piston |
Clean and check pipettes regularly |
The US FDA sets high standards for ensuring that calibrated equipment meets pre-established
accuracy and precision limits:
“When accuracy and precision limits are not met, there shall be provisions for remedial action to evaluate whether there was any adverse effect on product quality.” 21 CFR 820.72 (b)
Remedial action requirements include repair and/or recalibration of the equipment and consideration of potential quality impacts in three particular areas. According to FDA Quality System Requirements Manual Part 7: Equipment and Calibration, this includes:
• on the product design or process validation parameters
or data;
• on the quality of existing components, in-process,
or finished products; and
• appropriate corrective action.
This regulation means that something as simple as a failed pipette could call
into question and place at risk any work done with that device since the last
prior calibration, and both validation data and products (in-process or finished)
can be held suspect. What constitutes appropriate corrective action can vary depending
on the situation. An isolated mechanical failure might be corrected with a simple
repair, while a pattern of recurring failures in a particular laboratory may require
process reengineering to prevent recurrences or to mitigate the associated risks.
Either way, it is desirable to avoid remedial action. Therefore, regular verification
of accuracy and precision are essential.
Lack of accuracy or precision is like a roller coaster ride. Results can be up
one minute and down the next. Attention to the accuracy and precision of pipettes
and other liquid handling devices improves confidence and quality in laboratory
analytical testing. When tests are correct and reliable, production is smooth
and laboratories are less likely to be troubled by regulatory compliance concerns.
A precise and accurate QC laboratory is a pleasant place to work – and a smooth
running operation leaves us more time for thrill-seeking outside of work.
References
1http://www.fda.gov/cder/news/pharmaco_studies/default.htm
About the Author: George Rodrigues, Ph.D., is Senior Scientific Manager at
ARTEL. Rodrigues earned his BS in Chemical Engineering at the U.C. Berkeley, and
a Ph.D. in Chemical Engineering at the University of Wisconsin. Dr. Rodrigues
can be contacted at (207) 854-0860 or grodrigues@artel-usa.com.
Look for these upcoming articles
in the Perspectives on Instrumentation column:
April - The Impact of One Microliter: The trend toward
smaller volumes has reached the point where a variation of one microliter
in one component of a test may cause the test to fail outright or to give
an erroneous result. Looking at common assay methods and test applications,
we will review possible impacts of a one microliter variation in a sample.
September - Calibration vs. Verification: New regulations,
rules, guidelines and methods may set new standards for how and when each
of these two related concepts apply.
November- FAQs on Liquid Handling: Here, we will answer
your questions about liquid delivery methods, instruments, applications
and techniques. Email your questions to grodrigues@artel-usa.com.
|
|
Addressing the Challenges Contract Manufacturing Organizations are... 
