Transcription

Test Method Validation and VerificationExcerpts from CAP All Common Checklist, August 21, 2017 releaseInspector Instructions: Policies and procedures for the introduction of new tests, methods, or instrumentsSampling of assay validation and verification studies with emphasis on tests introduced in the past twoyears, especially high volume tests and tests with the highest risk to patientsSampling of patient reports for laboratory-developed assaysWhich laboratory tests or instruments have been implemented in the past two years, particularly those thatare not FDA-cleared/approved?Do you follow the manufacturer's instructions exactly for all FDA-cleared/approved diagnostic kits ordevices?How does your laboratory validate or verify assay performance prior to test implementation?How does your laboratory verify or establish reference intervals?How does your laboratory validate clinical claims made by the laboratory about LDTs?Select at least one validation or verification study performed for each type of instrument or methodintroduced during the past two years.In addition, select assays for evaluation if recurrent problems have been identified in proficiency testingresults, quality control, competency assessment, or physician complaints regardless of how long theassay has been in place.Review validation or verification records to confirm that appropriate studies were performed using anadequate number of cases, and data were reviewed. If the data showed discordances or unacceptablevariations, investigate how they were resolved. If a study was not performed or is missing requiredcomponents, cite the appropriate related requirement(s) (e.g. COM.40300, COM.40350, COM.40400).Confirm that there is a written assessment of each component (accuracy, precision, interferences, etc.) ofthe validation or verification studies with laboratory director (or qualified designee) approval prior to theinitiation of clinical testing. If the study assessment was not signed by the laboratory director or designee,cite COM.40000.**NEW** 08/17/2016COM.30980 Waived Test Implementation and Approval Phase IIFor each waived test, the laboratory follows manufacturer's instructions for theintroduction of the instrument or device and there are records that the test(s) is approvedfor use by the laboratory director, or designee meeting CAP director qualifications, priorto use in patient testing.NOTE: Waived testing must be performed following the manufacturer's instructions as written.If the laboratory modifies a waived test, the checklist requirements for high complexity testingapply, including the requirements for validation of the method performance specifications.The laboratory director's signature on the written test procedure may be used to show approval ofthe test for use in patient testing.Evidence of Compliance: Records of test approval**REVISED** 08/21/2017COM.40000 Method Validation and Verification Approval - Nonwaived Tests Phase IIFor each nonwaived test, there is an evaluation of the test method validation orverification study (accuracy, precision, etc.) signed by the laboratory director, ordesignee meeting CAP director qualifications, prior to use in patient testing to confirm theacceptability of the data and approve each nonwaived test for clinical use.NOTE: This checklist requirement is applicable only to nonwaived tests implemented after JuneAll Checklists are 2017. College of American Pathologists. All rights reserved.Page 1 of 9

Test Method Validation and Verification15, 2009; however, all nonwaived tests must have records of completed analytical validation orverification, regardless of their implementation date.If multiple identical instruments or devices are in use, there must be records showing that themethod performance specifications have been separately verified for each test and instrument ordevice. The evaluation and data must clearly support the decision to approve the test for clinicaluse.The evaluation must include: 1) a written assessment of each component of the validation orverification study, including the acceptability of the data; 2) a signed approval statement, suchas, "I have reviewed the verification (or validation) data for accuracy, precision, reportable range,and reference interval studies (insert other components, as required) for the (insert instrument/test name), and the performance of the method is considered acceptable for patient testing." Ifdata include discordant results, there must be a record of the discordance and investigation ofany impact on the approval of the test for clinical use.If a validation or verification study (accuracy, precision, reportable range, etc.) was not performedor is missing required components, the appropriate, related checklist requirements must also becited (e.g. COM.40300, COM.40350, COM.40400).For an FDA-cleared/approved test, an evaluation of the verification data must address analyticalperformance specifications, including analytical accuracy, precision, interferences, and reportablerange, as applicable.In addition, for modified FDA-cleared/approved tests or LDTs, the evaluation must addressanalytical sensitivity, analytical specificity, and any other parameter that is considered important,to assure the analytical performance of a test (e.g. specimen stability, reagent stability, linearity,carryover, and cross-contamination, etc. as appropriate and applicable).If the laboratory makes clinical claims about its tests, the evaluation must address the validationof these claims.See the Method Performance Specifications section for details concerning validation and verification.Evidence of Compliance: Evaluation of validation and verification studies with review and approvalMethod Performance Specifications - Nonwaived testsNOTE: This subsection on METHOD PERFORMANCE SPECIFICATIONS does not apply to waivedtests.ANALYTICAL VALIDATION/VERIFICATIONLaboratories are required to perform analytical validation or verification of each nonwaived test, method, orinstrument system before use in patient testing, regardless of when it was first introduced by the laboratory,including instruments of the same make and model and temporary replacement (loaner) instruments. Thereis no exception for analytical validation or verification of tests introduced prior to a specific date. Thelaboratory must have data for the validation or verification of the applicable method performancespecifications and retain the records as long as the method is in use and for at least two years afterdiscontinuation.The method performance specifications (e.g. accuracy, precision, reportable range) must be validated orverified in the location in which patient testing will be performed. If an instrument is moved, the laboratory isresponsible for determining that the method performance specifications are not affected by the relocationprocess or any changes due to the new environment (e.g. refer to the manufacturer's manual regardingcritical requirements, such as set-up limitations, environmental conditions, etc.). The laboratory must followmanufacturer's instructions for instrument set up, maintenance, and system verification. SeparateAll Checklists are 2017. College of American Pathologists. All rights reserved.Page 2 of 9

Test Method Validation and Verificationrequirements for verifying the performance of instruments and equipment to confirm that they functionaccording to expectations for the intended use and within the defined tolerance limits are found in theInstrument and Equipment Maintenance and Function Checks section (COM.30550, COM.30600). Not allmethod performance specifications apply to qualitative tests. For qualitative tests, the laboratory must verifyor establish the method performance specifications that are applicable and clinically relevant.LABORATORIES SUBJECT TO US REGULATIONS: For unmodified FDA-cleared or approved tests, the laboratory may use information from manufacturers, orpublished literature, but the laboratory must verify such outside information on accuracy, precision andreportable range. For modified FDA-cleared or approved tests and laboratory-developed tests (LDTs), the laboratory mustestablish accuracy, precision, analytical sensitivity, interferences, analytical specificity, and reportable range,as applicable; data on interferences may be obtained from manufacturers or published literature, asapplicable.LABORATORIES NOT SUBJECT TO US REGULATIONS: The laboratory must verify or establish analytical accuracy, precision, analytical sensitivity, analyticalspecificity (interfering substances) and reportable range for each test. Laboratories may use information frommanufacturers, published literature, or studies performed in other laboratories, but should verify such outsideinformation, whenever practical.LABORATORY-DEVELOPED TESTS:For the purposes of interpreting the checklist requirements, a laboratory-developed test (LDT) is defined asfollows: A test used in patient management that has both of the following features:1. The test is performed by the clinical laboratory in which the test was developed wholly or in part; AND2. The test is neither FDA-cleared nor FDA-approved.EMERGENCY USE AUTHORIZATION (EUA)For laboratories subject to US regulations, an emergency use authorization (EUA) is the legal mechanismused by the FDA to allow the use of an unapproved medical product (e.g. diagnostic device) or anunapproved use of an approved medical product during an emergency to diagnose, treat, or prevent aserious or life threatening disease condition caused by a chemical, biological, radiological, or nuclear agent(CBRN).A laboratory that uses an EUA assay may not be able to establish accuracy, precision, analytical sensitivity,interferences, analytical specificity, and reportable range studies. Laboratories using an EUA assay mustfollow the assay or test system's protocol without modification and document the alternative mechanismemployed to ensure accurate test results.COM.40100 Intermittent Testing Phase IIWhen a test is put back into production, the following requirements must be met:1. PT or alternative assessment performed within 30 days prior to restarting patient testing2. Method performance specifications verified, as applicable, within 30 days prior to restarting patienttesting3. Competency assessed for analysts within 12 months prior to restarting patient testingNOTE: This requirement applies to tests that are taken out of production for a time (for example, seasonaltesting for influenza). A test is considered to be taken out of production when (1) patient testing is not offeredAND (2) PT or alternative assessment, as applicable, is suspended. It does not apply to situations where aproficiency testing challenge is not performed due to a temporary, short-term situation, such as a reagentback order or an instrument breakdown. In those situations, the laboratory must perform alternativeassessment for that testing event.The laboratory should have written procedures for putting intermittent tests into production.For tests for which PT is required by CAP, if a PT challenge is not offered during the 30-dayAll Checklists are 2017. College of American Pathologists. All rights reserved.Page 3 of 9

Test Method Validation and Verificationperiod prior to restarting patient testing, the laboratory may perform an alternative assessmentof the test. The laboratory must participate in the next scheduled PT event, if the LaboratoryAccreditation Program requires external PT for that analyte.COM.40200 LDT and Modified FDA-cleared/approved Test List Phase IThe laboratory maintains a list of laboratory-developed tests (LDTs) and modified FDAcleared/approved tests implemented by the laboratory.NOTE: The list must include tests developed in-house, and for laboratories subject to USregulations, tests using analyte-specific reagents (ASRs), and FDA-cleared/approved tests thathave been modified by the laboratory. A form is available on the CAP website that may be used formaintaining this list and can be downloaded from the CAP website (http://www.cap.org) through e-LABSolutions Suite.COM.40250 Manufacturer's Instructions Phase IIThe laboratory follows manufacturer's instructions or provides validation records if thetest has been modified.NOTE: Following manufacturer's instructions includes performing quality control, calibration, calibrationverification, and related functions as applicable to the scope of testing. Reagents, fluids, and disposablematerials supplied by the laboratory must meet the specifications in the instructions.If the laboratory modifies manufacturer's instructions, the test is no longer an FDA-cleared/approved test,and the modification(s) must be validated by the laboratory. Changes in the specimen type or collectiondevice are examples of common modifications (see "modification of manufacturer's instructions" in thedefinition of Terms). Additional requirements for validation/verification may be found in the discipline-specificchecklists.For waived and moderately complex tests, if manufacturer instructions are modified,requirements for high complexity testing apply.Evidence of Compliance: Validation records of established performance specifications (accuracy, precision, analyticalsensitivity, analytical specificity, interferences, reference interval(s), and reportable range) ofany test that has been modified.REFERENCES1) Department of Health and Human Services, Centers for Medicare and Medicaid Services. Clinical laboratory improvementamendments of 1988; final rule. Fed Register. 2003(Jan 24): [42CFR493.1252]**REVISED** 08/21/2017COM.40300 Accuracy Phase IIThe laboratory verifies or establishes analytical accuracy for each test using a sufficientnumber of characterized samples.NOTE: Accuracy is verified or established by comparing results to a definitive or referencemethod, or an established comparative method. Use of matrix-appropriate referencematerials, patient specimens (altered or unaltered), or other commutable materials with knownconcentrations or activities may be used to verify or establish accuracy. The use of routine qualitycontrol materials or calibrators that were used to calibrate the method is not appropriate.Evidence of Compliance: Written procedure for determining method performance characteristics, including accuracyAND Records of verification or establishment of accuracy for each testAll Checklists are 2017. College of American Pathologists. All rights reserved.Page 4 of 9

Test Method Validation and Verification**NEW** 08/21/2017COM.40310 Precision Phase IIThe laboratory verifies or establishes analytical precision for each test using a sufficientnumber of characterized samples with repeated analysis.NOTE: Precision is verified or established by repeat measurement of samples at varyingconcentrations or activities within-run and between-run over a period of time.Evidence of Compliance: Written procedure for determining method performance characteristics, including precisionAND Records of verification or establishment of precision for each testCOM.40350 Accuracy - Modified FDA-cleared/approved and LDTs Phase IIFor modified FDA-cleared/approved and laboratory-developed tests (LDTs), validation ofanalytical accuracy includes testing with an appropriate number of samples.NOTE: An appropriate number of samples is defined as the following: For quantitative tests, a minimum of 20 samples with analyte concentrations distributed across theanalytical measurement range should be used. Proportionate mixtures of samples may be used tosupplement the study population. For qualitative tests, a minimum of 20 samples, including positive, negative, and low positive samples withconcentrations near the lower level of detection should be used; equivocal samples should not be used. For certain methods that test multiple analytes (e.g. next-generation sequencing, FISH, HPLC, GC-MS,MALDI-TOF, etc.), analytic accuracy may be established for each method (not necessarily each analyte), asappropriate.If the laboratory uses fewer samples, the laboratory director must record the criteria used to determine theappropriateness of the sample size. In many cases, a validation study with more samples is desirable.For modified FDA-cleared/approved tests and LDTs in use prior to July 31, 2016, for which limited validationstudies are recorded, ongoing data supporting acceptable test performance may be used to meet the aboveminimum sample requirement, unless the laboratory director has recorded the criteria used to determine theacceptability of a smaller sample size. Examples of such ongoing data include records of proficiency testing,alternative performance assessment, and quality control.This checklist requirement does not apply to LDTs that employ the following methods: Manual microscopy (e.g. histopathologic and cytologic interpretation, microscopicexamination of blood or body fluids, Gram stains) Conventional microbiologic cultures and disc/broth/tube susceptibility studiesEvidence of Compliance: Written procedure for determining method performance characteristics, including accuracyAND Records of establishment of analytical accuracy for each testCOM.40400 Analytical Sensitivity - Modified FDA-cleared/approved and LDTs Phase IIFor modified FDA-cleared/approved tests or laboratory-developed tests (LDTs), thelaboratory establishes the analytical sensitivity (lower detection limit) of each assay, asapplicable.Evidence of Compliance: Written procedure for determining method performance characteristics, including analyticalsensitivity AND Records of establishment of analytical sensitivity for each assayAll Checklists are 2017. College of American Pathologists. All rights reserved.Page 5 of 9

Test Method Validation and VerificationCOM.40450 Analytical Specificity - Modified FDA-cleared/approved and LDTs Phase IIFor modified FDA-cleared/approved tests or laboratory-developed tests (LDTs), the resultsof each validation study include a sufficient number of samples to establish the test's analyticalspecificity.NOTE: The analytical specificity refers to the ability of a test or procedure to correctly identify or quantify anentity in the presence of interfering or cross-reactive substances that might be expected to be present.Laboratories are encouraged to review the published literature for guidance and provided confidenceintervals to estimated analytical specificity.Evidence of Compliance: Records of validation studies and published references used to establish analytical specificityCOM.40500 Analytical Interferences Phase IIThe laboratory understands the analytical interferences for each test, and has anappropriate plan of action when they are present.NOTE: Interfering substances may pose a significant problem to the clinical laboratory and healthcareproviders who may be misled by laboratory results that do not reflect patient clinical status. The laboratorymust be aware of common interferences by performing studies or referencing studies performed elsewhere(such as by the instrument-reagent manufacturer).Evidence of Compliance: Written procedure for determining method performance characteristics, including analytical interferencesAND Document listing known interferences for each test and plan of action when they are present**REVISED** 08/21/2017COM.40600 Reportable Range Phase IIThe reportable range is verified or established for each analytical procedure beforeimplementation.NOTE: The reportable range of an assay is the range of values that the laboratory reports for that assay. Theanalytical measurement range (AMR) is the range of analyte values that a method can directly measure onthe specimen without any dilution or concentration. In some cases, clinically relevant limits may be narrowerthan the analytical measurement range, and the clinically relevant limit is used as the limit for the reportablerange.Expanded definitions and details of the AMR are provided in