In the Spotlight
Test Method Validation: What’s in the Pipeline for 2008
Published: November 14, 2008
To facilitate the replacement of existing toxicity tests with new ones, national and international authorities have developed processes and criteria for evaluating new toxicity test methods.
The major steps for replacing an existing regulatory test with a new method are:
- Test method development
- Pre-validation/protocol optimization
- Inter-laboratory validation study
- Peer review/independent assessment
- Regulatory acceptance
The inter-laboratory validation of a test method is a major bottleneck in the overall process owing to the time and resources required. Furthermore, some validation studies have failed to demonstrate adequate performance to substantiate the use of the alternative method for regulatory safety assessments, resulting in a large investment, but little progress.
Once an inter-laboratory study has been completed, a new or revised method can be submitted to one of the validation Centers for an assessment of its scientific validity. In some cases, a Center may become directly involved in conducting or coordinating the inter-laboratory study. All of the Centers are now often involved in the validation process from test development to peer review/assessment, although the details of their processes may vary.
Validation Pipeline in 2008
ICCVAM 2008
An ongoing policy of NICEATM-ICCVAM has been to provide an open, transparent test method evaluation process and opportunity for stakeholder involvement. Their updated website enhances the ability of stakeholders to locate information on NICEATM-ICCVAM activities as well as to locate relevant procedures and historical documents (Winters, et al., 2007). All of the methods reviewed by ICCVAM are detailed on the NICEATM-ICCVAM website, which provides a summary table which is very helpful for viewing the overall status of the various toxicity test methods that have undergone evaluation.
At the June 2008 meeting of the Scientific Advisory Committee on Alternative Toxicological Methods (SACATM), Dr. William Stokes, Director of NICEATM, provided information on test methods that NICEATM-ICCVAM is currently or planning to evaluate (Stokes, 2008). Test methods currently being assessed include methods that are undergoing test development/optimization, inter-laboratory assessment, submission evaluation, independent peer review, and/or awaiting ICCVAM recommendations (Table 1). Test methods planned for assessment are summarized in Table 2. According to Dr. Stokes, each test method evaluation is expected to include a public peer review assessment. Other updates on activities of NICEATM-ICCVAM, such as work on test method optimization, harmonization, and document preparation, are detailed on the specific test method pages on the website.
Additional information on NICEATM-ICCVAM can be found on AltTox under the following sections: Validation; Regulatory Acceptance; US Validation Centers.
Table 1.
Toxicity test methods currently in some active stage of assessment by NICEATM-ICCVAM as of mid-2008.
Test Method | Toxicity Endpoint | Status* |
| Five In vitro Pyrogenicity Test Methods | Pyrogenicity | Submitted to ICCVAM; ICCVAM Peer Review completed; ICCVAM test method recommendations forwarded for transmittal to relevant agencies |
| Integrated Testing Strategy: Bovine Corneal Opacity and Permeability (BCOP), EpiOcular, and Cytosensor Microphysiometer for Eye Irritation Potential of Antimicrobial Cleaning Products | Eye Irritation/Corrosion | Submitted to ICCVAM; currently under evaluation by NICEATM and ICCVAM |
| BCOP and ICE OECD Test Guidelines | Eye Irritation/Corrosion | Accepted by US agencies; Draft submitted to OECD; distributed for national comments |
| ICE, BCOP, Isolated Rabbit Eye (IRE), and Chicken Egg (HET-CAM) test methods | Eye Irritation/Corrosion | Background review documents in preparation; peer review in 2009 for predicting non-, mild, and moderate eye irritants (coordinated with ECVAM and JaCVAM; ECVAM will assess other in vitro methods for same endpoints) |
| 4 In vitro Eye Irritation Test Methods – Fluorescein Leakage, Neutral Red Release, Cytosensor Microphysiometer, and Red Blood Cell Haemolysis | Eye Irritation/Corrosion | Coordinated international evaluation with ECVAM and JaCVAM; ECVAM-led international peer review in 2009. ICCVAM-NICEATM on Validation Study Management Team to perform initial review of these test methods on October 14-15, 2008 |
| NTP Two-year Rodent Bioassay | Carcinogenicity | Nominated to ICCVAM; assigned a low priority for evaluation by ICCVAM in May 2008, which was agreed to by SACATM in June 2008 |
| New Versions and Applications of LLNA methods: LLNA for potency determinations reduced LLNA test method updated LLNA applicability domain three non-radiolabeled LLNA methods development of LLNA performance standards | Skin Sensitization | Nominated to ICCVAM; ICCVAM Peer Review completed; ICCVAM review resulted in request for additional existing data and further evaluation by Peer Review Panel for use of the LLNA for potency determinations, the LLNA applicability domain, and the three non-radiolabeled LLNA methods. Final versions of the ICCVAM Test Method Evaluation Report for the reduced LLNA and the ICCVAM Recommended LLNA Performance Standards expected in November/December 2008 |
| MCF-7 Cell Proliferation Assay | Endocrine Disruptors | Nominated to ICCVAM; prescreened by NICEATM; ICCVAM recommended the assay be given a high priority for additional validation; NICEATM is working closely with CertiChem, Inc. to develop and implement validation study protocols |
| LUMI-CELL Estrogen Receptor Transcriptional Activation Assay | Endocrine Disruptors | Nominated to ICCVAM; NICEATM-ICCVAM completed protocol standardization; NICEATM-ICCVAM is lead for international validation study in collaboration with ECVAM and JaCVAM – currently in Phase 2 of 4 Study Phases |
| Human Skin Model Systems for identifying skin irritants | Skin Irritation | Providing comments to OECD on a draft test guideline; working with ECVAM on data analyses to support a limited applicability domain for international test guidelines |
| In vitro Micronucleus Assay | Genetic Toxicology | Providing comments on a draft OECD test guideline and working with UK scientists and OECD to identify the most appropriate mechanism for evaluating/calculating cytotoxicity in the assay |
Sources of information: ICCVAM Nominations and Submissions and Summary of Test Methods Evaluated by ICCVAM
*Definitions
–evaluation: NICEATM and ICCVAM conduct a preliminary evaluation of the submission for completeness and adherence with ICCVAM guidelines
—test method nominations and submissions
Table 2.
Toxicity test methods planned for assessment (such as evaluation or peer review) by NICEATM-ICCVAM as of mid-2008.
Test Method | Toxicity Endpoint | Activities |
| ICE and BCOP test methods | Eye Irritation/Corrosion | Evaluate BCOP alternative corneal holder |
| Up-and-Down Procedure | Dermal Acute Systemic Toxicity | Develop and evaluate procedure |
| 3T3 NRU cytotoxicity assay | Oral Acute Systemic Toxicity | Evaluate usefulness for predicting starting doses of mixtures |
| 3T3 NRU cytotoxicity assay | Oral Acute Systemic Toxicity | Evaluate ability to identify nontoxic substances. ECVAM has the lead; NICEATM-ICCVAM is participating as an observer during analyses after data has been collected |
| Alternative method for potency testing of veterinary vaccines for Leptospirosis | Biologics Testing | Evaluation planned when USDA study completed |
| EpiDerm and EPISKIN | Skin Corrosivity and Irritation | Evaluate the ability of EpiDerm and EPISKIN irritation protocols to identify substances that were false negative in the corrosivity protocols |
Source of information: Stokes, W. (2008). NICEATM-ICCVAM update. Powerpoint presentation at June 2008 SACATM meeting. Available here.
ECVAM 2008
Following Thomas Hartung’s departure from ECVAM earlier this year, ECVAM is reportedly being reorganized, but the details have yet to be announced publicly.
In recent years, ECVAM has been focused on validating methods to address the requirements of the Cosmetics Directive, as well as the new EU chemicals testing legislation, Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH).
The European Commission, of which ECVAM is a part, has funded a number of research programs for the development of alternative toxicity test methods. ECVAM is participating in many of these scientific projects, including the following Integrated Projects funded within the 6th Framework Programme (FP6): ACuteTox, Predictomics, Pulmonet, carcinoGENOMICS, COMICS, INVITROHEART, ReProTect, Sens-it-iv, TOXDROP, and VITROCELLOMICS. Detailed information on these programs is available on each of the program websites.
The ECVAM website is a resource for information on ECVAM activities as well as relevant literature such as reports on ECVAM workshops. ECVAM, however, does not post details on their peer review meetings or the related documents (such as the Background Review Documents developed by ICCVAM). Instead, they work primarily through closed meetings and publish the outcomes of their workshops and peer evaluations in the journal Alternatives to Laboratory Animals (ATLA). This approach, while effective at distributing historical information, makes it difficult for interested parties not directly involved with ECVAM to learn about their current activities.
The information provided in Table 3 is a summary of ECVAM test method validation activities from public sources such as their presentation at the June 2008 SACATM meeting (Linge, 2008) and Integrate Program websites.
Additional information on ECVAM can be found on AltTox under the following sections: Validation; Regulatory Acceptance; and EU Validation Centers.
Table 3.
Toxicity test methods planned for or currently in some active stage of assessment by ECVAM as of mid-2008.
Test Method | Toxicity Endpoint | Status |
| EpiDerm and EpiSkin IL-1alpha endpoint | Skin Irritation | Evaluation of reproducibility of IL-1 alpha endpoint planned |
| SkinEthic assay | Skin Irritation | Evaluation in peer review |
| Optimized EpiDerm assay | Skin Irritation | Validation study submitted April 2008; currently in peer review |
| Cytotoxicity/cell function-based assays: NRR, RBC, FL, CM | Eye Irritation | Retrospective validation in 2008 |
| Human reconstituted tissue models: SkinEthic and EpiOcular | Eye Irritation | Planning of validation study in collaboration with COLIPA |
| Reduced LLNA (animal refinement method) | Skin Sensitization | Retrospective evaluation of new chemicals database data |
| Non-radioisotope LLNA (animal refinement method) | Skin Sensitization | Preparing background review documents for assessment of validity |
| In vitro methods: Peptide-binding assay; Dendritic cell-based assays (U937 CD86, hCLAT); Vitosens | Skin Sensitization | Draft study plan prepared and submitted to COLIPA |
| Genotoxicity assays in 3D skin models: Micronucleus Test and Comet Assay (partial replacement for use in test strategy) | Genotoxicity | Pre-validation study being coordinated by COLIPA |
| SHE and Balb/c 3T3 cell transformation assays (CTA) (partial replacement for use in test strategy) | Carcinogenicity | ECVAM is lead lab in international pre-validation study to be completed in 2008 in collaboration with ICCVAM and JaCVAM |
| Toxiline-DSP Test for detection of okadaic acid and dinophysistoxins in shellfish | Biologics Testing | ECVAM on management team of validation study that will be submitted for ESAC review |
| In vitro trout S9 fraction assay for bioaccumulation (reduction method) | Ecotoxicology | Validation study |
| PANVERA Estrogen Receptor Binding Test | Endocrine Disruptors | Outsourced validation study in progress |
| HeLa Estrogen Receptor Antagonist Assay | Endocrine Disruptors | Outsourced validation study in progress |
| LUMI-CELL Estrogen Receptor Transcriptional Activation Assay | Endocrine Disruptors | International validation effort in collaboration with NICEATM-ICCVAM and ECVAM |
| 3T3 NRU cytotoxicity assay | Acute Oral Toxicity | Study to evaluate usefulness for discriminating toxic from non-toxic substances – was scheduled to start in late 20071 |
| ACuteTox testing strategy | Acute Toxicity | Pre-validation of derived testing strategy was scheduled to begin June 20082 |
Linge, J. (2008). ECVAM update. Powerpoint presentation at June 2008 SACATM meeting. Available here.
Various methods under development in FP6 research projects and optimization of existing testing strategies are not listed here unless specific information on their validation assessment status was found.
1Poster from ECVAM website
2ACuteTox Newsletter, February 2008
JaCVAM 2008
The Japanese Center for the Validation of Alternative Methods (JaCVAM), the newest of the three validation authorities covered in this review, works collaboratively with ICCVAM and ECVAM in assessing the scientific validity of toxicity test methods (Ohno, 2004). JaCVAM also conducts and coordinates validation studies, and assesses alternative test methods.
JaCVAM is part of Japan’s National Institute of Health Sciences (NIHS), the national agency that oversees the quality, safety, and efficacy of pharmaceutical and medical devices, foods, and chemicals. JaCVAM/NIHS in collaboration with the Japanese Society for Alternatives to Animal Experiments (JSAAE) were major sponsors and organizers of the 6th World Congress on Alternatives and Animals in the Life Sciences held in Tokyo, Japan in 2007.
At the June 2008 SACATM meeting, Dr. Hajime Kojima described JaCVAM’s involvement and progress with respect to ongoing international validation studies, new validation studies, independent scientific peer reviews, and recommendations to regulatory agencies (Kojima, 2008). Test methods planned for or currently under review by JaCVAM are summarized in Table 4.
Additional information on JaCVAM can be found on AltTox under the following sections: Validation and Other Regional & International Organizations.
Table 4.
Toxicity test methods planned for or currently in some active stage of assessment by JaCVAM as of mid-2008.
Test Method | Toxicity Endpoint | Status |
| In vivo Comet Assay | Genetic Toxicity | International validation study; Phase III in 2008 in collaboration with NICEATM-ICCVAM and ECVAM |
| In vitro Comet Assay | Genetic Toxicity | International validation study; Phase II in 2008 in collaboration with NICEATM-ICCVAM and ECVAM |
| LUMI-CELL ER Agonist and Antagonist | Endocrine Disruptors | International validation effort in collaboration with NICEATM-ICCVAM and ECVAM |
| STTA Assay | Endocrine Disruptors | International validation study of estrogen antagonists started in 2008; in collaboration with ECVAM |
| Cell Transformation Assay using Bhras cells | Carcinogenicity | International validation effort in collaboration with NICEATM-ICCVAM and ECVAM (based on SHE cells or Balb/c 3T3 CTA); will start January 2009 |
| BCOP and ICE | Eye Irritation | JaCVAM Independent Peer Review |
| Cytotoxicity test of Short Time Exposure (STE) Assay using SIRC cells | Eye Irritation | Validation study beginning June 2008 |
| EPISKIN | Skin Irritation | JaCVAM Independent Peer Review |
| LabCyte 3-dimensional Epidermal model | Skin Irritation | Validation study beginning June 2008 (based on EPISKIN method using Japanese cell model) |
| h-CLAT using THP-1 cells | Skin Sensitization | Validation study beginning 2009; in collaboration with ECVAM |
| LLNA-DA | Skin Sensitization | JaCVAM Independent Peer Review or ICCVAM Peer Review |
| LLNA-BrdU | Skin Sensitization | JaCVAM Independent Peer Review or ICCVAM Peer Review |
| In vitro Pyrogenicity | Pyrogenicity | JaCVAM Independent Peer Review |
| Battery System: Yeast growth inhibition phototoxicity assay and red blood cell photohemolysis assay | Phototoxicity | JaCVAM Independent Peer Review |
Kojima, H. (2008). JaCVAM update. Powerpoint presentation at June 2008 SACATM meeting. Available here.
Conclusions
Our attempt to pull together the information in this article illustrated (to us at least) how the three Centers provide different types and degrees of accessibility to information about their validation activities (procedures, documents, meetings, etc.). Stakeholders arguably should have access to all of the relevant activities and documents of these organizations to be able to assess progress and the potential availability of alternative methods for meeting their future testing needs.
Even with excellent access to validation documents, such as provided by NICEATM-ICCVAM, stakeholders still face many challenges, such as the interpretation of the study and peer-review results, as well as the use of the information in planning a testing strategy. Similarly, determining the relevant details needed to effectively use any method endorsed as scientifically valid still requires delving through lengthy documents for protocols, identifying applicable types of test substances, understanding other potential test limitations, and determining which agencies might accept data from the new test method. Consequently, the process often results in ongoing uncertainty about the current status and possible uses of a particular ‘validated’ method by its potential users. Ideas and strategies for reducing these challenges to the use of validated and accepted new test methods are topics for which AltTox would welcome further discussion.
Considering the many types of toxicity test methods that are used worldwide by the many regulatory authorities, the current throughput of non-animal test method development, validation, and regulatory acceptance is limited in its ability to dramatically impact the number of animals used for regulatory testing purposes in the short-term. Validation centers are prioritizing methods and conducting an increasing number of reviews each year, but the current bureaucracy and validation processes are very labor intensive, time consuming, and expensive. Funding remains insufficient for test development, optimization, and inter-laboratory assessment. Government testing programs are creating an increasing demand for validated and accepted alternative toxicity test methods. Innovative ways to create efficiencies need to be identified and implemented in an ongoing manner.
Many non-animal test methods that show promise in the development stage are not moved forward for optimization and pre-validation due to lack of funding and/or lack of the investigator understanding the developmental requirements for regulatory use of the test method. Funding, education, and technological advancements specifically designed to promote in vitro toxicology methods are some of the critical factors needed for progress.
In addition to these procedural and logistical issues, many scientific and technical barriers remain. There are insufficient methods being optimized, evaluated, and submitted for validation. Part of this is due to the biological complexity of replicating the intact animal response using in vitro and in silico test systems. Another problem is the use and availability of in vivo animal data as the gold standard for new test method validation. The animal data, when available, is often not of a form or sufficient quality to be useful in a quantitative validation assessment. However, the major problem is that the animal data may not represent human tissue responses, in which case correlative results would be misleading regardless of the quality of the in vivo data.
Even when an alternative method has been endorsed as valid, there have typically been a number of limitations on its use, meaning it needs further refinement or evaluation, or possibly even another method needs to be validated for evaluating the full range of types of test substances. The toxicity endpoints being ‘replaced’ with integrated testing schemes provide even more complexities, including the need for procedures to evaluate their scientific validity.
The three validation centers have increased their cooperative efforts on validation studies and other validation and acceptance related activities. Regulatory acceptance of methods endorsed as scientifically valid by the centers is still lagging, but all of the organizations appear to be taking steps to address this issue. The pace of overall throughput will continue to be a problem, and practical strategies and approaches for resolving this need to be identified and implemented.
Acknowledgements
We asked all three validation Centers mentioned above to provide feedback on the accuracy and completeness of the test methods under review (Tables 1-4). We thank ICCVAM and JaCVAM for taking the opportunity to do so.
