Preclinical statistics: enabling ethical decisions Peter Konings BigChem Training School 10 May 2019 2 Overview Drug development pipeline In silico
In vitro In vivo clinical market Drug development pipeline 4 https://ncats.nih.gov/translation/maps Goals
Move working drugs forward as quickly and efficiently as possible Patient benefit Limited patent protection Stop non-working/unsafe drugs as early as possible No harm to patients Free up resources 5 In-vivo models Regulatory and ethical requirements to demonstrate efficacy and safety in animals before clinical phases Model organisms need to be well characterized Specific phenotypes Useful even if different from humans
Rodents vs. non-rodents 6 Ethics in clinical trials Well developed framework Reaction to war crimes in WW II Informed consent Voluntary participation Right to withdraw at any time for any reason 7 Ethics in animal testing Differences with human trials:
Informed consent impossible Involuntary enrolment No possibility to withdraw 3R framework: Replacement Reduction Refinement 8 9 Good Statistical Practice Good Statistical Practice Framework
1. Appropriate design 10. Monitoring 9. Blinding 2.Appropriate reference groups 8. Appropriate order for sample processing and analysis 3. Planned statistical analysis 4. Justification for
animal numbers 7. Appropriate processing order for treatment, sampling and termination 5. Blocking 6. Randomisation to treatment groups 10 Experimental design Good design can be analysed with simpler models Different from human trials Typical example:
Negative control (sham or vehicle) Positive control (no treatment) A number of dose groups A reference group (Compound with known reaction) 11 Statistical analyses need to be pre-planned Both Bayesian and frequentist inference only valid for pre-planned analyses Garden of forking paths, Data dredging, 12 Sample size calculations
Need to achieve scientific/biological goals Minimize number of animals Allow for dropout: Toxicity Termination for ethical reasons 13 Power Result of test No diff No diff Correct! 95% chance
ERROR False +ve (@ 5%) ERROR False -ve (@ 20%) Correct! (@ 80%) True State Difference
Power 14 Difference Power POWER is the probability that the test will detect a difference or effect if one is present 6 pieces of information of which we need to know 5 and estimate 1: -
15 Size of difference Variability within group Significance level (usually 5%) 1 or 2 sided test Power (usually 80%) Sample size Dont want too big or too small group sizes Loading control group appropriately
Sample size calculations Often simulation-based Need to specify assumptions upfront Various scenarios calculated; sensitivity analysis 16 Avoiding Bias: blinding and blocking Confounders: Known confounders: blocking Unknown confounders: blinding Single blinding vs double blinding 17
Blocking: example Case 1 Day 1: Control Group (N=10) Day 2: Treated Group (N=10) Case 2 Day 1: Control Group (N=10) Treated Group (N=10) Case 3 Day 1: Control Group (N=5) Treated Group (N=5) Day 2: Control Group (N=5) Treated Group (N=5)
Treatment difference ~ Day difference Blocking by study day (especially useful where results are not so reproducible)
Example Scatter Plot Key : 20 Day 1 18 Day 2 16 14 12
Day & Treatment effects 10 8 6 T V Treatment
Blocking Scatter Plot Key : 20 Day 1 18 Day 2 16 14
Day effects removed Less variability for judging treatment effects 12 10 8 6 T V
Treatment Appropriate processing order for treatment, sampling and termination The prescribed necropsy order for the main study animals starts with the lowest numbered male animal in each group, followed by the lowest numbered female in each group, with groups appearing in the order: vehicle control, high, medium and low dose. Animal Number 1 61 41 21 11 71 51
31 2 62 42 22 etc Group 1 4 3 2 1 4 3
2 1 4 3 2 etc vehicle control high dose medium dose low dose vehicle control high dose medium dose low dose
vehicle control high dose medium dose low dose Sex M M M M F F F F M
M M M etc Appropriate order for sample processing and analysis This refers to the order that samples taken from animals are going to be processed and/or analysed or assessed. Check that this order is not going to introduce bias. Plate 1 Plate 2 The variability is affected by assessment occasion which has impact to treatment effect.
Sometimes difference for the same measured area is almost 2-fold. Evidence of systematic effects dependent on order of assessment Randomization of assessment order introduced to reduce bias Appropriate order for sample processing and analysis As with allocating animals / cages to study groups, and performing in-life procedures, the goal is to keep the risk of bias in the study as low as possible, and to maximise the chance of generating meaningful estimates of treatment effects, whilst processing tissues/samples Monitoring Assurance that the model continues to perform satisfactorily, that we will quickly spot any changes in performance and any opportunities to reduce unwanted variation.
25 Trends Trends Move away from a null hypothesis testing framework Estimation Bayesian statistics Best practices from clinical statistics Pre-registration More complex designs Meta-analytic techniques Reproducibility
Confidentiality Notice This file is private and may contain confidential and proprietary information. If you have received this file in error, please notify us and remove it from your system and note that you must not copy, distribute or take any action in reliance on it. Any unauthorized use or disclosure of the contents of this file is not permitted and may be unlawful. AstraZeneca PLC, 1 Francis Crick Avenue, Cambridge Biomedical Campus, Cambridge, CB2 0AA, UK, T: +44(0)203 749 5000, www.astrazeneca.com 27
