Impact of Censoring Data Below an Arbitrary Quantification Limit on Structural Model Misspecification

W. Byon 1, C. V. Fletcher 2, R. C. Brundage 3

Pfizer Global Research and Development, New London Connecticut, USA 1, University of Colorado Denver, Colorado, USA 2. University of Minnesota, Minneapolis, Minnesota, USA 3

ABSTRACT

5.000

Full data

BQL censoring

BQL data

NONMEM VI 5FOCEI / YLO with LAPLACIAN )

$EST Advan1 Trans2 for 1 Compartment model

1.50

2.00

2.50

3.00

3.50

4.00

Time After Dose

Figure 1. Simulation flow chart using a typical simulated concentrationtime profile from scenario 2

10.2

0.0

2

5

0.0132

0.0884

< 20
17.3
0.2
3
5
0.0187
0.1250
< 20
26.7
0.4
4
5
0.0265
0.1768
< 20
37.6
0.9
5
5
0.0374
0.2500
< 20
49.1
1.8
Simulation plan 2
10
6
5
0.0132
0.2640
< 10
51.3
0.2
7
5
0.0132
0.0294
< 50
3.1
0.2
8
5
0.0132
0.0139
< 100
1.1
0.2
SIMULATION & ESTIMATION
Each simulated data set was designated as Full data 5no BQL censoring). This data set
was then used to generate a second data set that excluded data below the relevant LLOQ
to the scenario, which was designated BQL data.
The simulations and population analyses were performed using a nonlinear mixed effects
model implemented in NONMEM VI [7] using Compaq Visual Fortran version 6.5. The
preparation of BQL censored datasets was performed using SPLUS 7.0 5Insightful
Corporation).
BQL data and Full data were analyzed with a one-compartment model using ADVAN1 and
TRANS2 and a two-compartment model using ADVAN3 and TRANS4. When the twocompartment model was tested, the peripheral volume of distribution and the intercompartmental clearance were added into the model without between subject variability on
them. The FOCEI was used for this estimation. Additionally, a new conditional likelihood
estimation feature in NONMEM VI 5YLO) was evaluated..
20
30
40
50
1.0
BQL data
data from
from all
all 500 runs
runs
Full data
data from all 500 runs
runs
0.8
0.6
< 20
CONCLUSION
10
20
Percent of median censored data
30
40
50
Percent of median censored data
Figure 2. Type I error rates at the 5 % 5left) and 1 % 5right) alpha levels in simulation plan 1 for
BQL data 5solid lines) and Full data 5dashed lines).
BQL data from all 500 runs
Table 2. Type I error rates when testing YLO
at the 5 % alpha level
BQL data from all 500 runs
10 20
50
CV at LLOQ 5%)
100
Scenario
No.
10 20
50
CV at LLOQ 5%)
100
Figure 3. Type I error rates at the 5 % 5left) and
1 % 5right) alpha levels in simulation plan 2
The censoring of concentrations as BQL can lead to structural model
misspecification in population PK analyses. Furthermore, relaxing the current
practice of censoring data with less than 20% precision can help prevent this
misspecification.
With the nave cut-off values in the 2-distribution at two degrees of freedom, the
type I error rates from Full data 5without any BQL censoring) in simulation plan 1
were lower than the nominal value at both the 5% and 1% alpha levels in scenario 1
3. This is a known result under the constrained one-sided test using log likelihood
ratio test under a boundary condition [8].
A trend in type I error rate in Full data was that it increased across scenarios 1
through 5. This is suspected to result from the simulated non-positive data which
were removed from the parent datasets.
The maximum conditional likelihood estimation minimized the elevation of type I error
across all scenarios. Therefore, in a PK analysis that includes a substantial fraction
of data being censored, the use of YLO options should be strongly considered to
avoid any model misspecification
0.4
0.0625
Type I error rate at the alpha level of 0.01
0.0093
BQL data from
from all 500
500 runs
Full data
data from
from all 500
500 runs
Type I error rate at the alpha level of 0.01
0.2
0.4
0.6
0.8
1.0
5
1.0
Median of percent data Median of percent data
set censored as
set censored as
BQL and negative
negative
Concentrations
concentrations
1
RESULTS
Simulation plan 1
1.00
0.8
error
CV at LLOQ
5%)
0.25 0.5
$EST Advan3 Trans4 for 2 Compartment model
0.6
LLOQ
LLOQ
0.4
Scenario
No.
Additive
Type I error rates were elevated when datasets included BQL censoring compared
to when all the data were available across all the scenarios. The increasing trend in
type I error rate was observed as the median of percent censored data increased
when BQL data were estimated at both alpha levels.
When the rules of successful minimization, successful covariance step, and
reasonable results were applied, the type I error rates were nearly identical to the
results from all 500 runs.
Type I error rates in Full data without BQL censoring generally stayed close or lower
than the expected 5 or 1%. However, the trend was observed that the error rate
slightly increased as the median of percent censored data increased.
Restricting the CV to 10% caused a higher type I error rate compared to the 20%
CV, while the error rate was reduced to the nominal value as the CV increased to
100%
When the YLO option was implemented with both one-compartment and twocompartment models for BQL data, the type I error rate for structural model
misspecification was close to nominal values.
DISCUSSIONS
0.500
Simulated Concentrations
$SIM SUBPROBLEMS=500
Table 1. Summary of simulation plans for eight scenarios
Proportional
Error
5%)
0.2
0.0
Scenario 2
5LLOQ=0.0884 AE=0.0132)
0.050
NONMEM VI
In simulation plan 1, scenarios 1 to 5 examined the influence of the percentage of data
censored on the structural model decision when the LLOQ had no greater than a 20% CV.
Five different LLOQ values were defined as the concentration at 2, 2.5, 3, 3.5, and 4 half-lives
using typical parameter values. Once the LLOQ was decided for each scenario, an additive
error was chosen so that the CV at the LLOQ was no more than 20%.
In simulation plan 2, scenarios 6 to 8 evaluated the impact of allowing more and less precise
CVs at the LLOQ than the current practice of 20%. This was conducted as variations of
scenario 2. Three different CV values were chosen as 10, 50, and 100%, and these were
analyzed in addition to the 20% CV which was tested as scenario 2.
For each scenario, 500 simulations were conducted. Each simulation consisted of 50 subjects
with 9 PK observations at 0.25, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, and 4 units of time.
0.005
SCENARIOS
0.001
OBJECTIVE
Assess the impact of the percentage of data censored as BQL on the PK structural model
decision
Evaluate the effect of different CV values to define the LLOQ on the structural model decision
Evaluate the use of a maximum conditional likelihood estimation method available in
NONMEM VI 5YLO/LAPLACIAN).
exp 0.693t
The between-subject variability on CL and V were assumed to follow a log-normal distribution
with an exponential error model, and both were set to a 20% CV.
The residual unexplained variability was chosen as a combined proportional/additive error
model to represent an analytical proportional component 5constant CV), and an absolute
additive component 5constant standard deviation) of measurement noise. The proportional
error component was set to a 5% CV. A different additive error was chosen for each scenario
to control the CV at the LLOQ according to the following plans and scenarios.
0.0
CL
t
V
0.2
Dose
C(t)
exp
V
A type I error was declared when the OFV was significantly lower for the two-compartment
model compared to the one-compartment model using the standard likelihood ratio test.
The error rate was determined at a level of significance of 5% and 1%; with two degrees of
freedom, the associated drops in OFV from a 2 table were 5.99 and 9.21, respectively.
The type I error rate was determined from 500 simulations per each scenario with the
following rules.
1. All runs
2. Runs with a successful minimization
3. Runs with a successful minimization and a successful covariance step
4. Runs with reasonable results for the two-compartment model in addition to a successful
minimization and covariance step where a reasonable result was defined as the alphaphase half-life 56t1/2) had to be greater than 0.25 5the first sampling time), and the betaphase half-life 59t1/2) had to be less than 10 units 5considering concentrations were
sampled over 4 units of time).
An intravenous one-compartment pharmacokinetic model was chosen for the simulation. The
clearance 5CL) and volume of distribution 5V) were 0.693 and 1, respectively. A single unitvalued dose was administered at time zero. The PK model becomes and the units of time can
be regarded as half-lives 54).
0.0
It is not uncommon that some concentrations are censored by the bioanalytical laboratory
since those concentrations are below the lower limit of quantification 5LLOQ). The acceptance
of a LLOQ in analytical methods development is nearly universal.
In an effort to report only those concentrations that are considered to have acceptable
precision, the laboratory determines a LLOQ and truncates a standard curve so that no
concentrations are reported below that limit. The LLOQ is often defined in practice as the
lowest concentration on the standard curve that is associated with a CV 5coefficient of
variation) of no more than 20%. The 20% CV is suggested by the FDA Guidance for Industry
Bioanalytical Method Validation and other reports [1, 2]. Typically, any sample associated with
a signal less than LLOQ is not reported quantitatively, but as below the quantification limit
5BQL).
Although these standard operating procedures are well intentioned, the policy of censoring
observations below the LLOQ violates one of the assumptions PK/PD modelers often make.
When using the maximum likelihood estimation method in fitting models to data, it is assumed
that residual errors are independent and normally distributed with zero mean and a variance.
However, censoring data below the LLOQ truncates the tail of this normal distribution and
violates the assumption of residual errors.
The impact of censoring has been examined in population PK settings and procedures for
handling BQL information have been suggested [3-6]. However, these references have
focused on bias and precision of parameter estimates when some data were censored as
BQL. Since the BQL censoring occurs more frequently at later time points, a visual
examination of the cloud of concentration-time data can appear to be associated with a
multiple-compartment drug. To our knowledge, structural model misspecification related to
BQL censoring has not been examined.
TYPE I ERROR RATE
Type I error rate at the alpha level of 0.05
PK MODEL
Type I error rate at the alpha level of 0.05
0.2
0.4
0.6
0.8
1.0
INTRODUCTION
RESULTS
0.0
Objectives: The current simulation study investigated the impact of the percentage of data
censored as BQL on the PK structural model decision; evaluated the effect of different coefficient of
variation 5CV) values to define the LLOQ; and tested the maximum conditional likelihood estimation
method in NONMEM VI 5YLO).
Methods: Using a one-compartment intravenous model, data were simulated with 10 to 50% BQL
censoring, while maintaining a 20% CV at LLOQ. In another set of experiments, the LLOQ was
chosen to attain CVs of 10, 20, 50 and 100%. Parameters were estimated with both one- and twocompartment models using NONMEM VI 5GloboMax LLC, Hanover, MD). A type I error was
defined as a significantly lower objective function value for the two-compartment model compared
to the one-compartment model using the standard likelihood ratio test at alpha=0.05 and
alpha=0.01.
Results: The type I error rate substantially increased to as high as 96% as the median of percent
censored data increased at both the 5% and 1% alpha levels. Restricting the CV to 10% caused a
higher type I error rate compared to the 20% CV, while the error rate was reduced to the nominal
value as the CV increased to 100%. The YLO option prevented the type I error rate from being
elevated.
Conclusions: This simulation study has shown that the practice of assigning a LLOQ during
analytical methods development, although well intentioned, can lead to incorrect decisions
regarding the structure of the pharmacokinetic model. The standard operating procedures in
analytical laboratories should be adjusted to provide a quantitative value for all samples assayed in
the drug development setting where sophisticated modeling may occur. However, the current level
of precision may need to be maintained when laboratory results are to be used for direct patient
care in a clinical setting. Finally, the YLO option should be considered when more than 10% of
data are censored as BQL.
METHODOLOGY
Median of percent data set Type I
censored as
error rate
BQL and negative
concentrations
1
10.2
0.00
2
17.3
0.02
3
26.7
0.02
4
37.6
0.05
5
49.1
0.06
This simulation study has shown that the practice of assigning a LLOQ during analytical
methods development, although well intentioned, can lead to incorrect decisions regarding the
structure of the pharmacokinetic model. The standard operating procedures in analytical
laboratories should be adjusted to provide a quantitative value for all samples assayed in the
drug development setting where sophisticated modeling may occur. However, the current
level of precision may need to be maintained when laboratory results are to be used for direct
patient care in a clinical setting. Finally, the YLO option should be considered when more
than 10% of data are censored as BQL.
REFERENCES
1. FDA guidance for industry bioanalytical method validation. Available from http://www.fda.gov/cder/guidance/4252fnl.pdf Accessed on
may 2001
2. Shah VP, Midha KK, Findlay JW, Hill HM, Hulse JD, McGilveray IJ, McKay G, Miller KJ, PatnaikRN, Powell ML, Tonelli A,
Viswanathan CT, Yacobi A 52000) Bioanalytical method validationa revisit with a decade of progress. Pharm Res 17512):15511557
3. Hing JP, Woolfrey SG, Greenslade D, Wright PM 52001) Analysis of toxicokinetic data using NONMEM:impact of quantification limit
and replacement strategies for censored data. J PharmacokinetPharmacodyn 2855):465479
4. Beal SL 52001) Ways to fit a PK model with some data below the quantification limit. J Pharmacokinet Pharmacodyn 2855):481504
5. Duval V, Karlsson MO 52002) Impact of omission or replacement of data below the limit of quantification on parameter estimates in a
two-compartment model. Pharm Res 19512):18351840
6. Beal SL 52005) Conditioning on certain random events associated with statistical variability in PK/PD. J Pharmacokinet Pharmacodyn
3252):213243
7. Beal SL, Sheiner LB, Boeckmann AJ 5eds) 519892006) NONMEM users guides. Icon development solutions. Ellicott City
8. Stram DO, Lee JW 51994) Variance components testing in the longitudinal mixed effects model. Biometrics5054):11711177