False Negative ECG Device Results May Increase the Risk of Adverse Events in Clinical Oncology Trials (2024)

Abstract

Keywords: Oncology trials, Core lab, ECG algorithm, False negative, QRS, QTcF

Introduction

Accurate measurements of the ECG intervals during clinical trials are crucial for making appropriate inclusion/exclusion and dosing decisions for individual patients, as well as to allow an accurate evaluation of a new drug’s effects on heart rate, cardiac depolarization, and repolarization. The QT interval represents a global measure of the duration and uniformity of ventricular repolarization [1]. QTc thresholds (i.e. QT interval corrected for heart rate) are commonly used in clinical trial inclusion/exclusion criteria to avoid administering a drug whose QT effect has not been adequately defined to a study participant whose QTc is already prolonged [2]. The Common Terminology Criteria for Adverse Events (CTCAE) terminology, which classifies QTc prolongation criteria into 4 grades, are frequently used during clinical oncology trials to inform dosing decisions, and in particular order to avoid dosing patients whose QTc has increased since entry into the trial [3]. The QTc interval is used as a surrogate marker for the detection of increased risk of drug-induced Torsades de Pointes (TdP), a potential lethal form of polymorphic ventricular tachycardia [4, 5]. Correct measurement of the QRS duration is equally important, as drugs may delay cardiac depolarization, and study protocols may exclude patients with baseline QRS prolongation. Drug induced prolongation of the QRS interval, which represents slowing of cardiac depolarization and intracardiac conduction, may represent drug-induced block of rapid or late sodium current or a direct slowing of myocyte to myocyte conduction, and may also be a marker for increased risk of ventricular proarrhythmia [6, 7].

Cardiac intervals are assessed by recording 12-lead ECGs via a standard resting ECG device or continuous Holter recording. Most modern ECG devices include automated algorithms which can print out device generated ECG measurements and interpretive statements. Many clinicians, both in clinical practice as well as within clinical trials, do not have expertise in ECG evaluations, and may rely on the ECG device generated measurements and interpretations for clinical decision-making. The authors have reviewed cases of sudden death during which drug-induced QTc prolongation was not identified by the ECG device automated algorithm and therefore was not recognized by the prescribing physician until the ECGs were reviewed retrospectively by an experienced electrocardiographer (unpublished personal observations).

We have previously reported on the high rate of false positive QTc measurements generated by ECG device algorithms in clinical oncology trials (i.e. device reported QTc values higher than measurements performed at a core lab) [8]. The current research addresses the issues of ECG device algorithm false negative QTcF (QT interval corrected for heart rate by the Fridericia method) and QRS assessments (i.e., device reported interval measurements lower than centralized measurements), and the overall risk of relying on ECG device measurements during clinical oncology trials.

Materials and Methods

From a set of 1,000,000 ECGs collected during a wide range of clinical drug development trials utilizing Clario (Philadelphia, PA) as a centralized core lab, we selected all ECGs collected during clinical oncology trials. All ECGs were collected digitally on calibrated ECG devices provided to the sites by Clario. ECG devices were tested using a calibrated simulator prior to shipment to the investigative sites. The ECG devices were manufactured and validated by Mortara Instruments, which utilized the VERITAS algorithm to generate ECG device measurements, or by GE Healthcare, which utilized the 12SL algorithm. ECGs were collected by the site staff and transmitted digitally to Clario or were recorded on continuous digital 12-lead Holter monitors and were stored on digital flashcards, from which Clario extracted 12-lead ECGs that were processed using the Mortara VERITAS algorithm prior to measurement by Clario personnel. The ECG device algorithm measurements were stored in the Clario database but were not available to the Clario personnel performing the ECG measurements, except for protocols that used a global median beat measurement methodology. When the latter methodology was used, the device-based measurements were available to the staff.

ECG measurements were performed in the Clario EXPERT system using either a semi-automated measurement of 3 consecutive beats on a single lead (typically lead II), or with a global median beat methodology, in which measurements are performed on a superimposition of one median beat from each lead. Nearly all ECG device algorithms, including the GE12SL and Mortara VERITAS algorithms, use a global median beat methodology, with proprietary method of median beat formation and weighting of the various leads. With either measurement methodology, approximately 60% of ECGs required manual adjustment by Clario technicians of 1 or more caliper positions.

Measurements were performed using a semi-automated process combining an algorithm for initial caliper placement followed by review of all ECGs by a team of highly trained technician and a limited number of Clario cardiologists. The Clario technicians adjusted ECG algorithm caliper placements judged to be incorrect. As an additional step to insure correct ECG measurements, all ECGs with measurements outside the normal range, all ECGs with less than good quality, and 5% of all other ECGs selected at random went through an additional review (and if necessary, adjudication) by a second set of trained technicians. Finally, all ECGs were reviewed by a limited group of Clario cardiologists, who also had the opportunity to revise measurements.

The patient randomization status (pre- or post-randomization) was available for most ECGs, but the details of the trial design and the randomization codes were not known to Clario, and for purposes of patient confidentiality, none of the clinical characteristics nor demographics of the patients were known. Thus, the prior cardiac history and information about concomitant medications were not available.

For change from baseline QTcF (ΔQTcF) analysis, screening or baseline ECGs recorded within a 15-min time interval were considered part of the same timepoint and the average QTcF across replicates was considered as the timepoint estimate. The baseline for a given patient was defined as the pre-dose timepoint closest to dosing. Individual ECGs collected while on treatment were compared to this baseline.

Results

A total of 261,572 ECGs had QTcF and QRS measurements from both the ECG device and the core lab available for comparison. The dataset included ECGs collected from 17,475 individual patients participating in 285 clinical oncology trials; 20,786ECGs were recorded at screening, 19,384ECGs were recorded at baseline, 364 ECGs at the time of randomization, 207,065 ECGs were recorded during treatment, 1694 ECGs were recorded at trial termination or during follow-up, and for 12,281 ECGs the randomization status was unknown. The age, gender, prior medical history, oncologic indication, and concurrent medications were not known for any patient. There were 217,908 ECGs measured using 3beats in a single lead, and 43,664 measured using a global median beat methodology. The dataset for the change from baseline analysis contained 13,465 baseline timepoints and 152,066ECGs recorded during treatment with a change from baseline QTcF value, both centrally read and based on the device algorithm.

The mean differences between the core lab and the automated algorithm QTcF and QRS measurements were + 1.8 ± 16.0ms and − 1.0 ± 8.8ms, respectively; on average, centralized QTcF measurements were longer than those from the ECG device algorithm, while the centralized QRS measurements were shorter (Table ​(Table1).1). Treatment did not seem to have a significant impact as the results pre- and post-randomization showed similar results. Across the pre-dose ECGs, the mean differences between the centrally read QTcF and QRS measurements and the automated algorithm were + 1.3 ± 14.7ms and − 1.3 ± 8.8ms. Across the post-dose ECGs, the mean differences between the centrally read QTcF and QRS measurements and the automated algorithm were + 1.9 ± 16.0ms and − 0.9 ± 8.8ms. When using the global median beat methodology, the mean differences between the QTcF and QRS measurements from the core lab and the automated algorithm were + 1.9 ± 12.6ms and − 0.6 ± 4.8ms, respectively, versus + 1.7 ± 16.6ms and − 1.1 ± 9.4ms when measurements were performed using the single lead methodology.

There were many false negative ECG machine algorithm measurements. Among the 17,239 ECGs with a centralized QTcF value > 450ms, 6817 ECGs (39.5%) had a device reported QTcF value ≤ 450ms, and of the 3916 ECGs with a centralized QTcF > 470ms, 1872 ECGs (47.8%) had a device reported QTcF value ≤ 470ms (Table ​(Table2).2). Out of the 330 ECGs with a centrally measured QTcF > 500ms, 184 ECGs (55.8%) had a device reported value ≤ 500ms. Across the ECGs with a false negative QTcF machine reading, the mean centralized versus (vs.) automated QTcF values for QTcF > 450, 470 and 500ms were 460.2 vs. 432.3ms, 480.4 vs. 443.8ms, and 514.6 vs 452.4ms respectively. The standard deviation of the machine measurements was significantly higher than for the centralized measurements indicating a larger variability in the QTcF measurements.

The differences between the centralized and ECG machine QRS measurements were of smaller magnitude. Out of the 12,177 ECGs with a centrally measured QRS > 110ms, 964 (7.9%) had a device reported QRS value ≤ 110ms, and of the 8543 ECGs with a centrally measured QRS > 120ms, 624 ECGs (7.3%) had a device reported QRS ≤ 120ms (Table ​(Table2).2). Across the ECGs with a false negative QRS machine measurement, for centralized QRS measurements of > 110 and > 120ms, the mean centralized and machine measurements were 116.2 vs. 104.8 and 126.8 vs. 113.2ms respectively. Similar trends were seen when evaluating only ECGs recorded prior to the first drug administration (Table ​(Table33).

Table ​Table44 shows the agreement between the core lab and device algorithm measurements when grading the ECGs using the CTCAE QTc prolongation criteria. The agreement was 55.6, 38.5 and 44.2% for grade 1, 2 and 3 respectively. ECGs measured at the core lab using the global median beat showed a greater agreement with the device readings compared to the ECGs measured using the 3beats on a single lead methodology. For the global median beat methodology, the agreement was 78.0, 60.1 and 51.2% for grade 1, 2 and 3 respectively, versus 49.4, 29.7 and 39.9% for the three beats on a single lead methodology.

Based on the centralized measurements, the QTcF change from baseline (ΔQTcF) was > 30ms for 10,857ECGs (Table ​(Table5).5). The device algorithm based ΔQTcF based was ≤ 30ms for 48.3% of these cases. Out of 707 ECGs for which the core lab identified a ΔQTcF > 60ms, 381 ECGs (53.9%) had ΔQTcF ≤ 60ms based on the device algorithm measurements.

Figures1 and ​and22 show examples of ECGs where the end of the T-wave was incorrectly identified by the ECG machine algorithm, resulting in under-reporting of the QT and QTc intervals. Figure1 shows an ECG where the QT was centrally read on three consecutive beats on a single lead. The mean centralized QT measurement was 492ms, while the device algorithm undermeasured the QT at 344ms. Figure2 shows an ECG for which the centralized measurements were performed using the global median beat methodology. The ECG machine algorithm under-measured the QT interval by 95ms. Figure3 shows an ECG for which the ECG machine algorithm markedly undermeasured the QRS duration. The mean centrally measured QRS duration was 101ms, while the ECG machine algorithm reported a QRS duration of 39ms.

Discussion

The accurate evaluation of the cardiovascular safety profile is an important step during the development of any new investigational drug. Depending on the direct and indirect mode of action of the drug, tests may include cardiac imaging, analysis of cardiac biomarkers, blood pressure assessments and electrocardiographic monitoring [9, 10]. Although there are numerous different classes of cardiac toxicities, among the most serious is ventricular proarrhythmia which may result in sudden cardiac death. The mechanism for this rare but lethal event is now known to be Torsades de Pointes (TdP), an unusual type of ventricular tachycardia that has a characteristic morphology and is a consequence of alterations in ventricular repolarization. For non-antiarrhythmic drugs that produce TdP, the incidence is low enough that it is not feasible to detect an excess of TdP during clinical trials, and drug developers have therefore been forced to rely on surrogate markers for assessing a new drug's risk of producing TdP. The surrogate marker that is currently utilized for the detection of increased risk of drug-induced TdP is prolongation of the QTc interval [4, 5]. All drugs that are known to produce TdP have been demonstrated to prolong QTc (as measured on the surface ECG) despite different chemical structures and potentially different mechanisms of prolonging cardiac repolarization. The current paradigm for detection of drug-induced QTc prolongation, as described in the International Conference for Harmonization (ICH) E14 Guidance for Industry and its subsequent Q&A releases, relies on detection of a drug-induced mean QTc prolongation of about 5ms (as evidenced by an upper bound of the 95% confidence interval around the mean effect on QTc of 10ms) [11]. For drugs with relatively wide therapeutic indices, the QTc assessment may be performed in a dedicated QT/QTc study or during the single and multiple ascending dose studies performed early in a drug's development. These studies are usually performed in healthy volunteers under highly controlled clinical trial settings and involve serial replicate ECGs collection time-matched to the PK sampling timepoints, the use of precise measurement techniques and limitations in the use of concomitant medications [12]. Many oncologic drugs, however, cannot be administered to healthy volunteers in supratherapeutic or even therapeutic doses, and QTc assessments must therefore be performed in the clinical trials performed with oncology patients who are typically older, may have additional comorbidities, are prone to electrolyte shifts, and on average, have a higher QTc than healthy volunteers [13]. The precision of QTc measurements is therefore very important as greater precision results in increased statistical power of the trial to accurately assess a drug's effect on QTc.

Twelve-lead ECGs are collected in nearly all clinical trials during the screening or baseline assessments and during the treatment phase. The inclusion/exclusion criteria typically include QTc (and to a lesser extent QRS) threshold criteria to avoid administering a drug whose effect on cardiac conduction has not been adequately defined to a subject with preexisting QTc or QRS prolongation. Many oncologic agents are known to prolong the QTc interval. The QRS is an integral part of the QT interval, and therefore drug-induced QRS prolongation can result in a direct increase in the QT interval, related to the increased QRS duration, independent of any real effect on ventricular repolarization [14]. The CTCAE classification guideline describes QTc prolongation criteria as grade 1 (QTc 450–480ms), grade 2 (QTc 481–500ms), grade 3 (QTc > 500ms or QTc change from baseline > 60ms) and grade 4 (Torsade de pointes; polymorphic ventricular tachycardia; signs/symptoms of serious arrhythmia) [3]. In many oncology studies, a grade 2 QTc prolongation may result in withholding drug dosing until the QTc prolongation resolves. A false positive report of QTc prolongation based on the ECG machine algorithm values may lead to increased ECG monitoring or withholding of doses. In a clinical oncology trial, unnecessary withholding of doses may have deleterious effects both for the patient as well as for the drug development program. During the dose escalation portion of a Phase I oncology trial, this may also affect decisions about dose escalation. Grade 3 QT prolongation may lead to patient discontinuation from the trial or may be considered a dose limiting toxicity that precludes further dose escalation. The risk of TdP is not a linear function of the duration of the QTc interval, nor of the extent of QTc prolongation during drug therapy [15, 16]. However, progressive prolongation of the QTc interval increases the risk for TdP, and the risk increases markedly when the QTc interval exceeds 500ms [17–19]. Data from Congenital Long QT Syndrome studies indicate that a QTc > 500ms is associated with a much higher risk for TdP. Likewise, case reports and small series of patients with drug-induced TdP show similar increased risk when the threshold of QTc > 500ms is exceeded [20].

False negative results—underreporting of QTc prolongation by automated ECG algorithms—poses even greater risks. A false negative QTc value (a QTc value reported by the ECG machine algorithm that is substantially lower than the true QTc value) exposes the patient to the risk that further drug-induced QT prolongation may result in TdP, potentially with lethal consequences. While false positive findings tend to be disruptive for the site and patient, they rarely impact patient safety, and their effects can be mitigated following recognition that the finding is a false positive. In contrast, false negative results may expose patients to safety risks and may result in patient deaths before they are recognized.

Accurate cardiac intervals assessments are thus of critical importance in oncology trials, both to maintain patient safety as well as to allow program wide assessments of the QTc effects of a new drug. Many clinical trials in oncology do not utilize a central ECG lab to perform the cardiac interval measurements and rely upon the sites to evaluate the ECGs. Many clinicians, both in clinical practice as well as within clinical trials, do not have expertise in ECG evaluations [21, 22]. Furthermore, those who are familiar with measuring ECG intervals may still use the ECG machine algorithm measurements, which can contain errors and cannot match the precision of core lab measurements performed using digital ECG waveforms measured at high magnification [23, 24]. Most ECGs are printed with a paper speed of 25mm/s and have a pen width of 5–10ms. Using metal calipers, the resolution of QT measurements ranges from 10 to 20ms. In contrast, centralized interval measurements of digital ECG waveforms can typically achieve a resolution of 1ms.

We previously reported on the presence of false positives when comparing ECG device measurements with device measurements. The present study extends these results by evaluating ECG device algorithm false negative QTc and QRS assessments, by analyzing a dataset of 261,572 ECGs collected during oncology clinical trials.

The mean QTcF and QRS differences between the core lab and the device algorithm measurements were small, + 1.8 and − 1.0ms respectively. On average the core lab QTcF measurements were slightly longer than those from the ECG device algorithm, while the core lab’s QRS measurements were shorter, independent of the measurement methodology used by the core lab or randomization status. However, the confidence bounds around these point estimates were larger based on device algorithms, reflecting the large numbers of automated measurements that were significantly too high or too low. Exclusion thresholds commonly used in clinical trials include QTcF > 450ms and QTcF > 470ms. Our analyses found that ~ 40% and ~ 48% of the ECGs with a centrally measured QTcF value > 450 and > 470ms, respectively, had a device measurement below these thresholds. Furthermore, the differences in measurements often were not trivial. Among ECGs with a centrally measured QTcF > 450ms, the mean centrally measured QTcF was 460ms, compared to a mean automated measurement of 432ms. For the ECGs with centrally measured QTcF > 470ms, the mean centrally measured QTcF was 480ms compared to a mean automated measurement of 444ms. These effects were independent of the investigational drug being evaluated, as the results were similar for both pre-randomization and post-randomization ECGs. For the ECGs with a centralized QTcF > 500ms, 56% had an automated measurement ≤ 500ms, i.e., the CTCAE grade 3 QTc prolongation agreement was 44%. The mean QTcF for these ECGs was 515 and 452ms for centralized and automated measured QTcF, respectively, with a standard deviation of 16 and 47ms, respectively.

ECG machine automated measurements also failed to detect many instances in which QTcF had increased substantially compared to baseline. Out of 707 ECGs for which the core lab identified a QTcF increase from baseline > 60ms, only 46.1% were correctly identified by the ECG machine measurements. Reliance on ECG machine measurements may therefore result in patients who have already had a large QTc increase continuing to receive a drug that may have been responsible for the QTc prolongation. The large categorical and central tendency difference between the centrally read and automated measurements may be due to these ECGs being more noisy or having challenging T waves (flat or biphasic T wave), which confound the ability of the algorithms to accurately determine the end of the T wave. However, as illustrated in Figs.1, ​,22 and ​and3,3, ECG algorithms may generate false negative QRS and QT measurements even for good quality ECGs.

False negative QTc data may result in the inclusion in a trial of patients who may be at increased risk when receiving an investigational drug whose effect on repolarization is not yet known. Under-measurement of the QRS duration may also expose patients to proarrhythmic risk for both QTc and QRS prolonging drugs. Since ECG devices algorithms use the QRS duration as a criterion to determine the presence of a bundle branch block, complete bundle branch block may be missed by sites who heavily rely on their ECG device generated measurements and interpretations for clinical decision-making.

Conclusion

Our analysis shows that the use of automated measurements from ECG devices poses a potential risk to patients participating in oncology studies. False negative QTcF and QRS automated measurements may lead to the inappropriate inclusion of high-risk patients into a trial and during a trial may lead to patient exposure to a drug that has already produced QTc or QRS prolongation. ECGs should therefore always be carefully reviewed and measured at the site or at a central core lab to detect measurement errors.

Author Contributions

All authors were involved in the analysis of the data, the interpretation of the results and the preparation of the manuscript.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article. The authors would like to thank Kalvin Connor for providing statistical analysis support.

Declarations

References