Two-graph receiver operating characteristic (TG-ROC): a Microsoft-EXCEL template for the selection of cut-off values in diagnostic tests

doi:10.1016/0022-1759(95)00078-O

Journal of Immunological Methods

Volume 185, Issue 1, 11 September 1995, Pages 145-146

https://doi.org/10.1016/0022-1759(95)00078-O Get rights and content

Abstract

TG-ROC, a template for Microsoft-EXCEL, represents a novel, easy-to-handle approach for selecting cut-off values in quantitative diagnostic tests. In addition to graphical representations of test efficiency, Youden index and likelihood ratios as functions of the preselected cut-off value, the software supports the definition of an intermediate range of test results. For this purpose, two cut-off values are established that realise a pre-selected accuracy level (e.g., 90 or 95% sensitivity and specificity) which can be specified by the user.

References (5)

M. Greiner et al.
Notes about determining the cut-off value in enzyme-linked immunosorbent assay (ELISA) — Reply
Prev. Vet. Med.
(1994)
M. Greiner et al.
A modified ROC analysis for the selection of cut-off values and the definition of intermediate results of serodiagnostic tests
J. Immunol. Methods
(1995)

There are more references available in the full text version of this article.

Cited by (108)

Validation of laboratory developed serology assays for detection of IgG antibody to severe acute respiratory syndrome coronavirus 2 in the South African population
2022, Journal of Virological Methods
Citation Excerpt :
In many instances a cut-off is determined using two or three standard deviations above and below the mean value however this has limitations as it assumes a normal distribution of test values in the target population. The application of a TG-ROC provides an alternative method for determining a cut-off (Greiner, 1995). In this validation data normalization was achieved by expressing OD values as a percentage of a known positive control included in each run.
Serological assays for detection of IgG, IgM or IgA against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) play an important role in surveillance, antibody persistence, vaccine coverage and infection rate. Serological assays, including both ELISA and rapid lateral flow assays, are available commercially but the cost limits their accessibility for low resource countries. Although serological assays based on mammalian-expressed SARS-CoV-2 spike protein have been previously described these assays need to be validated using samples from local populations within the continent, or country, in which they will be used. Interpretation of results could be influenced by differences in specificity and potential for pre-existing cross-reactive antibodies. In this study, we investigated two laboratory developed serological assays, an enzyme linked immunosorbent assay (ELISA) and an immunofluorescent assay (IFA), developed using recombinant SARS-CoV-2 spike protein, for use in South African populations. The tests were compared with commercially available and South Africa Health Products Regulatory Authority (SAPHRA) approved assays. A panel of 100 residual diagnostic serum samples, collected prior to the pandemic, were tested on three separate occasions to determine a suitable cut-off value for differentiation of positive from negative samples. Specificity of 96 % and 100 % for ELISA and IFA respectively was demonstrated. A total of 82/89 serum samples collected between days 2–94 after onset of illness from patients with a positive molecular result were positive for IgG antibody. The sensitivity of the laboratory developed assays on samples collected > one week after onset of illness was shown to be 100 % and 98.8 % for ELISA and IFA respectively. Positive predictive values were 92.1 % for ELISA and 91.0 % for IFA using characterization of samples as positive based on confirmation of infection using RT-PCR. Serum samples (n = 62) collected from RT-PCR positive patients infected with either ancestral, or emerging variants such as Beta or Delta, tested positive for IgG antibody (62/62) using the laboratory developed assays confirming application of the assays regardless of currently circulating variant during the time of evaluation. High concordance was demonstrated between the laboratory developed assays and the commercial immunoassay among samples collected from South African populations, although the small sample size, especially for the comparison with commercial assays, must be noted. If all quality assurance controls are in place, the use of local laboratory developed assays for high-throughput screening in resource-constrained environments is a realistic alternative option.
Comparison of protectiveness of recombinant Babesia ovis apical membrane antigen 1 and B. ovis-infected cell line as vaccines against ovine babesiosis
2020, Ticks and Tick-borne Diseases
Citation Excerpt :
Each plate consisted of four replicates for each of negative, positive and conjugate controls. The mean of the OD plus twice-standard deviation (mean OD + 2SD) was accepted as “cut-off” value for each plate (Greiner, 1995). Positive and negative threshold percentage positivity (PP) values were chosen arbitrarily to obtain the optimal sensitivity and specificity using a set of post infection serum samples as a known positive and a pool of pre-infected serum samples (n = 10) was use as a negative control.
Babesiosis is a disease complex caused by unicellular Babesia parasites and among them, malignant ovine babesiosis caused by B. ovis has a devastating economical impact on the small ruminant industry. The control of disease is mainly based on chemotherapy and preventing animals from tick infestation and to date no vaccine is available against ovine babesiosis. The requirement for vaccination against B. ovis infection in endemically unstable regions is necessary for implementation of effective disease control measures. The aim of the present study was to evaluate the effectiveness of different immunisation protocols against disease in sheep experimentally vaccinated with recombinant B. ovis apical membrane antigen-1 (rBoAMA-1) and/or live, a B. ovis-infected cell line. Sheep were divided into four experimental groups, plus a control group. Animals were immunised either with the B. ovis stabilate, or with rBoAMA-1, or with both rBoAMA-1 and the B. ovis stabilate. Western blots and ELISAs indicated that immunisation with rBoAMA-1 resulted in generation of a specific response against the recombinant protein, but the degree of antibody response did not correlate with the level of induced protection against challenge. The strongest immune response was induced in animals co-immunised with the live B. ovis stabilate plus rBoAMA-1. Both the hematological and parasitological findings indicated that this co-immunisation regimen has vaccine potential to limit losses incurred by ovine babesiosis in endemic countries.
Preoperative multiplex nomogram for prediction of high-risk nonorgan-confined upper-tract urothelial carcinoma
2019, Urologic Oncology: Seminars and Original Investigations
Accurate risk stratification prior to radical nephroureterectomy remains a challenge with upper-tract urothelial carcinoma (UTUC). Herein, we generated an optimized preoperative tool predicting high-risk nonorgan-confined (NOC)-UTUC.
Retrospective evaluation of 699 patients undergoing radical nephroureterectomy at 3 academic centers. Multiplex preoperative patient, imaging, endoscopic, and laboratory values were evaluated. Model derivation and validation were based on a split-sample method. Patients were divided randomly into a development (training) cohort (70% of patients) and validation (test) cohort (30% of patients). Univariate and multivariate logistic regression addressed the prediction of NOC disease (pT3/pT4 and/or pN+) based on training cohort. A backward stepdown selection process achieved the most informative nomogram. The ROC analysis identified a cut-off point predicting high-risk disease. The test cohort served as “external” validation to verify the findings based on the training cohort. Bootstrap resampling was conducted for both internal and “external” validation to evaluate the model fitting.
Total of 566 patients included for analysis, mean age 69.7 years, 85% Caucasian, 64% male, 62% high grade. NOC-UTUC was found in 184 (32.5%) patients on final pathology. Of 184 patients with NOC-UTUC, an equal number of renal pelvis and ureter only tumors (n = 74; 40.2% for each location) were noted; 36 (19.6%) had tumors in both locations. Multivariate model based on development cohort (n = 396) demonstrated clinical stage (odds ratio [OR] 14.0, P < 0.01), biopsy tumor grade (OR 3.3, P = 0.01), tumor architecture (OR 2.65, P = 0.09), and Hgb (OR 0.8, P = 0.02) level were independently associated with NOC disease. A preoperative nomogram incorporating these 4 variables achieved 82% accuracy, 48% sensitivity, and 95% specificity in predicting NOC-UTUC. The cut-off point for predicting high-risk disease was ≥0.49.
We established and validated an accurate tool for the prediction of locally advanced NOC-UTUC. This preoperative nomogram can be used to more optimally select patients for preoperative systemic chemotherapy, and facilitate clinical trial enrollment.
Metaperiodate deglycosylation of Strongyloides venezuelensis larvae: Immunochemical characterization and antigen production for human strongyloidiasis diagnosis
2018, Acta Tropica
Citation Excerpt :
Two-way ANOVA with the antigens (SE and MP) and IgG subclasses factors, followed by the Šidák multiple comparison test, were applied to determine how the response/detection was affected by two factors. Receiver operating characteristic (ROC) and two-graph ROC (TG-ROC) curves were built to describe the tests and define the optimum point of each reaction condition (cut-off) (Greiner, 1995; Greiner et al., 1995). Tests were considered positive when OD > cut-off.
Strongyloidiasis is an important helminthiasis affecting million people worldwide. The aim of this study was to use sodium metaperiodate (MP) treatment to immunochemically characterize Strongyloides venezuelensis filariform larvae and use MP-treated heterologous antigen to detect IgG and subclasses in serum. Samples from individuals with definitive diagnosis of strongyloidiasis (n = 50), other parasitic diseases (n = 60) and negative endemic (n = 50) were tested. TG-ROC and two-way ANOVA were applied. MP-treatment resulted on differential localization of carbohydrates at larval structure and no carbohydrate content in saline extract (SE). Electrophoretic profiles were similar before and after treatment. ELISA sensitivity and specificity were: 90%; 88.2% for SE and 92.0%; 94.6% for MP, respectively. When using MP treated antigen we observed reduction in IgG1 and IgG3 detection in strongyloidiasis group and decrease of cross reactions in control groups. Our data demonstrate the role of carbohydrate residues in cross reactions and on the recognition of anti-Strongyloides IgG and its subclasses.
Missing values and inconclusive results in diagnostic studies – A scoping review of methods
2023, Statistical Methods in Medical Research
Objective Cost-Informed Cutoff Criteria Improve the Utility of Machine Learning Models of Environmental Hazards: A Case Study of Groundwater Arsenic Distribution in India
2023, Exposure and Health

View all citing articles on Scopus

^∗: Tel.: +49 30 8108 2317; Fax: +49 30 8108 2323

View full text

Software noticeTwo-graph receiver operating characteristic (TG-ROC): a Microsoft-EXCEL template for the selection of cut-off values in diagnostic tests

Abstract

Prev. Vet. Med.

J. Immunol. Methods

Software notice
Two-graph receiver operating characteristic (TG-ROC): a Microsoft-EXCEL template for the selection of cut-off values in diagnostic tests