You are here

  • Home
  • Archive
  • Volume 99, Issue 12
  • Development and validation of a prediction rule for recurrent vascular events based on a cohort study of patients with arterial disease: the SMART risk score

Article Text

Article menu
Prognosis research
Original article
Development and validation of a prediction rule for recurrent vascular events based on a cohort study of patients with arterial disease: the SMART risk score
  1. Johannes A N Dorresteijn1,
  2. Frank L J Visseren1,
  3. Annemarie M J Wassink1,
  4. Martijn J A Gondrie2,
  5. Ewout W Steyerberg3,
  6. Paul M Ridker4,
  7. Nancy R Cook4,
  8. Yolanda van der Graaf2,
  9. on behalf of the SMART Study Group
  1. 1Department of Vascular Medicine, University Medical Center Utrecht, Utrecht, The Netherlands
  2. 2Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
  3. 3Department of Public Health, Center for Medical Decision Making, Erasmus Medical Center, Rotterdam, The Netherlands
  4. 4Division of Preventive Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA
  1. Correspondence to Dr Frank L J Visseren, Department of Vascular Medicine, University Medical Center Utrecht, PO Box 85500, Utrecht 3508 GA, The Netherlands; F.L.J.Visseren{at}umcutrecht.nl

Abstract

Objectives To enable risk stratification of patients with various types of arterial disease by the development and validation of models for prediction of recurrent vascular event risk based on vascular risk factors, imaging or both.

Design Prospective cohort study.

Setting University Medical Centre.

Patients 5788 patients referred with various clinical manifestations of arterial disease between January 1996 and February 2010.

Main outcome measures 788 recurrent vascular events (ie, myocardial infarction, stroke or vascular death) that were observed during 4.7 (IQR 2.3 to 7.7) years’ follow-up.

Results Three Cox proportional hazards models for prediction of 10-year recurrent vascular event risk were developed based on age and sex in addition to clinical parameters (model A), carotid ultrasound findings (model B) or both (model C). Clinical parameters were medical history, current smoking, systolic blood pressure and laboratory biomarkers. In a separate part of the dataset, the concordance statistic of model A was 0.68 (95% CI 0.64 to 0.71), compared to 0.64 (0.61 to 0.68) for model B and 0.68 (0.65 to 0.72) for model C. Goodness-of-fit and calibration of model A were adequate, also in separate subgroups of patients having coronary, cerebrovascular, peripheral artery or aneurysmal disease. Model A predicted <20% risk in 59% of patients, 20–30% risk in 19% and >30% risk in 23%.

Conclusions Patients at high risk for recurrent vascular events can be identified based on readily available clinical characteristics.

https://doi.org/10.1136/heartjnl-2013-303640

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    According to American and European guidelines, all patients with symptomatic arterial disease have 20% or more absolute 10-year risk of developing recurrent vascular events.1–3 However, clinicians intuitively know that actual risk is not the same for all patients. Identifying patients at highest risk is becoming more and more important, because potentially hazardous and expensive novel preventive treatments (eg, biologicals, immunomodulants and antithrombotics) may not be indicated for all patients with vascular disease. Although it should be appreciated that blood pressure and lipid management are required in most vascular patients, risk stratification could help clinicians determine which patients benefit most from aggressive therapy and expensive programmes for lifestyle improvement. Finally, patients may wish to be informed about their individual prognosis and risk factors for disease recurrence.

    Yet, although some prediction rules for risk stratification of patients with vascular disease exist, these are not commonly used in practice. Reasons may include that these risk scores are applicable to restricted subgroups of vascular patients only, predict short-time (eg, 2-year) prognosis or are made up of complicated point-scores with long lists of variables.4–9 Commonly used prediction models, such as the Framingham risk score, the Systematic Coronary Risk Evaluation score (SCORE) and the Reynolds Risk Score, have been developed in subjects without clinical manifestations of cardiovascular disease,10–13 and have not been validated in a population with clinically manifest atherosclerosis.14 Moreover, these commonly used risk scores lack valuable determinants of risk that are of particular importance in patients with established disease, such as the time since first diagnosis of vascular disease, vascular disease history (eg, coronary event or cerebrovascular event) and renal function.15 Different types of vascular imaging, alone or in addition to clinical risk factors, have also been proposed for identifying high risk patients among those with symptomatic arterial disease.15–18 We therefore aimed to develop and validate practical models for prediction of long term (ie, 10-year) risk of developing recurrent vascular events in individual patients with any type of arterial disease, based on clinical parameters, imaging or both.

    Methods

    Study population

    Patients originated from the SMART (Secondary Manifestations of ARTerial disease) study, an ongoing single-centre prospective cohort study at the University Medical Centre Utrecht, The Netherlands. A detailed description of the SMART study has been published previously.19 For the present analysis we used data of 5788 patients who were newly referred to the University Medical Centre between January 1996 and February 2010 for any clinical manifestation of arterial atherosclerosis (ie, cerebrovascular disease (CVD), coronary artery disease (CAD), peripheral artery disease (PAD) or abdominal aortic aneurysm (AAA)). CAD was defined as angina pectoris, myocardial infarction (MI) or coronary revascularisation (coronary bypass surgery or coronary angioplasty). Patients with CVD had had a transient ischemic attack, cerebral infarction, amaurosis fugax or retinal infarction, or a history of carotid surgery. PAD was defined as a symptomatic and documented obstruction of distal arteries of the leg or surgery of the leg (percutaneous transluminal angioplasty, bypass or amputation). Patients with AAA had a supra- or infrarenal aneurysm of the aorta (distal aortic anteroposterior diameter ≥3 cm, measured with ultrasonography) or a history of AAA surgery. Excluded were patients with a terminal malignancy, those not independent in daily activities or not sufficiently fluent in Dutch language, and those without planned clinical follow-up appointments in the study centre. In practice, patients who presented with an acute vascular event were treated by a medical specialist first and enrolled in the study only after they had reached a stable situation in the course of their disease. This was usually several months later. The study was approved by the ethics committee at the University Medical Centre Utrecht, and all participants gave their written informed consent.

    Baseline examination

    Patients completed a standardised vascular screening protocol, including a questionnaire on medical history, parental history of an MI <60 years, smoking status and current medication use.19 Office blood pressure was measured and fasting venous blood and urine samples were taken. Carotid intima-media thickness (cIMT) was measured at the left and right common carotid arteries with an ATL Ultramark 9 (Advanced Technology Laboratories, Bethel, Washington, USA) equipped with a 10-MHz linear array transducer. Stenosis of the common or internal carotid artery was detected by colour Doppler-assisted duplex scanning and defined as >50% occlusion.20 Evidence of plaque formation was included in the diagnostic criteria for stenosis.19

    Based on the questionnaire, the time since first diagnosis of clinically manifest atherosclerosis was calculated. When the patient's first vascular event occurred in the preceding year, the duration of disease was rounded down to zero years. A history of diabetes mellitus was defined as either a referral diagnosis of diabetes mellitus, self-reported diabetes mellitus, a known history of diabetes mellitus at the time of enrolment or a fasting plasma glucose ≥7 mmol/l. Glomerular filtration rate (eGFR) was estimated using the Modification of Diet in Renal Disease formula.21

    Follow-up

    Patients biannually completed a questionnaire on hospitalisations and outpatient clinic visits. The outcome of interest for this study was first occurrence of a major cardiovascular event, defined as cardiovascular death, ischaemic or haemorrhagic stroke, or myocardial infarction. Definitions of events are shown in online supplement 5. When a possible event was reported, hospital discharge letters and results of relevant laboratory and radiology examinations were collected. Death and cause of death were reported by relatives of the participant, the general practitioner or a medical specialist. All events were adjudicated by three members of the endpoint committee, composed of physicians from different departments. Follow-up duration was defined as the period between enrolment and first cardiovascular event or death from any cause, date of loss to follow-up or the preselected date of 1 March 2010. Events occurring after this date were ignored. Of the 5788 participants, 197 patients (3.4%) were lost to follow-up due to migration or withdrawal from the study; these patients were censored.

    Model derivation

    The models were developed on a random 60% of the study data (n=3489 patients). First, single imputation by weighted probability matching on the basis of multivariable regression using covariate and outcome data was used to reduce missing values for carotid artery stenosis (n=217; 4%), parental age in case of parental history of MI at unknown age (n=562; 10%) and parental history of MI <60 years (n=791; 14%). Biologically implausible values were considered missing values. Continuous predictors were truncated at the 1st and 99th percentile to limit the effect of outliers.22

    Next, three Cox proportional hazards models for the occurrence of major cardiovascular events were fit in the derivation cohort based on age and sex in addition to clinical parameters (model A, ie, the SMART risk score), carotid ultrasound findings (model B) or both (model C, ie, the SMART risk score PLUS). Clinical parameters considered for models A and C were based on the Reynolds Risk Score,10 ,11 with the addition of a few easy to measure characteristics of particular relevance in secondary prevention. As a result, clinical candidate predictors were: presence of diabetes mellitus, current smoking, systolic blood pressure, total cholesterol, high density lipoprotein-cholesterol, high-sensitivity C-reactive protein (hs-CRP), parental history of MI <60 years, years since first vascular event, history of CAD, CVD, PAD or AAA, and eGFR. Carotid ultrasound findings included in models B and C were: cIMT and presence of stenosis.

    To improve model fit, continuous predictors were transformed if the association between a continuous predictor and the outcome was not linear as evidenced by the model χ2.22 As a result, hs-CRP was log transformed and quadratic terms were added for age and eGFR. The model coefficients were estimated with penalised maximum likelihood with a restriction on the sum of the absolute coefficients of standardised predictors (ie, the Lasso method). The optimal penalty factor was determined by cross-validation of model A and this same penalty factor was used for all three models.22–24

    All three models were fitted for prediction of 7-year risk and extrapolated to obtain 10-year risk predictions through exponentiation of the models’ mean survival estimates. Because 30% of study patients completed at least 7 years’ follow-up as opposed to only 12% who completed 10 years’ follow-up, this increases the stability of the model predictions. The proportional hazard assumption was assessed by testing the correlations between scaled Schoenfeld residuals for the various predictors and time.

    Validation

    Performance was tested in the remaining 40% of the study population (n=2299). The discriminatory ability of the models (or the extent to which the model can separate those with and without a recurrent cardiovascular event) was expressed by the concordance statistic.25 The concordance statistic was computed for the overall population and within separate subgroups of patients with CAD, CVD, PAD or AAA. Model calibration, reflecting the precision of how close the predicted probabilities are to the actual (observed) risk, was demonstrated by calibration plots. In addition, validation cohort patients were divided into the following categories of 10-year predicted risk: <10%, 10 to <20%, 20 to <30%, 30 to <40% and ≥40%. Within each category, predicted risk was compared to actual observed Kaplan–Meier survival during follow-up.26 Overall goodness-of-fit of both models was assessed in the validation cohort with the Gronnesby and Borgan test.27 ,28 Finally, the net benefit of prediction-based decision-making was presented in a decision curve (see online supplement 1).

    All analyses were conducted with R statistical software V.2.11.1 (http://www.R-project.org) using add-on packages Design, Hmisc, survival, survcomp (http://www.bioconductor.org/packages/2.12/bioc/html/survcomp.html), stdca (http://www.decisioncurveanalyis.org) and penalised.23

    Results

    Table 1 shows baseline characteristics for participants in the derivation and validation cohorts. During a median follow-up of 4.7 years (IQR 2.3–7.7; 30 012 person-years), 788 cardiovascular events occurred in the total cohort (overall event rate: 2.6% per year; 483 in the derivation cohort and 305 in the validation cohort).

    View this table:
    Table 1

    Baseline characteristics

    Model derivation

    Penalised maximum likelihood estimation with the cross-validated optimal penalty factor (λ=3.2) reduced the coefficient for parental history of MI <60 years in models A and C to zero. The coefficients of the remaining predictors and HRs of models A, B, and C are presented in table 2. The computational formulas are presented in online supplement 6. Moreover, an interactive calculation sheet for model A is available (see online supplement 2). Some non-proportionality was observed with respect to the coefficient of history of PAD in models A and C (p=0.03 and p=0.02, respectively). Yet, because the coefficient remained consistently positive over time and non-proportionality was not observed for any of the other coefficients, we concluded that the proportional hazards assumption was sufficiently met.

    View this table:
    Table 2

    Model coefficients and HRs

    Validation

    In the split sample validation dataset, the concordance statistic of model A including clinical parameters only (ie, the SMART risk score) was significantly better (p<0.01) compared with the concordance statistic of model B including carotid ultrasound findings only (table 3). Although apparently trivial, the difference between the concordance statistics of models A and C (ie, the SMART risk score PLUS) was statistically significant (p=0.03), favouring model C. The concordance statistics within each subgroup of vascular patients support adequate discrimination of models A, B and C in all types of vascular patients (table 3).

    View this table:
    Table 3

    Summary statistics of performance

    The calibration plots of 10-year predicted versus observed event-free survival (ie, 1-risk) of models A, B and C are shown in figure 1. In addition, the results of the Gronnesby and Borgan tests confirm satisfying goodness-of-fit of models A (p=0.31), B (p=0.22) and C (p=0.88). Figure 2 shows that predicted and observed risk also match when patients are classified into clinically relevant risk categories. The SMART risk score predicted low or moderate risk (<10% or 10–20%) in 1347 (59%) patients, intermediate risk (20 to <30%) in 432 (19%) and high or extremely high risk (≥30%) in 520 (23%). Observed risk was similar to predicted risk, except in the group with 30 to <40% risk, in which 26% 10-year risk was observed (figure 2). Figure 2 shows similar findings based on models B and C. Furthermore, in online supplement 3, survival curves are shown of the above risk categories according to all three models, showing that the higher risk groups consistently demonstrated worse survival at all time points. Finally, calibration plots of the SMART risk score in subgroups of vascular disease patients are shown in online supplement 4. Finally, the decision curve in online supplement 1 demonstrates that model A predictions can improve net benefit of medical decisions when there is a need to select patients having >15% or higher 10-year risk for recurrent vascular events.

    Figure 1
    Figure 1

    Calibration plots of models A, B and C in the validation cohort. Predicted and observed survival (ie, 1-risk) within quintiles of predicted survival at 10-years according to model A (ie, SMART risk score based on clinical characteristics only, left), model B (ie, carotid ultrasound findings only, middle) and model C (ie, SMART risk score PLUS based on clinical characteristics and carotid ultrasound findings combined, right).

    Figure 2
    Figure 2

    Risk classification flow chart of validation cohort patients according to models A, B and C. Number of patients and actual (observed) risk in categories of predicted risk.

    Discussion

    In the present study of 5788 patients with clinically manifest arterial disease, we developed and validated three models for estimating the absolute risk for recurrent major vascular events based on clinical parameters, carotid ultrasound findings or both combined. We found that the discrimination of the model based on clinical parameters in a split sample validation cohort was superior to the discrimination of the model based on carotid ultrasound findings only and just a little smaller compared with the accuracy of the model based on clinical parameters and imaging combined. Furthermore, the model including clinical parameters only showed satisfactory goodness-of-fit and was able to stratify patients according to level of vascular risk. Because a model based on clinical parameters without the need for carotid ultrasound is easy to use, we propose this relatively simple model, the SMART risk score, to be used in practice for the prediction of 10-year risk for recurrent vascular events in patients with any type of symptomatic atherosclerotic vascular disease. Importantly, the performance of this model was reasonable in all subgroups of patients with different types of vascular disease (ie, CAD, CVD, PAD or AAA). Although the eligibility criteria of the SMART study were broad and patients with various manifestations of atherosclerotic disease were included, this means that the SMART risk score can be used reliably in any of those more specific patient subgroups. In addition, the SMART risk score can be used for patients with a history of multiple clinical manifestations of vascular disease as is often the case in clinical practice.

    The SMART risk score provides an opportunity for risk stratification of patients in the stable phase of symptomatic vascular disease. While guidelines still consider all vascular patients to be at equally high risk for recurrent events (ie, >20% in 10 years),1–3 identifying those at extremely high risk is becoming more and more important. Although most vascular patients benefit from lipid and blood pressure lowering therapy, tight compared to average control may come at the price of more adverse events and the need for more frequent monitoring and comprehensive programmes for lifestyle improvement and counselling, which are costly. Knowing which patients are most likely to benefit, could help to allocate such resources most efficiently. In addition, new preventive treatment modalities (eg, biologicals, immunomodulants and antithrombotics), may have more serious side effects and are costly. This will further increase the need for risk stratification in the future. Our analysis of the net benefit of decision-making on the basis of risk predictions (see online supplement 1) shows that the SMART risk score can be used for identifying those high risk patients.

    In addition, patients may wish to be informed about their individual prognosis. Atherosclerosis is a serious and chronic condition with considerable risk for recurrence. Individual risk prediction may increase patients’ understanding of their disease, risk factors for recurrence, and the need for preventive treatment. Finally, risk stratification may be desired for enrolment into trials, for example because exposure of low-risk patients to potentially dangerous experimental drugs may be unwanted. In addition, selective enrolment of high risk patients into trials may allow for smaller sample sizes without reduction of statistical power.

    What type of characteristics best discriminate between low and high risk individuals with symptomatic vascular disease is much debated. Known risk factors in the healthy population still determine risk for recurrent events in those who have already developed symptomatic disease,5 ,6 ,15 ,29 but may be supplemented or replaced by determinants that are specific for the secondary prevention setting. The REACH Registry, for example, showed that the cardiovascular event rate increases with the number of symptomatic arterial disease locations (ie, CAD, CVD and PAD), ranging from 12.6% in 1 year in patients with one, to 21.1% in patients with two, and to 26.3% in patients with more than two locations of symptomatic arterial disease.9 ,30 Others have found that inflammatory biomarkers and renal function may add incremental predictive value.15 ,29 Including more sophisticated information about the severity of disease obtained by various imaging techniques is also often proposed. Previous studies have, for example, shown that left ventricular ejection fraction, vessel wall anatomy, coronary artery calcium score, cIMT and the presence of carotid artery stenosis are important predictors of outcome.15–18 Yet, the additional costs, time and expertise needed to perform additional imaging in clinical patients severely compromise the clinical utility of any prediction score based on such characteristics. The present study shows that the incremental value of adding carotid ultrasound findings to a risk score including clinical characteristics is limited.

    We acknowledge several limitations of the study. First, although our prediction models performed well in our own validation dataset, they would benefit from further external validation in other datasets. Nonetheless, we minimised the risk for over-fitting by including well known risk factors only, limiting the number of covariates, and through penalised maximum likelihood estimation of the model coefficients. Because an optimal penalty factor did not exist for models B and C, the same penalty factor was used for all three models. Because more complex models require more shrinkage in general, this might have slightly disadvantaged the performance of model B, but certainly not model C. Second, we were able to obtain stable survival estimates for up to only 7 years and needed to extrapolate our predictions in order to compute clinically more relevant 10-year risk predictions. Extrapolation is based on the assumption that the hazard rate is constant and thus survival is exponential over time, which is usually true for cardiovascular disease risk. Extrapolation is, therefore, preferable to relying on unstable survival estimates at 10 years’ follow-up, which was only completed by 12% of our study population. Third, to ensure that the prediction model can be conveniently applied in clinical practice and to minimise the risk for over-fitting, we limited the number of predictors and selected readily available patient characteristics only. For example, biomarkers that are not routinely measured, such as N-terminal pro-brain natriuretic peptide and cardiac troponin, were not included in the SMART risk score. Also, we decided to omit medication use. Although the use of blood pressure and lipid lowering medication often prove to be prognostic indicators of vascular event risk, prescription rates have dramatically increased over the past decades, thus limiting the applicability of (historic) study findings to future patients. Instead we included four terms for vascular disease history. Although the necessary categorisation of different types of disease (eg, stroke and transient ischaemic attack were both classified as CVD) leads to loss of information, the terms for disease history were important discriminators of risk in this population. Finally, the concordance statistic of the SMART risk score (ie, 0.68) might be considered mediocre when compared to those of vascular risk scores in the general population. Importantly, the age distribution of vascular patients is more homogeneous, impeding discrimination on the basis of age. Still, figure 2 and online supplement 1 demonstrate that the model can be used to discriminate between low and high risk patients and to improve clinical decision making.

    In conclusion, among patients with clinically manifest atherosclerosis, the 10-year risk for myocardial infarction, stroke or vascular death ranges from low (<10%) to extremely high (>40%). The SMART risk score, containing 14 easy-to-measure clinical predictors, provides an opportunity to identify vascular patients at high risk for recurrent events. Carotid ultrasound findings add little to a risk score based on clinical characteristics and are inferior discriminators of risk when used in isolation. This may facilitate evidence-based treatment decisions and individualised patient care.

    Acknowledgments

    We gratefully acknowledge the contribution of the SMART research nurses, Rutger van Petersen (data manager) and Harry Pijl (vascular manager).

    References

    1. ↵
      1. Graham I,
      2. Atar D,
      3. Borch-Johnsen K,
      4. et al
      . European guidelines on cardiovascular disease prevention in clinical practice: executive summary. Eur Heart J 2007;28:2375–414.
      OpenUrlFREE Full Text
    2. ↵
      1. Grundy SM,
      2. Cleeman JI,
      3. Merz CN,
      4. et al
      . Implications of recent clinical trials for the National Cholesterol Education Program Adult Treatment Panel III guidelines. Circulation 2004;110:227–39.
      OpenUrlAbstract/FREE Full Text
    3. ↵
      1. Smith SC Jr.,
      2. Allen J,
      3. Blair SN,
      4. et al
      . AHA/ACC guidelines for secondary prevention for patients with coronary and other atherosclerotic vascular disease: 2006 update: endorsed by the National Heart, Lung, and Blood Institute. Circulation 2006;113:2363–72.
      OpenUrlFREE Full Text
    4. ↵
      1. Brown JR,
      2. Cochran RP,
      3. MacKenzie TA,
      4. et al
      . Long-term survival after cardiac surgery is predicted by estimated glomerular filtration rate. Ann Thorac Surg 2008;86:4–11.
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. D'Agostino RB,
      2. Russell MW,
      3. Huse DM,
      4. et al
      . Primary and subsequent coronary risk appraisal: new results from the Framingham study. Am Heart J 2000;139:272–81.
      OpenUrlPubMedWeb of Science
    6. ↵
      1. MacKenzie TA,
      2. Malenka DJ,
      3. Olmstead EM,
      4. et al
      . Prediction of survival after coronary revascularization: modeling short-term, mid-term, and long-term survival. Ann Thorac Surg 2009;87:463–72.
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. van Wijk I,
      2. Kappelle LJ,
      3. van Gijn J,
      4. et al
      . Long-term survival and vascular event risk after transient ischaemic attack or minor ischaemic stroke: a cohort study. Lancet 2005;365:2098–104.
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Weimar C,
      2. Diener HC,
      3. Alberts MJ,
      4. et al
      . The Essen stroke risk score predicts recurrent cardiovascular events: a validation within the REduction of Atherothrombosis for Continued Health (REACH) registry. Stroke 2009;40:350–4.
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Wilson PW,
      2. D'Agostino R Sr.,
      3. Bhatt DL,
      4. et al
      . An international model to predict recurrent cardiovascular disease. Am J Med 2012;125:695–703 e1.
      OpenUrlCrossRefPubMed
    10. ↵
      1. Ridker PM,
      2. Buring JE,
      3. Rifai N,
      4. et al
      . Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA 2007;297:611–19.
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Ridker PM,
      2. Paynter NP,
      3. Rifai N,
      4. et al
      . C-reactive protein and parental history improve global cardiovascular risk prediction: the Reynolds Risk Score for men. Circulation 2008;118:2243–51, 4p.
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. Wilson PW,
      2. D'Agostino RB,
      3. Levy D,
      4. et al
      . Prediction of coronary heart disease using risk factor categories. Circulation 1998;97:1837–47.
      OpenUrlAbstract/FREE Full Text
    13. ↵
      1. Conroy RM,
      2. Pyorala K,
      3. Fitzgerald AP,
      4. et al
      . Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J 2003;24:987–1003.
      OpenUrlAbstract/FREE Full Text
    14. ↵
      1. Empana JP,
      2. Ducimetiere P,
      3. Arveiler D,
      4. et al
      . Are the Framingham and PROCAM coronary heart disease risk functions applicable to different European populations? The PRIME Study. Eur Heart J 2003;24:1903–11.
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Wattanakit K,
      2. Folsom AR,
      3. Chambless LE,
      4. et al
      . Risk factors for cardiovascular event recurrence in the Atherosclerosis Risk in Communities (ARIC) study. Am Heart J 2005;149:606–12.
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Kitagawa K,
      2. Hougaku H,
      3. Yamagami H,
      4. et al
      . Carotid intima-media thickness and risk of cardiovascular events in high-risk patients. Results of the Osaka Follow-Up Study for Carotid Atherosclerosis 2 (OSACA2 Study). Cerebrovasc Dis 2007;24:35–42.
      OpenUrlCrossRefPubMed
    17. ↵
      1. Polonsky TS,
      2. McClelland RL,
      3. Jorgensen NW,
      4. et al
      . Coronary artery calcium score and risk classification for coronary heart disease prediction. JAMA 2010;303:1610–6.
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Garg S,
      2. Sarno G,
      3. Garcia-Garcia HM,
      4. et al
      . A new tool for the risk stratification of patients with complex coronary artery disease: the Clinical SYNTAX Score 1. Circ Cardiovasc Interv 2010;3:317–26.
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Simons PC,
      2. Algra A,
      3. van de Laak MF,
      4. et al
      . Second manifestations of ARTerial disease (SMART) study: rationale and design. Eur J Epidemiol 1999;15:773–81.
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. Elgersma OE,
      2. van Leersum M,
      3. Buijs PC,
      4. et al
      . Changes over time in optimal duplex threshold for the identification of patients eligible for carotid endarterectomy. Stroke 1998;29:2352–6.
      OpenUrlAbstract/FREE Full Text
    21. ↵
      National Kidney Foundation. K/DOQI clinical practice guidelines for chronic kidney disease: evaluation, classification, and stratification. Am J Kidney Dis 2002;39:S1–266.
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Steyerberg EW
      . Clinical prediction models: a practical approach to development, validation, and updating. New York: Springer, 2009.
    23. ↵
      1. Goeman JJ
      . L1 penalized estimation in the Cox proportional hazards model. Biom J 2010;52:70–84.
      OpenUrlPubMedWeb of Science
    24. ↵
      1. Tibshirani R
      . The lasso method for variable selection in the Cox model. Stat Med 1997;16:385–95.
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. Harrell FE Jr.,
      2. Lee KL,
      3. Mark DB
      . Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996;15:361–87.
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      1. Steyerberg EW,
      2. Pencina MJ
      . Reclassification calculations for persons with incomplete follow-up. Ann Intern Med 2010;152:195–6.
      OpenUrlCrossRefPubMedWeb of Science
    27. ↵
      1. Gronnesby JK,
      2. Borgan O
      . A method for checking regression models in survival analysis based on the risk score. Lifetime Data Anal 1996;2:315–28.
      OpenUrlCrossRefPubMed
    28. ↵
      1. May S,
      2. Hosmer DW
      . A simplified method of calculating an overall goodness-of-fit test for the Cox proportional hazards model. Lifetime Data Anal 1998;4:109–20.
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      1. Blankenberg S,
      2. McQueen MJ,
      3. Smieja M,
      4. et al
      . Comparative impact of multiple biomarkers and N-terminal pro-brain natriuretic peptide in the context of conventional risk factors for the prediction of recurrent cardiovascular events in the Heart Outcomes Prevention Evaluation (HOPE) Study. Circulation 2006;114:201–8.
      OpenUrlAbstract/FREE Full Text
    30. ↵
      1. Steg PG,
      2. Bhatt DL,
      3. Wilson PW,
      4. et al
      . One-year cardiovascular event rates in outpatients with atherothrombosis. JAMA 2007;297:1197–206.
      OpenUrlCrossRefPubMedWeb of Science

    Supplementary materials

    Footnotes

    • Collaborators Members of the Secondary Manifestations of Arterial Disease Study Group are as follows: A Algra, MD, YvdG, MD, DE Grobbee, MD and GEHM Rutten, MD, Julius Centre for Health Sciences and Primary Care; FLJV, MD, Department of Vascular Medicine; FL Moll, MD, Department of Vascular Surgery; LJ Kappelle, MD, Department of Neurology; WPTM Mali, MD, Department of Radiology; PA Doevendans, MD, Department of Cardiology; University Medical Centre, Utrecht, The Netherlands.

    • Contributors JAND designed and executed the data analyses, interpreted the results and completed the final manuscript; FLJV conceived the research question, contributed to data collection, designed the data analyses, interpreted the results and revised the manuscript for important intellectual content; AMJW contributed to data collection, designed and executed part of the data analyses, interpreted the results and drafted the manuscript; MJAG designed and executed part of the data analyses, interpreted the results and revised the manuscript for important intellectual content; EWS designed the data analyses, interpreted the results and revised the manuscript for important intellectual content; PMR conceived the research question, interpreted the results and revised the manuscript for important intellectual content; NRC designed the data analyses, interpreted the results and revised the manuscript for important intellectual content; YvdG conceived the research question, contributed to data collection, interpreted the results, supervised the project's progress and revised the manuscript for important intellectual content. Members of the SMART study group have designed the study and have supervised patient enrolment and data collection. All authors have read and approved the final version of this manuscript.

    • Funding The SMART study was supported by a grant of the Board of Directors of the University Medical Centre Utrecht, who had no role in the conduct of the analyses or drafting of the report. All statistical analyses were done by the investigators.

    • Competing interests FLJV's department receives grant support from Merck, the Netherlands Organisation for Health Research and Development, and the Catharijne foundation Utrecht; and speaker fees from Merck and AstraZeneca. PMR has served as a consultant to Merck, ISIS, Vascular Biogenics, Genzyme, Abbott and Boerhinger; has received research grant support from AstraZeneca, Merck, Novartis and Amgen; and is listed as a co-inventor on patents held by the Brigham and Women's Hospital related to the use of inflammatory biomarkers in cardiovascular disease and diabetes that have been licensed to Siemens and AstraZeneca.

    • Ethics approval The ethics committee at the University Medical Centre Utrecht.

    • Provenance and peer review Not commissioned; externally peer reviewed.