Show Summary Details
Page of

Large-scale randomized evidence: trials and meta-analyses of trials 

Large-scale randomized evidence: trials and meta-analyses of trials

Chapter:
Large-scale randomized evidence: trials and meta-analyses of trials
Author(s):

C. Baigent

, R. Peto

, R. Gray

, S. Parish

, and R. Collins

DOI:
10.1093/med/9780199204854.003.020303_update_002

Update:

Chapter reviewed June 2011—no substantial updates required.

Updated on 30 Nov 2011. The previous version of this content can be found here.
Page of

PRINTED FROM OXFORD MEDICINE ONLINE (www.oxfordmedicine.com). © Oxford University Press, 2015. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford Medicine Online for personal use (for details see Privacy Policy).

date: 30 March 2017

Reliable detection or refutation of realistically moderate effects on major outcomes often requires large-scale randomized evidence

As long as doctors start with a healthy scepticism about the many apparently striking claims and counter-claims that appear in the medical literature, trial results do make sense. The main enemy of common sense is over-optimism: there are a few striking exceptions where treatments for serious disease work extremely well, but many claims of vast improvements from new therapies turn out to be evanescent.

Clinical trials generally need to be able to detect or to refute realistically moderate (but still worthwhile) differences between treatments in long-term disease outcome. Large-scale randomized evidence should be able to detect such effects, but medium-sized trials or medium-sized meta-analyses can, and often do, yield false negative or exaggeratedly positive results. If the results of such studies seem too good to be true then they probably are; conversely, unpromising evidence can be misleading if it is from a study of inadequate size, or from one particular subgroup of a large study with a clearly favourable overall result. Realistically moderate expectations of what a treatment might achieve (or, if one treatment is to be compared with another, of how large any difference between the main effects of these two treatments is likely to be) should foster studies that can discriminate reliably between (1) a difference in outcome that is realistically moderate but still worthwhile, and (2) a difference in outcome that is too small to be of any material importance.

To assess moderate effects reliably, avoid both moderate biases and moderate random errors

To demonstrate or refute realistically moderate differences in outcome, studies must guarantee both (1) strict control of bias - which, in general, requires proper randomization and appropriate statistical analysis, with no unduly ‘data-dependent’ emphasis on specific parts of the overall evidence; and (2) strict control of the play of chance - which, in general, requires large numbers with the outcome of interest, rather than a lot of detail on each patient. The conclusion is obvious: moderate biases and moderate random errors must both be avoided if moderate benefits are to be assessed reliably. This leads to the need for large numbers of properly randomized patients with properly analysed data, which in turn should lead to some large but simple randomized trials (or ‘mega-trials’) and to large systematic overviews (or ‘meta-analyses’) of all related randomized trials.

Other forms of evidence may be untrustworthy

Non-randomized evidence, unduly small randomized trials, unduly small meta-analyses of trials and undue emphasis on particular subgroups (or on particular trials) are all much inferior as sources of evidence about current patient management or as foundations for future research strategies because they often cannot discriminate reliably between moderate (but worthwhile) differences and negligible differences in outcome, and the mistaken clinical conclusions that they engender could well result in the undertreatment, overtreatment, or other mismanagement of millions of future patients worldwide.

Benefits of large-scale randomized evidence

In contrast, many premature deaths each year could be avoided by seeking appropriately large-scale randomized evidence about various widely practicable treatments for the common causes of death, and by disseminating this evidence appropriately. The value of such large-scale randomized evidence is illustrated by the trials of fibrinolytic therapy for acute myocardial infarction; of anti-platelet therapy for a wide range of vascular conditions; of hormonal therapy for early breast cancer; and of drug therapy for lowering blood pressure. In these examples, proof of benefit that could not have been achieved by either small-scale randomized evidence or non-randomized evidence of benefit has led to widespread changes in practice that are now preventing hundreds of thousands of premature deaths each year, and appropriately large-scale randomized evidence could substantially improve the management of many important, but non-fatal, medical conditions.

Access to the complete content on Oxford Medicine Online requires a subscription or purchase. Public users are able to search the site and view the abstracts for each book and chapter without a subscription.

Please subscribe or login to access full text content.

If you have purchased a print title that contains an access token, please see the token for information about how to register your code.

For questions on access or troubleshooting, please check our FAQs, and if you can''t find the answer there, please contact us.