Levels of Evidence for Cancer Screening and Prevention Studies (PDQ®): Screening - Health Professional Information [NCI]

Introduction

The PDQ Screening and Prevention Editorial Board reviews human studies and summarizes its understanding of the level of the evidence for the magnitude of the effect (including absolute and relative risks) for both the benefits and the harms of implementing cancer screening and prevention interventions. The Editorial Board uses the term Level of Evidence to refer to the certainty of its estimate of the health effects of implementing the intervention in question.

Typically, varying levels of evidence support each PDQ screening or prevention summary. Evidence that indicates, within a narrow range, the direction and magnitude of the effect on health outcomes is termed solid evidence. Evidence that allows an estimate of the direction and magnitude of the health outcomes within a broader range, but still allows a reasonably certain estimate, is termed fair. Evidence that allows either no estimate at all or only an estimate that is very broad is termed inadequate.

The highest Level of Evidence (solid) is generally obtained from several well-designed and well-conducted randomized controlled trials (RCTs) in generalizable settings with generalizable populations that report evidence about both benefits and harms. It is, however, not always practical to conduct such trials to address every question within the field of screening and prevention, and solid evidence from alternative study designs is often not available. There are also instances where alternative study designs can generate this solid Level of Evidence. Additionally, the PDQ Editorial Board does consider fair evidence in estimating the benefits and harms of an intervention in the general population. When the evidence for the benefits and/or harms of an intervention is inadequate, the Board states this finding.

The PDQ Editorial Board evaluates evidence in 2 steps. The first step is to describe the evidence within 5 domains (see below); the second is to judge the overall Level of Evidence as solid, fair, or inadequate. The Board conducts the same process separately for potential benefits and potential harms of each intervention. The PDQ Editorial Board does not explicitly assess the net benefit (i.e., benefits minus harms) of interventions, nor does it make recommendations.

The question at issue for the Board is to determine the direction and magnitude of important health effects from introducing the intervention into the general population. This is a question of effectiveness rather than of efficacy only, and the Board seeks evidence about effectiveness as well as efficacy. When only efficacy evidence is available, the Board must judge the additional uncertainty involved with estimating the actual health impact of widespread implementation.

Evaluation of Evidence

The two steps in evaluating evidence are described below.

1. Description of the Evidence (The PDQ Editorial Board uses the same process for benefits and harms; the evidence referred to is the evidence relevant for answering the question of the magnitude of the health effects of widespread implementation.)

   Domains

   1. Study Design (evidence from the best studies available; ranked in descending order of strength)
      1. Evidence obtained from randomized controlled trials (see below).
      2. Evidence obtained from nonrandomized controlled trials.
      3. Evidence obtained from cohort or case-control studies.
      4. Evidence from ecologic and descriptive studies (e.g., international patterns studies, time series).
      5. Opinions of respected authorities based on clinical experience, descriptive studies, or reports of expert committees.
   2. Internal Validity: Quality of Execution Within the Study Design

      The Editorial Board uses design-specific criteria within each research design to assess the internal validity of the evidence. At present the Board uses the criteria developed by the U.S. Preventive Services Task Force (see Table 3 in [1]). These criteria may be modified over time as needed.

   3. Consistency (coherence)/Volume of the Evidence
      • One study (small vs. large number of participants; agree vs. disagree).
      • Multiple studies (small vs. large number of participants; agree vs. disagree).
   4. Magnitude of Effects on Health Outcomes (both absolute and relative risks; as quantitative as possible; may vary for different populations)
      • Small positive/negative magnitude (benefits/harms).
      • Larger positive/negative magnitude (benefits/harms).
   5. External Validity
      • Extent to which the intervention can be applied to usual practice with the same effects as in efficacy studies.
      • Effects among people in the general population, differences with study subjects.
2. Assessment of the Evidence
   1. The level of certainty (solid, fair, inadequate) of our understanding of the direction and magnitude of the health effects (both benefits and harms) of widespread implementation.
   2. Example: Statement of Benefits

      Option 1: "Based on [solid/fair] evidence, use of intervention X [among population Y, where appropriate] leads to a reduction/increase in (a specific benefit)." [In the Evidence of Benefit section, the actual evidence is detailed, including evidence and assessment of direction and magnitude of specific benefits.]

      Option 2: "The evidence is inadequate to make a clear determination of benefit." (To be used when evidence is inadequate in amount or quality.) Alternative format depending on situation: "The evidence is inadequate to determine whether (preventive service) reduces (health problem) to a clinically or public health important degree." (Further explanation or clarification as needed.) Need to state for what outcome the evidence is inadequate, i.e., to assess a mortality effect.

   3. Example: Statement of Harms

      Option 1: "Based on [solid/fair] evidence, use of intervention X [among population Y] leads to a reduction/increase in (a specific harm)." [In the Evidence of Harm section, the actual evidence is detailed, including evidence and assessment of the direction and magnitude of specific harms.]

      Option 2: "The evidence is inadequate to make a clear determination of benefit." (To be used when evidence is inadequate in amount or quality.) Alternative format depending on situation: "The evidence is inadequate to determine whether (preventive service) reduces (health problem) to a clinically or public health important degree." (Further explanation or clarification as needed.) Need to state for what outcome the evidence is inadequate, i.e., to assess a mortality effect.

References:

1. Harris RP, Helfand M, Woolf SH, et al.: Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med 20 (3 Suppl): 21-35, 2001.

Notes on Quality Assessment

1. Semantics: One can think of the level of a body of evidence as the certainty of the assessment allowed from the evidence. This may also be considered as the quality of the evidence to address a specific question. Because the word quality can have other meanings (e.g., the extent to which individual studies meet given criteria), the Editorial Board prefers the word level.
2. Certainty: Certainty can be considered in several ways, one of which is to use the heuristic of the confidence interval. This confidence interval does not come from a statistical calculation but is instead a conceptual confidence interval (CCI) representing the Editorial Board's assessment of the range of values for direction and magnitude of the effect of the intervention that is consistent with the overall evidence. This CCI is seeking to answer an effectiveness question (not simply an efficacy question). The question at issue is:

   What would the health effects be if the intervention were widely implemented in routine practice? Even well-conducted RCTs may not provide good evidence if they are not conducted in generalizable populations, using generalizable tests and treatments. The CCI for evidence based on RCTs conducted in highly selected populations might be wide rather than narrow (fair as opposed to good) unless there are reasons to believe the effects would be the same in the general population.

3. Benefits: Potential benefits include mortality reduction (and should note whether disease-specific or overall), improved quality of life, improved function, and reduced need for invasive procedures or interventions. If mortality reduction from an RCT is disease-specific with no trend toward overall mortality reduction, then the Editorial Board might consider the contribution of this disease-specific reduction to overall mortality to be less certain. As the question at issue is an all-cause mortality question, the CCI for a study of only disease-specific reduction without other indications of overall reduction may be considered wider than if there were indications of overall mortality reduction.
4. Harms: Harms may be considered in 3 categories:
   1. Psychological (often from labeling or anxiety after a false-positive test or a diagnosis of an intermediate condition of no clinical importance).
   2. Complications from diagnostic or monitoring tests (e.g., colonoscopy with perforation for a positive FOBT).
   3. Complications or side effects from treatment, especially treatment from which there are no benefits (e.g., in the case of overdiagnosis).
5. Extrapolation: Estimates of the presence and magnitude of either benefits or harms may come from extrapolation from indirect evidence; the degree of extrapolation determines the CCI for each estimate. For example, evidence shows that screening for ovarian cancer results in many false-positive tests (due to the low prevalence of the disease) for which the workup is an invasive procedure. Other evidence could provide an estimate of the effects of the invasive workup, including not only complications but also the discomfort, anxiety, and time of reduced usual activity. The evidence, then, for the harms of screening is indirect (i.e., not from an RCT of screening) but would still provide at least fair evidence of harms (i.e., intermediate width CCI).
6. Judgment: Judgment is involved at several steps in this process, including the assessment of quality (internal validity), consistency/coherence, external validity, and the overall Level of Evidence. It is important in each case that the rationale and conclusion be as explicit and transparent as possible. The reasoning behind the judgment of the overall Level of Evidence should be stated clearly.

Changes to This Summary (01 / 12 / 2022)

The PDQ cancer information summaries are reviewed regularly and updated as new information becomes available. This section describes the latest changes made to this summary as of the date above.

Introduction

Revised text to state that the PDQ Screening and Prevention Editorial Board reviews human studies and summarizes its understanding of the level of the evidence for the magnitude of the effect for both the benefits and the harms of implementing cancer screening and prevention interventions.

Revised text to state that it is not always practical to conduct well-designed and well-conducted randomized controlled trials to address every question within the field of screening and prevention, and solid evidence from alternative study designs is often not available. Also added text to state that there are also instances where alternative study designs can generate this solid Level of Evidence.

This summary is written and maintained by the PDQ Screening and Prevention Editorial Board, which is editorially independent of NCI. The summary reflects an independent review of the literature and does not represent a policy statement of NCI or NIH. More information about summary policies and the role of the PDQ Editorial Boards in maintaining the PDQ summaries can be found on the About This PDQ Summary and PDQ® - NCI's Comprehensive Cancer Database pages.

About This PDQ Summary

Purpose of This Summary

This PDQ cancer information summary for health professionals provides comprehensive, peer-reviewed, evidence-based information about the formal ranking system used by the PDQ Editorial Boards to assess evidence supporting the use of specific interventions or approaches. It is intended as a resource to inform and assist clinicians in the care of their patients. It does not provide formal guidelines or recommendations for making health care decisions.

Reviewers and Updates

This summary is reviewed regularly and updated as necessary by the PDQ Screening and Prevention Editorial Board, which is editorially independent of the National Cancer Institute (NCI). The summary reflects an independent review of the literature and does not represent a policy statement of NCI or the National Institutes of Health (NIH).

Board members review recently published articles each month to determine whether an article should:

• be discussed at a meeting,
• be cited with text, or
• replace or update an existing article that is already cited.

Changes to the summaries are made through a consensus process in which Board members evaluate the strength of the evidence in the published articles and determine how the article should be included in the summary.

Any comments or questions about the summary content should be submitted to Cancer.gov through the NCI website's Email Us. Do not contact the individual Board Members with questions or comments about the summaries. Board members will not respond to individual inquiries.

Levels of Evidence

Some of the reference citations in this summary are accompanied by a level-of-evidence designation. These designations are intended to help readers assess the strength of the evidence supporting the use of specific interventions or approaches. The PDQ Screening and Prevention Editorial Board uses a formal evidence ranking system in developing its level-of-evidence designations.

Permission to Use This Summary

PDQ is a registered trademark. Although the content of PDQ documents can be used freely as text, it cannot be identified as an NCI PDQ cancer information summary unless it is presented in its entirety and is regularly updated. However, an author would be permitted to write a sentence such as "NCI's PDQ cancer information summary about breast cancer prevention states the risks succinctly: [include excerpt from the summary]."

The preferred citation for this PDQ summary is:

PDQ® Screening and Prevention Editorial Board. PDQ Levels of Evidence for Cancer Screening and Prevention Studies. Bethesda, MD: National Cancer Institute. Updated <MM/DD/YYYY>. Available at: https://www.cancer.gov/publications/pdq/levels-evidence/screening-prevention. Accessed <MM/DD/YYYY>. [PMID: 26389496]

Images in this summary are used with permission of the author(s), artist, and/or publisher for use within the PDQ summaries only. Permission to use images outside the context of PDQ information must be obtained from the owner(s) and cannot be granted by the National Cancer Institute. Information about using the illustrations in this summary, along with many other cancer-related images, is available in Visuals Online, a collection of over 2,000 scientific images.

Disclaimer

The information in these summaries should not be used as a basis for insurance reimbursement determinations. More information on insurance coverage is available on Cancer.gov on the Managing Cancer Care page.

Contact Us

More information about contacting us or receiving help with the Cancer.gov website can be found on our Contact Us for Help page. Questions can also be submitted to Cancer.gov through the website's Email Us.

Last Revised: 2022-01-12