Pragmatic Free Trial Meta
Pragmatic Free Trail Meta is an open data platform that facilitates research into pragmatic trials. It collects and shares cleaned trial data and ratings using PRECIS-2 which allows for multiple and varied meta-epidemiological research studies to examine the effects of treatment across trials with different levels of pragmatism and other design features.
Background
Pragmatic trials are becoming more widely acknowledged as providing evidence from the real world for clinical decision making. However, the use of the term "pragmatic" is inconsistent and its definition and evaluation requires further clarification. Pragmatic trials are intended to guide clinical practices and policy decisions rather than verify a physiological hypothesis or clinical hypothesis. A pragmatic trial should try to be as close as it is to the real-world clinical practice which include the recruiting participants, setting up, delivery and execution of interventions, determination and analysis results, as well as primary analyses. This is a major distinction from explanation trials (as described by Schwartz and Lellouch1) which are designed to provide more thorough proof of a hypothesis.
Truly pragmatic trials should not blind participants or clinicians. This can lead to a bias in the estimates of treatment effects. Pragmatic trials will also recruit patients from various health care settings to ensure that their results can be applied to the real world.
Furthermore the focus of pragmatic trials should be on outcomes that are vital to patients, like quality of life or functional recovery. This is especially important for trials that involve surgical procedures that are invasive or may have dangerous adverse consequences. The CRASH trial29, for instance focused on the functional outcome to compare a 2-page case-report with an electronic system to monitor the health of patients admitted to hospitals with chronic heart failure. Similarly, the catheter trial28 used symptomatic catheter-associated urinary tract infections as its primary outcome.
In addition to these aspects pragmatic trials should also reduce the procedures for conducting trials and requirements for data collection to cut down on costs and time commitments. Additionally the aim of pragmatic trials is to make their results as applicable to current clinical practices as possible. official website can be achieved by ensuring their primary analysis is based on the intention-to treat approach (as described within CONSORT extensions).
Many RCTs that don't meet the criteria for pragmatism but contain features in opposition to pragmatism, have been published in journals of various types and incorrectly labeled pragmatic. This can lead to false claims of pragmatism and the usage of the term should be made more uniform. The creation of the PRECIS-2 tool, which offers a standard objective assessment of practical features, is a good first step.
Methods
In a pragmatic trial the goal is to inform policy or clinical decisions by showing how an intervention could be implemented into routine care. This is different from explanatory trials, which test hypotheses about the cause-effect relationship in idealised settings. In this way, pragmatic trials can have less internal validity than explanatory studies and are more susceptible to biases in their design as well as analysis and conduct. Despite their limitations, pragmatic studies can be a valuable source of information for decision-making within the context of healthcare.
The PRECIS-2 tool evaluates an RCT on 9 domains, ranging between 1 and 5 (very pragmatist). In this study, the recruit-ment organization, flexibility in delivery, flexible adherence and follow-up domains were awarded high scores, but the primary outcome and the procedure for missing data fell below the practical limit. This suggests that a trial can be designed with effective practical features, yet not harming the quality of the trial.
It is, however, difficult to judge how practical a particular trial is, since pragmatism is not a binary quality; certain aspects of a trial can be more pragmatic than others. Moreover, protocol or logistic modifications made during an experiment can alter its score on pragmatism. Additionally 36% of the 89 pragmatic trials discovered by Koppenaal et al were placebo-controlled, or conducted prior to approval and a majority of them were single-center. They are not close to the standard practice and can only be called pragmatic if their sponsors agree that these trials aren't blinded.
Another common aspect of pragmatic trials is that the researchers attempt to make their findings more relevant by analyzing subgroups of the sample. However, this often leads to unbalanced comparisons and lower statistical power, which increases the risk of either not detecting or incorrectly detecting differences in the primary outcome. In the instance of the pragmatic trials that were included in this meta-analysis this was a significant problem because the secondary outcomes were not adjusted to account for the differences in baseline covariates.
In addition practical trials can present challenges in the collection and interpretation of safety data. This is because adverse events are typically reported by participants themselves and are susceptible to reporting delays, inaccuracies or coding deviations. It is essential to increase the accuracy and quality of the results in these trials.
Results
Although the definition of pragmatism may not require that clinical trials be 100% pragmatic there are benefits of including pragmatic elements in trials. These include:
By incorporating routine patients, the results of the trial are more easily translated into clinical practice. However, pragmatic trials may also have disadvantages. The right kind of heterogeneity, like could allow a study to expand its findings to different patients or settings. However the wrong kind of heterogeneity can reduce the assay sensitivity, and therefore lessen the power of a trial to detect small treatment effects.
A variety of studies have attempted to classify pragmatic trials using different definitions and scoring methods. Schwartz and Lellouch1 developed a framework for distinguishing between research studies that prove the clinical or physiological hypothesis, and pragmatic trials that aid in the choice of appropriate therapies in real-world clinical practice. Their framework comprised nine domains, each scored on a scale of 1-5, with 1 indicating more lucid and 5 indicating more practical. The domains included recruitment, setting, intervention delivery and follow-up, as well as flexible adherence and primary analysis.
The initial PRECIS tool3 included similar domains and an assessment scale ranging from 1 to 5. Koppenaal et al10 devised an adaptation of this assessment, dubbed the Pragmascope that was simpler to use in systematic reviews. They discovered that pragmatic systematic reviews had higher average scores across all domains but lower scores in the primary analysis domain.
The difference in the primary analysis domain could be due to the fact that most pragmatic trials process their data in the intention to treat way however some explanation trials do not. The overall score for pragmatic systematic reviews was lower when the domains of management, flexible delivery and follow-up were merged.
It is important to note that a pragmatic trial doesn't necessarily mean a poor quality trial, and indeed there is an increasing number of clinical trials (as defined by MEDLINE search, but this is not specific or sensitive) that employ the term "pragmatic" in their abstracts or titles. The use of these terms in abstracts and titles could suggest a greater awareness of the importance of pragmatism however, it is not clear if this is manifested in the content of the articles.
Conclusions
In recent years, pragmatic trials have been gaining popularity in research as the value of real world evidence is becoming increasingly acknowledged. They are randomized studies that compare real-world treatment options with clinical trials in development. They involve patient populations more closely resembling those treated in regular care. This approach can overcome the limitations of observational research, like the biases that are associated with the reliance on volunteers as well as the insufficient availability and the coding differences in national registry.
Pragmatic trials offer other advantages, including the ability to draw on existing data sources and a higher likelihood of detecting meaningful differences from traditional trials. However, pragmatic trials may have some limitations that limit their credibility and generalizability. Participation rates in some trials may be lower than expected due to the healthy-volunteering effect, financial incentives, or competition from other research studies. The requirement to recruit participants in a timely fashion also limits the sample size and impact of many pragmatic trials. Additionally certain pragmatic trials lack controls to ensure that the observed differences aren't due to biases in the conduct of trials.
The authors of the Pragmatic Free Trial Meta identified RCTs that were published between 2022 and 2022 that self-described as pragmatism. The PRECIS-2 tool was employed to evaluate the degree of pragmatism. It covers domains such as eligibility criteria as well as recruitment flexibility and adherence to intervention and follow-up. They discovered that 14 of the trials scored highly or pragmatic sensible (i.e., scoring 5 or higher) in one or more of these domains and that the majority of them were single-center.
Trials with high pragmatism scores tend to have more lenient criteria for eligibility than traditional RCTs. They also contain populations from various hospitals. According to the authors, can make pragmatic trials more useful and relevant to the daily clinical. However, they cannot ensure that a study is free of bias. Moreover, the pragmatism of trials is not a definite characteristic; a pragmatic trial that does not contain all the characteristics of an explanatory trial may yield valuable and reliable results.