Pragmatic Free Trial Meta
Pragmatic Free Trail Meta is an open data platform that allows research into pragmatic trials. It is a platform that collects and shares clean trial data and ratings using PRECIS-2 which allows for multiple and varied meta-epidemiological studies to compare treatment effects estimates across trials with different levels of pragmatism, as well as other design features.
Background
Pragmatic trials provide evidence from the real world that can be used to make clinical decisions. The term "pragmatic" however, is not used in a consistent manner and its definition and evaluation need further clarification. Pragmatic trials must be designed to inform clinical practice and policy decisions, rather than to prove an hypothesis that is based on a clinical or physiological basis. A pragmatic study should strive to be as close as possible to actual clinical practices which include the recruiting participants, setting, design, delivery and implementation of interventions, determination and analysis outcomes, and primary analysis. This is a major distinction from explanation trials (as described by Schwartz and Lellouch1), which are designed to provide more thorough proof of the hypothesis.
The trials that are truly pragmatic should avoid attempting to blind participants or the clinicians, as this may result in distortions in estimates of the effects of treatment. The pragmatic trials also include patients from different health care settings to ensure that the results can be applied to the real world.
Additionally, pragmatic trials should focus on outcomes that are important to patients, such as quality of life or functional recovery. This is particularly important in trials that involve invasive procedures or those with potential for dangerous adverse events. The CRASH trial29, for instance, focused on functional outcomes to compare a 2-page case-report with an electronic system to monitor the health of patients in hospitals suffering from chronic heart failure. Similarly, the catheter trial28 used urinary tract infections that are symptomatic of catheters as the primary outcome.
In addition to these features, pragmatic trials should minimize the trial's procedures and data collection requirements in order to reduce costs. Finaly these trials should strive to make their findings as relevant to actual clinical practices as possible. This can be achieved by ensuring that their analysis is based on the intention-to treat method (as described in CONSORT extensions).
Despite these requirements, many RCTs with features that defy the concept of pragmatism have been mislabeled as pragmatic and published in journals of all kinds. This could lead to misleading claims of pragmatism, and the usage of the term needs to be standardized. The creation of a PRECIS-2 tool that can provide an objective, standardized evaluation of pragmatic aspects is the first step.
Methods
In a practical trial the goal is to inform clinical or policy decisions by demonstrating how the intervention can be implemented into routine care. Explanatory trials test hypotheses regarding the cause-effect relationship within idealised environments. Consequently, pragmatic trials may be less reliable than explanatory trials, and could be more susceptible to bias in their design, conduct and analysis. Despite these limitations, pragmatic trials may provide valuable information to decision-making in the context of healthcare.
The PRECIS-2 tool evaluates an RCT on 9 domains, with scores ranging between 1 and 5 (very pragmatic). In this study, the domains of recruitment, organisation as well as flexibility in delivery flexible adherence, and follow-up received high scores. However, the primary outcome and method of missing data were scored below the practical limit. This indicates that a trial can be designed with good practical features, yet not compromising its quality.
However, it's difficult to determine how practical a particular trial is since pragmaticity is not a definite characteristic; certain aspects of a trial may be more pragmatic than others. A trial's pragmatism can be affected by modifications to the protocol or the logistics during the trial. In addition 36% of the 89 pragmatic trials discovered by Koppenaal et al were placebo-controlled, or conducted prior to licensing and most were single-center. They are not in line with the usual practice and can only be referred to as pragmatic if their sponsors agree that such trials are not blinded.
Additionally, a typical feature of pragmatic trials is that researchers try to make their results more valuable by studying subgroups of the sample. This can lead to unbalanced analyses with lower statistical power. This increases the possibility of omitting or misinterpreting differences in the primary outcomes. In the instance of the pragmatic trials included in this meta-analysis this was a major issue since the secondary outcomes weren't adjusted for the differences in the baseline covariates.
Furthermore, pragmatic studies may pose challenges to collection and interpretation of safety data. This is because adverse events are typically reported by participants themselves and are susceptible to reporting errors, delays or coding deviations. It is therefore crucial to improve the quality of outcomes for these trials, and ideally by using national registries rather than relying on participants to report adverse events on the trial's database.
Results
While the definition of pragmatism may not require that all trials be 100 percent pragmatic, there are advantages to incorporating pragmatic components into clinical trials. These include:
Incorporating routine patients, the trial results can be translated more quickly into clinical practice. However, pragmatic trials may also have disadvantages. For instance, the right type of heterogeneity can help a trial to generalise its findings to a variety of patients and settings; however the wrong type of heterogeneity could reduce assay sensitivity and therefore reduce the power of a study to detect small treatment effects.
More suggestions have attempted to categorize pragmatic trials, with various definitions and scoring systems. Schwartz and Lellouch1 created a framework to distinguish between explanatory studies that prove a physiological or clinical hypothesis, and pragmatic studies that inform the choice for appropriate therapies in clinical practice. Their framework comprised nine domains, each scored on a scale of 1-5, with 1 indicating more explanatory and 5 indicating more practical. The domains covered recruitment and setting up, the delivery of intervention, flexible adherence and primary analysis.
The initial PRECIS tool3 included similar domains and scales from 1 to 5. Koppenaal et. al10 devised an adaptation of the assessment, called the Pragmascope that was simpler to use for systematic reviews. They found that pragmatic reviews scored higher on average across all domains, however they scored lower in the primary analysis domain.
This difference in primary analysis domains could be explained by the way that most pragmatic trials analyze data. Certain explanatory trials however, do not. The overall score for systematic reviews that were pragmatic was lower when the domains of management, flexible delivery and following-up were combined.
It is important to note that a pragmatic trial doesn't necessarily mean a low quality trial, and there is an increasing number of clinical trials (as defined by MEDLINE search, but this is neither specific nor sensitive) which use the word 'pragmatic' in their abstract or title. These terms could indicate a greater understanding of pragmatism in abstracts and titles, but it isn't clear whether this is reflected in the content.
Conclusions
As appreciation for the value of real-world evidence grows widespread the pragmatic trial has gained traction in research. They are clinical trials randomized that compare real-world care alternatives instead of experimental treatments in development. They have patients that more closely mirror the ones who are treated in routine care, they employ comparators that are used in routine practice (e.g. existing drugs) and rely on participant self-report of outcomes. This method could help overcome the limitations of observational studies that are prone to biases associated with reliance on volunteers, and the limited accessibility and coding flexibility in national registries.
Other benefits of pragmatic trials include the ability to use existing data sources, as well as a higher probability of detecting significant changes than traditional trials. However, they may be prone to limitations that compromise their reliability and generalizability. For instance the rates of participation in some trials might be lower than expected due to the healthy-volunteer effect as well as financial incentives or competition for participants from other research studies (e.g. industry trials). The need to recruit individuals quickly limits the sample size and the impact of many practical trials. Some pragmatic trials also lack controls to ensure that observed differences aren't caused by biases during the trial.
The authors of the Pragmatic Free Trial Meta identified RCTs that were published between 2022 and 2022 that self-described themselves as pragmatic. 프라그마틱 무료체험 메타 -2 tool was used to determine pragmatism. It covers domains such as eligibility criteria and flexibility in recruitment and adherence to intervention and follow-up. They discovered that 14 of these trials scored as highly or pragmatic practical (i.e. scores of 5 or more) in any one or more of these domains, and that the majority were single-center.
Trials that have a high pragmatism score tend to have broader eligibility criteria than traditional RCTs which have very specific criteria that aren't likely to be used in the clinical setting, and include populations from a wide variety of hospitals. The authors claim that these traits can make pragmatic trials more effective and relevant to everyday practice, but they do not necessarily guarantee that a pragmatic trial is free from bias. The pragmatism is not a fixed attribute and a test that does not have all the characteristics of an explanatory study can still produce reliable and beneficial results.