Data Integrity in Clinical Research: Audit Trail Review as a Key Tool
A recent position paper from the eClinical Forum and the Society for Clinical Data Management highlighted the value of audit trail review as a key tool to maintaining a high level of data integrity, particularly in high-risk scenarios when audit trails are used with regularity.
The paper is particularly timely as the availability of large data sets collected electronically increases, leading to concerns around the integrity of that data and standards around ensuring timely review of data sets.
Before diving into the paper’s recommendations, we must first define audit trails in the context of clinical research.
What is an Audit Trail?
Put simply, an audit trail refers to the "who," "what," "when," "where," and "why" of activity on an electronic record. Provided in chronological sequence, audit trails allow for documentation and a record of events when further analysis due to concerns around data integrity arise.
In a clinical setting, audit trails are a requirement in clinical trial management systems that maintain a wide variety of patient information and data around the trial itself. The evaluation of that data is vital, as that data is then used to answer critical questions about patient care, quality of life, and healthcare policy, among other applications.
An audit trail review is one tool available to ensure a high level of data integrity from the outset and mitigate risk factors associated with clinical research overall.
Learn the difference between Audit Trails and Audit logs
The Importance of Audit Trail Review
As the availability of big data increases and access to digital data is increasingly available from third parties, organizations must focus on ensuring that data is accurate and reliable. Regulators have increased their attention on reviewing audit trails as well.
Consider the number of health apps on the market collecting data on individuals’ health and wellness alone. That electronic data capture, the trends tracked and monitored, alongside existing pressures to keep up with the need for therapies to improve patient outcomes, has certainly led to even more reliance on these avenues for data collection.
When used on a routine basis and appropriately, by a well-trained staff familiar with an organization’s operations, audit trail reviews have helped identify inaccuracies and inefficiencies in the large swath of data collection and collection methods out there. Reviews also allow for a closer look at whether those datasets meet regulatory requirements, as they are coming from sources that may not be as well-versed in that area.
Audit trail review is then one strategy in a clinical research team’s toolkit to identify errors in data collection and reporting and ensure a high level of data integrity, all critical when discussing patient outcomes and healthcare research.
The Key Components of Audit Trail Review
The authors identified five key components of audit trail review. Each component is important in the implementation of a successful audit trail and identifying areas for review. Key components are as follows:
Process
Processes across roles within an organization should be clearly defined ahead of an audit trail review to ensure the review is executed with consistency. Key questions around processes include who is in charge of identifying audit trails for review and how routine those reviews will be at an organization.
Roles
People within an organization should be given roles that are directly related to aspects of the processes they were involved in. Third parties may need to get involved for additional oversight, but first steps should start within relevant teams.
Technology
Technology is the most important component in the data collection used in audit trail reviews and how that data is prepared and presented. The authors recommend how the systems should be set up in a way that mitigates risks to data integrity at the outset and prevents the possibility of the creation, deletion, and modification of records outside of valid processes.
Standards
As the role of third parties in clinical settings increases, particularly as it relates to data collection and monitoring, it is important that organizations have set standards around how audit trail data is presented and accessed when up for review. Ensuring usable and readable access to electronic data is also important if regulators need to access audit trails.
Regulatory Expectations
Regulators are increasing scrutiny with regard to audit trails as the world transitions from paper to digital in all aspects of clinical research. There is an expectation in clinical settings that all stakeholders are implementing strategies to maintain a high level of data integrity. While regulators have not spelled out exactly how organizations should be conducting audit trail reviews, there is the expectation that audit trails are routinely accessible to regulators, and that organizations are putting processes into place that promote data integrity and high-quality data sets.
Areas of Risk to Data Integrity
The use of audit trail review has traditionally been forensic up to this point, although the authors urge routine audit trail reviews to ensure the health of an organization’s data integrity. Based on data integrity guidance from the MHRA, the authors identified five data integrity risk areas where audit trails are used as a primary tool, and where reviews could be used to mitigate those risks. Those are as follows:
System Access
Data access concerns include unauthorized users given system access, a lack of training around audit trails or systems, a lack of oversight in login compliance leading to incomplete data, system integration failures, and a lack of PI (principal investigator) access at a site.
Data Changes
Changes to an audit trail that can cause errors in reporting include:
- Data deletion without corresponding explanation
- Changes to inclusion/exclusion criteria in a trial or other critical data
- Inconsistencies in data or changes after identified timepoints
- Excessive changes
- The timing of those changes to critical data
Data Collection
Risk factors include:
- Data collected outside of noted protocol timing
- Electronic data not collected at the same time as a given event or with suspicious timing
- Inaccurate data based on varying durations of data collection
- Data reported outside of required timing
- Missing data
- Sponsor changes as data are collected
- Inconsistencies in patient response data.
Reporting
Reporting concerns around data integrity include duplicate data sets during the same reporting date and data changes made during the migration of that data.
Device Concerns
Risk scenarios involving device (e.g. electronic Patient Reported Outcomes) concerns include inaccurate date/timestamps due to technical malfunctions and device changes that lead to the inadvertent merging of data.
Conclusions and Next Steps
With risk areas identified, audit trail review becomes an important tool in not only mitigating these risks but becoming a routine at an organization to create efficient, high-quality processes with data integrity in mind. Audit trail reviews will only grow in importance as data sets continue to grow in size, and the need for additional oversight becomes critical with the addition of more stakeholders on given projects.
The success of audit trail reviews as a tool in improving data integrity relies on those who implement the strategy and strong project management support with audit trails impacting almost every facet and function in clinical research. The people behind the strategy must have a strong knowledge base in audit trail review, a handle on the data and how the data was gathered, and a willingness to reach out across stakeholders – regulators, sites, vendors, sponsors and contract research organizations – to tap into the effectiveness of audit trail review.
It is with that willingness to continue to improve these processes when audit trail review truly becomes a tool that promotes a high level of data integrity.
