How to evaluate patient safety event reporting software

What the software does

Event reporting software is where staff record incidents, near misses and unsafe conditions, and where the patient safety team turns those reports into learning and action. A system succeeds or fails on whether frontline staff will use it, and whether the analysis it holds keeps the protections your organization intends. This guide sets out what to ask and test, and names no vendors.

Most products in this category cover the same core jobs:

• Reporting from any workstation or phone, with short forms that vary by event type, and anonymous reporting where your policy allows it.
• Triage by the patient safety team: confirming the category and harm level, routing the event to the manager of the area involved, and flagging serious events for leadership.
• Investigation, such as root-cause analysis with a timeline, interviews, contributing factors and causal statements.
• Action tracking, so corrective actions have owners and due dates and are checked for effect.
• Feedback to reporters, so staff see that reporting leads somewhere.
• Export to a Patient Safety Organization (PSO), often in a structure aligned with the AHRQ Common Formats, and reporting for committees and the board.

Who should be on the evaluation team

• The patient safety officer and the staff who triage reports every day.
• Risk management, for events that may lead to a claim, and quality staff who run root-cause analyses.
• Frontline reporters: nurses, physicians, pharmacists and support staff on different shifts, who will judge the form in the first minute.
• Department managers who follow up events in their areas.
• Your PSO contact, to review what the export contains.
• Counsel, for how patient safety work product is designated and protected.
• Privacy and IT, for access controls, EHR patient lookup and single sign-on.

Questions to ask and things to test

Ask each vendor to show these with your own scenarios rather than a prepared demonstration:

• Reporting a near miss. Ask frontline staff to report one on a phone and on a workstation, and listen to what they say about the form. How long did it take, and which questions did they skip or misread?
• Triage. Classify a set of sample reports. Do the categories and harm levels match how your team classifies events, and can they be configured when they do not?
• Root-cause analysis. Run one sample analysis from opening to an approved action plan, then follow an action to closure and check how its effect is recorded.
• Protection of reporters and analyses. Check, role by role, who can see a reporter's identity, and how the system marks information as part of your patient safety evaluation system and records when it entered and any removal before reporting to a PSO.
• The PSO export. Review a sample export with your PSO. Which version of the Common Formats is it mapped to, and how is the mapping updated when AHRQ releases a new one?
• Role separation. Confirm who can triage, who can approve analyses and who can close actions, and that the audit trail shows who viewed or changed an event.
• Patient lookup. If reporters select the patient from the EHR, test the interface and what happens when it is down.

Limits to probe

Every product in this category has boundaries. Ask vendors to state theirs in writing:

• Software is not a PSO. A reporting system can support your patient safety evaluation system and prepare data for the PSO you contract with, but it is not the PSO.
• Privilege is not a setting. Whether a report or analysis is protected as patient safety work product depends on your PSO arrangements, your policies and advice from counsel. Software records the designation you make.
• Common Formats alignment. Formats are versioned by AHRQ. Being structured to align with them is not the same as being mapped to the version your PSO uses.
• Judgment stays with people. Harm levels, causes and actions are decided by your patient safety team, not by the system.
• External reporting. Many products do not submit state adverse event reports or accreditor reports for you; your team prepares them from the data held.
• Anonymous reports. They cannot be followed up with the reporter, and their value depends on what the reporter writes.

Regulations and standards to understand

Confirm what applies to you in the current official text and with your counsel and PSO. The main sources:

• The Patient Safety Act. The Patient Safety and Quality Improvement Act of 2005 (42 U.S.C. 299b-21 to 299b-26) authorized PSOs, which work with providers on confidential, aggregated patient safety data.
• The Patient Safety Rule. 42 CFR Part 3 defines a patient safety evaluation system as the collection, management or analysis of information for reporting to or by a PSO, and defines patient safety work product, which can include root-cause analyses. It excludes the medical record and other original patient or provider information, and information kept separately from the patient safety evaluation system. Subject to exceptions, patient safety work product is privileged and confidential, and knowing or reckless disclosure of identifiable work product in violation of the rule can bring a civil money penalty.
• AHRQ Common Formats. AHRQ publishes Common Formats so patient safety data can be compared across providers. Under 42 CFR 3.102, a PSO must collect work product in a standardized manner to the extent practical and appropriate; for continued listing it certifies that it uses the Common Formats or an alternative that permits valid comparisons, or explains why neither is practical.
• CMS Conditions of Participation. Under 42 CFR 482.21, a hospital's quality program must track medical errors and adverse patient events, analyze their causes, and implement preventive actions with feedback and learning throughout the hospital.
• HIPAA. Event reports often contain protected health information. The business associate definition in 45 CFR 160.103 includes patient safety activities, so check the agreements with your PSO and your software vendor.
• Accreditation and state law. Accredited hospitals also follow The Joint Commission's Sentinel Event Policy, and many states have their own adverse event reporting laws. Use your accreditor's current policy and your state's statute.

Implementation risks

• A form nobody finishes. Long forms suppress reporting. Pilot with frontline staff and cut questions the safety team does not use.
• Taxonomy changes. Moving to new categories or harm scales breaks trend lines. Decide how historical events are mapped, and tell the committees that read the trends.
• Protections set up late. Agree your patient safety evaluation system policy with counsel before go-live, so the system's designations match it from the first report.
• Access drift. Managers and committee members change roles. Review who can see reporter identities and analyses on a schedule.
• Silence after reporting. If reporters never hear back, reporting falls. Plan who gives feedback.
• Migration. Decide whether open investigations and actions move to the new system or are finished in the old one, so nothing is lost in between.

Related reading on this site

Requirement pages: CMS Conditions of Participation for hospitals (42 CFR Part 482), The Joint Commission standards, HIPAA.

In the glossary: Medicare Conditions of Participation, Joint Commission, HIPAA, PHI, EHR and EMR.

This guide describes the category, not any one product. It is general information, not legal advice: for what a rule requires of your organization, read the rule itself or ask your counsel.